I think the real problem is config files either in hard-to-parse-correctly custom ad-hoc formats or even "config files" written in a scripting language (-> impossible to parse).
All config files should use the same standard format. I'd say "like YAML", but I'm not aware of a widely-used format with standard features like file includes or data types beyond "int" and "string" (e.g. for time intervals; these really shouldn't be "just a string... with a custom format").
I think it would be better to just start with a Turing complete language. I think they should use Lua. It has very simple general data structures that are self explanatory, and its very lightweight and sandboxable.
The only issue is combining config files with other programs. You don't want to strip the comments or formatting when you modify it with another program. Also wish there was a way to specify metadata, like what the possible values for a variable are allowed to be. Or descriptions and help info. With that you could easily convert config files into GUIs.
No! Turing complete config files are even worse than ad-hoc config files.
If your config format is turing complete, you couldn't correctly modify config files automatically. (You wouldn't even know how long until you're done evaluating it.)
If you need more logic, put it in your program or have a plugin system or write a program that generates the config file, but don't put it in the config file.
Many projects start out with just simple config files. But then they realize they need to do logic, and hack that into it. Then they realize they need more complex logic, and hack even more stuff in. And it's just a mess. It would have been much cleaner if they just started out with a scripting language.
Whether you should be putting logic in the config file is a different issue, but as long as people do it or have a need to do it, it's much better than crazy ad hoc solutions.
See these discussions on the issue: https://stackoverflow.com/questions/648246/at-what-point-doe... https://medium.com/@MrJamesFisher/configuration-files-suck-6...
But as soon as scripting is supported, it's impossible to write tools that process the config files and always work, especially with untrusted config files. You can't have both.
So the question is: can code be separated from config (like I've proposed above) (the code can still be inlined in the config file, as long as the "root" of the config file is declarative and the boundaries are well-defined)?
If no, which is more important: parseability or flexibility? It's a tradeoff.
My personal opinion is almost exactly the opposite. If the program's config file requires anything more complicated than a regular language to specify, it's doing too much.
Not sure how this compares http://raml.org/
1. Lack of adption: It's not widely used yet. Implementations for few languages.
2. No formal spec; the "spec" is very imprecise and seems to try very hard to leave as much as possible to the implementation.
3. The spec is very, very Java specific. No clear separation between "core HOCON" and "Java extensions".
4. It probably doesn't lend itself well to automatic changes in a way that preserves structure of an original file (-> without just rendering out JSON).
It's clear that HOCONs only real focus is being easy to edit manually.
It's a nice idea, but it's definitely not the long term solution we need.
I can't think of a single format that would lend itself to all cases, but I agree that config files should be in a standard format (i.e. a format that is supported by parsing tools).
YAML++ for being very readable.
The complexity this brings can be overwhelming compared to an ad-hoc config file format.
But your complexity comparison is unfair. Ad-hoc file formats are overwhelming. Users don't see the complexity because there is no spec and they just write config files by example. Developers generally either give up or write something that doesn't behave quite the same. A fair comparison would be:
""""
XML looks like this:
<section>
<subsection>
<key>value</key>
</subsection>
</section>
"""
Making documentation for graphical programs often requires screenshots and sentences like "click the third radio button on the right-hand section" that add nothing to the documentation and are easily misunderstood by hapless users. Then the developer changes the layout of the dialog box and you need a new set of screenshots. I'll grant that GUIs are more discoverable to a casual user.
I seem to recall that it's possible to automate GUI software but it seems fraught with peril in a way that automation of a commandline and text config system simply is not.
Try automating the deployment of a new IIS server 8 years ago. Hell, try it today.
> Try automating the deployment of a new IIS server 8
> years ago. Hell, try it today.
"Download this, and this. Then run the Installation Wizard and select this and this and this, and this if you want it but you might not need it. Then install this and reboot. Then run the New Server Configuration Wizard Utility Package. Then ..." (cont. 94 pages) -- every tutorial.
Contrast that with Ubuntu, where installing F# took all of 5 minutes, with one command, 200MB and I had a full IDE and F# support.
The one concession I'll make is the one you yourself seem ignorant of - ease of use pertains to your expertise with the system. If you grew up on Windows, you may get its idiosyncrasies.
Unless you want the latest version then you are cloning a few repos, compiling and hunting dependencies because building mono + ide isn't straightforward.
OpenSUSE Tumbleweed is rolling-release and we use all of the same (free software) QA and build systems we use for OpenSUSE Leap and SLE to build and test it. Not to mention that SUSE essentially mirrors packages between OpenSUSE Leap and SLE (our enterprise distribution).
I've been told by some of the people working on Tumbleweee that there's been a lot of people switching from Arch to OpenSUSE Tumbleweed because the packages are released much faster (which appears to be the case from my usage of Arch and TW). But if you're looking at having minimalist installs, there's still some work left to do (minimal "server" installs are still a bit too bloated, and --no-recommends isn't the default).
But yes, there is a rolling-release distribution backed by a company.
Imagine mobile OSs with just the Unix permissions system, the malware spread on those would be so humongous, it'd almost be funny again (arguably this was a long-time problem anyway privacy-wise, with software requiring privileges that couldn't be faked (e.g. giving the application a fake address book instead of your own), but at least apps couldn't easily nuke/hijack all your personal files.)
And i suspect they did it more to get onto government approval lists than anything else (though it may also placate the *AAs).
As long as you only do the things you're allowed to do. I just replaced my 6-year old Windows gaming machine, the only win machine I have. I can't even change the windowing theme - it has to be MS's preselected graphics. I wanted to turn off all the 'phone home' stuff except updates and windows defender; these are spread through half-a-dozen locations. I hadn't even got to install my first bit of software yet (firefox) and already I was limited over what can normally be done with a desktop.
"Just works" isn't really an argument when it's paired with "but you can only do these things".
Not to mention that back in the server world that lightweight virtual servers are an impossibility with Windows arena. ^nix servers are small in volume size, can run on fumes, and are largely disposable. Windows servers are (relatively) huge, slow to launch, require much more system resources, and require licensing. That isn't "just works" for me.
> [text config] This is often ok, but it does make it hard to integrate with GUI tools, hence the lack of them.
Only if your GUI tools are written from the viewpoint that nothing else should touch the config file. After all, if I can write a bash script that upserts a config value in a text ini file, why can't a GUI programmer?
Ha, I'm server tech at a hosting company and this is spit-take worthy.
I really can't see how anyone could possibly think this.
Unless you're talking end-users local PC's this is wrong, and even then it generally isn't 'Linux bugginess' it's 'user ineptness'
On my desktop system I want apps installed in isolated directories, with few or no central dependencies. Even if it means I have some unpatched vulnerability in 10 places.
For a server system I don't mind having to tinker, and central libraries can even be a security win.
Use it for a month and all manner of oddities bubble to the surface.
Also by "just works" I meant mostly being able to get binaries from a random website and installing without having to use a package manager or failing to have the right deps. If you go to the 100 biggest app sites (Skype, Spotify, ...) and try to set up from a download, the "just works" is probably a lot better on win and Mac. This is of course a lot due to the size of the market, but it's no secret that standard cross-distro/desktop-env prebuilt binary installers for GUI apps are still not exactly a strong point on Linux.
I don't see how you could consider package management as a bad thing. Why do you consider "downloading binaries from a random website" to be a "good thing". Not to mention that those binaries almost never update themselves and how well they deal with dependencies depends on what $500 installer builder they used.
Package managers allow you to always keep your system up to date and you have a single database of all software that has been installed, what it's dependencies are and what files it did install (so you can uninstall it). They are definitely one of the awesome things about Linux package managers. OS X has homebrew, but it's not well integrated into the system because it's a third-party library of software. BSD's package managers are at least 10 years behind Linux's (they're still working on packaging the base system). Windows has nothing AFAIK. Things like OBS allow you to automate the release of new versions, OpenQA allows you to do automated QA testing to make sure there's no regressions in graphical or terminal services.
I especially don't understand why you think that not having a way to update the libraries on your system is a good idea. Packaging the same DLL in 30 different places is not a good thing.
2. Shared libraries are only good for saving space (largely irrelevant on desktop) and for security. Different side by side versions of libs work poorly when you reach "system" level is my experience. Example: having apps that require different incompatible glibc versions is painful. For example: see the accepted answer to this question http://stackoverflow.com/questions/847179/multiple-glibc-lib... "The absolute path to ld-linux.so.2 is hard-coded into the executable at link time" WAT?
I think the difference in mindset between what desktop computing is, and what a "system" is, is very different bewteen a linux user and a windows user (i.e. one that just wants an OS that is a dumb layer for running binary compiled shitware that must be built and distributed by its creator because it will never be in a repo).
"What Windows users know" doesn't mean that it's a good thing. Windows users also know to run everything with administrative privileges. For sufficiently sophisticated build systems (read: OBS) you can automatically rebuild packages. The reason why packaging takes time is because there is a testing process (which can also be automated with things like OpenQA), but there's lots of other maintainence that goes on when curating packages. Believe it or not, but sometimes upstream is downright irresponsible when doing version bumps and it's the maintainer's job to deal with it. It's fairly thankless work, to be honest, because you're not working on the new hot stuff. Sure, "just download a binary" works until you have multiple components that depend on each other.
> 2. Shared libraries are only good for saving space (irrelevant on desktop) and for security. Different side by side versions of libs work poorly when you reach "system" level is my experience. Several libc versions etc is painful.
"Only good for [...] security" is enough reason for me. Tell me how Windows programs deal with updates to critical libraries that everyone uses separately. I'm guessing the answer is "not well at all". And if you're going though your package manager, then no package should require a specific version of libc (besides, this problem can be mitigated somewhat with symbol versioning). The gains far outweigh the perceived costs IMO.
> "What Windows users know" doesn't mean that it's a good thing.
I know (I also removed it). It's patently stupid. Let me rephrase it if you want one way of distributing apps that anyone can use, it's basically the only working way. Download a binary from the creators' site. Otherwise you end up with the utterly broken method of "check if it's in a tree in some package repo, if not, you can add more package sources to your repo, or if not, you check if you can find a downloadable package for it, if not, you build from source".
> Tell me how Windows programs deal with updates to critical libraries that everyone uses separately
They don't. It's both a bug and a feature. OS libraries are updated of course (by windows update) but I don't necessarily consider e.g. a C++ runtime to be an OS library, even if it's microsofts' own redist. I prefer my applications to ship their own copy of their c++ runtime and keep it local because it limits problems. Even at the cost of having an unpatched one somewhere.
> Windows users also know to run everything with administrative privileges.
Well, accidentally answering "yes" to the UAC prompt is about as likely as accidentally sudoing something imo.
On the other hand sometimes in the internet I see "advices" like "Windows should be reinstalled every six months" and wondering what the hell these people are doing with their computers?
Why?
We aren't there yet, but Linux is trending towards xdg-app and appstream-esque projects producing a "common" nomenclature for software. Then you can write once install anywhere sandboxed app packages. All you really need are abstractions for both the package manager specific naming conventions and the system specific MAC filter.
1. Legacy and convention. Why do my 64-bit system files live in C:\Windows\system32? Why is the first volume on my system C and not A? Why are there multiple 'global' window stations on my system? Why is the real path to my disk drive \GLOBAL??\PhysicalDrive0, but for some reason I have to use a different path (\\.\PhysicalDrive0) in my programs or else it won't work; a path that I can't discover by myself but have to be told to use by scouring the darkest depths of MSDN? What the heck is ipv6-literal.net and why do I have to refer to some weird third-party domain to connect to a file share on IPv6 within my own network? Making these changes would be disruptive for the software that has to make the change, and impossible for the software that can not be modified. Distributions exist that try to improve the hierarchy (GoboLinux) but no one adopted them. We're currently seeing a push to unify the / and /usr hierarchies, so at least in the future things will get a bit simpler here.
2. Legacy and convention. In the days when storage was scarce, you could keep the contents of /usr/share and /usr/lib on central file servers; a single export for the former could serve all your clients, and you'd only need a single instance of the latter for each architecture in use, rather than having a separate copy on each machine. Besides, even in a world where each program lives in its own directory, as soon as you want one program to install a component for the other to consume, you have to bring in a package manager to remember the fact that /app/A installed a plugin into the /App/B/Plug-Ins directory... not to mention the unusably long PATH environment variable that would result... I find the package manager approach is overall superior to unreproducible the crap-fest you get when applications arbitrarily dump files all over the system.
3. Perhaps I'm in the minority, but I've never had problems relying on glibc via dynamic linking; you just have to build against the oldest version that you want to support, which is a bit of a pain but it's hardly the end of the world. The tradeoff with musl is, that you can no longer rely on dynamically loaded NSS and gconf modules. If you don't need these, fine, knock yourself out--but you should be aware of the tradeoffs when you switch your libc implementation out; namely that you can no longer use mdns, myhostname, ipv6literal, winbind, LDAP, etc. for looking up hosts, users, groups and so on.
4. System-wide configuration is better kept in text form where it can be read by a human, contain comments, and be kept in Git. Desktop programs can store their config in dconf, or otherwise do whatever they want as long as it lives in ~/.config and I don't have to care about it.
5. As long as you use “set -eu” with an understanding of its shortcomings, and know how to quote variables properly, writing small and medium systems in shell is a great tradeoff between development speed and robustness. Components can be rewritten in a real systems programming language once they stabilize or require better integration than can be had by parsing the output of other commands.
So, I was looking this up as I havn't used windows in forever. I then did a `whois ipv6-literal.net` and was surprised that Microsoft doesn't own it. That seems weird for them to use it in such a way?
Domain Name: Ipv6-literal.net
Registry Domain ID: 1915314004_DOMAIN_NET-VRSN
Registrar WHOIS server: whois.NameBright.com
Registrar URL: http://www.NameBright.com
Updated Date: 2015-09-26T00:00:00.000Z
Creation Date: 2015-03-31T18:16:33.000Z
Registrar Registration Expiration Date: 2016-03-31T00:00:00.000Z
Registrar: DropCatch.com 577 LLC
Registrar IANA ID: 2057
Registrar Abuse Contact Email: [email protected]
Registrar Abuse Contact Phone: +1.720.496.0020
Domain Status: clientTransferProhibited
Registry Registrant ID:
Registrant Name: lirong shi
Registrant Organization: www.Juming.comIf you want paths up to 32k you need to use back-slashes.
I think this is the reason they are not switching e.g. Visual Studio over to max 32k paths.
That's so accurate. I used to write shell scripts with
command $1
foo() {
for arg in "$@" ; do
echo "arg is \"${arg}\""
done
}
foo 'bar baz' 'spaces in filename.txt'
When writing shell scripts it's a good idea to use the -- option whenever possible
stupid_backup() {
cp -a -- "$@" /stupid/backup/dir/
}
# copies 2 files
stupid_backup --files '-with -leading -dashes'Comparability is important, and not just for existing tools. Choosing something different has a learning cost, and if you have to care about both versions you have ongoing mental effort costs as well.
There may be better names for "/etc", but it's not worth the effort.
What about things like encap/gnu stow which symlink files from one single package dir.
"Unnecessary" for you maybe, but have you considered the possibility that there is a reason why the directory structure is the way it is - and that the problems originally addressed may still be relevant? Here is a hint: partitions. Partitioning the files roughly by usage pattern allows you to tune performance, safety and security in a way that you would have a very hard time doing otherwise. Your suggested names make me think that you aren't very clear on the directory's actual purpose [0]. While longer names may have helped you out in understanding their purpose, once you actually learn it you're stuck with an unnecessarily long PWD that wraps each prompt.
> 2. Programs are mushed together and scattered...
Again, consider why it is the way it is. Do you really want to have a PATH that includes every directory for every binary on the system, or manage individual file permissions within all those directories? You want to do that with libraries as well? Just consider the complexity of what you're proposing and how you'd address: system defaults, setuid, per user preferences, dependencies, build environments... Years ago I basically did what you're suggesting, when Redhat was my daily driver. I'd build from source and install into ~/bin. Try it out for a while, you'll hate it.
> 4. Emphasis on text-based configuration files... hard to integrate with GUI tools...
What are you suggesting, a windows like registry? The text-based configs are no more difficult to use through a GUI than a binary representation, they're both a library function call away. Unless you are suggesting a windows like registry... but you've already pointed out how developers don't want to spend time on portability - so that can't be it.
> 7. General bugginess... Windows is a lot more reliable than Linux...
Ah, well try out an OS that is closer to Unix than Linux - maybe one of the BSDs. I'd put my Freebsd workstation against any flavor of Windows in a contest of uptime and performance, that is a bet I'd be happy to take. As far as Windows just working, that is true for the majority of tasks for the majority of users. But if you fall outside of that happy band of the target market, you are SOL. Consider the whole Windows telemetry issue. Also, I just noticed in my network logs that Windows update is sending out IPv6 dns requests despite the fact that I've disabled it on the network interface and tweaked several registry variables... there is nothing I can do about it. It would be a pretty simple fix for any opensource OS though.
No way. But I'd love to gt ride of the plain PATH, replacing it with a hierarchical PATH with a convention. Maybe with every 'bin' directory inside that path getting in the search path, or maybe something that let me nest things deeper.
One can not do this in Unix, and that's the point.
One certainly can, very easily actually: edit you shell rc file in /etc to modify your PATH based on PWD, boom - hierarchical PATH the Unix way. Don't want it system wide? Edit your shell rc in the user home directory. Want something more complex? The posix shell source code is a lot simpler than you'd think. I wanted the same fancy git repo status PS1 stuff found in bash rc scripts, but without the performance impact - and in tcsh. It only took an hour of work to integrate libgit2. I don't think it would have been as easy with cmd.exe.
And I'm hierarchising my ~/bin :)
/etc contains more than just config files so /config would be a misleading (or at least overly specific) name. If we went that route we'd need dozens of top level directories to cover everything. I do prefer OSX's more user friendly directory layout but the traditional directory structure has been around for decades. It works fine.
Any why would you need dozens of top level directories? I can't imagine that you could name even one dozen completely orthogonal aspects of program and system configuration that can't be put into a _some_ meaningful hierarchy.
> How about defending these?
> 1. Unnecessary and confusing directory structure. `/etc`? Why not `/config`?
> `/usr` instead of `/system`, `/var` instead of ... well who knows. The maximum
> directory name length is no longer 3 characters.
Why do I need to edit c:\Windows\System32\Drivers\etc\hosts (in what way is hosts related to bit-length or a driver?)
Why .htm rather than .html?
Why programiwanttorun.exe rather than programiwanttorun
/system is as overloaded a word as any in IT. /config doesn't accurately reflect what /etc is about (but in any case, the latter is not a great barrier to entry)
If you're complaining about limits on directory entries .. you're skating around on very thin ice if you're on the NTFS lake (compared to any of xfs, btrfs, reiserfs, ext2/3/4fs, etc)
> 2. Programs are mushed together and scattered through the filesystem rather
> than stored in separate locations. This basically means applications are
> install-only. Yeah, package managers try to keep track of everything, but
> that is just hacking around the problem, and most developers don't want to
> spend hours creating 5 different distro packages.
Applications are not install-only. Because package managers (or, rather, distributions) managed to solve this problem elegantly more than a decade ago, I don't know how you can credibly make this claim.
Is 'mushed together and scattered' a contradiction?
Building packages for a variety of distros is a solved problem (again, it has been for a decade or more).
> 3. Not strictly Unix, but the mess of glibc with respect to ABI compatibility,
> static linking, etc. is ridiculous. Musl fixes most of this fortunately.
More pragmatically, I rarely (in twenty years) have had glibc issues. I think perhaps 3 times. All easily solved.
> 4. Emphasis on text-based configuration files. This is often ok, but it does
> make it hard to integrate with GUI tools, hence the lack of them.
GUI tools do not intrinsically have an issue dealing with text-based configuration files.
I posit the problem you're describing is the fact that text-based configuration files often contain useful human-readable components (which non-text-file config systems, such as 'the registry' lack) which is slightly harder to maintain using automated tools. But only slightly. As noted elsewhere, these are typically only a library call away.
> 5. Emphasis on shell scripts. Fortunately this is starting to change, but
> doing everything with shell scripts is terribly bug-prone and fragile.
I don't know many people who have significant experience with apt|rpm && sccm (for example) ... but I know a couple, and the fragility of shell scripts leads to fewer expletives than the alternative.
> 6. X11. 'nuff said about that. When is Wayland going to be ready again?
What problems have you had with X11 that aren't dwarfed by citrix / rdp / single-user consoles / etc?
> 7. General bugginess. I know stuff works 90% of the time, but that 10% is
> infuriating. Windows is a lot more reliable than Linux at having things
> "just work" these days.
Network transparency is mostly irrelevant on modern desktop systems, X11 remotes modern apps poorly at best, and if you really need remote desktop access you know where to find RDP and SPICE.
The Wayland switch will be a win because X11 is almost pure cruft. The good parts are what was kept in Wayland.
That depends on your perspective. Why are you accepting connections from malicious X clients?
> Network transparency is mostly irrelevant on modern desktop system
For you, maybe. That's an opinion that many of us do not share.
> X11 is almost pure cruft
It's only cruft if you limit your use cases to stuff like GTK+ that decided to only use X as a dumb framebuffer.
> The good parts are what was kept in Wayland.
Except for a long list of features, such network transparency, support for copying PRIMARY selections in addition to CLIPBOARD, or overriding input events of arbitrary programs. Until Wayland supports these, it isn't compatible with a lot of my tools.
Just because something is "old" or has features that you personally don't mean they are "bad" features that should be removed. There are more use cases than those in you definition of "desktop".
I don't understand this point. Care to explain, please?
GUI tools tend to manage everything at once. Imagine the giant options box from MS Word 2003.
Mind you, there are plenty of GUI applications that get this right anyway. But usually for the vast majority of users there is never a need of mangling configuration files by other means.
My point was really that some/most of those complaints about "Unix" don't apply to OS X for most people and sound more like the pain that users running Linux have to deal with, even though it's not typically an official Unix. I hope this was more clear.
1. Unnecessary and confusing directory structure. `/etc`? Why not `/config`? `/usr` instead of `/system`, `/var` instead of ... well who knows. The maximum directory name length is no longer 3 characters.
2. Programs are mushed together and scattered through the filesystem rather than stored in separate locations. This basically means applications are install-only. Yeah, package managers try to keep track of everything, but that is just hacking around the problem, and most developers don't want to spend hours creating 5 different distro packages.
3. Not strictly Unix, but the mess of glibc with respect to ABI compatibility, static linking, etc. is ridiculous. Musl fixes most of this fortunately.
4. Emphasis on text-based configuration files. This is often ok, but it does make it hard to integrate with GUI tools, hence the lack of them.
5. Emphasis on shell scripts. Fortunately this is starting to change, but doing everything with shell scripts is terribly bug-prone and fragile.
6. X11. 'nuff said about that. When is Wayland going to be ready again?
7. General bugginess. I know stuff works 90% of the time, but that 10% is infuriating. Windows is a lot more reliable than Linux at having things "just work" these days.