News for hackers. Stuff that matters.
Re:Posix Abstractions in Modern Operating Systems: The Old, ... by ZenoArrow on Sunday May 08, 2016 @05:51AM (#11653390)
I would argue passing around all data as text is suboptimal, not only from a performance point of view but also in terms of robustness. Parsing plain text using regular expressions and suchlike can lead to some useful outcomes but it's easy to derive false positives from this. On the other hand a binary-based object-oriented approach can give better performance and make it easier to parse data.

This is just one example of where the old ways aren't necessarily the best. I'm fairly confident we could come up with something better than POSIX if we were willing to make the effort.

Re:Posix Abstractions in Modern Operating Systems: The Old, ... by agoodboy on Sunday May 08, 2016 @10:57AM (#11654285)
> On the other hand a binary-based object-oriented approach can give better performance and make it easier to parse data

please, no. OOP approach means that my process needs to know how to communicate with other processes via very specific protocols, which aren't very well defined (any process could describe its type of object). This is the exact opposite of flexibility. The usefullness of tools like grep or sed would drop drastically, and we would fall back to big blobs of software.

Since I'm just passing data, do I really need the behavior attached to it? How would state persistence be handled?

The text (bytes as ASCII until line ending) approach may seem ugly and dirty, but in fact you can pass list, tuples, maps, trees or just text and is up to the receiver the responsibility to make sense out of the data.

Need data from A but B can't understand it? Use C (which operate on text) to format A's output as B needs. With object how many "translator" (C in the example) would you need to acheive the same result?

Re:Posix Abstractions in Modern Operating Systems: The Old, ... by ZenoArrow on Sunday May 08, 2016 @11:43AM (#11654471)
> "please, no. OOP approach means that my process needs to know how to communicate with other processes via very specific protocols, which aren't very well defined"

It's not as hard as you make out. Take a look at how PowerShell works for an idea about how a OOP-based approach can work for CLIs.

http://www.computerworld.com/article/2954261/data-center/und...

Re:Posix Abstractions in Modern Operating Systems: The Old, ... by zvrba on Sunday May 08, 2016 @12:21PM (#11654579)
> OOP approach means that my process needs to know how to communicate with other processes via very specific protocols, which aren't very well defined (any process could describe its type of object).

That's why we have IDL -- interface description language for RPC calls. An IDL-to-X (usually C) compiler generates the necessary glue so that anybody can talk to the program in question. IDL is also the basis of MS COM, which I quite like from the design standpoint.

> The usefullness of tools like grep or sed would drop drastically, and we would fall back to big blobs of software.

So you teach grep to take an IDL file, invoke an IDL compiler and dynamically load the parser for the protocol in question. Also, if the broker were a standardized, perhaps in-kernel component (dbus, kdbus), you could attach "idlgrep" to any process to trace its calls. You wouldn't be restricted to pipes.

> Since I'm just passing data, do I really need the behavior attached to it? How would state persistence be handled?

The parent's wording was a bit unfortunate. You can have interfaces and interface inheritance and versioning (the "OOP" part), but there's no behavior send between processes.

> With object how many "translator" (C in the example) would you need to acheive the same result?

Exactly one: the IDL compiler.

Re:Posix Abstractions in Modern Operating Systems: The Old, ... by majewsky on Sunday May 08, 2016 @02:26PM (#11655088)
> So you teach grep to take an IDL file, invoke an IDL compiler and dynamically load the parser for the protocol in question. Also, if the broker were a standardized, perhaps in-kernel component (dbus, kdbus), you could attach "idlgrep" to any process to trace its calls.

So you introduce a huge load of accidental complexity because the text interface has some perceived inefficiency? I'll choose simplicity and accessibility over this mess anytime.

Re:Posix Abstractions in Modern Operating Systems: The Old, ... by digi_owl on Monday May 09, 2016 @03:24AM (#11658119)
Ok, now i am confused. On the one hand you rail against complexity, while on the other you favor systemd.
Re:Posix Abstractions in Modern Operating Systems: The Old, ... by majewsky on Monday May 09, 2016 @05:24AM (#11658471)
Yes, systemd is complex. But much of its complexity is justified. I rail against unnecessary complexity.
Re:Posix Abstractions in Modern Operating Systems: The Old, ... by gaius on Sunday May 08, 2016 @01:04PM (#11654727)
Heh, you will only be able to openly advocate IDL once the last CORBA programmer is dead. We've been down that route. It didn't work.
Re:Posix Abstractions in Modern Operating Systems: The Old, ... by pjmlp on Sunday May 08, 2016 @02:29PM (#11655105)
It is called REST and micro-services nowadays.
Re:Posix Abstractions in Modern Operating Systems: The Old, ... by JdeBP on Tuesday May 10, 2016 @03:11PM (#11669736)
Actually, as noted parenthetically above, it's called DBUS (Desktop Bus) and ... erm ... IDL.

* http://dwheeler.com/dbus/

Re:Posix Abstractions in Modern Operating Systems: The Old, ... by Avshalom on Sunday May 08, 2016 @12:05PM (#11654544)
>>process needs to know how to communicate with other processes via very specific protocols

>>and is up to the receiver the responsibility to make sense out of the data

That is exactly the same thing.

Either way you're throwing bytes from one process to another and hoping the second one can do something useful with it.

Re:Posix Abstractions in Modern Operating Systems: The Old, ... by gaius on Sunday May 08, 2016 @06:40AM (#11653488)
It's about the failure modes you anticipate. If a text file becomes corrupt a human can probably still make some sense of it. If a binary file does, you may well be completely out of luck. I mean try it for yourself, truncate a 4G core dump by a couple of bytes and watch GDB try to open it. Now imagine that all your system and application logs are vulnerable to this.

In a world where things are "fixed" by just blowing away a VM and spinning up a new one, you might not care but in that case why bother to log anything at all...?

Re:Posix Abstractions in Modern Operating Systems: The Old, ... by ZenoArrow on Sunday May 08, 2016 @06:54AM (#11653511)
> "truncate a 4G core dump by a couple of bytes and watch GDB try to open it"

This comes down to a failure of the binary design. It's possible to design a binary format that is uniform enough to handle truncation of a few bytes. To give you one example, you have a header for the binary with pointers to the start and end of each data block and you can keep multiple copies of this header. Another alternative is to specify that each block needs to specify where the data in that block ends. That way, even if you have partial corruption you can read the uncorrupted data blocks.

Re:Posix Abstractions in Modern Operating Systems: The Old, ... by lisivka on Sunday May 08, 2016 @08:54AM (#11653842)
It is already invented by ASCII.
Re:Posix Abstractions in Modern Operating Systems: The Old, ... by ZenoArrow on Sunday May 08, 2016 @11:39AM (#11654448)
No it's not. ASCII doesn't give you enough metadata to permit for efficient and reliable parsing of data, you can do a lot better without using ASCII.
Re:Posix Abstractions in Modern Operating Systems: The Old, ... by lisivka on Monday May 09, 2016 @04:26PM (#11662854)
ASCII solved problem of "few bits/bytes lost", isn't? Binary flow is divided into bytes. Bytes are divided into control sequences and text characters. Control sequences contains sequences to separate columns (tab character) and records (new line character). Moreover, all that was standardized across various platforms. It was huge improvement. It is why UNIX sticks to ASCII: to be portable.
Re:Posix Abstractions in Modern Operating Systems: The Old, ... by 1_player on Sunday May 08, 2016 @06:02AM (#11653408)
> passing around all data as text is suboptimal

POSIX is not Unix.

Passing data around as text is a tenet of the Unix philosophy, while AFAIK POSIX doesn't mandate any of that.

Re:Posix Abstractions in Modern Operating Systems: The Old, ... by danieldk on Sunday May 08, 2016 @07:28AM (#11653584)
Actually it does, POSIX also specifies many of the standard (text-based) UNIX utilities:

http://pubs.opengroup.org/onlinepubs/9699919799/utilities/co...

Re:Posix Abstractions in Modern Operating Systems: The Old, ... by ZenoArrow on Sunday May 08, 2016 @06:18AM (#11653443)
> "POSIX is not Unix."

"This is just one example of where the old ways aren't necessarily the best."

Re:Posix Abstractions in Modern Operating Systems: The Old, ... by VLM on Sunday May 08, 2016 @11:21AM (#11654376)
Performance in an absolute number vs scaling.

The lowest hanging fruit gets picked. "Lets dump absolutely everything and NIH the whole thing for a single time 5% performance increase" doesn't sell well when that one time gain is expressed in the amount of time it takes hardware or network capacity to improve 5%, or fixing poorly scaling algos. Also insert the usual analogy of the ratio of the cost of microscopically faster hardware vs the labor cost of extremely expensive rockstar ninja programmers.

Also its assumed that change will lead to improvement because anecdote, or because change is always good. However, "the thing that won uses text, so naturally we gotta get rid of text" doesn't sound like a wise plan.

Re:Posix Abstractions in Modern Operating Systems: The Old, ... by ZenoArrow on Sunday May 08, 2016 @12:06PM (#11654549)
> "However, "the thing that won uses text, so naturally we gotta get rid of text" doesn't sound like a wise plan."

I'd argue the Unix command line ecosystem has only 'won' in the sense that many developers are familiar with it. I don't think it is technically the best we could do if we were starting from a blank slate.

Re:Posix Abstractions in Modern Operating Systems: The Old, ... by thwarted on Sunday May 08, 2016 @12:57PM (#11654697)
Maybe many developers are familiar with it because it is text and thus accessible. Technical excellence matters not if no one is using it. Although I think that accessibility to humans is a major contributor to technical excellence.
Re:Posix Abstractions in Modern Operating Systems: The Old, ... by ZenoArrow on Sunday May 08, 2016 @01:33PM (#11654840)
Look at how Bash scripts are used. You can hack together something quick with them, but at the time you want to do something harder or better people end up using a language like Python or Perl.

Now look at what happens on Windows. Devops can use PowerShell to hack together something quickly, but if they want to do something more complex you have the option to easily add new PowerShell commands and data types because it's all based on .NET so you can pull in any .NET code you want, whether that's something you write yourself or from an existing library.

You could do the same with Bash, but people don't do as often, I would suggest this is because it would tend to rely on plain text and regular expressions, and programmers tend to prefer using better specified data types.

Re:Posix Abstractions in Modern Operating Systems: The Old, ... by Blackthorn on Sunday May 08, 2016 @10:53AM (#11654260)
Microsoft PowerShell does this, and it's pretty neat. Of course it's less popular than bash so now they're bringing bash and text-based pipes onto Windows! Ergh.
Re:Posix Abstractions in Modern Operating Systems: The Old, ... by stepvhen on Sunday May 08, 2016 @11:46AM (#11654490)
Bash + coreutils is powerful. Putting that on windows is a very good thing. The "performance increase" is irrelevant compared to the ease of use, available documentation (man pages), interoperability, and maturity.
Re:Posix Abstractions in Modern Operating Systems: The Old, ... by jschwartzi on Sunday May 08, 2016 @01:33PM (#11654843)
It's worth noting that the traditional UNIX text-based utilities are designed for reporting to a user. Using text to generate reports is the most intelligent thing they could have done, because humans universally process text. It's not a superior programming environment, but it's a superior reporting environment, which is helpful when you're trying to reverse-engineer some third-party ELF binary or debug why your shared memory object sometimes causes clients to crash. That we sometimes take strings from our reports and pass them into our shell is accidental.
Re:Posix Abstractions in Modern Operating Systems: The Old, ... by pjmlp on Sunday May 08, 2016 @02:26PM (#11655087)
The same can be told of the REPL environments of Smalltalk, Interlisp-D, Lisp Machines, Mesa/Cedar and Oberon.

All more powerful than UNIX shell languages while using structured data.

Coming from Spectrum, MS-DOS and Amiga, the UNIX shell seemed powerful until I discovered how the development workstation should look like, from the eyes of Xerox PARC.

Re:Posix Abstractions in Modern Operating Systems: The Old, ... by JdeBP on Tuesday May 10, 2016 @03:08PM (#11669701)
At which point you switched to emacs and never looked back? (-:

Amiga REXX should have militated against that view of Unix shells slightly.

Re:Posix Abstractions in Modern Operating Systems: The Old, ... by ktRolster on Sunday May 08, 2016 @02:26PM (#11655090)
the biggest problem with bash on windows is that it has poor interop between Windows tools (to the point of almost no interop). In that sense, cygwin is already better than the recent Microsoft offering.
Re:Posix Abstractions in Modern Operating Systems: The Old, ... by eru on Sunday May 08, 2016 @07:23AM (#11653570)
What does binary vs text have to do with object orientation?
Re:Posix Abstractions in Modern Operating Systems: The Old, ... by ZenoArrow on Sunday May 08, 2016 @11:48AM (#11654492)
It doesn't, but it gets around the extra overhead you get with the added metadata.

OO on its own is slower to parse than plain text, but binary is faster to parse than plain text. If you combine the both can you get the best of both words, something that's both reliable and fast.

Re:Posix Abstractions in Modern Operating Systems: The Old, ... by eru on Monday May 09, 2016 @08:39AM (#11659313)
I see that binary might be interesting. But why add OO to the mix? There are other worlds to mix the best from.

As one silly example, you can consider Google's protocol buffers. They are not particularly OO-y. (I have great sympathies for functional languages, but I don't think they offer too much insight into how to format your data files. And neither does OO?)

Re:Posix Abstractions in Modern Operating Systems: The Old, ... by woodman on Sunday May 08, 2016 @12:02PM (#11654535)
It is a "enemy of my friend is my enemy" sort of thing, the minimization of a shortcoming. OOP stumbles on text (because of object methods) in a way that no other programming paradigm does. So obviously that means that text is stupid and we never really liked the benefits of text anyway!