I think this is a human thing, not necessarily just kids today.
Can we reform it, perhaps into a "D-shell," that is an LR-parsed language?
AWK had a yacc grammar for many years. Why can't this be the language of the shell, at least for the sake of maintainability and comprehension?
https://archive.fosdem.org/2018/schedule/event/code_parsing_...
Also see scsh and rash for scheme
Fish for non posix but sane syntax
PowerShell?
$ ll /bin/dash
-rwxr-xr-x 1 root root 85368 Jan 5 09:52 /bin/dash
The POSIX shell is common, from the largest supercomputer down to the smallest ARM.
I don't think that there will be any new developments covering that full scope.
It's stupid and almost always leads to unintended (and usually negative) consequences, but it's probably something strongly ingrained in our instincts because it rears its ugly head every time there's a generational turnover.
Sooner or later after you've had your fun refusing and denying your predecessors, you realize why they do things the way they do, become wiser, and go back and undo and redo all the fuckery you caused with your refusing and denying.
It would be nice if we could get that realization without first having to fuck everything up, y'know?
As the saying goes: "good decisions come from experience, and experience comes from bad decisions". It would be great if we could all come out of the womb with enough education and experience to function as responsible adults in modern society, but we don't. Society (rightfully) allows people some leeway in doing dumb shit so that they can learn firsthand why it's dumb.
It's all cute if this just happens during our childhood years, where most actions hold no real consequence. It's when this extends into the real world with very real consequences that it gets really wasteful.
Licensing and training has a very real cost as well of course, and for many topics the cost of forcing everyone to get a license would outstrip the benefits. So you either have to spend on training, or you have to spend to fix the mistakes of untrained people, or a mix of both. There is no way to bring the waste to zero.
Discussed 13 months ago:
https://news.ycombinator.com/item?id=30070757 (142 comments)
Its opposite is "cargo culting".
If there is no documentation on it, no-one around to pro-actively explain why it’s there, then the most economical way to see if it had any positive value is to stash it and see what happen.
That doesn’t mean it’s always the most rational thing to do, of course. Throwing POSIX away doesn’t qualify with a situation where there is a lake of provided documentation or a missing obvious purpose.
The fallacy is well confined in these two sentences:
>However, before they decide to remove it, they must figure out why it exists in the first place. If they do not do this, they are likely to do more harm than good with its removal.
Actually, assuming you don’t know consequences of removing the fence, you also have to assume that you don’t know consequences of letting the fence in place, which might just as well turn to a dramatic situation. What is for sure, if nothing around can explain its purpose, is that it was placed here by people to negligent to induce the right clues about it. So, who know to which level their laxity might have reach?
Remove the fence unless someone know why it was put up in the first place, and the rational still hold.
In the case of POSIX it might hold, since probably the worst case is that some volunteers throw away a few years of their life hacking away on yet another OS that will never gain traction. Humanity spends a lot more time on a lot less worthy endeavors.
You still don't escape the requirement to understand the problem before fixing the problem. Even if the explanation for the current solution is lacking, you still need to understand enough to articulate why it's lacking.
Whomever wants to tear the fence down has to do basic research -- visit city hall for the plans, talk to old-timers, check the old newspapers, etc... If they do it, this qualifies for "understanding". And yes, maybe the fence was put by Crazy Old Bill who was paranoid about neighbour's goats... then yes, tear it down. But make sure it is really him first, and not some better reason.
> assuming you don’t know consequences of removing the fence, you also have to assume that you don’t know consequences of letting the fence in place,
Note that at some moment there were no fence, and yet someone has expended non-trivial effort to put the fence up. This means someone had a reason do to so, so whomever that past person was, they knew more than you did right now. Therefore, unless you believe that you are smarter than most other people, you should defer to their past decision.
(And if you believe that you are smart that most other people, and yet want to tear down the fences you don't understand... I have bad news for you :) )
Well, for every fence at the middle of a path that have for example a a commemorative value, that seems a good start. And "warning do not remove" is a rather common thing on places where a danger is expected otherwise.
>Whomever wants to tear the fence down has to do basic research -- visit city hall for the plans, talk to old-timers, check the old newspapers, etc...
I think that qualifies in "someone know why it was put up in the first place, and the rational still hold".
>Therefore, unless you believe that you are smarter than most other people, you should defer to their past decision.
I don’t think smartness matter that much here. It’s all about blindly rel invariably in a cargo cult fashion about everything or keep critical thought vivid when it seems possibly relevant. Someone can be extremely smart and still follow rigorously the cult of the day.
-----------
Some thought experiment now. :)
Say you won a new house, congratulation!
In the middle of the garden, crossed by a path, there is a fence with no door, easy to circumvent, serving no apparent purpose. The planes you where given with the house, duplicate of those from the city hall, don’t mention it. Now, there might many explanation for this situation. One possible rationale would be, the fence used to be larger, possibly even forming a closure some time in the past. But no one know. Will you let let the fence disrupting the path in the middle of the garden? Will you call some expert to diagnose the soils, just in case it might collapse would someone remove the fence?
Also, in the middle of the living room there is a thick wall up to the ceiling, with large space to pass on both side. It’s not mentioned in any plane. Having a unified big living room would be fare more nicer to you. Will you destroy the wall without delay? Or call an expert to determine if this is a bearing wall first, without which the house would collapse?
So this is really largely dependent on context, as usual.
That assumption reads as false to me. There should generally be historical data around the costs and consequences of maintaining the fence. E.g. if you're shutting down a server, it's generally easy to figure out how much the hardware costs, how many man-hours you spend on maintaining it, etc. None of that requires knowing why the fence exists.
Those costs and consequences tend to be small, because large costs have to be continually validated as a good use of resources so they have a living memory of sorts.
> What is for sure, if nothing around can explain its purpose, is that it was placed here by people to negligent to induce the right clues about it.
Again, I don't think this is sure. I've seen lots of cases where docs probably did exist at some point, but the writers left and successors lost the knowledge of where the docs are or failed to migrate them to a new system or etc. They're lost to attrition, not to negligence.
> Remove the fence unless someone know why it was put up in the first place, and the rational still hold.
It doesn't, because it's trading known risks for unknown and unbounded risks. That's usually a bad trade, especially when the known costs tend to be small. Nobody wins a promotion for causing a million dollar outage while trying to get rid of a $50/month maintenance cost.
More broadly my underlying point was more as "don’t blindly follow the herd obedience".
If there is a small black box in multi-million generator stream process, no one of course want to be the guy unplugging it just to see what happens.
On the other hand, if really no-one knows the real impact of such a black box, given the resources at play, risk management should induce an audit and possibly a replacement management. To my mind that sounds very different from "no one know why this thing is there, those who dare to ask will be chastised with a burden of proof to carry alone".
Around here, on HN, Chesterton's fences pops up so many times as some type of ageless nugget of wisdom.
But another way to look at it is that it's just the lazy guy's excuse to allow bit rot. Because, let's be honest, we only care about Chesterton's fence in the context of software development.
Who among us did not work with software with dark corners, with files with no commits in the last 10 years, with playbooks that people just go through mechanically, because there's no one from the "old guard" who remembers why things are the way they are?
Who would not love to live in an alternate reality, where each Chesterton's fence was removed at the first opportunity, rather than left there to burden future generations of unlucky toilers? Where code is clean, documentation is up to date, tests complete, colleagues who know what's going on? Well, that alternate reality does not exist, of course.
But at least let's stop thinking we are wise whenever we postpone deprecating something we don't understand, just because some semi-famous essayist came up with a nice figure of speech one hundred years ago.
Reinventing things without learning from the past - and if you point it out people gets so defensive.
This opinion piece is the epitome of a negative impact work. It's easy to accuse random things of being all wrong, but it's far harder to actually present something you feel is right.
So to me it seems like criticism without suggestions is very valuable, it is called data. Suggestions from amateurs are usually bad and misleading, just highlighting a problem so that the expert can come up with a solution is the most efficient way to work in most cases.
If you don’t know why the customer was unhappy, what do you change? You can fire the customer service rep, but then what do you do when the next one gets reviews just as bad because they are following the same script?
If you bitch about how "this food is terrible" but you do nothing to make it more palatable or order something else and instead just stand there eating while complaining and reordering and asking for seconds... The food is not terrible, and you just enjoy bitching.
Talk is cheap. If even that same critic feels that his solution to his problem is not worth pursuing, the criticism adds nothing and creates no value. At best it's nitpicking about stuff that falls in Eisenhower's "not important"/"not urgent" box.
2 sides of the same coin.
Also, it can be fun to explore existing ideas and build upon those too - nobody forces you to maintain anything existing "for fun".
1. Build inside the ecosystem as it was designed. Eg, make a static webpage using HTML. Make a linux binary using the standard linux tooling + apt (or whatever). This is almost always a good idea when you can, but sometimes the ecosystem is a bit shonky, or misaligned with what you're building. Eg, I want my program to have a fixed execution environment and instead I have debian and macos. I want a react-like API but the browser doesn't provide that. Etc.
2. Build a new thing on top of the ecosystem as it exists. Eg, build docker on top of linux. Build web frameworks. Invent the web browser, or couchdb views, or electron and build your software on top of that. The problem with this approach is that if each generation layers their own rubbish on top of the underlying layers then software will keep getting slower, more complex and more buggy over time.
3. Change the underlying system to work how you want it to work. Instead of using docker on top of ubuntu, replace ubuntu with nixos. Instead of using dpdk for higher storage performance, add io-uring to the linux kernel.
I think I would prefer it if more people did option 3 rather than option 2. I think there's lots of ways modern computing systems could be better. I'd rather if linux gained the ability to run binaries from the web (for example) than we invent a new kind of binary in userland (docker containers) and run those. I don't want people in 100 years to still be using linux (with all the bad decisions it contains), but with 10 more layers of different generations' ideas stacked on top. (Oh, its linux - but with docker, and then most people run this specific container to run languageX, and that lets us run electo2050 apps, which in turn you can use to run IRC. It only uses 50 gigs of ram. So small and light!)
Option 3 may require us to throw out or deprecate parts of the POSIX API. That may be overdue.
1. Can you do X with existing tooling?
If not
2. Can you add a tool to your set to do X?
If not
3. Get a new tool set.
2 should also serve as testing to show the utility and iron out the links of things that make it to 3. Jumping straight to 3 likely means you end up with an ill thought out solution that soon needs another 3.
All of docker’s features could be built into the Linux kernel. And it would probably be better as a result, and have ancillary benefits to existing Linux programs. But docker was built as a separate userland tool instead.
Building a new thing is usually easier, since you don’t need to dive into the politics of an existing community. But if each new generation of software engineers builds new software by layering on top of the old stuff, then we end up with layers on layers of cruft and our computers run slower and are more buggy as a result.
The POSIX apis aren’t very good in lots of ways. With a lot of complex effort in userland I can make a database with atomic write semantics. But maybe we should just add atomic file operations to Linux instead. In the long run I think it would be safer and result in faster programs in most cases.
If you just put every half thought out, poorly implemented idea straight in the kernel, you'd have a 5 petabyte kernel.
[1] https://opensource.apple.com/source/Git/Git-26/src/git-htmld...
Of course, you must already have some intuition that something could go wrong. :D
If that sounds difficult just create a branch prior to merging so that you know where to reset your failed branch to.
I don't think he's wrong that the POSIX synchronous IO model isn't a great fit for modern hardware, though. He doesn't really go into it much but (Windows) NT's async-everything IOCP model really seems to be the winner, as far as generic abstractions that have stood the test of time.
Opinions don't really matter. Solutions to real-world problems do.
To me, the fact that no one up to this day felt that the issues in this area was significant enough to warrant a fix or an alternative tells me that it's noise about nothing.
Do you not view io_uring or Windows IOCP as alternatives? Or the plethora of other non-POSIX IO extensions in Linux, such as splice and sendfile? SPDK and DPDK? There’s also Google’s Fuschia operating system, which is not Posix. I think it’s pretty clear that people are investing in a variety of alternative approaches.
To look at things from another angle, what if having had POSIX all these years actually made things more uniform, in the sense that we didn't have variance between systems as large as e.g. between a BSD Unix and a Lisp machine? What if that were our choices today. As it is the largest mainstream departure is maybe Windows.
They're great tools but they're not great abstractions. If you use them directly for all I/O, then every application in every programming language will end up looking JavaScript, which even with Promises is quite ugly. Promises aren't new; language designers avoided them as best they could for decades because they suck as abstractions.
Languages like Go do a great job at providing abstractions, and tools like io_uring and IOCP are useful for implementing those abstractions. But they're not ideal, either, nor are they necessary. Their value is diminished. Note that both io_uring and IOCP are implemented internally using thread pools that perform synchronous operations, not much different from how Go works when using traditional BSD Sockets interfaces to implement its abstractions. io_uring and IOCP have access to some optimizations, but the basic model remains the same.
If you read between the lines (discussion of pthread complexity, emphasis on virtual memory, which is fundamentally blocking) I think the author is implicitly making the same argument--that better abstractions would make synchronous threading more efficient and easier, and these should be available as a basic application primitives, rather than relying on languages and third-party runtime engines to build atop more coarse-grained primitives, io_uring and IOCP included.
I don't simply due to the fact that no one ever bothered to put something together on UNIX-like systems that resembles any of those options.
If anyone felt that those were worthy ventures, in these last few years we would already have seen something resembling that effort. But we haven't.
What do you mean? io_uring is a Linux kernel interface, and there are several userspace projects that are building upon it.
yes, let's just ignore the mountains of people who end up literally reimplementing complete network stacks in userspace to handle events in a modern way
Who are they, exactly, and why haven't anyone from that mountain thought it would be a worthy investment of their time to pull it off?
I see you complaining that people have to implement things, but why has no one ever implemented the thing you perceive to be so invaluable?
Don't get me wrong, there are lots of room for improvements in POSIX IO interfaces. For example, POSIX doesn't define any modern event systems (epoll, kqueue, etc). But what's the result? We use libevent/libev.
This sort of seems like the result he's asking for in diverging from POSIX, which we ... already have. There are a lot of pretty great non-posix standard interfaces available too!
Most forms of async programming are also unstructured, which is bad for correctness, but also for performance since it can lead to priority inversions.
I for one think that the 'absurd thing' is that IPC is not built into OS as a core feature. That, and process isolation. Both sockets and shared memory are quite problematic and the challenge of 'true IPC' that works nicely with threads etc. is real.
POSIX and its derivatives build IPC into the OS as a core feature. In particular, POSIX is built around memory mappings and file descriptor inheritance, which means it is extraordinarily easy to make processes communicate.
I honestly have no idea what you mean by this statement.
Unix domain sockets are (1) fast, (2) primitive (just a file), (3) widely available, and (4) can be used between multiple processes extremely easily (use SOCK_DGRAM).
If it were easy than everyone would be doing it the same way and there wouldn't be a discussion about it.
With shared memory in particular, the issue gets tricky around signalling and locking aka indicating to other processes when new info is available, and when another process is accessing the data.
If anyone took just a moment to design an API that was user/developer centric, and worked back from there, it would look materially different from what is provided today.
Sockets aren't designed for IP networking. They date back to a time when it wasn't even clear TCP/IP would win vs. IPX/SPX. Or DECnet. AF_UNIX sockets have things that IP sockets can't even dream of doing (the very poorly named SCM_RIGHTS). And sockets don't really cover IP networking all that well, e.g. MPTCP and SCTP shenanigans to get multi-address & multi-stream connections.
That said, both sockets and mmap are low-level APIs. A higher level "user/developer centric" API indeed looks different.
POSIX has mutex and condition variable, is this not enough to do those things?
Moreover, it's a bit of a secondary thing.
The API would look something like a call function with a receive function/lambda on the receiving side, with memory possibly being 'handed over' or whatever. And so the level of abstraction would be slightly higher.
And it would be a no-brainer, well established thing. Searching for how to do that today and it's always complicated.
POSIX IPC has withstood the test of time. There is no other system that offers as rich a set of primitives.
Windows provides most (all?) of the same mechanisms, and a number that POSIX does not.
https://learn.microsoft.com/en-us/windows/win32/ipc/interpro...
Example Windows concepts that Unix doesn’t have (AFAIK):
Transactions on named pipes (request/response): https://learn.microsoft.com/en-us/windows/win32/ipc/transact...
Parents can control which handles child processes inherit: https://learn.microsoft.com/en-us/windows/win32/ipc/pipe-han...
Remote Procedure Call (RPC) IPC: https://learn.microsoft.com/en-us/windows/win32/ipc/interpro...
ACL-based permissions on objects that kernel APIs operate on, including pipes. (ACLs are a much more sensible way of specifying permissions on resources than owner/group). https://learn.microsoft.com/en-us/windows/desktop/SecAuthZ/a...
Since windowing is built into the NT kernel, it also has support for powerful clipboard operations as one crucial type of IPC in a graphical environment (see 1st link).
Windows I/O Completion Ports (IOCP) also provide a high-performance way to implement kernel-managed asynchronous I/O operations which has no parallel in POSIX: https://learn.microsoft.com/en-us/windows/win32/fileio/i-o-c...
I’ve been developing professionally on Linux for >15 years, and while I do like its simple aesthetic, the consistency and power of NT kernel APIs are something I miss.
I think "the [BSD] socket API" was designed for a thousand or so peers (FD_SETSIZE). That's why the API evolved. Now (2023) there are a lot of APIs and different choices you can make (blocking, polling readiness, sigio, aio, iocp, iouring), none of which is best at everything, but the non-POSIX apis are much faster than the POSIX api -- especially the ones that are harder to mix with the POSIX api.
> "We should be able to assume that the data we want exists in main memory without having to keep telling the system to load more of it."
> Yes, this is exactly what mmap() already does. mmap() is part of POSIX. Use it!
This is not what mmap() does, but the opposite. mmap() sets up page-tables for demand-paging. Those little page-faults actually trigger a check (demand) to see if the page is "in memory" (this is what Linux calls the "block cache"), an update to the page-tables to point to where it is and returns (or kick off a read IO operation to the backing store). These page-faults add up. What the author is looking for in Linux is mremap() and Darwin is mach_vm_remap() and definitely not in POSIX.
> * Shared memory - mmap() is about as generic of a memory sharing interface as one could hope for. It certainly works just fine with arbitrary numbers of clients. There are again countless examples of using shared memory with arbitrary participants -- for example all the various local file databases: sqlite, leveldb, berkeley db, etc.
But none of those "local file databases" can handle hundreds of thousands of clients, and aren't great even for the low hundreds(!), that's why most "big" database vendors avoid "just" using mmap(), and instead have to perform complex contortions involving things like mach_vm_remap/mremap/O_DIRECT/sigaction+SIGSEGV. userfaultfd and memfd are other examples of recent evolutions in these APIs.
You are looking for truth? Evolving APIs are evidence that people are unhappy with these interfaces, and these newer APIs are better at some things than the old, demonstrating that the old APIs are not ideal.
So we have evidence our (programming) model is not ideal, are we to be like Copernicus and look for a better model (better APIs)? Or are we to emulate Tolosani and Ingoli?
That tracks, but this does not necessarily follow:
> demonstrating that the old APIs are not ideal.
I've been around the tech industry long enough to see several cycles of practitioners going back and forth between technologies and approaches. E.g. strong type systems (C++, Java) -> loose type systems (Python, JS) -> strong type systems (Rust, TypeScript). And each time the tide shifts there are always plenty of arguments trying to show why the previous approach was objectively worse and the new one is better.
That's not to say that nothing is moving forward, but the fact that people are unhappy with a technology doesn't mean that technology is objectively inferior to a proposed replacement. Sometimes it's just fashion.
I apologise for writing so much that you couldn't read all of it, but this part, whilst buried at the end of the first paragraph was really important:
> > the non-POSIX apis are much faster than the POSIX api -- especially the ones that are harder to mix with the POSIX api.
That they are better in this objective way is why this particular kind of unhappiness demonstrates what I say, even if you're not otherwise interested in people being happy.
So POSIX as a technology may be lacking but honestly what’s impressive about it to me is that it’s also a standard, not perfect there either, of course.
I think any conversation about replacing POSIX must include an effort to update or replace the standard. Deprecate the parts that no longer work well and add things that other systems have de-facto agreed upon, at least.
I think a more accurate framing is that in the 80s POSIX was designed for a thousand or so peers, and later versions have expanded and scaled accordingly. Literally every interface has these types of growth patterns so this shouldn't be a surprise.
But your point, which is the same point I made below, is that POSIX is but one of several interfaces on a modern system: https://news.ycombinator.com/item?id=34905405
As I noted below, we have semi-standardized extensions from POSIX. We don't really need POSIX to codify a new event system because we have libev/libevent. This is another point the parent article gets wrong when it says we need to get POSIX "off our necks." Nowhere does POSIX compliance stand in the way of using libev, or even using mremap(). I don't agree with your conclusion around mremap() above but it doesn't matter because the salient point is that mremap() works absolutely fine within the paradigm offered by mmap() and POSIX. libev works just fine within the paradigm offered by the file/socket api.
"But none of those "local file databases" can handle hundreds of thousands of clients"
This is absolutely not true. Those systems work just fine with even a million reader threads.
Maybe you mean the databases don't perform well under parallel writes, but this has nothing to do with the shared memory interface (how could it?). It's purely due to the design and intent of each system as primarily read-focused systems. mmap will work just as well as any other shared memory interface when it comes to reading and writing. Pick nearly any write-performant database - it's using mmap.
"Evolving APIs are evidence that people are unhappy with these interfaces"
Not quite. The actual truth is that POSIX plays very well with extensions to its interfaces and this is yet another dimension in which the above article is demonstrated to be very, very silly.
I suppose it depends how you define a "client". If you define it as a query (or an HTTP request or something like that), then I suppose you're right, but you're also not mmap()ing all the time, which is what we're actually talking about here. But if you define it the way databases traditionally do, it is itself a full-featured application, so 100 clients doing 1m queries a second is 100m queries per second, and no, mmap() isn't fast enough to do that, and since we're talking about mmap() I didn't think I needed to explain that.
> But your point, which is the same point I made below, is that POSIX is but one of several interfaces on a modern system: https://news.ycombinator.com/item?id=34905405
I must be reading a different comment, because I don't understand exactly what that has to do with anything, but I think if you're not reading "every word" of the article, maybe I'm thinking you're not reading "every word" my comments either. I was only under the impression you thought the sockets API and mmap() were ideal interfaces.
What position do you think I have that you think I need to change?
> Not quite. The actual truth is that POSIX plays very well with extensions to its interfaces and this is yet another dimension in which the above article is demonstrated to be very, very silly.
I don't understand. That POSIX is so big is one of the first complaints in the article given by implementors of embedded systems, and the "extension-like" nature of POSIX works directly against that. Then, once you have this nice "extension" like iouring with its provided buffers and files, which is basically like a whole io-state-machine operating system in zero system calls, and you know, maybe it can be part of the POSIX standardization process and make POSIX better, or maybe it just becomes popular enough there's a kevent ring extension for the bsds and there's a libiocp4 that unifies the differences, and you think that's all basically POSIX anyway, so what's the point?
Well, the point is that shouldn't stop people from imagining a system that didn't do anything else -- how much smaller the code would be, and cheaper the hardware (less ram, less rom), and maybe if you can organise the kernel with this model in mind, how much faster it would be. These sorts of ideas you can really only explore once you've mentally prepared yourself for there not being a POSIX.
The article defines client as another thread or process - the argument is that these interfaces only support two processes and not many. There isn't any functional limit on how many processes can map the same region of memory.
"but you're also not mmap()ing all the time, which is what we're actually talking about here"
No, I don't think that's at all what we're talking about. We're talking about the ability of the architecture to scale -- not the efficiency of calling one specific syscall.
"I was only under the impression you thought the sockets API and mmap() were ideal interfaces."
I haven't said that they're ideal, only that they're demonstrably capable of enabling more than just two processes to interact.
"and you think that's all basically POSIX anyway, so what's the point?"
The point is that the article's conclusion that POSIX is preventing progress is nonsense.
As you say, standards emerge.
"how much smaller the code would be, and cheaper the hardware (less ram, less rom), and maybe if you can organise the kernel with this model in mind, how much faster it would be."
These aren't points raised by the article -- in fact so far we've been discussing the opposite (extensions and features, not cutting features).
Could things be cut from POSIX? Sure. But is implementing stuff like select() really a "yoke?" It's not exactly difficult, and I don't think you or anyone has articulated how it might hamper the inclusion of more advanced interfaces. The resources needed to implement POSIX are not significant.
What specifically do you think POSIX's architecture is preventing us from implementing?
It does no such thing. The word "client" is given once in the text, and little is said except it isn't a thread due to the fact their support is fraught with peril and often poorly provided for by programming languages
> I haven't said that they're ideal, only that they're demonstrably capable of enabling more than just two processes to interact.
So was carrier pigeon, but sometimes we want to send messages faster than that.
> These aren't points raised by the article -- in fact so far we've been discussing the opposite (extensions and features, not cutting features).
>... about the ability of the architecture to scale -- not the efficiency of calling one specific syscall.
> What specifically do you think POSIX's architecture is preventing us from implementing?
Erm, they are the points raised by the article, that's kind-of why I think you still haven't read it yet. Here's the paragraph right after the one where it defines a client as not-a-thread (and after talking about one specific syscall):
The death of any of these systems is when a user asks, "What about Posix? How can I port my XXX program to run on your system?" Providing Posix-like semantics in these systems is their death knell, because the Posix way of thinking is so narrow and providing its varied illusions requires so many hidden changes and adherences to age-old assumptions that there is no way to have a flexible innovative system and serve the Posix elephant. This means that no matter how innovative and clever a system is, once it is tainted by Posix support, it becomes just another Posix system.
> But is implementing stuff like select() really a "yoke?" It's not exactly difficult, and I don't think you or anyone has articulated how it might hamper the inclusion of more advanced interfaces
Could you show me? I don't think it's possible to implement a select() that works with Linux's fancy new iouring that uses the provided files and buffers features easily, but I'd like to be proven wrong.
It does. It says quite clearly, quote "If you look at most inter-process communication mechanisms, you can see that they are most often used by only two programs — usually a client and server." Two processes on a local machine, using shared memory or other IPC. IPC, not network calls.
"So was carrier pigeon, but sometimes we want to send messages faster than that."
Don't be disingenuous. You haven't articulated a faster mechanism and in fact there isn't one.
"Erm, they are the points raised by the article,"
No, they are not. Based on your replies it seems you may fundamentally not understand the points raised by the article.
"Could you show me? I don't think it's possible to implement a select() that works with Linux's fancy new iouring that uses the provided files and buffers features easily, but I'd like to be proven wrong."
You've already provided an example. The Linux kernel provides both select, a posix interface, and io_uring. Nothing about providing posix interfaces precludes providing specialized interfaces as well.
Two processes! Not two threads!
> Don't be disingenuous.
How would you know? You still haven't read the article, so you're arguing with a straw-man!
> You haven't articulated a faster mechanism and in fact there isn't one.
"la la la my fingers are in my ears I can't hear you!" is the best you got? You seem to admit iouring exists, isn't POSIX, but what? You just don't believe it's faster now? Bull shit.
> You've already provided an example. The Linux kernel provides both select, a posix interface, and io_uring. Nothing about providing posix interfaces precludes providing specialized interfaces as well.
If you think a reimplementation of Linux is "easy" you need to have your head examined: Microsoft has near-infinite money and eventually gave up, deciding POSIX was too hard, and just implemented another hypervisor.
And I wrote "thread or process" because, architecturally regarding IPC, the distinction is unimportant.
"How would you know?"
Due to the inaccuracy of your analogy, with regard to actual systems interfaces.
"You seem to admit iouring exists, isn't POSIX, but what? You just don't believe it's faster now? Bull shit."
io_uring is not faster than shared memory via mmap.
"If you think a reimplementation of Linux is "easy" you need to have your head examined"
I think now we're touching on the limits of your experience and expertise. Implementing select(2) on Linux is indeed quite simple. Have you ever read through select.c?
Have you ever even implemented a posix subsystem? I have.
"Microsoft has near-infinite money and eventually gave up, deciding POSIX was too hard, and just implemented another hypervisor. "
This isn't how things work on Windows.
As an aside, I've noticed that your ignorance has shifted into rudeness. Don't bother to respond again. This conversation is over.
"The Socket API, local IPC, and shared memory pretty much assume two programs"
There's no truth to this whatsoever - it's an utterly absurd statement because:
* The socket API as applied to networking is empirically designed for arbitrary numbers of peers -- there are unlimited real world examples. The socket API as applied to local IPC is of course similarly capable.
* Shared memory - mmap() is about as generic of a memory sharing interface as one could hope for. It certainly works just fine with arbitrary numbers of clients. There are again countless examples of using shared memory with arbitrary participants -- for example all the various local file databases: sqlite, leveldb, berkeley db, etc.
"We should be able to assume that the data we want exists in main memory without having to keep telling the system to load more of it."
Yes, this is exactly what mmap() already does. mmap() is part of POSIX. Use it!
"Kode Vicious, known to mere mortals as George V. Neville-Neil,"
Oh, ok.