Except for the 64 handle limit, which makes it largely useless for anything that involves server applications where the number of handles grows with the number of clients. So then you'd spawn "worker" threads, each handling just 64 handles. And that's exactly where your code starts to go sour - you are forced to add fluff the sole purpose of which is to work around API limitations. What good is the super-select if I need cruft to actually use it in the app?
And don't get me started on the clusterfuck of the API that is IOCP.
> Yes, Windows gets a lot wrong, but it gets a lot right.
Oh, no, it doesn't.
Specifically, what Windows does not get is that it's other developers that are using its services and APIs, not just MSSQL team that can peek at the source code and cook up a magic combination of function arguments that does not return undocumented 0x8Fuck0ff. I have coded extensively for both Linux and Windows, including drivers and various kernel components, and while Windows may get things right at the conceptual level, using what they end up implemented as is an inferior and painful experience.
Don't leave us hanging... what examples are you referring to?
For WINE, if you are going to write an emulator that runs Microsoft code (like the .net framework, or notepad.exe), you need to find the correct behavior of these undocumented APIs by trial and error.
For a developer on Windows, it is perhaps frustrating that MS identifies and solves problems with it's platform, but doesn't publicly release these solutions to you.
I work for some random company that makes an SDK, although I mostly make products using it. Many of our best features are hidden - we don't ship the header files with the SDK, but the symbols are there in the binaries. Someone doesn't think they are useful enough to justify the testing expense for a full release. I guess MS have the same thing.
Ever since the monopoly rulings against Microsoft, they have had extremely strict internal controls preventing non-OS code from using undocumented APIs. If a DLL or EXE does not ship on the Windows client install disk, it cannot reference APIs which are not documented in MSDN. Full stop.
You're referencing an API that inadvertently didn't get documented, but everyone knows about it, has blogged about it and lots of articles mentioning to use it? Too bad. Either stop using the API or delay release by 3 months for the next MSDN update.
"undocumented APIs" != "a magic combination of function arguments"
Any function that's not documented can be called an undocumented API, any program has them -- Linux, Windows, whatever.
But you were talking about the argument combinations being undefined, not the functions themselves. What are some examples of that?
Wine Is Not an Emulator.
tl;dr: compatibility layer
WINE is a translator that allows Windows programs (those that have an .exe extension) to talk to other Operating Systems.
or perhaps:
Many people need wine if you want them to work. A windows program on Linux is no exception.
I'm sure the aforementioned freshmen would like a no-bullshit explanation for why "emulator" hurts the WINE people's feelings but "translator" does not. It's always been a mystery to me.
Hah! Not really possible, especially considering that the project used to be called windows emulator:
http://www.faqs.org/faqs/windows-emulation/wine-faq/
I'd say they've changed the meaning so they can reinforce that it's not just another virtual machine ( and that it's faster than one).Which is true, it doesn't emulate another hardware architecture's calls, just the kernel and OS magic needed.
In that regard, think of say a console game or arcade game emulator on a computer- that will have to emulate the whole architecture of said game system as well as the game.
Contrast that to a game program for the the x86 architecture, but not for windows- you could run it natively on windows providing you change everything the game asks for that's part of the Gamesystem380 system into something windows understands, and vice versa.
The problem with Windows is that there isn't any such super-select, unlike on Linux (see epoll, signalfd, timerfd...). You should take a look at my SO answer which lists six different ways to use sockets in Windows: http://stackoverflow.com/questions/11830839/when-using-iocp-...
And expecting a modern OS to have an uncomplicated O(1) API for monitoring sockets is clearly too much to ask.
- To send data, user calls Socket::send(const char *data, size_t length). This starts sending the given data; the buffer must remain available while data is being sent. IOCP implementation will initiate I/O using WSASend() or similar.
- When send operation is complete, the socket implementation calls a Done() callback of the user, and from that point on, the user is allowed to call Socket::send() again. With proper implementation, send() can directly be called from the callback.
This interface is very easy to implement on Linux. But on Windows there's a complication, because it's non-trivial to just stop an IOCP I/O operation that is in progress, in case you decide you don't need the socket any more and want to red rid of it NOW. If you just forget about it, there could be a crash when you release your buffer but Windows is still using it. A simple solution is to CancelIo() the socket and do blocking GetQueuedCompletionStatus()s until you get an event inidicating the completion of the pending I/O operation - taking care to queue up any unrelated results you may have gotten, for later processing. However, a more efficient solution is to use reference counting or similar with your buffers, so they only get released when Windows is done with them.
I didn't say it's easy, but it's possible.
> I can't really use IOCP, because I need keep things reasonably portable
LOL!
You outright dismiss the solution because it isn't "portable". What does "portable" mean here? Portable in a particular platform?!
Portable between what exactly? IOCP has been there since Windows NT 3.51 -- it's "portable" between the Windows-es.
Your epoll, signalfd, timerfd, etc. are Linux functions They're "portable" between the Linux-es.
i.e. The "portable" solutions you like are just as non-portable as the "non-portable" solutions you outright dismiss, making Microsoft's Windows designers look like idiots.
You're never going to find a solution while considering the Linux platform to be the "portable platform" and Windows to be the weird kid on the block.
Sadly, you're not the only person who thinks like this.
So "LOL!" yourself, bud.
It's funny, you mention "epoll, signalfd, timerfd" as an example of a portable API, and then when I point out they're Linux APIs, you suddenly start talking about "a scalable variation" of the BSD sockets API. As for your "scalable variation of BSD sockets API", why don't you tell us the name so we know what you're referring to? Are you talking about Kqueue for example, or something else?
> an abstraction that is both simpler
I/O completion ports are just read-write queues... you ReadFile(), and when it's done, an item goes into the queue. (You can insert your own tasks into the queue, too, if you want.) Now just pop off the next item in the queue and off goes your thread. What part of that was so complex compared to your portable API (which for some reason you didn't feel was appropriate to mention the name of)?
> and more universal
Can't reply to it without knowing what "scalable variation" you're referring to, and why you didn't actually mention it as an example of portability when you were actually talking about the subject. Is portability actually a concern for you or no? (If it is, why did you bring up non-portable examples?!)
It wasn't me who mentioned it.
My comment remains essentially unchanged, though, with the same question as before:
What's this great portable API you're referring to?
Update: And I forgot to mention the font parsing security issues (look up Duqu), and the issues with user mode callbacks too.
Genuine question (because I really don't know) -- how do Linux GUI frameworks work without kernel-mode callbacks? What do they use instead, when they need to send messages to other windows?
Asynchronous communication is achieved using syscalls like select and poll which take a set of file descriptors (files, special files, sockets) and block the calling thread until some descriptor from this set has new data available for reading, starts accepting writes or signals an error.
GUI toolkits repeatedly poll the X server descriptor, parse incoming messages and call application-defined callbacks.
That sounds like it's basically the same thing as Windows (GetMessage or MsgWaitForMultipleObjectsEx anyone?)... only that now you don't have the ability to use SendMessage, and need to rely exclusively on PostMessage instead.
Is that correct?
In all boils down to message-passing: in Windows, the win32k is the trusted third party that mediates interactions between different window-system clients. In the X11 case, the X server and the window manager work together to do the same job. The architecture are fundamentally similar.
Why does Windows need kernel-to-user callbacks in the first place then? Now I'm thinking it shouldn't even be necessary at all...
Wouldn't that completely avoid needing to call back into user-mode from kernel mode?
Okay, you're in charge. Are you sure you can't find a better way to deploy shareholder capital than using it to remove win32k?
Could that be why developing an OS inside of a shareholder-owned corporation is inherently hobbled compared with developing it as an open source project?
True, but resource allocation is distributed between volunteer contributors, sponsored contributors who may have different sponsors, downstream consumers of a project, and people involved in project governance. That can make it easier to try new approaches.
Not all commercial projects are run like Windows, and it seems to me that most of their conservatism doesn't come from lazy or shoddy or shortsighted developers and managers, but rather their large number of hardware and software partners who take dependencies on undocumented implementation details (see Raymond Chen's blog for a virtually infinite supply of such anecdotes).
That said, if you're writing an app you might be able to go higher level and use XPC services, which can be much less pain.
See, in Windows, processes are first class kernel objects. You have handles (read: file descriptors) that refer to them. Processes have POSIX-style PIDs too, but you don't use a PID to manipulate a process the way you would with kill(2): you use a PID to open a handle to a process, then you manipulate the process using the handle.
This approach, at a stroke, solves all the wait, wait3, wait4, SIGCHLD, etc. problems that plague Unixish systems to this day. (Oh, and while you have a handle to a process open, its process ID won't be re-used.)
It's as if we live in a better, alternate universe where fork(2) returns a file descriptor.
You can wait on process handles (the handle becomes signaled and the wait completes when the process exits). You can perform this waiting using the same functions you use to wait on anything else, and you can use WaitForMultipleObjects as a kind of super-select to wait on anything.
If you want to wait on a socket, a process, and a global mutex and wake up when any of these things becomes available, you can do that. The Unix APIs for doing the same thing are a mess. Don't even get me started on SysV IPC.
Another thing I really like about NT is job objects (http://msdn.microsoft.com/en-us/library/windows/desktop/ms68...). They're a bit like cgroups, but a bit simpler (IMHO) to set up and use.
You can apply memory use, scheduling, UI, and other restrictions to processes in job objects. Most conveniently of all, you can arrange for the OS to kill everything in a job object if the last handle to that job dies --- the closest Linux has is PR_SET_PDEATHSIG, which needs to be set up individually for each child and which doesn't work for setuid children.
(Oh, and you can arrange for job objects to send notifications to IO completion ports.)
Yes, Windows gets a lot wrong, but it gets a lot right.