News for hackers. Stuff that matters.
Re:How to Write an Operating System by kenjackson on Sunday August 15, 2010 @11:00AM (#1605224)
Either I'm missing something or this tutorial is seriously lacking.

I'd much recommend Tannenbeum's classic text that covers implementing Minix: http://www.amazon.com/Operating-Systems-Implementation-Prent...

But since this text is rather old, it doesn't have some of the later developments in OS. So for that additional material, I'd recommend his latest book to have at your side: http://www.amazon.com/Modern-Operating-Systems-Andrew-Tanenb...

Re:How to Write an Operating System by xpaulbettsx on Sunday August 15, 2010 @08:47PM (#1606411)
Windows Internals is also a good book for learning about operating systems, even if you're not specifically interested in Windows. Mac OS X is more detailed since the source code itself can be published, but it's not as easy to read (gets pretty dry at times). I also enjoyed reading Robert Love's "Linux Kernel Development".

One of the things that really helped me learn more was reading both of those books - understanding two very different kernels (NT vs. Linux) really helps you grok the problems an OS has to solve, and to separate "This is a NT thing" vs. "This is a hardware thing that every OS has". Kind of like learning a foreign language makes you think more about grammar in your native language (at least for me).

Re:How to Write an Operating System by mahmud on Monday August 16, 2010 @01:08AM (#1606919)
I wrote Love a personal thank you letter after I read LKD :-)
Re:How to Write an Operating System by michael_dorfman on Sunday August 15, 2010 @11:16AM (#1605246)
Actually, I'd recommend the latest version of OSDI, which is fairly recent, and contains the complete source for MINIX3 (in text form, and on CD): http://www.amazon.com/Operating-Systems-Design-Implementatio...
Re:How to Write an Operating System by kenjackson on Sunday August 15, 2010 @12:05PM (#1605317)
Thank you. I'd learned from the original text. I never knew he had released a couple more editions of this.

In fact, I may have to go buy this now!

Re:How to Write an Operating System by gruseom on Sunday August 15, 2010 @01:04PM (#1605425)
So Tanenbaum has two OS textbooks in print? How different are they? Why read one rather than the other?
Re:How to Write an Operating System by michael_dorfman on Sunday August 15, 2010 @01:53PM (#1605540)
OSDI is "the MINIX book". It shows, about as much detail as possible, how a microkernel-based OS is written, down to the actual (complete) source code.

Modern Operating Systems, on the other hand, is a bit higher level (but still fairly nitty-gritty) and examines two monolithic kernels (Linux/Unix and Windows) as case studies.

Re:How to Write an Operating System by mahmud on Monday August 16, 2010 @01:10AM (#1606921)
He has THREE. Don't forget the Amoeba book "Distributed Operating Systems" (not sure if it's in print though.)
Re:How to Write an Operating System by flatulent1 on Sunday August 15, 2010 @01:58PM (#1605552)
The tutorial certainly lacks any emphasis on security. It ought to be mentioned right up front. Does the choice of development environment matter? Say so!!! Trying to add security after the fact is painful and tends to be only marginally successful.
Re:How to Write an Operating System by mahmud on Monday August 16, 2010 @01:13AM (#1606928)
OS security is something that really aught to be discussed separately from OS texts. I mean, not all OSes are multiprogramming or need security. Earlier version of his Operating Systems book taught MS DOS, along with others, for example.

Security exposition is something best left to a separate text, and Matt Bishop's fat book does a good job (not the thin watered down one.)

Re:How to Write an Operating System by shoesfullofdust on Sunday August 15, 2010 @10:41AM (#1605201)
Since each chapter does not link to the next (the zeroeth one does, but not the others), here's a link to the index: http://www.acm.uiuc.edu/sigops/roll_your_own/
Re:How to Write an Operating System by nivertech on Sunday August 15, 2010 @12:08PM (#1605319)
I used XINU to study OS course: http://en.wikipedia.org/wiki/Xinu

Xinu is a small, elegant, multitasking Operating System supporting the following features:

    * Concurrent Processing
    * Message Passing
    * Ports
    * Semaphores
    * Memory Management
    * Buffer Pools
    * Uniform Device I/O
    * Shell
    * Tcl
    * TCP/IP
Xinu was originally designed as a vehicle for teaching Operating System design concepts and is used by many educational institutions for this purpose. Later versions supported TCP/IP, these versions are often used in Data Communications courses.
Re:How to Write an Operating System by mjschultz on Sunday August 15, 2010 @01:45PM (#1605523)
I also used Xinu in my O/S course. I'll mention that one of Doug Comer's (http://www.wikipedia.org/wiki/Douglas_Comer) intentions with Xinu was to purposefully make a different design decision than Unix when pedagogically relevant. For example, instead of treating everything like a file (Unix/Linux) everything is treated like a device.

Its various iterations over the year do not necessarily remain true to that principle, but it is something different about the O/S.

Re:How to Write an Operating System by xulescu on Sunday August 15, 2010 @12:23PM (#1605344)
Interesting project, but it's name sounds too much like Xenu: http://en.wikipedia.org/wiki/Xenu .
Re:How to Write an Operating System by crazydiamond on Sunday August 15, 2010 @12:38PM (#1605372)
Actually, it's Unix backwards, but surely you knew that.
Re:How to Write an Operating System by ralph on Wednesday August 18, 2010 @08:19AM (#1613887)
As well as being Unix backwards, it's also a recursive acronym; Xinu is not Unix.
Re:How to Write an Operating System by dgordon on Sunday August 15, 2010 @01:55PM (#1605547)
You say that like it's a bad thing. ;-)
Re:How to Write an Operating System by nivertech on Sunday August 15, 2010 @01:36PM (#1605508)
Aliens had much more advanced operating systems 75 million years ago. Imagine which operating systems they have NOW! :-)
Re:How to Write an Operating System by SkyMarshal on Sunday August 15, 2010 @03:13PM (#1605681)
Add Menuet, the OS written in assembly, to that list too:

http://www.menuetos.net/

Re:How to Write an Operating System by ehnus on Sunday August 15, 2010 @01:28PM (#1605486)
There's also LoseThos (http://www.losethos.com/), although it's a bit on the eccentric side.
Re:How to Write an Operating System by MrJagil on Monday August 16, 2010 @06:18AM (#1607341)
Don't forget http://www.loper-os.org/

by far my favourite.

Re:How to Write an Operating System by njn on Sunday August 15, 2010 @04:00PM (#1605797)
Hactar is "Possibly, an Operating System": http://anynowhere.com/bb/posts.php?t=410
Re:How to Write an Operating System by abless on Sunday August 15, 2010 @12:33PM (#1605366)
Some good tutorials can be found at http://www.osdever.net/tutorials/index - these are generally more hands-on than academic textbooks. Also, feel free to have a look at http://flick.cvs.sourceforge.net/flick/src/, my attempt at writing an OS (long time ago, now); the code is clear enough to give you a rough idea on how to get started.
Re:How to Write an Operating System by MaysonL on Sunday August 15, 2010 @01:30PM (#1605490)
For another look at OS design and implementation, try Project Oberon. Beautifully clean design, if somewhat dated by hardware advances since the 80s.

http://www-old.oberon.ethz.ch/WirthPubl/ProjectOberon.pdf

Re:How to Write an Operating System by donaq on Sunday August 15, 2010 @10:51AM (#1605212)
See, this is why I dream of making FU money. So I can spend all my time doing stuff like that.
Re:How to Write an Operating System by endlessvoid94 on Sunday August 15, 2010 @11:39AM (#1605280)
This was written by undergrads at ACM's UIUC. I went to school there.

You don't need FU money.

Re:How to Write an Operating System by Niten on Sunday August 15, 2010 @12:24PM (#1605347)
That's why I love the software industry. We have such a low financial barrier to if you want to play around with even the deepest systems programming techniques, or the highest-level functional programming languages. All you need is a $400 PC.
Re:How to Write an Operating System by pohl on Sunday August 15, 2010 @03:16PM (#1605693)
And time. Sometimes that's what the FU money would bring to the table.
Re:How to Write an Operating System by jff on Monday August 16, 2010 @09:26AM (#1607695)
These days, a $50 PC (or a free PC) will probably serve you better, because it's more likely to have a serial port.

You should probably try to find one that can either PXE boot or boot from a USB stick, since those are pretty much the easiest ways to quickly test your software.

Re:How to Write an Operating System by ygd on Sunday August 15, 2010 @03:47PM (#1605762)
Actually, I got away with a <$200 PC.
Re:How to Write an Operating System by donaq on Sunday August 15, 2010 @08:16PM (#1606343)
I am not in school. I have mouths to feed.
Re:How to Write an Operating System by dhume on Monday August 16, 2010 @11:56AM (#1608149)
On the other hand, weekends aren't work days anymore (not for me, at least).
Re:How to Write an Operating System by megablast on Monday August 16, 2010 @02:19AM (#1607023)
When you are an undergrad or recent graduate, you can do this.

When you are older, have a job, you need FU money generally. Or no life.

Re:How to Write an Operating System by hxr on Sunday August 15, 2010 @11:33AM (#1605271)
Makes me wonder, can I make that money doing stuff like this? I know systems research is irrelevant and all that (Rob Pike)... AFAIK the last successful operating systems startup was VMware (is that true?). Does the world have place for more OS startups these days?
Re:How to Write an Operating System by Aegean on Sunday August 15, 2010 @02:01PM (#1605556)
I tried to build one from scratch based on my own L4 microkernel. Given the bloated nature of monolithic kernels and their irrelevance to embedded space, I had the thought that if I wrote something small and neat it could gain traction for embedded systems. For that I took the BSD virtual memory design, and implemented a complete VM system from scratch. Then I wrote a VFS layer but that one was partially complete. I had demand paging implemented and supporting calls like fork() execve() etc. It was challenging and fun - but an incomplete project, also my biggest error from a startup/making money point of view. Basically having a new kernel is a hard sell nowadays, no matter how neat it is. It was more of an intellectual challenge for me rather than a startup - but still I was in the expectation that it could be a success.

About exactly a year ago, I changed my direction to virtualization. I ditched about %60 of the software that includes all that VM layer and VFS for which I put loads of effort (many man months...). I realized I was out of focus, so I changed my focus to enhancing only the microkernel itself. I decided upon three specific goals: 1) Adding multi-core support for latest ARM cores (e.g. the ones that will be on most high-end mobile phones and devices) 2) Adding fine-grain security and control based on capabilities. (in short, having control over system calls) 3) Adding virtualization support for the linux kernel. Right now I am done with the first two, and about to get done with the 3rd one.

My interest has always been to create something new in the OS/kernel space. After doing much research I decided that L4 microkernel design represents the most promising work in this field. This is because a) monolithic kernels are mature and good for what they are doing b) microkernels can solve certain new problems like virtualization.

Eventually I was convinced that embedded virtualization would be the most interesting problem I could solve. In my opinion there is still room for systems-level work particularly in Virtualization. The APIs are not set in stone. Not many people know how virtualization works, let alone design an interface for it from scratch.

I think new areas like this will come up but its quite important to spot an unsolved problem because if you dive in to write a kernel its an expensive journey.

Re:How to Write an Operating System by Niten on Sunday August 15, 2010 @12:48PM (#1605386)
There's definitely room for improvement over currently used server and desktop operating systems; the security models used by desktop Linux, OS X, Windows, and BSD don't meet the needs of users in a world of malicious software. iOS and Android were able to improve on the situation a good deal by adopting higher-level security models that protect applications (and their data) from one another, but the desktop lags behind.

As far as I know there's no technological reason the same couldn't be done with the server or desktop, it would just be an enormous shift in momentum to get there from where we are now. But if we're willing to stomach shifts in momentum, we could even go further and consider other, more radically different approaches to security than what's commonly used now, such as Microsoft's Singularity.

I don't know enough to say whether an operating system startup could be successful, but I think if it were, it would be less about basic operating systems research than about getting what research we already have into the hands of the masses on the desktop or the server. Presumably by targeting sectors where customers care enough about security to make the difficult switch to a fundamentally incompatible system.

Re:How to Write an Operating System by kragen on Monday August 16, 2010 @02:51AM (#1607068)
Systems software research was irrelevant ten years ago when Rob wrote that. Thanks in part to his efforts, there's a great deal of relevant systems software research now: Golang, MapReduce, Hadoop, Tahoe-LAFS, Caja, Chrome (the browser), LuaJIT, seL4, Capsicum, the CLR, HotSpot, V8, Nitro, Tracemonkey, GFS, Sawzall, Pig, Redis, Puppet, Chef, Tor, HTML5, ...

Most of this, but not all, comes from the traditional academic and industrial-lab research context.

Re:How to Write an Operating System by jff on Monday August 16, 2010 @09:27AM (#1607699)
How is Chrome "systems software"? It's just a damn web browser.
Re:How to Write an Operating System by kragen on Monday August 16, 2010 @12:12PM (#1608205)
In http://doc.cat-v.org/bell_labs/utah2000/ (the point of reference for this discussion), "systems" is defined as "Operating systems, networking, languages; the things that connect programs together." Examples of what he's talking about from other parts of those slides include:

> By contrast, a new language or OS can make the machine feel different, give excitement, novelty. But today that's done by a cool Web site or a higher CPU clock rate or some cute little device that should be a computer but isn't.

> Work on how systems behave and work, not just how they compare. Concentrate on interfaces and architecture, not just engineering.

> Only one GUI has ever been seriously tried, and its best ideas date from the 1970s. (In some ways, it's been getting worse; today the screen is covered with confusing little pictures.) Surely there are other possibilities. (Linux's interface isn't even as good as Windows!)

> There has been much talk about component architectures but only one true success: Unix pipes. It should be possible to build interactive and distributed applications from piece parts.

Chrome is an environment for running AJAX web applications and enabling them to talk to each other; among other services, it provides a graphical user interface toolkit (DHTML), a SQL database (SQLite), security mechanism and policy (via tab-per-process, the same-origin policy, incognito mode, and restrictions in the JS engine), a JIT compiler for a language, and process management (both at the user level, with its process viewer, and at the language level with Web Workers). It has a component architecture built in; several of them, actually: iframes, plugins, JSONP. It is possible in Chrome to build an interactive application from "piece parts"; this is currently called a "mashup".

So Chrome is right in the center of the issues Pike's talk was talking about. Much of this, of course, is made of ideas that don't come originally from Chrome; but Chrome is on the cutting edge of making new stuff possible.

Re:How to Write an Operating System by loewenskind on Monday August 16, 2010 @08:51AM (#1607611)
If this post were from, say 5 years ago I would have figured it was spot on. But today? Look at all the new OSes coming out now in the mobile/appliance space. Not to mention that Windows Vista/7 was enough of a shift that the learning cost of moving operating systems has gone down relatively (because you'll have to do it regardless).
Re:How to Write an Operating System by mwilliamson on Sunday August 15, 2010 @06:51PM (#1606180)
What about XenSource? I believe they were later than VMware, and they made a tidy sum when Citrix bought them.
Re:How to Write an Operating System by norswap on Sunday August 15, 2010 @12:01PM (#1605310)
I think there is. Every OS that exists is a mess. Make an OS whose inner workings I can undestand (and therefore are very well documented) while allowing to make commonly used software, and I will worship you.
Re:How to Write an Operating System by Niten on Sunday August 15, 2010 @12:58PM (#1605406)
There's no need for a new operating system if your criteria is just "inner workings I can understand". OpenBSD is very thoughtfully implemented, its documentation is top notch, it's open source, and yet it is a general purpose operating system that runs real world applications. It isn't a "mess"; in terms of its design, I think OpenBSD is about as clean as a real-life operating system can possibly be, so much so that my college operating systems professor even chose it over Minix for teaching the course.

I still think there's room for improving operating system security models, but I don't think it's possible to improve over OpenBSD in this particular dimension.

Re:How to Write an Operating System by norswap on Sunday August 15, 2010 @07:19PM (#1606232)
It's true I don't know BSD, I always thought I ought to investigate it and Minix. But well, there's only so much hours in a day :)
Re:How to Write an Operating System by mahmud on Monday August 16, 2010 @01:16AM (#1606931)
Minix is hiring driver developers and kernel hackers.
Re:How to Write an Operating System by jamii on Sunday August 15, 2010 @01:51PM (#1605535)
I don't know about OS research specifically but there are a few systems startups about. RethinkDB and Acunu are the first to come to mind.
Re:How to Write an Operating System by revorad on Sunday August 15, 2010 @11:09AM (#1605233)
You mean like Linus?
Re:How to Write an Operating System by djcapelis on Sunday August 15, 2010 @04:00PM (#1605791)
My friend was part of a group of students who decided to write XOmB. They just apparently decided it would be a neat project, now it's a pretty cool exokernel: http://xomb.net/index.php?title=Main_Page
Re:How to Write an Operating System by steveklabnik on Sunday August 15, 2010 @04:21PM (#1605838)
Hey, thanks for the shout-out.

Not only are we working on XOmB, (pronounced 'zombie') but we extracted all of the stuff "from power on 'till kmain" and called it XOmB Bare Bones, so that other people who want to write kernels in D don't have to worry about that ugliness: http://wiki.xomb.org/index.php?title=XOmB_Bare_Bones

Re:How to Write an Operating System by djcapelis on Sunday August 15, 2010 @04:41PM (#1605879)
That is great to hear. I've heard a few stories on how painful that stuff was to track down and write. Bring up always sucks. This sounds like an excellent way for people to dig into writing kernels without forcing people to write all the bootup code before they can do anything.

How do you do device handoff? Are people restricted to an exokernel model or can they handoff to a macrokernel kexec style?

(VMware used to use the Linux kernel to take care of init before they wrote it themselves, so there's definitely a need for this type of thing, even beyond hobbyist.)

Re:How to Write an Operating System by steveklabnik on Sunday August 15, 2010 @09:03PM (#1606440)
In classic exokernel fashion... you have a choice! Someone can write a libos to handle that stuff however they'd like, or you can just go do the work yourself.
Re:How to Write an Operating System by nivertech on Sunday August 15, 2010 @01:46PM (#1605526)
I implemented my own timesharing kernel in C and little inline x86 assembler.

Main ideas:

1. Write context switch routine, which will require a little inline x86 assembler code.

2. Map it to timer interrupt.

3. Use C setjmp and longjmp to switch context.

Re:How to Write an Operating System by ja30278 on Monday August 16, 2010 @11:33AM (#1608067)
Surprised not to see "Design and Implementation of the FreeBSD Operation System" mentioned yet (http://www.amazon.com/Design-Implementation-FreeBSD-Operatin...).

McKusick's 'Kernel Internals' class is based on this, and is well worth your time (though it's a bit pricey to purchase the videos on your own: https://www.mckusick.com/courses/advorderform.html)

Re:How to Write an Operating System by steveklabnik on Sunday August 15, 2010 @04:23PM (#1605841)
http://osdev.org is another great resource and small community for people trying to make hobby OSes.
Re:How to Write an Operating System by philwelch on Sunday August 15, 2010 @11:07AM (#1605232)
I took a class like this once except instead of i386 we implemented on 16-bit x86 so we had to fiddle with the stack and registers manually to switch kernel and user mode, or to switch processes. It was great fun despite a few eccentricities.
Re:How to Write an Operating System by krmboya on Sunday August 15, 2010 @10:49AM (#1605209)
Cool
Re:How to Write an Operating System by PidGin128 on Sunday August 15, 2010 @05:33PM (#1606005)
The comment doesn't meet the guidelines, certainly.

However, a lesson could have been gleaned from zero points. There is no need to dig a grave.