News for hackers. Stuff that matters.
Re:Writing an OS in Rust: Returning from Exceptions by adamnemecek on Wednesday September 21, 2016 @04:40PM (#12552018)
I think that now that we have a language suitable for writing kernels, we also need to have a discussion about what do we really want from a new os. Over the years there have been quite a few OSs and some of them were more advanced than the currently popular OSs. E.g. AS/400 didn't make the distinction between memory and hdd which seems to me like a good idea. BeOS has a fully async API which resulted in a much better user experience and better CPU utilization. Tandem had erlang like processes. None of the popular OSs have this. I'm not sure I want another UNIX implementation.

Furthermore, why are all the APIs so diverse? Why aren't there reactive operating systems (as in OS with reactive API)? All of these ideas can be explored in Rust but on some level I'm not sure what should be the feature set of the OS of the future.

The current driver models aren't that great either.

Re:Writing an OS in Rust: Returning from Exceptions by pjmlp on Thursday September 22, 2016 @02:35AM (#12554896)
> I think that now that we have a language suitable for writing kernels

We already had it in 1961 as ESPOL and NEWP.

http://www.liquisearch.com/burroughs_large_systems/algol/esp...

Followed by many others, before UNIX's adoption due to its source being available for free.

> I'm not sure I want another UNIX implementation.

Yeah, they just keep repeating what was already done and without much research besides adding mainframe features.

As Rob Pike puts it:

"We really are using a 1970s era operating system well past its sell-by date. We get a lot done, and we have fun, but let's face it, the fundamental design of Unix is older than many of the readers of Slashdot, while lots of different, great ideas about computing and networks have been developed in the last 30 years. Using Unix is the computing equivalent of listening only to music by David Cassidy."

from https://interviews.slashdot.org/story/04/10/18/1153211/rob-p...

What we really need are OSes that take the ideas from human-computer interaction from Xerox PARC and Viewpoints Research Institute and take them further into mainstream, using safe languages in the process.

Re:Writing an OS in Rust: Returning from Exceptions by DblPlusUngood on Thursday September 22, 2016 @08:27AM (#12556298)
> "...while lots of different, great ideas about computing and networks have been developed in the last 30 years."

> What we really need are OSes that take the ideas from human-computer interaction from Xerox PARC and Viewpoints Research Institute and take them further into mainstream, using safe languages in the process.

How are modern OSes prevented from using said ideas (which ideas exactly?) simply because they are a UNIX implementation?

Re:Writing an OS in Rust: Returning from Exceptions by pjmlp on Thursday September 22, 2016 @08:53AM (#12556446)
Because most UNIX users see only this:

http://pubs.opengroup.org/onlinepubs/9699919799/

Everything else is fluff, and could stay just like it was on UNIX System V.

Hence why for macOS, iOS, tvOS, Android and ChromeOS, having an UNIX kernel is just a matter of convenience, given their application models and programming languages.

Apple and Google could eventually migrate to something else as kernel while keeping their Objective-C, Swift, Java and JavaScript APIs.

NeXT based NeXTStep on UNIX as a door into the then new UNIX worksations market, but it never had a CLI culture like the other UNIX vendors.

GUI workflows like on Xerox PARC were always important to Steve.

Re:Writing an OS in Rust: Returning from Exceptions by DblPlusUngood on Thursday September 22, 2016 @09:11AM (#12556552)
I don't understand you.

Is your answer to my question: "Modern UNIX OSes cannot take advantage of said ideas because they are POSIX compatible."?

Re:Writing an OS in Rust: Returning from Exceptions by pjmlp on Thursday September 22, 2016 @10:42AM (#12557093)
No, my answer is that the UNIX clones only bother to achieve POSIX compatibility and POSIX doesn't provide anything related to GUIs.

So, each UNIX clone project ends up replicating POSIX and doesn't move beyond what is actually yet another TWM clone with pretty graphics.

GNOME, KDE, Unity are the only ones that try to somehow modernize the experience and tend to get pretty vocal pushback.

The UNIX culture, is the culture of the command line and something like XFCE is probably the GUI a TWM user is willing to accept to manage its XTerms.

Re:Writing an OS in Rust: Returning from Exceptions by legulere on Thursday September 22, 2016 @01:54AM (#12554714)
IMO a sane syscall API that was built with sandboxing in mind. The unix API gives you access to basically all of the computer or at least all of the user's data. Nowadays we live with executables we can't trust. Random binaries don't need to be dangerous to execute, only the syscalls that they can make on windows and unixes make them dangerous. This also relieves you from the need to sign most programs by a central authority
Re:Writing an OS in Rust: Returning from Exceptions by bluejekyll on Wednesday September 21, 2016 @11:37PM (#12554135)
> Why aren't there reactive operating systems (as in OS with reactive API)

This is something that's been interesting me lately. I've been working with the excellent futures and tokio libraries, and I've wondered what would it be like if that was the primitive for all OS interactions?

Basically, imagine a message passing system like Mach, but where the fundamental messages are all futures based. There might be some interesting things possible without holding locks in the kernel.

Re:Writing an OS in Rust: Returning from Exceptions by phil-opp on Thursday September 22, 2016 @10:01AM (#12556846)
I definitely plan to integrate the futures library in the kernel somehow. We just need some asynchronous things and threads first :).
Re:Writing an OS in Rust: Returning from Exceptions by bluejekyll on Thursday September 22, 2016 @11:18AM (#12557399)
cool! Do you have an overarching design your working off of? Or still too low level?
Re:Writing an OS in Rust: Returning from Exceptions by phil-opp on Saturday September 24, 2016 @06:48AM (#12570401)
I have some vague ideas for the next posts, but no overarching design. At our current level, there aren't many design choices anyway. It will get much more intesting when we create a threading/concurrency model in the future.
Re:Writing an OS in Rust: Returning from Exceptions by TickleSteve on Wednesday September 21, 2016 @04:43PM (#12552043)
Would be wiser to only change one variable at once...

Prove that an existing RTOS style library can be written with a simple scheduler working well.

That would go a long way to it gaining adoption in the embedded world, show that traditional solutions work correctly with the new language first.... then move on to an innovative OS.

Re:Writing an OS in Rust: Returning from Exceptions by adamnemecek on Wednesday September 21, 2016 @04:53PM (#12552132)
Sometimes you have to go top to bottom. I think that Alan Kay says something along the lines that you should start with the design of the API. Polya say something similar in how to prove it.
Re:Writing an OS in Rust: Returning from Exceptions by taeric on Wednesday September 21, 2016 @05:01PM (#12552201)
Polya says to start from the solution. The problem is you have to know a definitive solution for this to be applicable.

In the case of this hypothetical OS idea.... Most of what people "would want" from an OS is a made up thing they can't really express. There is no working backwards from a generalization. You work backwards from concrete ideas.

Realistically, the OS everyone wants is "one that runs all of their software." They would prefer one that "doesn't crash." However, starting over with no real software is a non-answer.

Re:Writing an OS in Rust: Returning from Exceptions by dman on Wednesday September 21, 2016 @05:23PM (#12552386)
Unfortunately the requirement of "one that runs all of their software" is the one that means that most new os's are posix/unix clones of some variety. There is too much human effort invested into the posix userspace for us to start from scratch. In fact even Microsoft seems to be struggling with this inspite of their entrenched position.
Re:Writing an OS in Rust: Returning from Exceptions by johncolanduoni on Wednesday September 21, 2016 @10:13PM (#12553832)
So the solution is Unix or (God help us) Win32 forever? I think the nature of the unix user space is one of the things that could stand improving. How about doing something a bit more structured than piping bytes (ascii? UTF8? Latin1? Some binary format? Who knows!) around? Or making tools that don't run in one-shot mode when that's not how they're used (compilers, etc.)?

This need not throw everything away that we have today. Why not make the equivalent of higher level bindings over a C API for command line programs? Or maybe any of a hundred other options that might improve over what we have now? Is unix really the end stage evolution for user space?

Re:Writing an OS in Rust: Returning from Exceptions by lmm on Thursday September 22, 2016 @06:14AM (#12555768)
> So the solution is Unix or (God help us) Win32 forever? I think the nature of the unix user space is one of the things that could stand improving. How about doing something a bit more structured than piping bytes (ascii? UTF8? Latin1? Some binary format? Who knows!) around? Or making tools that don't run in one-shot mode when that's not how they're used (compilers, etc.)?

The solution is incrementally improving these things. What you're talking about is absolutely the kind of thing I'm interested in. Arguably PowerShell is one piece of the right direction. Show me a better way with compilers etc. and I'm interested (I work in Scala and compile time is a big issue, but I couldn't stand to have SBT as my framework).

> This need not throw everything away that we have today. Why not make the equivalent of higher level bindings over a C API for command line programs? Or maybe any of a hundred other options that might improve over what we have now? Is unix really the end stage evolution for user space?

If you've got a way to introduce one of those things incrementally, I'm interested.

Re:Writing an OS in Rust: Returning from Exceptions by xj9 on Wednesday September 21, 2016 @10:56PM (#12554000)
I'm (very slowly) working towards having time to build the hobby OS I've been dreaming about for years. I've been especially inspired by Plan 9, BeOS, and seL4. I love simple, powerful things.
Re:Writing an OS in Rust: Returning from Exceptions by Razengan on Wednesday September 21, 2016 @11:48PM (#12554174)
At this point, any new OS that wants to breaks away from Unix/Windows and hopes to gain mass adoption, will need to come with an Unix/Windows emulation layer built-in.
Re:Writing an OS in Rust: Returning from Exceptions by adamnemecek on Wednesday September 21, 2016 @05:11PM (#12552278)
> Realistically, the OS everyone wants is "one that runs all of their software."

I want an OS that's easy to program to the point I don't actually have to rely on third party software too much. This might be a pipe dream but I'm not convinced it's not achievable on some level.

> They would prefer one that "doesn't crash." However, starting over with no real software is a non-answer.

I'm trying to open a discussion about what ideas have been tried in the past, seem good, but for one reason or another didn't catch on.

Re:Writing an OS in Rust: Returning from Exceptions by taeric on Wednesday September 21, 2016 @08:25PM (#12553398)
How does any modern computer not fit this ideal? Linux, in particular, is very easy to start hacking at.
Re:Writing an OS in Rust: Returning from Exceptions by adamnemecek on Wednesday September 21, 2016 @08:59PM (#12553532)
What percentage of the OS that you use every day was written by you?
Re:Writing an OS in Rust: Returning from Exceptions by taeric on Wednesday September 21, 2016 @11:03PM (#12554026)
Why would I even want that? Like, seriously. How much of the computer could I even make? I even studied this stuff in school and understand the basics. Somewhat beyond the basics, actually. Still, I would have zero capabilities to make my own.

Move that to other areas. My car? Forget about it. I could have a fighting chance at building a bike. However, to build a good one, not a chance.

So... why is this something that is even desirable in the computer?

Now. Do I want to be able to fix what I can? Of course. I run emacs, in part because I like that I can pull up the source, quite easily, of any component I am using. I still rely majority on code written by others. And have no shame or concern in doing so.

Re:Writing an OS in Rust: Returning from Exceptions by adamnemecek on Wednesday September 21, 2016 @11:37PM (#12554136)
> Still, I would have zero capabilities to make my own.

That's the point, you don't have much capability to alter the OS you are running. But you should. It's not unachievable but the languages, tools and OS (and maybe HW) have to come together to allow for this.

> Move that to other areas.

Computer is different from any of those since it's fundamentally a machine to model human mind. I always feel like my OS is somewhat restrictive as to what it lets me to do easily and what alterations it lets me to make.

> Now. Do I want to be able to fix what I can? Of course. I run emacs, in part because I like that I can pull up the source, quite easily, of any component I am using. I still rely majority on code written by others. And have no shame or concern in doing so.

Right. But the languages and tools that we use currently don't lend themselves to terseness and correctness. I think that that's the direction of future languages.

Re:Writing an OS in Rust: Returning from Exceptions by taeric on Thursday September 22, 2016 @12:18AM (#12554316)
I disagree. It is very easy for me to modify Linux. I just don't have the skills to do deep modifications. I used to compile my own kernel. Even made modifications to it. None were worth keeping.

And i don't think languages really help here. Given that many advanced languages have products that are, again, beyond me.

Directly, what sorts of edits do you wish you could do, but feel prevented from doing?

Re:Writing an OS in Rust: Returning from Exceptions by adamnemecek on Thursday September 22, 2016 @12:48AM (#12554445)
> I disagree. It is very easy for me to modify Linux. I just don't have the skills to do deep modifications.

Part of this is that setting up a development environment is kind of tricky.

> And i don't think languages really help here.

They really do actually.

> Directly, what sorts of edits do you wish you could do, but feel prevented from doing?

Imagine an OS where you can do something like "view source" with html, you can edit it, and immediately re-execute it. I think that the list of things I wouldn't look into tweaking would be shorter.

Re:Writing an OS in Rust: Returning from Exceptions by taeric on Thursday September 22, 2016 @10:13AM (#12556914)
Emacs basically has this. It is why I mentioned it. There are a set of functions that are compiled c code, but the majority of emacs is just a click away from my looking at it and modifying if I wanted to.

And that is the thing, I rarely want to. Or, I'll want to, but it isn't easy enough to do that I can get it done. Often, the thing I would like to do is a melpa install away. And the amount of code involved with many features is far beyond my skillset, it usually seems.

Re:Writing an OS in Rust: Returning from Exceptions by lmm on Thursday September 22, 2016 @06:17AM (#12555774)
> I think that the list of things I wouldn't look into tweaking would be shorter.

So can you give maybe a top 3 concrete examples of things you would change?

Re:Writing an OS in Rust: Returning from Exceptions by Retra on Thursday September 22, 2016 @01:30AM (#12554595)
>Imagine an OS where you can do something like "view source" with html, you can edit it, and immediately re-execute it.

Do you mean view the kernel's source? Or user program source? Either way, this already exists, it's just often unsuitable for OS software.

Re:Writing an OS in Rust: Returning from Exceptions by tveita on Friday September 23, 2016 @05:52AM (#12563420)
Sounds like you want a Smalltalk OS. If you haven't already I suggest playing around with Squeak.

http://squeak.org/

Re:Writing an OS in Rust: Returning from Exceptions by justaaron on Thursday September 22, 2016 @09:00AM (#12556490)
I would like to see a very simple OS, built for single-usage, allowing a singular application full control of hardware resources (multi-"threaded", but then let's expose "kernel threads" and "processes" via a rather simple/brutal/direct scheduling API.)

I basically want a simple RTOS and HAL/Driver body. I want timers and clocks. I want some block devices and perhaps file-system-primitives etc. I want TCP/IP and a USB stack. I don't want anything nearly as complex as a Unix, nor need it be Posix compliant etc. I would like it to be runnable from uboot, as it's ARM time now, and arm64 support should be a given.

I would think there is a substantial market for a minimalistic ultra-simple braindead-easy alternative to the "embedded linux" (LOL) wave that seems to be proliferating on all "a" class arm processors...

i need determinacy more than thoroughput, but a 64k ram microcontroller with 500mips isn't going to cut it either. there is a serious hole in the OS market for CPU's where *nix is overkill or unsuitable (and no, RTAI nor a realtime patched kernel count. the complexity of the entire codebase is still there)

I've heard about MirageOS from the oCaml people, but I have not been able to find out any info about timing and concurrency etc...

any info on edgy projects highly appreciated, please!

Re:Writing an OS in Rust: Returning from Exceptions by _pmf_ on Thursday September 22, 2016 @06:45AM (#12555882)
Off the top of my head:

- strongly typed syscalls.

- easier way to circumvent the kernel / OS (unikernels are very active and tangible space, see HFT) while potentially staying relatively safe

- safe zero copy IPC

- projects like https://zinc.rs/ which fold Embedded Linux' Device Tree concept into a coherent declarative HW description that is consumable by the bootstrapper

Re:Writing an OS in Rust: Returning from Exceptions by BinaryIdiot on Wednesday September 21, 2016 @06:33PM (#12552853)
> The current driver models aren't that great either.

Driver developer is soooo far out of my frield but it's always fascinated me. I'm curious why hardware companies didn't go with something like an interface for products versus these highly complex drivers; wouldn't it be possible for each type of device a group comes together, decides on an interface, and now anytime you plug a device in it would work (and if a specialized driver could be better you could install that separately but the goal is making everything magically work, immediately).

But since no one has done it I feel like my idea is either horrible or short sighted. But I do want to look into how it all works one day.

Re:Writing an OS in Rust: Returning from Exceptions by sedachv on Wednesday September 21, 2016 @07:52PM (#12553289)
That is how the original IBM PC BIOS worked with basic hardware, more or less. Both Open Firmware (https://en.wikipedia.org/wiki/Open_Firmware) and EFI (http://www.uefi.org/sites/default/files/resources/EBC_Driver...) include a bytecode interpreter for portable device drivers - the idea is that the hardware device stores the driver as bytecode, the Open Firmware controller loads that driver bytecode when the device is plugged in and uses the CPU to execute it, and provides a uniform API to the OS. From what I read EFI was developed at Intel to be a deliberately incompatible Open Firmware clone for the Itanium.

There really is not that much savings or advantage to this approach so there are not a lot of Open Firmware and EFI drivers out there. You really do not want hardware manufacturers writing drivers either - they tend to be terrible and full of bugs. Having the manufacturer publish the hardware specifications so that drivers can be easily implemented by developers is a much better approach, sadly a lot of manufacturers cannot be bothered with this.

Re:Writing an OS in Rust: Returning from Exceptions by wott on Wednesday September 21, 2016 @09:28PM (#12553661)
During DOS days, standards or more often de-facto standards had appeared. So chip makers or board makers would make their chip or board compatible with another existing model, and you would just need one driver per kind of device to get most of the expected functions. I am thinking of :

-- sound cards being compatible with AdLib and/or SoundBlaster and/or SoundBlaster Pro;

-- network cards being compatible with NE2000;

-- video cards being compatible with VESA standards.

Then Windows took over, buses changed, the shitfest started and never stopped. And now the shitfest comes with NDAs too.

Re:Writing an OS in Rust: Returning from Exceptions by pcwalton on Wednesday September 21, 2016 @10:13PM (#12553834)
Well, we do have things like USB, Bluetooth, Intel HD Audio, ACPI, and so forth, all of which are standards. VESA is still good enough to get an unaccelerated framebuffer set up and going. Network adapters are usually at least supported by Linux, so there's some open-source reference implementation somewhere. (NDISwrapper is mostly a thing of the past, thank goodness...)

Accelerated graphics is probably the biggest remaining disaster, honestly. At least with Vulkan/DX12 we might be getting thinner drivers...

Re:Writing an OS in Rust: Returning from Exceptions by stusmall on Wednesday September 21, 2016 @06:51PM (#12552984)
You aren't shortsighted, they have. Look at USB class drivers.
Re:Writing an OS in Rust: Returning from Exceptions by paulsutter on Wednesday September 21, 2016 @09:31PM (#12553670)
Security.
Re:Writing an OS in Rust: Returning from Exceptions by xj9 on Wednesday September 21, 2016 @10:57PM (#12554004)
Genode on seL4.