News for hackers. Stuff that matters.
Re:Redox – A Unix-Like Operating System Written in Rust by adamnemecek on Tuesday November 13, 2018 @12:38PM (#18442876)
Redox is cool but I'm pumped for the OS' that really take advantage of Rust to create new architectures.
Re:Redox – A Unix-Like Operating System Written in Rust by johannes1234321 on Tuesday November 13, 2018 @12:58PM (#18443082)
The issue with such an OS is that there aren't won't be any libraries and tools. So the OS vendor might need a custom shell, custom utilities, custom libraries for everything ... or they offer POSIX-like APIs. One area where one can see this is IncludeOS - they had their custom APIs for a long time to leverage their architecture, but are now focussed on providing POSIX compatibility ... and that's only for single purpose unikernel systems. If the OS should also be universal this is even more of a problem. (Especially if you also want a graphical desktop ...)
Re:Redox – A Unix-Like Operating System Written in Rust by AnIdiotOnTheNet on Tuesday November 13, 2018 @01:09PM (#18443182)
With that attitude you'll never be able to ditch legacy cruft, and there's quite a lot of it in POSIX. I agree with the other poster that a compatibility layer in the form of virtualization and/or emulation should be sufficient, as long as your new OS brings something to the table that's desirable. Shunt the legacy crap off into its own contained environment and build a nice new clean one for new stuff to use.
Re:Redox – A Unix-Like Operating System Written in Rust by nixpulvis on Tuesday November 13, 2018 @01:12PM (#18443209)
I still think there are a plenty of reasons to support POSIX in many places. As someone who's been running fish for a while, I can appreciate the common ground it provides across the current UNIX ecosystem, but it's not like translating a bash script to fish is impossible.

There should always be a supported standard, but there should be nothing forcing you too it. This is the freedom we need to demand of our OSes.

Re:Redox – A Unix-Like Operating System Written in Rust by johannes1234321 on Tuesday November 13, 2018 @01:22PM (#18443315)
Yeas, in a imaginary world replacing POSIX would be great. But that's not an easy endeavor. 30 years of legacy tied to it.

As long as you don't have POSIX only special interest users will use your system.

I believe the only way forward is to start on POSIX and then move on step by step and deprecating part by part.

Re:Redox – A Unix-Like Operating System Written in Rust by AnIdiotOnTheNet on Tuesday November 13, 2018 @02:00PM (#18443730)
In the same imaginary world, creating a new POSIX OS that isn't Linux is a giant waste of time just from gap in drivers alone. If you're going to ditch all that hardware support then you may as well ditch POSIX and its crap while you're at it.
Re:Redox – A Unix-Like Operating System Written in Rust by pjmlp on Wednesday November 14, 2018 @03:36AM (#18448439)
Which is a reason why despite disliking what Google did to Sun and how hard they make to use the NDK, in the end I appreciate their Android design.

No POSIX for userspace as official API, rather ISO C and C++ APIs plus Android native APIs.

Re:Redox – A Unix-Like Operating System Written in Rust by yjftsjthsd-h on Tuesday November 13, 2018 @06:24PM (#18445858)
Unless you have some portable standard to target, you just tied the entire universe to exactly 1 OS implementation. I would like to not do that.
Re:Redox – A Unix-Like Operating System Written in Rust by AnIdiotOnTheNet on Tuesday November 13, 2018 @10:46PM (#18447302)
Bull. People developed software for multiple completely different computer architectures throughout the 80s and 90s. People do it today between different game console platforms in addition to operating systems. Even if you try and target something like SDL or a web browser your abstraction won't save you from a platform's quirks once you reach a certain level of complexity and then you'll have to work around it anyway.

Hell, even between supposedly POSIX systems there's a lot of #ifdef going on to make things work.

Re:Redox – A Unix-Like Operating System Written in Rust by ansible on Tuesday November 13, 2018 @02:13PM (#18443872)
My issues with POSIX stem from the fact that writing completely correct code which handles signals, interruptable operating system calls, and threads is hard. There are plenty of little details that are easy to get wrong. And you won't know you've gotten something wrong until much later when some confluence of events occurs.

I don't know if deprecating parts of POSIX is going to work any better than deprecating parts of C++. If all the bad stuff is still there waiting to be misused...

Re:Redox – A Unix-Like Operating System Written in Rust by monocasa on Tuesday November 13, 2018 @02:58PM (#18444318)
Or successfully pull off the library OS concept with POSIX as a first class citizen. Z/OS is most of the way there. NT has tried a couple times with the POSIX subsystem first, and now their Windows Subsystem for Linux work came out of MS Research's drawbridge library OS work.
Re:Redox – A Unix-Like Operating System Written in Rust by pjmlp on Wednesday November 14, 2018 @03:38AM (#18448447)
And SQL Server on Linux makes use of library OS concept as well.
Re:Redox – A Unix-Like Operating System Written in Rust by monocasa on Wednesday November 14, 2018 @10:48AM (#18450722)
That's heavily overstated.
Re:Redox – A Unix-Like Operating System Written in Rust by pjmlp on Wednesday November 14, 2018 @03:30AM (#18448420)
On the real mobile OS world that is already a fact for user space apps.
Re:Redox – A Unix-Like Operating System Written in Rust by karr0n on Tuesday November 13, 2018 @01:32PM (#18443419)
Virtualization/Emulation seems to be in the making. Let's see how it goes
Re:Redox – A Unix-Like Operating System Written in Rust by nerdponx on Tuesday November 13, 2018 @02:39PM (#18444131)
As a user, I personally wouldn't mind a "fresh start" when it comes to userspace. Just look at Haiku -- yes, it's technically not a new design, but it sure ain't Unix.
Re:Redox – A Unix-Like Operating System Written in Rust by waddlesplash on Tuesday November 13, 2018 @03:13PM (#18444467)
> yes, it's technically not a new design, but it sure ain't Unix.

Except we are? We are pretty POSIX compliant all the way into the kernel, we have "/dev", filemodes, etc. We don't have X11 or other UNIX staples, sure, but we are pretty UNIXy.

Re:Redox – A Unix-Like Operating System Written in Rust by nerdponx on Tuesday November 13, 2018 @07:15PM (#18446149)
I didn't know that. Thanks for clarifying.

By "userspace" I was more talking about "the programs and interfaces that a normal user interacts with". Haiku is pretty unique in that regard.

Re:Redox – A Unix-Like Operating System Written in Rust by waddlesplash on Tuesday November 13, 2018 @11:19PM (#18447453)
> By "userspace" I was more talking about "the programs and interfaces that a normal user interacts with". Haiku is pretty unique in that regard.

In terms of GUI apps... sorta? We use extended attributes and IPC messaging more than most Linux desktops do, that's true, and our UI/UX is often different.

But if you're talking CLI, then, also no. Bash is the default shell, coreutils are installed by default, sshd is activated by default, etc.

Re:Redox – A Unix-Like Operating System Written in Rust by pjmlp on Wednesday November 14, 2018 @03:39AM (#18448451)
I don't recall BeOS being that POSIX friendly though.

Haiku own extensions to the original design?

Re:Redox – A Unix-Like Operating System Written in Rust by waddlesplash on Wednesday November 14, 2018 @10:52AM (#18450776)
Yes, indeed; we implement most of the mainline POSIX specification where BeOS did not at all (it didn't even have pthreads.)
Re:Redox – A Unix-Like Operating System Written in Rust by ptman on Wednesday November 14, 2018 @10:28AM (#18450544)
BeOS supported enough posix to use gcc.
Re:Redox – A Unix-Like Operating System Written in Rust by pjmlp on Wednesday November 14, 2018 @10:31AM (#18450570)
ISO C and POSIX are orthogonal. GCC also supports non-POSIX platforms.
Re:Redox – A Unix-Like Operating System Written in Rust by adamnemecek on Tuesday November 13, 2018 @01:06PM (#18443153)
> The issue with such an OS is that there aren't won't be any libraries and tools.

This might not be as big of a deal. Rust increases your productivity quite a bit and I'm really impressed with the pace of progress in the community. I can imagine that new, better & more integrated tools will be made.

Re:Redox – A Unix-Like Operating System Written in Rust by nixpulvis on Tuesday November 13, 2018 @01:02PM (#18443122)
Don't even need to implement POSIX if you just emulate another OS inside, this seems to be a pretty viable solution for some of the cases.

I'm more concerned with drivers/firmware, which could be handled the same way, but seems less appropriate.

Re:Redox – A Unix-Like Operating System Written in Rust by pjmlp on Wednesday November 14, 2018 @03:29AM (#18448415)
Ditching POSIX is all a matter of money and willingness, as proven by iOS, Android and ChromeOS.
Re:Redox – A Unix-Like Operating System Written in Rust by empath75 on Tuesday November 13, 2018 @04:17PM (#18444985)
Not really relevant to rust, but if you squint, Kubernetes is basically an operating system.
Re:Redox – A Unix-Like Operating System Written in Rust by k__ on Tuesday November 13, 2018 @12:45PM (#18442947)
Like?
Re:Redox – A Unix-Like Operating System Written in Rust by protomyth on Tuesday November 13, 2018 @01:53PM (#18443644)
My guess is someone will try to have an OS based on containers of Web Assembly apps. There are quite a few APIs that have been built over the years that are familiar to programmers. I do believe this will cut down on the pain of having to new develop system level tools to manage such a beast.
Re:Redox – A Unix-Like Operating System Written in Rust by bluejekyll on Tuesday November 13, 2018 @02:26PM (#18444013)
https://github.com/nebulet/nebulet

"Nebulet is a microkernel that executes WebAssembly modules in ring 0 and a single address space to increase performance. This allows for low context-switch overhead, syscalls just being function calls, and exotic optimizations that simply would not be possible on conventional operating systems. The WebAssembly is verified, and due to a trick used to optimize out bounds-checking, unable to even represent the act of writing or reading outside its assigned linear memory."

Re:Redox – A Unix-Like Operating System Written in Rust by zaarn on Wednesday November 14, 2018 @04:25AM (#18448597)
WebAssembly has a few more advantages when deployed as part of the kernel; you can run things in ring 0. Even better, you can transparently remap things in memory. And even better than, you can realize IPC as a simple function pointer. Safely.

Of course, you can't have everything as WebAssembly, some core drivers will need to run some critical machine code, but those could be tightly enough integrated that the overhead is almost zero (ie, by using WA imports you can turn this into a function call overhead)

Re:Redox – A Unix-Like Operating System Written in Rust by adamnemecek on Tuesday November 13, 2018 @12:58PM (#18443090)
* entirely async api

* more light weight memory manager (possible thanks to borrow checker)

* GPU first OS

* a better shell. I know I can run whatever shell I want on unix but a better shell being native would go far.

Re:Redox – A Unix-Like Operating System Written in Rust by nixpulvis on Tuesday November 13, 2018 @01:07PM (#18443161)
I agree with the first two points, not sure how Rust and GPUs are really related yet. I mean I know you can bind into GL/etc libs, but there's something more profound about Rust's type system, and the parallelism / memory model of a GPU (or CPU/heterogeneous computation in general). AFAIK, there's no way to write GPU shader code that shares the static analysis from the Rust CPU code. It would be very interesting to be able to talk about the move semantics across the full modern computing architecture.

If anyone knows work being done in this area I'd be curious to read more personally.

As for a better shell, I also completely agree, but I'm not sure it needs to break POSIX. Shameless little plug, I recently started a shell in Rust myself: https://github.com/nixpulvis/oursh

Re:Redox – A Unix-Like Operating System Written in Rust by eridius on Tuesday November 13, 2018 @01:30PM (#18443391)
As a fish user myself, I would love to see a new shell that retains many of the UI features of fish (like the excellent autocompletion behavior while typing) but with an actual usable modern fast scripting language.

POSIX compatibility at the scripting layer is beneficial for being able to run existing shell scripts, but the sh scripting language sucks in many ways.

What I'd really like to have is a shell that supports both a POSIX compatibility mode for running existing scripts, alongside a more powerful and modern scripting language for use in writing scripts targeting the new shell directly. I'm not sure how to identify which mode an arbitrary script should run in though, or which mode should be used at the command line.

Re:Redox – A Unix-Like Operating System Written in Rust by nixpulvis on Tuesday November 13, 2018 @01:32PM (#18443422)
I've started allowing just that `{#! }` also as a means to write other languages inside the shell syntax.

Take a look at: https://nixpulvis.com/oursh/oursh/program/index.html

Re:Redox – A Unix-Like Operating System Written in Rust by eridius on Tuesday November 13, 2018 @02:15PM (#18443897)
Oh great! I hope you take good care in designing the "modern" language, because once people start writing scripts for your shell, it becomes very hard to fix mistakes (this has been a problem for fish's scripting). I wish I had the time to be involved in designing a new shell scripting language, as it's something I'd really like to see done right, I just have no time to spend on that.

Incidentally, the link to the `modern` module is broken, it's just program::modern (which is of course not a valid link). Given that I don't see a `modern` module in the TOC I'm assuming the module doesn't actually exist yet?

Re:Redox – A Unix-Like Operating System Written in Rust by nixpulvis on Tuesday November 13, 2018 @02:22PM (#18443978)
I still living in the 70s for now... Maybe one day I can be a modern man living in a modern world.

Until then I struggle with background jobs, because they are a fucking pain in the ass.

Re:Redox – A Unix-Like Operating System Written in Rust by eridius on Tuesday November 13, 2018 @02:37PM (#18444116)
Oh man, a lot of the ideas I've thought about for improving handling of shell scripts have problems in the presence of background jobs, and in particular the ability to background a job that's currently foregrounded.

On a related note, here's something I've been thinking about:

I want to be able to insert shell script functions in the middle of a pipeline without blocking anything. Or more importantly, have two shell functions as two different components of the pipeline. I believe fish handles this by blocking everything on the first function, collecting its complete output, then continuing the pipeline, but that's awful. Solving this means allowing shell functions to run concurrently with each other. But given that global variables are a thing (and global function definitions), we need a way to keep the shell functions from interfering with each other. POSIX shells solve this by literally forking the shell, but that causes other problems, such as the really annoying Bash one where something like

  someCommand | while read line; do …; done
can't modify any variables in the while loop because that's running in a subshell.

So my thought was concurrently-executed shell functions can run on a copy of the global variables (and a copy of the global functions list). And when they finish, we can then merge any changes they make back. I'm not sure how to deal with conflicts yet, but we could come up with some reasonable definition (as this all happens in-process, we could literally attach a timestamp to each modification and say last-change-wins, though this does mean a background process could have some changes merged and some discarded, so I'm not sure if this is really the best approach; we could also use the timestamp the job was created, or finished, or we could also give priority to changes made by functions later in a pipeline so in e.g. `func1 | func2` any changes made by func2 win over changes made by func1).

When I first started typing this out I thought that this scheme didn't work if the user started a script in the foreground and then backgrounded it, but now that I've written it out, it actually could work. If every script job runs with its own copy of the global environment, and merges the environment back when done, then running in the foreground and running in the background operates exactly the same, and this also neatly solves the question of what happens if a background job finishes while a foreground job is running; previously I was thinking that we'd defer merging the background job's state mutations until the foreground job finishes, but if the foreground job uses the same setup of operating on a copy of the global state, then we can just merge whenever. The one specialization for foreground jobs we might want to make is explicitly defining that foreground jobs always win conflicts.

Re:Redox – A Unix-Like Operating System Written in Rust by nixpulvis on Tuesday November 13, 2018 @03:26PM (#18444567)
This is along the lines of things I was thinking myself. I'm currently aiming to get the POSIX programs working 100%, which I don't believe could allow this. But, the framework for running managing foreground and background jobs should support both the POSIX and Modern syntax and something like this. This is EXACTLY the kind of thing I want to add to the new "modern" language!

Also, the ability to "rerun" previous commands from a buffer without actually re-executing anything would be a cool somewhat related feature.

If you want to chat about shells anytime shoot me an email or something: [email protected]

Re:Redox – A Unix-Like Operating System Written in Rust by pjmlp on Tuesday November 13, 2018 @03:15PM (#18444479)
Entirely async api and GPU first OS were fulfilled by Windows Phone, sadly did not work out well.
Re:Redox – A Unix-Like Operating System Written in Rust by GordonS on Tuesday November 13, 2018 @01:07PM (#18443165)
A bit terse, since I'm on mobile:

> * entirely async api

Surely async only makes sense for IO?

> * GPU first OS

What does this actually mean? The GPU can't be used for everything

> * a better shell. I know I can run whatever shell I want on unix but a better shell being native would go far.

What do you mean by 'native'? Do you just mean 'ships with the OS'?

Re:Redox – A Unix-Like Operating System Written in Rust by sam0x17 on Tuesday November 13, 2018 @12:50PM (#18442999)
As in something different from unix I take it
Re:Redox – A Unix-Like Operating System Written in Rust by nixpulvis on Tuesday November 13, 2018 @12:54PM (#18443034)
I feel like a new fresh look at what UNIX is today could be valuable. I wouldn't want to give up a lot of the philosophy around it. Redox does in fact do this in a number of places, for example I think the expansion on "everything is a file" [1] is a pretty awesome idea.

[1]: https://doc.redox-os.org/book/design/url_scheme_resource/eve...

Re:Redox – A Unix-Like Operating System Written in Rust by sam0x17 on Tuesday November 13, 2018 @02:00PM (#18443732)
I'd love to see a "there are no paths, file system is a db" like it was originally rumored Vista was going to be. I'd also love to see more opinionated OS integration e.g. per-application volume sliders, better file system search indexing, real-time built-in file and directory watching API that can be blocking or non-blocking, standardized storage of program files instead of installed app locations being ambiguous, standardized system and app settings via a neat API, etc., etc., an easy and sane git-based package manager, etc
Re:Redox – A Unix-Like Operating System Written in Rust by GordonS on Tuesday November 13, 2018 @02:14PM (#18443891)
Windows 10 does per-application volume control.

Chrome OS has a fixed app install location, as does Windows 10 in 'S' mode (since you can only install store apps).

I remember being intrigued by the database-as-filesystem idea when it was first touted - has any OS actually implemented this? I'd be interested to see how it works in practice.

Re:Redox – A Unix-Like Operating System Written in Rust by TickleSteve on Tuesday November 13, 2018 @02:42PM (#18444177)
"pick"

https://en.wikipedia.org/wiki/Pick_operating_system

Everything old is new again... this was from the mid-60s and was relatively popular for its time.

Re:Redox – A Unix-Like Operating System Written in Rust by gvx on Tuesday November 13, 2018 @03:32PM (#18444625)
Very interesting. The one thing that stood out to me was this, though:

> It is named after one of its developers, Dick Pick.

What an unfortunate name.

Re:Redox – A Unix-Like Operating System Written in Rust by gregw2 on Tuesday November 13, 2018 @09:42PM (#18447015)
OS/400 did this, very cool tech that was never "cool"
Re:Redox – A Unix-Like Operating System Written in Rust by pjmlp on Wednesday November 14, 2018 @03:45AM (#18448469)
Given the way Android and UWP apps work, I always have a warm fuzzy feeeling of carrying a subset of OS/400 design ideas with me.
Re:Redox – A Unix-Like Operating System Written in Rust by adamnemecek on Tuesday November 13, 2018 @02:35PM (#18444102)
These are all very good ideas I’ve also thought about.

WinFS was ahead of its time. Also re file watching, I think I want a reactive api for a lot ornate operations.

Re:Redox – A Unix-Like Operating System Written in Rust by dralley on Tuesday November 13, 2018 @01:08PM (#18443180)
That only makes things more confusing because Redox is explicitly not a traditional Unix. It's more like Plan9, except it goes even further than Plan9 in few places, and things like the Ion shell in Redox don't attempt to be Posix compatible.
Re:Redox – A Unix-Like Operating System Written in Rust by pjmlp on Wednesday November 14, 2018 @03:46AM (#18448473)
What about comparing Redox with Inferno instead?

The actual end of the line for UNIX improvements done by the original designers.

I never get the mistic around the middle stop instead of the end of the line.

Re:Redox – A Unix-Like Operating System Written in Rust by stcredzero on Tuesday November 13, 2018 @12:51PM (#18443007)
If an operating system was built to run lambdas instead of processes, what would that look like?
Re:Redox – A Unix-Like Operating System Written in Rust by nixpulvis on Tuesday November 13, 2018 @12:59PM (#18443096)
What is a lambda to you in this case? That lambda will need to be scheduled, it will need to maintain it's scope... All the same general issues could exist. There have been plenty of machines that simply run functions, in fact before there was things we call OSes today, the machines that ran the code would typically not have an notion of time sharing, and would map even more closely to a pure lambda evaluator.

Without specifics about what differences you mean, lambda = function = process = thread = fiber = service = worker = ...

Re:Redox – A Unix-Like Operating System Written in Rust by stcredzero on Tuesday November 13, 2018 @01:18PM (#18443281)
What is a lambda to you in this case?

Pure functions.

That lambda will need to be scheduled, it will need to maintain it's scope... All the same general issues could exist.

But no users would have to be able to start processes. Instead, lambdas could be associated with persistent storage of state, and processes would be started by the OS to apply the lambdas in a simulation loop, but users wouldn't directly start processes.

Perhaps thinking of those as processes isn't quite right either.

Re:Redox – A Unix-Like Operating System Written in Rust by chc4 on Tuesday November 13, 2018 @02:06PM (#18443799)
Have you looked at Urbit? It uses this idea - programs are state machines that you give to the runtime, which pumps them with events (IO, RPC calls, etc) that return [new-state events-out]. All programs are purely functional and deterministic, so you can replay events or serialize the app to transfer somewhere else.
Re:Redox – A Unix-Like Operating System Written in Rust by stcredzero on Tuesday November 13, 2018 @07:20PM (#18446178)
All programs are purely functional and deterministic, so you can replay events or serialize the app to transfer somewhere else.

Interesting. Actually, I have one of my processes serializing its state and exporting it to a different process, while all of the clients continue playing the MMOsteroids game they're logged into.

Re:Redox – A Unix-Like Operating System Written in Rust by nixpulvis on Tuesday November 13, 2018 @01:21PM (#18443310)
The only truly pure functions exist in the imaginary world of mathematics, if you even believe that.

But I think I kinda get your point, but would challenge you to think about this issue with the frame of mind of access control and permissions a bit. I think you'll find the need for some kind of process like task. Maybe not...

Re:Redox – A Unix-Like Operating System Written in Rust by stcredzero on Tuesday November 13, 2018 @01:44PM (#18443562)
The only truly pure functions exist in the imaginary world of mathematics, if you even believe that.

Note I already mentioned persistent stores of state.

I think you'll find the need for some kind of process like task. Maybe not...

Maybe re-read. I've already said that there would be a process like task. Lambdas will need to be associated with state. Users won't have to start processes. Instead, processes will be more like processors.

Re:Redox – A Unix-Like Operating System Written in Rust by macintux on Tuesday November 13, 2018 @02:01PM (#18443743)
This sounds like the Erlang VM.
Re:Redox – A Unix-Like Operating System Written in Rust by stcredzero on Tuesday November 13, 2018 @07:28PM (#18446236)
I'll take that as positive feedback.
Re:Redox – A Unix-Like Operating System Written in Rust by nickpsecurity on Tuesday November 13, 2018 @05:05PM (#18445357)
I'd look at Haskell-based stuff first. I was told, though, by Haskellers that things like House operating system are imperative in style even if done in Haskell. So, a quick search for functional programs for OS's gave these possibilities:

https://news.ycombinator.com/item?id=7166755

You were actually in that thread. Did the commenter's work not actually use FP or you not see it?

Re:Redox – A Unix-Like Operating System Written in Rust by stcredzero on Tuesday November 13, 2018 @07:28PM (#18446232)
You were actually in that thread.

That's kinda trippy. It's like I was a different person. Also, I was working on the predecessor system to the one I'm currently working on. Back then, the thing was written in Clojure. I later ported it to Go. The design philosophy has changed a heck of a lot as well. Back then, I was going to have everything on the same very large and fast virtual instance, with modest goals for the largest population/scale. Now my system is scalable by adding more "workers," which I've spread out onto small AWS instances.

Re:Redox – A Unix-Like Operating System Written in Rust by abecedarius on Tuesday November 13, 2018 @05:44PM (#18445622)
"A security kernel based on the lambda calculus" http://mumble.net/~jar/pubs/secureos/

http://cap-lore.com/CapTheory/upenn/

Re:Redox – A Unix-Like Operating System Written in Rust by ASipos on Tuesday November 13, 2018 @12:57PM (#18443071)
If K-A-T doesn't spell 'cat', what does it spell?
Re:Redox – A Unix-Like Operating System Written in Rust by Crespyl on Tuesday November 13, 2018 @03:05PM (#18444392)
'kat'
Re:Redox – A Unix-Like Operating System Written in Rust by someguydave on Tuesday November 13, 2018 @05:18PM (#18445456)
It would look like a LISP machine from Symbolics...