News for hackers. Stuff that matters.
Re:OBNC Oberon Compiler by ainar-g on Sunday September 02, 2018 @05:56AM (#17896389)
I've always been fascinated by Wirthian languages. The radical simplicity, the focus on fast compilation and programmer productivity.

One question I haven't researched enough is, how did the people at ETH Zürich used Oberon for OS programming. AFAIK, all Wirthian languages have garbage collection, and the common narrative is that you cannot use a GC'ed language in an OS kernel. How did Wirth et al. work around this in the Oberon OS family?

Re:OBNC Oberon Compiler by badsectoracula on Sunday September 02, 2018 @07:43AM (#17896637)
You can check it here, the first ~100 lines are about memory management:

https://www.inf.ethz.ch/personal/wirth/ProjectOberon/Sources...

The exposed procedures (exposed symbols have a star after their name in their declaration) are New (it is treated in a special way by the compiler to remain type safe even if the language itself wouldn't allow it normally), Mark and Scan. It seems that it uses a block allocator that the garbage collector uses itself privately. These procedures use special "GET"and "PUT" procedures which are translated to raw memory access by the compiler (these are also used to access the hardware which is memory mapped). A test for garbage collection is called periodically as seen below (check PROCEDURE GC):

https://www.inf.ethz.ch/personal/wirth/ProjectOberon/Sources...

The collection occurs either when the system is in low memory or some countdown is reached (set by a call to Collect, although looking at the code it seems other bits can change it too).

The rest of the OS just uses the above facilities.

Re:OBNC Oberon Compiler by pjmlp on Sunday September 02, 2018 @11:27AM (#17897295)
That common narrative is urban myths from modern OS Luddites.

There are many examples of system programming languages, used to implement OSes with GC at kernel level.

- Interlisp-D/Lisp Machines

- Smalltalk

- Mesa/Cedar

- Modula-2+

- Modula-3

- Oberon, Oberon-2, Active Oberon, Component Pascal, Oberon-07

- Sing# and System C#

- D

The only problem is having a company willing to put enough money so that the naysayers don't have other option other than adopting the platform.

However given the short sight of business regarding roadmaps, there isn't a company willing to go full speed on it.

So we get the half-way solution, with Swift, Android, UWP, ChromeOS, .NET. The kernels might not have a GC, but almost everything on userspace is done in GC enabled languages.

Re:OBNC Oberon Compiler by coldtea on Sunday September 02, 2018 @11:59AM (#17897419)
>There are many examples of system programming languages, used to implement OSes

Yeah, but are there any examples of "system programming languages, used to implement OSes with GC at kernel level" where said OSes are fast, practical, and used by way more people and businesses than 100 researchers and hard core fans?

Re:OBNC Oberon Compiler by pjmlp on Sunday September 02, 2018 @12:22PM (#17897497)
Yes, Oberon was one of them, used by the complete IT department at ETHZ, including normal employees. Were they more than 100, I don't know, never counted them.

Or IoT deployments from Oberon Systems and Astrobe, with Astrobe still focused on selling Oberon compilers for embedded development.

As mentioned, it is always a matter of money and business willingness.

Joe Duffy mentions on his Rustconf talk how his team had to deal with lots of Luddites, even though they had Midori running in front of them.

If the likes of Apple, Google and Microsoft were willing to spend the same money as they did/are making JavaScript JIT compiliers fast, we would already get something in mainstream.

Instead they focus on the easy path, closing down the kernel access, and making the userspace available to GC only languages.

Re:OBNC Oberon Compiler by coldtea on Sunday September 02, 2018 @03:25PM (#17898198)
>Yes, Oberon was one of them, used by the complete IT department at ETHZ, including normal employees. Were they more than 100, I don't know, never counted them.

Well, I think this falls in the "researchers and hard core fans" of my range.

Re:OBNC Oberon Compiler by pjmlp on Sunday September 02, 2018 @04:44PM (#17898546)
I doubt the people working on the secretariat were "researchers and hard core fans".

Then again I am used to people not believing in a future with modern safe OSes.

Re:OBNC Oberon Compiler by coldtea on Sunday September 02, 2018 @04:50PM (#17898570)
>I doubt the people working on the secretariat were "researchers and hard core fans"

No, they were just forced to use it for the researchers to prove a point.

Nothing about this speaks to its practically and success. Is it still in use in that organization? If not, why not?

Was it fast enough? Did it actually crash less or just under different conditions? Did it make it easy to connect and use the huge existing ecosystem of code available for other major OSes? Did it have hangups unique to it?

>Then again I am used to people not believing in a future with modern safe OSes.

Well, "belief" is not that scientific or pragmatic a criterion, is it?

Re:OBNC Oberon Compiler by pjmlp on Sunday September 02, 2018 @04:58PM (#17898608)
It is up to each one that cares about safe computing to delve on our history.

All those answers are available on relevant papers.

A few decades ago a company believed JavaScript could be made into a fast language and here we are.

Midori was a very real OS, not welcomed by management, that is hardly a scientific criterion.

Re:OBNC Oberon Compiler by srean on Sunday September 02, 2018 @02:53PM (#17898079)
Given your knowledge on the subject could comment on how Modula-2 compares with Go. Modula-2 had coroutines, did it have a notion of a channel ?

I have feeling that Go is a Modula-2/Pascal with a garbage collector and may be with a better concurrency story. Would that be a fair assessment ?

Re:OBNC Oberon Compiler by pjmlp on Sunday September 02, 2018 @04:40PM (#17898525)
No, Modula-2 did not had channels as such.

However you should look into Active Oberon, where active objects are similar to co-routines. Messages are passed around active objects via message handlers.

https://oberoncore.ru/_media/wiki/lang/reali_p.active_oberon...

http://www.ocp.inf.ethz.ch/wiki/Documentation/Front

Re:OBNC Oberon Compiler by eterps on Sunday September 02, 2018 @02:57PM (#17898095)
Re:OBNC Oberon Compiler by bewuethr on Sunday September 02, 2018 @04:18PM (#17898436)
That page doesn't show up in preview for me, but I'm guessing it's the diagram showing the origin of Go as a blend of Alef, Oberon and C. The authors also say that the package concept was inspired by Modula-2, the syntax for packages/imports/declarations by Oberon-2 and the syntax for method declarations my Object Oberon.

As the influence for channels, though, they mention Alef.

The diagram is on page xii of the publicly available preview: http://www.gopl.io/ch1.pdf

Re:OBNC Oberon Compiler by vidarh on Sunday September 02, 2018 @02:09PM (#17897919)
The main issue with GC for system level use is avoiding GC pauses in latency critical parts of the code. You can achieve that a large number of ways, but one of the simplest is to simply disable the GC in latency critical sections (such as interrupts) and ensure you set aside enough memory to prevent needing a collection to let the critical sections complete.

If you need dynamic allocation in such sections, you can still do so as long as you set safety limits to ensure sufficient memory will be available, or e.g. let the latency critical code allocate from separate parts of the heap or using a different allocation scheme (e.g. pre-allocated arenas).

There are lots of details to get right, but the "common narrative" basically assumes you're not able to disable the collector when you want to and no way of safely pre-empting the collector. But making it possible to selectively disable a GC and ensure your latency critical code can safely do so is fairly straightforward.

Re:OBNC Oberon Compiler by eterps on Sunday September 02, 2018 @02:24PM (#17897966)
Do you know examples cases of such critical parts in system level code where a GC pause would be problematic?
Re:OBNC Oberon Compiler by vidarh on Saturday September 08, 2018 @04:31PM (#17942617)
Pretty much any interrupt code, and lots of hardware drivers.
Re:OBNC Oberon Compiler by lazyjones on Sunday September 02, 2018 @06:17AM (#17896434)
> the common narrative is that you cannot use a GC'ed language in an OS kernel. How did Wirth et al. work around this in the Oberon OS family?

OS do not consist only of a kernel (nitpicking, sorry). Oberon hides the low-level functionality in special modules where implementations are provided by the compiler (in assembly language if needed), i.e. „builtins“. In other words, what is part of the kernel in e.g. Linux can be implemented as a language feature (system module) in the compiler and the resulting code can run on „bare metal“.

Re:OBNC Oberon Compiler by todd8 on Sunday September 02, 2018 @12:46PM (#17897573)
Wirth’s early languages, Euler, Algol-W, Pascal, and Modula did not have garbage collection—memory management was up to the programmer. Later languages Modula-3, Oberon, and Component Pascal do have garbage collection.
Re:OBNC Oberon Compiler by srean on Sunday September 02, 2018 @02:28PM (#17897983)
Wirth had nothing to do with Modula-3 though.

[Prior to edit:] Modula-3 had nothing to do with Modula-3 though.

Re:OBNC Oberon Compiler by pjmlp on Sunday September 02, 2018 @04:51PM (#17898572)
Yes he had, although it is not a direct creation from him.

Modula-3 is the evolution of Mesa/Cedar, after the researchers went to DEC and created Modula-2+, an improved version of Mesa/Cedar.

They collaborated with Wirth on their work.

"The design of the programming language Modula-3 was a joint effort by the Digital Systems Research Center and the Olivetti Research Center, undertaken with the guidance and inspiration of Niklaus Wirth."

http://lucacardelli.name/papers/modula3typesystem.a4.pdf

Re:OBNC Oberon Compiler by newnewpdro on Sunday September 02, 2018 @03:48PM (#17898287)
> Modula-3 had nothing to do with Modula-3 though.

What?

Re:OBNC Oberon Compiler by srean on Sunday September 02, 2018 @03:58PM (#17898344)
I snorted my drink out after reading it myself. Thank you for catching that.
Re:OBNC Oberon Compiler by nickpsecurity on Sunday September 02, 2018 @01:04PM (#17897649)
What they do is make things safe by default with a keyword to turn it off. Ada and PL/S (IBM's) do this, too. They also have other keywords to tell the compiler how to handle some statements that might be undefined in C. For unsafe, the module will either use the same language without safety or GC, or assembly language, or some combo of that. Developers using such features often have type checks and input validation on their arguments since their usage is dangerous. So, the majority of the code is using safe primitives with GC while specific parts that can't are behind protected interfaces.

The next thing you can do is use lightweight formal methods, static analysis, etc on the unsafe code. So, those primitives are verified in unsafe HLL or in assembly. You know they'll probably act right given preconditions are met. Then, the rest of code is protected by type- and memory-safety. Combining best tool for each job makes your whole system safe despite unsafe code.

One person I know had a successful test of using SPARK Ada for fast, otherwise-unsafe components of Go applications. My Brute-Force Assurance idea would likely, given most use C language underneath, do these in Frama-C compiled with CompCert or C0 after I threw every static/symbolic analyzer, concurrency checker, and fuzzer at it. Very unlikely to have trouble out of anything that passes all that if the spec or mental model itself is correct. Rest is safe, memory-managed code. It also helps to keep same data types, calling conventions, and so on to prevent abstraction-gap attacks. There's ongoing work on abstraction-safe, verified integration of components in different languages, though.

Also, note that you can use different memory managers in different apps or even parts of an app. The runtime has to be designed to do that. JX OS in Java is one that is. There are GC's optimized for throughput, average latency, hard real-time, size, and so on. A standard allocator gives the code a block to use, they assign to or divide it up for the other GC's, and they work from there. There might also be some work to do on scheduling how they run, too. I can't remember how easy or difficult that is on a common platform. It was mostly done on custom OS's.

Re:OBNC Oberon Compiler by stewbrew on Sunday September 02, 2018 @09:23AM (#17896915)
Oberon (originally) wasn't just a language but also a single-threaded environment with cooperative multitasking (somewhat similar to Smalltalk but with a fast & efficient compiler).
Re:OBNC Oberon Compiler by pankajdoharey on Sunday September 02, 2018 @08:57AM (#17896809)
Actually before Apple implemented ARC in its OS Around Mac OS X Lion and iOS 5, the Objective-C runtime in Mac OS X was a garbage collection runtime.
Re:OBNC Oberon Compiler by cturner on Sunday September 02, 2018 @09:19AM (#17896901)
Wikipedia describes GC as being automatic memory management. I don't think it is valid to describe the traditional (pre-2007) osx/objective-c model of manual reference counting as a form of garbage collection.

Darwin itself is mostly unix+C. Hence, this does not respond to the parent, asking about GC in the OS itself.

Re:OBNC Oberon Compiler by Volt on Sunday September 02, 2018 @11:08AM (#17897229)
>Darwin itself is mostly unix+C. Hence, this does not respond to the parent, asking about GC in the OS itself.

You're right, but just to clarify WRT your first paragraph, Obj-C did briefly have a GC'd runtime between the eras of manual and automatic ref counting.

Re:OBNC Oberon Compiler by cturner on Monday September 03, 2018 @05:12AM (#17901428)
As you and neighbour inferred, I had not know of this. Thanks.
Re:OBNC Oberon Compiler by pjmlp on Sunday September 02, 2018 @11:36AM (#17897332)
Objective-C 2.0 got a tracing GC, which was later replaced by ARC, because it was a failure.

Although it was possible to link frameworks compiled in mixed modes, it lead most of the time to crashs and the C semantics did not help.

Hence the pragmatic solution of ARC, which basically has the compiler inserting the retain/release calls that used to be done manually.

While not the same, it was a good enough solution.

https://developer.apple.com/documentation/foundation/object_...

Re:OBNC Oberon Compiler by cellularmitosis on Monday September 03, 2018 @01:35AM (#17900547)
That was true for Mac OS but not for iOS. You had to manage memory manually with alloc and release.