Does that run on all the BSDs and nicely on low-performance embedded ARM boards, which are explicitly targeted by NTPsec? Does it talk to low-level OS APIs for time management without complicated wrappers?

It's a replacement for one of the many background UNIX services, not a "primary" service that can own a machine.

You know SIM cards run Java, right?

They run something called Java Card, which is a very limited subset of Java (missing things like threads, enums or floating point numbers). And they are highly standardized, so you can be sure it is not just available and supported, but optimized for running these applets.

That doesn't really say anything about viability for an open-source project that's supposed to run on all kinds of embedded and general-purpose hardware.

Java would be disqualified due to GC:

> It is worth noting that Go came nearest disqualifying itself entirely here. If it were not possible to lock out Go’s GC in critical regions Rust would win by default.

I think the assumption is that, as ntpsec is "soft real time" i.e. not hard real time, sufficiently low pauses mean you wouldn't need to lock out the GC at all.

I mean, technically anything the system or runtime does on your behalf can be thought of as a 'pause'. What if you lose your timeslice half way through a function?

What technique is used to lock the GC out?

i'd guess a negative arg passed to https://golang.org/pkg/runtime/debug/#SetGCPercent

I hope not, because this way a critical section requires two non-concurrent GCs (SetGCPercent calls runtime.GC() [1] and runtime.GC() triggers a non-concurrent GC [2]).

If this is the option, I hope the critical sections in ntpsec are few and far apart...

[1] https://golang.org/src/runtime/debug/garbage.go?s=3706:3740#...

[2] https://golang.org/src/runtime/mgc.go?s=33685:33694#L844

dunno- just copying what was in the original article :)

It's not pauseless.

> Shenandoah is an ultra-low pause time garbage collector that reduces GC pause times by performing more garbage collection work concurrently with the running Java program

Funny enough, the term "pauseless" was in fact invented precisely because of people like you, who abused the original terms ("concurrent" or "asynchronous" [0]) used for garbage collectors that never "stop the world", and used it for algorithms that merely run in parallel with the mutator threads, pausing them every so often. The next term used was "fully-concurrent" and now "pauseless". But if you keep abusing these terms to mean not fully pauseless/concurrent GCs, then it will become just as useless as all the terms before it.

[0] http://gallium.inria.fr/~xleroy/publi/concurrent-gc.pdf

> the term "pauseless" was in fact invented precisely because of people like you, who abused the original terms

This language seems unnecessarily personal and accusatory.

I put "pauseless" in quotes because I know it's not entirely pauseless. It's described as "ultra low pause time" by the original project.

However, whether we like it or not, Azul has established the term "pauseless" as the way to describe their own collector which before Shenandoah was the only GC that can realistically be described that way, and which Shenandoah is very similar to. This is despite the fact that Zing still pauses. Specifically, the GC moves objects whilst the program runs.

I am unaware of any GC algorithms that are truly, completely pauseless in every situation. So I don't know what you want the term to be reserved for.

The question is what sort of pauses do you care about. I don't know about Shenandoah but the original Pauseless GC that Azul developed (and the current C4 on x86) have smaller GC pause times then thread switches.

So, if you are operating in a environment where you don't care about thread switching, you might also not care about GC anymore.

You are correct however, its not pauseless, that just a marketing term.

AFAIK Azul is pauseless - it never "stops the world". Sure, it stops each thread (that's pretty much required for any synchronization mechanism, especially if you also need to determine the thread's roots), but it never stops all threads at once ("the world").

It never stops the world for full heap collection, but there is a synchronization point where all threads have to check in. This is the only thing required, so you have GC pauses, just really, really small ones. That's why I said, that practically, GC pauses are no longer relevant.

That's not correct. Unless it changed recently, Zing still has safe points as it's basically a fork of HotSpot and HotSpot pauses programs for all kinds of reasons beyond GC.

There's a talk on YouTube where they discuss their work to drive down safe point pause times. They eventually made the pauses so fast it hardly matters, but it wasn't technically completely pauseless.

Yes, Azul's C4 paper states that they do use some "synchronisation" stop-the-world phases (although their previous GCs that had hardware/kernel read barriers might not have), but IIRC Doligez-Leroy GC doesn't (although I'm not sure it was ever properly implemented).

On the other hand, Shenandoah has an explicit synchronisation phase in its algorithm (not sure if that's a fundamental property of the algorithm or just a JVM limitation though). From [0] (that was 2 years ago, things might have improved since, but I can't find a more recent version).

> Pauses only long enough to scan root set

[0]: https://rkennke.files.wordpress.com/2014/02/shenandoahtake4....

They had synchronization points on the hardware platform as well. The difference is that they had some amount of hardware support that made the thread check-in extremely fast.

I think Zing has always had safe points.

Calling a GC that does much of its work while not-blocking-your-code seems like an appropriate use of the word "concurrent". Concurrent almost never implies "will never block", as far as I've seen people use it. If anything, that would be a "lock-free concurrent GC" or something.

Has he GP edited his comment? "Pauseless" in quotes seems to suggest recognition that the GC is not truely pauseless. I interpretted the comment as being in contrast to Go's GC, which also purports to be very low pause.

No, it had quotes from the start.

GC must also reallocate used memory to avoid fragmentation of free memory. Do you know a true fully pauseless algorithm for that?

Downvoters - WTF? Memory fragmentation is the problem for long standing processes with frequent allocations/deallocations of various sizes.

http://stackoverflow.com/questions/3770457/what-is-memory-fr...

IIRC Azul's pauseless GC uses kernel-level memory barriers to catch accesses to recently moved objects.

Azul still have pauses (10-100ms).

> Azul Systems has built a custom system (CPU, chip, board, and OS) specifically to run garbage collected virtual machines. The custom CPU includes a read barrier instruction. The read barrier enables a highly concurrent, parallel and compacting GC algorithm. The Pauseless GC algorithm is simple, efficient (low mutator overhead), and has no Stop-The-World pauses.

Azul is very similar to Shenandoah we are talking about. Updating of a wrong pointer still needs time, which can be unlucky when write trap is triggered in the middle of moving of large object.

It's the "stop-the-world" pauses that are problematic and that we've been talking about. Single-thread pauses are a fact of life, completely unavoidable with any dynamic memory allocation system even without GC (e.g. when malloc searches the free-lists).

Shenandoah has pause times ~10ms and is recommended for heaps 20GB+ for vs sub millisec in Go. NTPSec does not seem to have such high end memory requirements. Also now Java based app will need JVM availability on all platforms where NTPSec runs.

I think sometimes it makes sense not to shove Java where it is not applicable.

http://www.slideshare.net/RedHatDevelopers/shenandoah-gc-jav...

Yes, but that's because it's only really large heaps where pause times become problematic. On small heaps even dumb algorithms can give tiny pauses (a few msec or less) on modern hardware.

It's true you'd need a JVM on the platforms where ntpsec runs but that's also true of having a port of the Go runtime.

can go (or rust) compile for targets that don't have a jvm?

As far as Go goes, I believe neither the Plan 9 OS nor the MIPS CPU architecture have a production-ready JVM. Go supports both.