News for hackers. Stuff that matters.
Re:Modern garbage collection by twic on Tuesday December 20, 2016 @07:51AM (#13219354)
An interesting shift in the wind for modern garbage collection comes from microservices.

The work on JVM garbage collectors over the last ten years has increasingly been about supporting bigger and bigger heaps efficiently, as physical machines have got bigger, and programs' (or at least application servers') heaps have grown to fill them. Even Shenandoah, a current prototype collector, is designed around large heaps.

But with microservices, we use our cavernous machines to house many small heaps instead. That leaves us with rather different garbage collection challenges.

So, it's possible that the Go developers will get to ignore decades of progress, as they love to do, and still be competitive, because that progress was towards a different future to the one we arrived at.

Re:Modern garbage collection by jblow on Tuesday December 20, 2016 @01:35PM (#13222566)
All the Hacker News kids are crazy about microservices right now.

In three years, people will have realized it wasn't such a good idea, and moved on to the next "this will solve everything" fad.

Once you have seen enough of these fads go by, it's pretty obvious.

I'd be careful about extrapolating future directions of computer science from something that seems cool this year.

Re:Modern garbage collection by bsder on Tuesday December 20, 2016 @02:17PM (#13222953)
> All the Hacker News kids are crazy about microservices right now.

> In three years, people will have realized it wasn't such a good idea, and moved on to the next "this will solve everything" fad.

Microservices are simply the youngsters rediscovering that encapsulation is a damn good idea.

As I explain to people, microservices do NOT make your life easier. If you can exist in a monolith, that's way easier to implement.

However, once things start getting complicated, microservices make certain things possible.

Re:Modern garbage collection by jeffdavis on Tuesday December 20, 2016 @02:42PM (#13223236)
Microservices go beyond encapsulation in important ways, and regress in other ways.

Build/deploy/release cycles become independent with microservices, which can be great. Also you can write microservices in different languages.

But the APIs can become less rich. Try passing callbacks between microservices written in different languages. And it can be more of a pain to change APIs. And if there's any state to worry about, you need to manage that and maybe use a stateful protocol (which are harder to get right).

Re:Modern garbage collection by twic on Tuesday December 20, 2016 @07:03PM (#13225562)
> Build/deploy/release cycles become independent with microservices, which can be great.

I accept that coarse-grained services can have independent lifecycles, but not microservices. I'm working on a microservice-based project, and two things we've learned are that there are almost no features we can implement in just one microservice, and that if we don't test microservices together, we get bugs. So, in practice, their lifecycles are tightly coupled.

Some people ding this for being a 'distributed monolith' rather than some kind of 'true microservices', but i don't see how we could factor this application into micro-sized bits that would not be tightly coupled.

> Also you can write microservices in different languages.

I worked somewhere that tried that for a while (although small services, rather than microservices). It turns out that it makes it really hard to share people, libraries, and learning between teams, and that the task-specific advantages of particular languages mostly aren't great enough to make that worthwhile. We managed okay with some bits in Java and some in Scala, but adding Clojure and Groovy was too much. I think there are some bits in R now, but they're owned by a completely separate part of the organisation, so Conway's law allows it.

Re:Modern garbage collection by braveo on Tuesday December 20, 2016 @11:18PM (#13226755)
> It turns out that it makes it really hard to share people, libraries, and learning between teams

The idea is that you don't do that. It's a matter of duplicating functionality across teams so that each team isn't beholden to another team. pushing that library doesn't suddenly break things, for example.

Whether you agree with it or not, when you choose microservices, you choose to avoid sharing libraries like that. In the small it's probably ok (across microservices in 1 team, depending on circumstances), but in the large the gain isn't worth the added maintenance hassle.

Re:Modern garbage collection by twic on Wednesday December 21, 2016 @08:40AM (#13228959)
Libraries, sure. But if you've chosen an approach where you can't move people and learning between teams, you've chosen failure.
Re:Modern garbage collection by braveo on Wednesday December 21, 2016 @10:15AM (#13229712)
I don't even know what that means.
Re:Modern garbage collection by acveilleux on Tuesday December 20, 2016 @03:34PM (#13223652)
The independence of microservice is not always achieved. You often end up with a mess of interrelated services depending on fairly narrow version ranges to retain compatibility. The more homogeneous your production environment is, the less that's a problem. The worst case is when you have a dozen (or hundreds...) of production clusters that are all independent and not necessarily in-synch as far as deployed version goes.
Re:Modern garbage collection by jeffdavis on Tuesday December 20, 2016 @03:55PM (#13223849)
Absolutely.

Developers often gloss over how truly difficult it is to make good API choices. And when the wrong choices are made, it goes wrong quickly.

Microservices probably amplify this effect, but it's hard to say.

Re:Modern garbage collection by neeleshs on Tuesday December 20, 2016 @02:25PM (#13223041)
And if its a business application, and the company survives long enough, it will invariably get complicated.
Re:Modern garbage collection by SixSigma on Tuesday December 20, 2016 @04:03PM (#13223945)
Do they still use FastCGI ?

I doubt it

You want to make your way in the CS field? Simple. Calculate rough time of amnesia (hell, 10 years is plenty, probably 10 months is plenty), go to the dusty archives, dig out something fun, and go for it. It’s worked for many people, and it can work for you.

        — Ron Minnich 

If you don't know Ron - http://www.socallinuxexpo.org/scale8x/blog/interview-ron-min...

People today could do well reading

Computer Architecture and Parallel Processing Paperback – International Edition, January 1, 1986

by Kai Hwang (Author)

https://www.amazon.com/Computer-Architecture-Parallel-Proces...

Re:Modern garbage collection by user5994461 on Tuesday December 20, 2016 @06:14PM (#13225247)
Services are a good idea. SOA (serviced oriented architecture) has been out for more than 20 years and it's not going away.

Micro-services is just a new hype buzzword. Classic case of "smaller is cuter" http://www.smbc-comics.com/?id=2932

Re:Modern garbage collection by eva1984 on Tuesday December 20, 2016 @03:43PM (#13223728)
Who said microservices don't need large heaps? The prefix 'micro' means the share of responsibility and boundary of functionality, not any hints at memory size.

Of cause JVM needs to focus on large heaps. Now it still stucks somewhere at 32GB for performance reason. And for heavy-lifting application like HBase/ElasticSearch, that is currently OK, but still makes it quite inconvenient to utilize larger memory(you might need deploy multiple instances on the same box, etc).

You can't invent a new buzzword and tell other people they are solving the wrong problem.

Re:Modern garbage collection by user5994461 on Tuesday December 20, 2016 @06:19PM (#13225297)
> Now it still stucks somewhere at 32GB for performance reason

Short version: No it's not.

Long version: Heap is under 30 GB or over 40 GB. Never between. There is a JVM optimization that allows it to use 4 byte pointers up to 30GB of heap.

Re:Modern garbage collection by eva1984 on Tuesday December 20, 2016 @06:40PM (#13225424)
What is the reasoning for this? I think JVM still has problem handling real large heaps, the GC stop will be unbearable..Does thing change recently?
Re:Modern garbage collection by user5994461 on Tuesday December 20, 2016 @08:18PM (#13225998)
There is no problem to handle large heaps.

There is simply a gap: A 30 GB heap (with 4 bytes pointers) may be similar in capacity to a 40 GB heap (with 8 bytes pointers).

Guides and documentation (elastisearch) are simplifying this as "java can't do more than 30 GB heap", which is factually incorrect. You can do a 60GB if you want.

Re:Modern garbage collection by bad_user on Wednesday December 21, 2016 @07:43AM (#13228625)
> There is no problem to handle large heaps.

That's not true, dealing with large heaps is problematic in any garbage collected language because due to fragmentation you end up in stop-the-world full GC phases. Which for a big heap (i.e. bigger than 4 GB!), you can end up with latencies measured in seconds or even minutes.

The article mentions the CMS GC for Java, which has this problem. The G1 GC, introduced in Java 7 I think, is much better, however from what I've read it too isn't immune to full GCs. Of course for Java you also have the Azul "pauseless GC", which is supposed to never do this, but it is a commercial solution. And the article also mentioned another GC currently being contributed by Red Hat, which is good to hear.

Re:Modern garbage collection by user5994461 on Wednesday December 21, 2016 @12:18PM (#13230785)
Define large? I never had problem by increasing a heap from 10 to 20 GB.

Sure, if you go from 300MB to 30GB, the GC might behave differently and maybe some applications can't take it.

Re:Modern garbage collection by baus on Tuesday December 20, 2016 @12:40PM (#13222009)
I don't agree with the snarky tone of this comment. I'm not a Go programmer, but I've seen many concepts touted as "progress" over the past 20 years which have been dead ends.

I think it is reasonable for designers to curate advances to address the problems they are trying to solve.

Re:Modern garbage collection by refulgentis on Tuesday December 20, 2016 @02:22PM (#13223004)
The comment doesn't come off as snarky, but instead a milder version of the content of the article.

Namely: I don't agree with Go marketing their GC as if it magically provides no trade offs. Since reading the original design doc for it, I've assumed that Google was simply lazy/didn't care about Android, because the Android GC didn't provide this magic.

Re:Modern garbage collection by Merovius on Tuesday December 20, 2016 @02:38PM (#13223200)
> I don't agree with Go marketing their GC as if it magically provides no trade offs.

I don't believe this to be true. They always point out painstakingly, that the advances made on pause times come with less throughput, for example.

What the go people are doing (and with which I tend to agree and I believe it's hard to argue against the success they're having), is to say that they believe it's possible to build a GC which is good enough for most modern use cases and doesn't come with a lot of tuneables.

(However, even that comes with tradeoffs; it limits the use cases that go can be used for. For large enterprises, being able to minutely tune a GC might be an important thing)

Re:Modern garbage collection by refulgentis on Tuesday December 20, 2016 @03:05PM (#13223449)
Apologies for the brusque contrarianism, but this simply isn't true. I spent hours pouring over all the material they published when starting on this journey, because I was astounded that a team with a reputation for purposely avoiding groundbreaking work in favor of productivity today, had done something truly groundbreaking.

It wasn't remotely clear that the advancement here was "We don't provide settings, because we're guesstimating your workload matches the standard Go use case!", and all the histrionics further masked this truth.

At this point I'm just repeating the article, though. :)

Re:Modern garbage collection by stcredzero on Tuesday December 20, 2016 @03:07PM (#13223462)
So, it's possible that the Go developers will get to ignore decades of progress, as they love to do, and still be competitive, because that progress was towards a different future to the one we arrived at.

Are other fields as bad as programming? Do engineering people get to do the equivalent of having a paper published or getting to present at a conference because they implemented something in language A which was already done three years ago in language B? Do other fields have people grousing about how no one uses X, even though it is clearly "better," where "better" is basically an unsubstantiated opinion?

Some embarrassingly large number of "best practices" in our field are pretty much based on a feeling it "seems right" combined with a "well, it worked a couple times before" level of empiricism. Vital decisions about tooling, such as choice of programming language, are made by some combination of: following the herd, following one's own prejudices, or being enamored of a nifty idea.

One thing a company at the scale of Google could probably afford to do, would be to collect real data about the effects of programming practices.

Re:Modern garbage collection by flogic on Tuesday December 20, 2016 @04:12PM (#13224048)
Go isn't a research project. It's more a manifesto on programming philosophy born of "old man rage" in language form. This works out mostly because the old men in question happen to include Rob Pike and Ken Thompson. However, that results in a very specific feel and set of tradeoffs in Go.
Re:Modern garbage collection by stcredzero on Tuesday December 20, 2016 @04:36PM (#13224281)
Go isn't a research project.

Never said it was.

It's more a manifesto on programming philosophy born of "old man rage" in language form.

I suspect this has something to do with why I like it so much!

Re:Modern garbage collection by baus on Tuesday December 20, 2016 @04:25PM (#13224176)
You might not agree with the design decisions, but to say one of the most successful contemporary systems programming languages was born out "old man rage" is insulting
Re:Modern garbage collection by vvanders on Tuesday December 20, 2016 @05:28PM (#13224820)
Small nit but many of us wouldn't consider Go a systems programming language. I'd say "service programming langauge" is more appropriate. It's missing many things that would make it appropriate for interacting with hardware and limited resources.
Re:Modern garbage collection by stcredzero on Tuesday December 20, 2016 @04:37PM (#13224293)
to say one of the most successful contemporary systems programming languages was born out "old man rage" is insulting

Should I be insulted by your finding it insulting? My thought on reading that phrase was, "Right on!"

Re:Modern garbage collection by baus on Tuesday December 20, 2016 @05:16PM (#13224712)
With all the discussions of ageism and sexism in tech, classifing a significant contribution to the industry as the result of "old man rage" is inappropriate IMHO
Re:Modern garbage collection by flogic on Tuesday December 20, 2016 @05:22PM (#13224774)
I can see how it would imply that negative connotation.
Re:Modern garbage collection by stcredzero on Tuesday December 20, 2016 @06:04PM (#13225155)
But being classifiable as an "old man" myself, I rather find it more annoying that it should be taken as a negative. Maybe "old man rage" should be taken as an indicator that the lessons of experience have been neglected? Instead, in our culture, it is taken to mean something foolish. I think this says something about our culture.
Re:Modern garbage collection by flogic on Wednesday December 21, 2016 @01:12AM (#13227139)
To be fair, I meant it in both contexts at the same time. A more polite term might be "opinionatedly retro". Go's conservative bend is both a positive and negative.
Re:Modern garbage collection by stcredzero on Wednesday December 21, 2016 @02:36PM (#13231952)
Somewhat like the reviews my lovers give me.
Re:Modern garbage collection by blilbs on Wednesday December 21, 2016 @01:55PM (#13231615)
What about in your un-honest opinion?
Re:Modern garbage collection by eptcyka on Tuesday December 20, 2016 @06:56PM (#13225520)
I would rather people called it an application programming language. It relies heavily on it's runtime, and I've yet to see a piece of go code running on bare metal that isn't a hack or a proof of concept. However, I've seen multiple applications that rely on the OS to have nice abstractions over low-level inter-computer IPC (ethernet, tcp/ip).
Re:Modern garbage collection by pjmlp on Tuesday December 20, 2016 @05:17PM (#13224717)
I keep wondering how much success it would have had if the design team was still at AT&T.
Re:Modern garbage collection by specialist on Tuesday December 20, 2016 @06:57PM (#13225527)
Are other fields as bad as programming?

K-12 education for sure. Completely fad (procurement) driven, utterly disconnected from reality (best available science) with egos and bent morality added for giggles.

Mind the feedback loops and incentives.

And don't trust the opinion of anyone who hasn't maintained code they (personally) wrote and deployed +5 years ago.

Re:Modern garbage collection by jeffdavis on Tuesday December 20, 2016 @02:10PM (#13222900)
Partially true, and Joe Armstrong (creator of erlang) makes similar arguments. But that doesn't work well for everything.

Erlang is more clear about their assumptions and goals. They don't claim to have an amazing new GC, they just say they have a simple generational GC, and that's all they need because they assuming many small heaps.

Re:Modern garbage collection by qaq on Thursday December 22, 2016 @10:57PM (#13242725)
it's also not an STW GC so the pauses affect only small number of processes.
Re:Modern garbage collection by osi on Tuesday December 20, 2016 @09:18AM (#13219950)
Where is the data kept?

The very large heaps are partially driven by distributed in-memory storage systems.

Re:Modern garbage collection by twic on Tuesday December 20, 2016 @09:36AM (#13220097)
Hmm, well if the processing is microservices, presumably the data is in a NoSQL database.

More seriously, though, isn't most large in-memory data storage done off the garbage collected heap? I have to confess i'm actually not sure what anyone is doing with >20 GB of normal objects.

Re:Modern garbage collection by sheeshkebab on Tuesday December 20, 2016 @02:12PM (#13222917)
Search (elastic/solr), Cassandra, presto, and most other java backend storage and data processing engines need 20+Gb (at least) to be mildly useful. It's not uncommong to go with 64gb+ heaps there too.

Microservices/API won't help with lowering these.

Re:Modern garbage collection by osi on Tuesday December 20, 2016 @03:51PM (#13223814)
yes, exactly this. and by having scalable GC algos so you don't need to do "off heap" tricks in java, things are much easier.
Re:Modern garbage collection by dkersten on Tuesday December 20, 2016 @11:33AM (#13221320)
I'm sure people do use huge heaps, but most web applications I know of (microservice or not) store the bulk of their data in a (separate) database. So, unless your application is a database, which might want a huge heap so it can cache a lot in-memory, I don't think huge heaps are really necessary. IMHO, at least.
Re:Modern garbage collection by dom0 on Tuesday December 20, 2016 @12:38PM (#13221987)
Since it's not really possible to do reliable giant user space caches in the POSIX model of things you'd usually delegate to the page cache.
Re:Modern garbage collection by agumonkey on Tuesday December 20, 2016 @08:53AM (#13219726)
I'd love to know about other GC schemes. Between no GC and whole VM GC. By type GC, localized sub tree GC.

edit: thank you all for your answers, much appreciatedly

Re:Modern garbage collection by mike_hearn on Tuesday December 20, 2016 @11:45AM (#13221452)
Check out Singularity. It was a research effort from Microsoft Research to build a whole OS in a modified .NET

Singularity was a single address space OS, thus there were no "processes" in a traditional sense. They still had a notion of isolated spaces though partly to allow different heaps to be GCd independently, and to use different GC algorithms (with different barriers) exactly due to the lack of a one-size-fits-all approach. Messaging between these process-like-things was done by moving objects onto a "transfer heap" and back again, which boiled down to just a simple pointer write. There were no syscalls.

A very nice architecture, from a theoretical perspective. However it's not clear to me that for standard desktop-sized machines you really need segmented GC spaces these days. With an incremental collector you can probably get pause times in the milliseconds range for a 32GB heap and I guess 32GB is as much RAM as you need on a Pro-level machine. For servers of course you go bigger.

One interesting GC algorithm I came across did thread-local collections. Objects were tracked with write barriers to discover when they became 'globally reachable' and moved to a global heap. Threads could stop independently whilst others continued running to collect their thread local heaps. That opens up the possibility in desktop/mobile apps of having a main thread that's doing an animation of some kind in a low-garbage way, whilst the background thread does work as quickly as possible, with the background thread collecting independently whilst the animation thread still runs.

Re:Modern garbage collection by micmus on Tuesday December 20, 2016 @09:20AM (#13219966)
Erlang has a per-process GC - each Erlang process has it's own heap and it's own garbage collector. There's no VM-wide GC. This usually means the GC pauses apply only to a small part of your system, while the other parts are running completely normally. There's a good article that describes it in depth: https://www.erlang-solutions.com/blog/erlang-19-0-garbage-co...
Re:Modern garbage collection by noss on Tuesday December 20, 2016 @11:30AM (#13221281)
Also if you have a lot of data you use an ets table which is more or less an in-process but off-gc-heap key-value database.
Re:Modern garbage collection by staticassertion on Tuesday December 20, 2016 @12:08PM (#13221686)
Thanks, I've been wondering how Erlang does GC.
Re:Modern garbage collection by AstralStorm on Tuesday December 20, 2016 @12:24PM (#13221829)
Of course, the cost is extra heap fragmentation.
Re:Modern garbage collection by dfox on Tuesday December 20, 2016 @12:36PM (#13221974)
In Erlang case, that is not so much of a problem as most allocated objects have essentially same size. On the other hand this causes another problem that GC's and allocators in Erlang VMs tend to have enormous overhead because most most GC'd objects are very small with sizes comparable to per-object GC overhead.
Re:Modern garbage collection by agumonkey on Tuesday December 20, 2016 @05:54PM (#13225085)
I joked once that in lisp, a vast majority of memory allocation are basically malloc(2). I wondered if it made fragmentation irrelevant.
Re:Modern garbage collection by mpweiher on Tuesday December 20, 2016 @11:57AM (#13221582)
You might be interested in:

"A Unified Theory of Garbage Collection"

https://pdfs.semanticscholar.org/91dc/a25f8cb407fc68218f7d5a...

Basic idea is that tracing and reference counting are duals of each other. Reference counting finds dead objects, tracing finds live objects. Optimized variants of each of these tend to approach the other side.

Re:Modern garbage collection by twic on Tuesday December 20, 2016 @10:10AM (#13220418)
I used to idly browse papers from programming language conferences, and there was often interesting GC stuff. PLDI is probably your bet bet - here's 2016:

http://dl.acm.org/citation.cfm?id=2908080&picked=prox

When there were two GC papers:

Idle time garbage collection scheduling - "We implemented idle time garbage collection scheduling in V8, an open-source, production JavaScript virtual machine running within Chrome. We present performance results on various benchmarks running popular webpages and show that idle time garbage collection scheduling can significantly improve latency and memory consumption."

Assessing the limits of program-specific garbage collection performance - "We demonstrate that there is considerable promise to reduce garbage collection costs by developing program-specific collection policies."

Or going back to 2012:

And then there were none: a stall-free real-time garbage collector for reconfigurable hardware - "We present the first implementation of a complete garbage collector in hardware (as opposed to previous "hardware-assist" techniques), using an FPGA and its on-chip memory. Using a completely concurrent snapshot algorithm, it provides single-cycle access to the heap, and never stalls the mutator for even a single cycle, achieving a deterministic mutator utilization (MMU) of 100%."

That's by David Bacon, one of the runtime implementation greats, who has been after low-pause GC for years. A golden oldie from him is:

A Pure Reference Counting Garbage Collector - "When processor or memory resources are limited, the Recycler usually runs at about 90% of the speed of the mark-and-sweep collector. However, with an extra processor to run collection and with a moderate amount of memory headroom, the Recycler is able to operate without ever blocking the mutators and achieves a maximum measured mutator delay of only 2.6 milliseconds for our benchmarks. End-to-end execution time is usually within 5%."

As for by-type GC, i remember reading a paper years (decades?) ago, perhaps from Microsoft, where they'd tried putting every class in its own region, the idea being that when you wanted to collect some particular region, you could exploit your knowledge of types and their fields to only scan regions which could possibly have pointers to it (eg to collect BlogPost, you have to scan User). Clever, but didn't work well in practice, because so much garbage is lists or strings that can be pointed to from a lot of places.

Re:Modern garbage collection by scott_s on Tuesday December 20, 2016 @10:28AM (#13220608)
ISMM, http://dl.acm.org/event.cfm?id=RE149, is also likely to have papers on garbage collection. The table of contents from 2016: http://dl.acm.org/citation.cfm?id=2926697
Re:Modern garbage collection by twic on Tuesday December 20, 2016 @07:36PM (#13225759)
Nice! There's a paper in there called 'Block-free concurrent GC: stack scanning and copying' which opens by claiming that "on-the-fly Garbage Collectors (GCs) are the state-of-the-art concurrent GC algorithms today". I'd never heard of on-the-fly GC, so i've learned something!

However, on googling, i find that Edsger Dijkstra wrote about it in 1975:

https://www.cs.utexas.edu/users/EWD/transcriptions/EWD05xx/E...

Hey, maybe that's old enough that Go can use it?

Re:Modern garbage collection by staticassertion on Tuesday December 20, 2016 @12:11PM (#13221713)
I believe MS does in fact use a typed heap in IE, but for security reasons (some sort of UAF mitigation iirc).
Re:Modern garbage collection by osi on Tuesday December 20, 2016 @09:19AM (#13219959)
Look at what Erlang's BEAM VM does. It has per-"process" heaps iirc.
Re:Modern garbage collection by me2i81 on Tuesday December 20, 2016 @02:41PM (#13223231)
Take a look at The Garbage Collection Handbook by Jones et. al. ISBN-13 is 978-1420082791.
Re:Modern garbage collection by AnimalMuppet on Tuesday December 20, 2016 @07:11PM (#13225609)
> So, it's possible that the Go developers will get to ignore decades of progress, as they love to do, and still be competitive, because that progress was towards a different future to the one we arrived at.

Progress in the wrong direction is not progress.

Ignoring "progress" that was in the wrong direction is progress.

Re:Modern garbage collection by geodel on Tuesday December 20, 2016 @12:00PM (#13221600)
A lot of 'decades of progress' in software is similar to progress in College tuitions, Health Insurance premiums and Cable TV bills. Yes advanced solutions have been developed and now price has to extracted by any means irrespective of user needs or agreement.