News for hackers. Stuff that matters.
Re:Anatomy of a Reducer by lmkg on Tuesday May 15, 2012 @04:30PM (#3978640)
I'm not an expert, but here's my take on it: The point of this library is to decouple things that are usually coupled but don't need to be, and provide a uniform interface for combining them back together. The hardest part of this is looking at things from a slightly different perspective and getting used to the new division of responsibilities, but when you get used to it the overall picture is actually simpler.

The idea is that you have four types of things: you have items, you have collections that hold items, you have ways of operating on those collections, and you have operations you want to perform on collections. The library deals with separating the latter two, since separating elements from a container is basically a solved problem.

A "reducer" is a view into a collection, which is responsible for knowing how to apply collection operations (map, filter, reduce) to its elements.

The collection operations (usually map, filter, and reduce) are separate, called mappings and filterings in this article. These are real entities, which are composable. I think of them similar to defining a database query, but not executing it.

Because reducers and operations are decoupled, you can always take an arbitrary "query" and throw it at at arbitrary reducer, and expect the collection to be able to handle the query, and always return the same result regardless of collection type.

Fork/join is an example use-case: you can create a collection type which is just a thin wrapper over fork/join[1]. You can then wrap this collection type as a view into your data, and run collection queries against it. This allows you to use fork/join as a drop-in replacement for naive data structures. It also keeps tinkering with fork/join separate from both the underlying data structure, and the work being processed.

The other primary use case seems to allow defining, storing, and composing queries without running them. This ties in heavily to lazy evaluation. The composability is a key feature because it allows you to pass around partial queries, rather than partial results, and saves greatly on time and space by not creating intermediate data structures (cf. stream fusion in Haskell).

[1] Fork/join would have to be a "foldable" rather than "reducable" collection, which as far as I can tell just means it's a little more lax with how you combine results back from a reduce. These relaxations allow for parallelization.

Re:Anatomy of a Reducer by pron on Tuesday May 15, 2012 @05:28PM (#3978959)
Fork-join is not a collection (or any data structure for that matter). It is a parallel computation process meant to make good use of all available processors. You can use fork-join to process a collection like an array or a tree in parallel, and you can use it to factor a single integer in parallel.