I agree the code you have there is very readable, but it's not really an example of what that sentence you quoted is referencing... However I didn't spell out exactly what I meant, so please allow me to clarify.
For me, roughly 5 calls in a chain is where things begin to become harder to read, which is the length of the example I used.
For the meaning of "multiple", I intended that to mean if there are nested chains or if the type being operated on changes, that can slow down the rate of reading for me.
Functional programming constructs can be very elegant, but it's possible to go overboard :)
This isn't just about readability. Chaining or FP is structurally more sound. It is the more proper way to code from a architectural and structural pattern perspective.
given an array of numbers
1. I want to add 5 to all numbers
2. I want to convert to string
3. I want to concat hello
4. I want to create a reduced comma seperated string
5. I want to capitalize all letters in the string.
This is what a for loop would look like: // assume x is the array
acc = ""
for(var i = 0, i < x.length; x++) {
value = x[i] + 5
value += 5
stringValue = str(value).concat(hello)
acc += stringValue + ","
}
for (var i = 0, i < acc.length; i++) {
acc[i] = capitalLetter(acc[i])
}
FP: addFive(x) = [i + 5 for i in x]
toString(x) = [str(i) for i in x]
concatHello = [i + "hello" for i in x]
reduceStrings(x) = reduce((i, acc) = acc + "," + i, x)
capitalize(x) = ([capitalLetter(i) for i in x]).toString()
You have 5 steps. With FP all 5 steps are reuseable. With Procedural it is not.Mind you that I know you're thinking about chaining. Chaining is eqivalent to inlining multiple operations together. So for example in that case
x.map(...).map(...).map(...).reduce(...).map(...)
//can be made into
addFive(x) = x.map(...)
toString(x)= x.map(...)
...
By nature functional is modular so such syntax can easily be extracted into modules with each module given a name. The procedural code cannot do this. It is structurally unsound and tightly coupled.It's not about going overboard here. The FP simply needs to be formatted to be readable, but it is the MORE proper way to code to make your code modular general and decoupled.
if you have two functions, they're not coupled. you change one, the other stays as-is
if you refactor it so that they both call a third function, they're now coupled. you can't change the part they have in common without either changing both, or uncoupling them by duplicating the code
(you often want that coupling, if it lines up with the semantics)
The "anti-functional Tourette's" comment was partly a response to how completely random and unjustified it seemed in that part of the article, and also that this feels like a very common gut reaction to functional programming from people who aren't really willing to give it a try. I'm not only arguing directly against you here, but that attitude at large.
Your funcA vs. funcB example doesn't strike me as "functional" at all. No functions are even passed as arguments. That "fluent" style of long chains has been around in OO languages for a while, independent of functional programming (e.g. see d3.js*, which is definitely not the oldest). Sure, breaking long "fluent" chains up with intermediate variables can sometimes help readability. I just don't really get how any of this is the fault of functional programming.
I think part of the reason funcB seems so much more readable is that neither function's name explains what it's trying to do, so you go from 0 useful names to 3. If the function was called "getNamesOfVisibleNeighbors" it'd already close the readability gap a lot. Of course if it were called that, it'd be more clear that it might be just trying to do too much at once.
I view the "fluent" style as essentially embedding a DSL inside the host language. How readable it is depends a lot on how clear the DSL itself is. Your examples benefit from additional explanation partly because the DSL just seems rather inscrutable and idiosyncratic. Is it really clear what ".data()" is supposed to do? Sure, you can learn it, but you're learning an idiosyncrasy of that one library, not an agreed-upon language. And why do we need ".nodes()" after ".connected()"? What else can be connected to a node in a graph other than other nodes? Why do you need to repeat the word "node" in a string inside "graph.nodes()"? Why does a function with the plural "nodes" get assigned to a singular variable? As an example of how confusing this DSL is, you've claimed to find "visibleNames", but it looks to me like you've actually found the names of visible neighborNodes. It's not the names that are not(.hidden), it's the nodes, right? Consider this:
function getVisibleNeighborNames(graph) {
return graph
.nodeByName(name)
.connectedNodes()
.filter(node => !node.isHidden)
.map(node => node.name)
}
Note how much clearer ".filter(node => !node.isHidden)" is than ".not('.hidden')", and ".map(node => node.name)" versus ".data('name')". It's much harder to get confused about whether it's the node or the name that's hidden, etc.Getting the DSL right is really hard, which only increases the benefit of using things like "map" and "filter" which everyone immediately understands, and which have no extrinsic complexity at all.
You could argue that it's somehow "invalid" to change the DSL, but my point is that if you're using the wrong tool for the job to begin with, then any further discussion of readability is in some sense moot. If you're doing a lot of logic on graphs, you should be dealing with a graph representation, not CSS classes and HTML attributes. Then the long chains are not an issue at all, because they read like a DSL in the actual domain you're working in.
*Sidenote: I hate d3's standard style, for some of the same reasons you mention, but mainly because "fluent" chains should never be mutating their operand.
I see this quite often with builders, actually, and I don't mind it so much there.
FooBuilder()
.setBar(bar)
.setBaz(baz)
.setQux(qux)
.build()in c or go if you dont like variadics just pass in a params struct.
its just a fad.
Which, again, is fine if your language supports named args for your params struct.
C++ didn't have this until C++20, despite C having it for decades prior.
Java still doesn't have this.
If your language doesn't have named args or designated initializers or whatever it calls them, then what, perform your function calls with argument comments of the form f(/*foo=*/1, /*bar=*/2, /*baz=*/true)? That's error-prone even in the best-case scenario.
It might be a language thing as well. In Python often people take list-comprehensions too far and it becomes an undecipherable mess of nested iterators, casts and lists.
There are always exceptions :)
However the GPs example of map filter distinct I find lovely.
Mind you, I learned to program is a functional language so I probably have a different perspective to most.
o_node := graph.GetNodeByName(name)
var ret []string
for _, node := range o_node.connectedNodes() {
if !node.isHidden {
ret = append(ret, node.name)
}
}
return retYou've added a temp variable for the result, manual appending to that temp variable (which introduces a performance regression from having to periodically grow the array), loop variables, unused variables, multiple layers of nesting, and conditional logic. And the logic itself is no longer conceptually linear.
You don't really think that functional languages aren't appending things, using temp vars, and using conditional logic behind the scenes, do you? What do you think ".filter(node => !node.isHidden)" does? It's nothing but a for loop and a conditional by another name and wrapped in an awkward, unwieldy package.
>which introduces a performance regression from having to periodically grow the array
This is simply ridiculous, do you just believe that the magic of Lisp/FP allows it to pluck the target variables out of the sky in perfectly-sized packages with zero allocation or overhead?
But the whole point of higher-level languages is that you don't have to think about what's going on behind the scenes, and can focus on expressing intent while worrying less about implementation. Just because a HLL is eventually compiled into assembler, and so the assembler expresses everything the HLL did, doesn't mean the HLL and assembler are equally readable.
(And I think that your parent's point is that "awkward, unwieldy package" is a judgment call, rather than an objective evaluation, based, probably, on familiarity and experience—it certainly doesn't look awkward or unwiely to me, though I disagree with some of the other aesthetic judgments made by your parent.)
You "can't see and don't have access to" `if`, `range`, or `append` but somehow you don't find this a problem at all. I wonder why not?
> You don't really think that functional languages aren't appending things, using temp vars, and using conditional logic behind the scenes, do you?
By this metric all languages that compile down to machine instructions are equivalent. After all, it winds up in the same registers and a bunch of CMP, MOV, JMP, and so on.
`.distinct()` could sort the result and look for consecutive entries, it could build up a set internally, it could use a hashmap, or any one of a million other approaches. It can even probe the size of the array to pick the performance-optimal approach. I don't have to care.
> [".filter(node => !node.isHidden)" is] nothing but a for loop and a conditional by another name and wrapped in an awkward, unwieldy package.
This is honestly an absurd take. I truly have no other words for it. map, filter, and friends are quite literally some of the clearest and most ergonomic abstractions ever devised.
Well, this is probably why functional programming doesn't see a lot of real use in production environments. Usually, you actually do have to care. Talk about noticing a performance regression because I was simply appending to an array. You have no idea what performance regressions are happening in ANY line of FP code, and on top of that, most FP languages are dead-set on "immutability" which simply means creating copies of objects wherever you possibly can... (instead of thinking about when it makes sense and how to be performant about it)
The usual map/filter/reduce is everywhere in production. Python, java, js, ruby, c#...
You could even argue that lack of generics hurt Go's popularity for a while precisely for that usecase.
Great, please make all the software even slower than it already is. I am overjoyed to have to purchase several new laptops a decade because they become e-waste purely due to the degradation of software performance. It is beyond ridiculous that to FP programmers daring to mutate variables or fine-tune a for loop is an exceptional scenario that you don't do "unless you have to" and which requires "knowing what you're doing". Do you know what you're doing? How can you be a software engineer and think that for loops are too difficult of a construct, and that you need something higher-level and more abstract to feel safe? It's insane. Utterly insane. Perhaps even the root of all evil, Code Golf manifested as religion.
It is more or less trivial to emit essentially optimal autovectorized and inlined machine code from functional iterators.
Rust does this, for example.
Also countless real-world production environments run on Python, Ruby, JS etc, all of which are significantly slower than a compiled FP program using filter & map.
> FP languages are dead-set on "immutability" which simply means creating copies of objects
Incorrect. The compiler can make it mutable for better performance, and that gives you the best of both worlds: immutability where a fallible human is involved, and mutability where it matters.
Well, we already know that no pure FP language can match the performance of a dirty normal imperative language, except for Common Lisp (which I am happy to hear an explanation for how it manages to be much faster than the rest, maybe it's due to the for loops?). And another comment here already mentioned how those "significantly slower" scripting languages have a healthy dose of FP constructs -- which are normally considered anti-patterns, for good reason. The only language that competes in speed in Rust, which just so happens to let you have fast FP abstractions so long as you manually manage every piece of memory and its lifetime, constantly negotiating with the compiler in the process, thereby giving up any of the convenience benefits you actually get from FP.
cepstrum: inverse spectrum
lifter: inverse filter
saphe: inverse phase
quefrency alanysis: inverse frequency analysis
gisnal orpcessing: inverse signal processing
False equivalence. You're saying that the statement "both for.. append and .map() executing the _same steps_ in the _same order_ are the same" is equivalent to saying that "two statements being composed of cmp,jmp, etc (in totally different ways) are the same" That is a dishonest argument.
> Distinct could sort and look for consecutive, it could use a hashmap
People love happy theories like this, but find me one major implementation that does this. For example, here is the distinct() implementation in the most abstraction-happy language that I know - C#
https://github.com/dotnet/runtime/blob/main/src%2Flibraries%...
It unconditionally uses a hashset regardless of input.
Edit: found an example which does different things depending on input
https://github.com/php/php-src/blob/master/ext/standard/arra...
This does the usual hashset based approach only if the array has strings. Otherwise, it gets the comparator and does a sort | uniq. So, you get a needless O(nlogn), without having a way to distinct() by say, an `id` field in your objects. Very ergonomic...
On the other hand...
seen := map[string]bool{}
uniq := []string{}
for _, s := range my_strings {
if !seen[s] {
uniq = append(uniq, s)
seen[s] = true;
}
}
Let us say you want to refactor and store a struct instead of just a string. The code would change to... seen_ids := map[string]bool{}
uniq := []MyObject{}
for _, o := range my_objects {
if !seen_ids[o.id] {
uniq = append(uniq, o)
seen_ids[o.id] = true;
}
}
Visually, it is basically the same, with clear visual messaging of what has changed. And as a bonus, it isn't incurring a random performance degradation.Edit 2: An SO question for how to do array_unique for objects in php. Some very ergonomic choices there... https://stackoverflow.com/questions/2426557/array-unique-for...
This follows what I like to call macro optimization. I don't care if a hash set is slightly slower for small inputs. Small inputs are negligibly fast anyway. The hash set solution just works for almost any dataset. Either it's too small to give a crap, midsized and hash set is best, or huge and hash set is still pretty much best.
That's why we default to the best time/space complexity. Once in a blue moon someone might have an obscure usecase where micro optimizing for small datasets makes sense somehow, for the vast majority of use cases this solution is best.
> Visually, it is basically the same, with clear visual messaging of what has changed.
In order to do this you had to make edits to all but a single line of logic. Literally only one line in your implementation didn't change.
Compare, with Ruby:
# strings
uniq = strings.uniq()
# structs, if they are considered equal based on the
# field in question (strings are just structs and they
# implement equality, so of course this is the same)
uniq = structs.uniq()
# structs, if you want to unique by some mechanism
# other than natural equality
let uniq = structs.uniq(&:id)
With Rust: # strings
let uniq = strings.unique();
# structs, if they are considered equal based on the
# field in question (strings are just structs and they
# implement equality, so of course this is the same)
let uniq = structs.unique();
# structs, if you want to unique by some mechanism
# other than natural equality
let uniq = structs.unique_by(|s| s.id);
You cannot tell me with a straight face that your version is clearer. You'll note that both languages have essentially the exact same code, which is to say: nearly none at all.My initial comment was a response to your comment
> Distinct could sort and look for consecutive, it could use a hashmap. It can even probe the size of the array to pick the performance-optimal approach. I don't have to care.
which says that you just use an abstraction and it transparently "does the right thing". The C# example showed how abstractions actually don't do that, and instead simply provide a lowest common denominator implementation. The rust examples you used also uses a hashmap unconditionally. The PHP example was showing how when abstractions attempt to do that it ends up even worse - when you move from strings to structs, you get a slowdown.
In practice, different strategies are always implemented as different functions (see python `moreitertools` `unique_justseen` and `unique_everseen`), at which point its no longer an abstraction (which by definition serves multiple purposes) and it just becomes a matter of whether or not the set of disparate helper functions is written by you, the standard library, or a third party. In rust, you would do `vec.sort()`, `vec.dedup()` for one strategy, and call `v.into_iter().unique().collect()` for the other strategy. That is not an "abstraction" [which achieves what you claimed they do].
This is a key difference between imperative programming and other paradigms.
> You don't really think that functional languages aren't appending things, using temp vars, and using conditional logic behind the scenes, do you?
A key concept in a FP approach is Referential Transparency[0]. Here, this concept is relevant in that however FP constructs do what they do "under the hood" is immaterial to collaborators. All that matters is if, for some function/method `f(x)`, it is given the same value for `x`, `f(x)` will produce the same result without observable side effects.
> What do you think ".filter(node => !node.isHidden)" does?
Depending on the language, apply a predicate to a value in a container, which could be a "traditional" collection (List, Set, etc.), an optional type (cardinality of 0 or 1), a future, an I/O operation, a ...
> It's nothing but a for loop and a conditional by another name and wrapped in an awkward, unwieldy package.
There is something to be said for the value of using appropriate abstractions. If not, then we would still be writing COBOL.
What? Golang append()s also periodically grow the slice.
> Conditional logic
it's just a single if, really, the same thing is there in your filter()
> Multiple layers of nesting
2... You're talking it up like it's a pyramid of hell. For what it's worth, I've seen way way more nesting in usual FP-style code, especially with formatting tools doing
func(
args
)
For longer elements of the function chain.If you already know the size of the result (there are no filtering operations), the functional approach can trivially allocate the resulting array to already have the correct capacity. This happens with zero user intervention.
IIRC the Rust optimizer basically emits more or less optimal machine code (including SIMD) for most forms of iteration.
If overallocating is indeed OK for your usecase, then you can do so yourself
uniq := make([]MyObject, 0, len(my_objects))Yes, you can always just write more and more and more code to fix these things. Or you could just… not write more code and still get optimal performance.
But the abstractions don't really adapt themselves to be performant in each usecase the way you imagine. I gave an example of c# distinct() above. Sure, they can. But do they? No. They only save anything at all for trivial usecases, where the imperative code is also obvious to parse.
I for example prefer a well chosen one-liner list comprehension in python over a loop with temporary variables and nested if statements most of the time. That is because usually people who use the list comprehension do not program it with side effects, so I know this block of code, once understood stands for itself.
The same is true for the builder style code. I just need to know what each step does and I know what comes out in the end. I even know that the object that was set up might become relevant later.
With the traditional imperative style that introduces intermediate variables I might infer that those are just temporary, but I can't be sure until I read on, keeping those variables in my head. Leaving me in the end with many more possible ways future code could play out. The intermediate variables have the benefit of clarifying steps, but you can have that with a builder pattern too if the interface is well-chosen (or if you add comments).
This is why in an imperative style variables that are never used should be marked (e.g. one convention is a underscore-prefix like _temporaryvalue — a language like Rust would even enforce this via compiler). But guess what: to a person unfamilar with that convention this increases mental complexity ("What is that weird name?"), while it factually should reduce it ("I don't have to keep that variable in the brain head as it won't matter in the future").
In the end many things boil down to familiarity. For example in electronics many people prefer to write a 4.7 kΩ as 4k7 instead, as it prevents you from accidentally overlooking the decimal point and making an accidental off-by-a-magnitude-error. This was particularly important in the golden age of the photocopier as you can imagine. However show that to a beginner and they will wonder what that is supposed to mean. Familiarity is subjective and every expert was once a beginner coming from a different world, where different things are familiar.
Something being familiar to a beginner (or someone who learned a particular way of doing X) is valuable, but it is not necessarily an objective measure of how well suited that representation is for a particular task.
Over the years I've come to firmly believe that readability is highly subjective. And familiarity is a key contributor to that, but not the only one. There are other factors that I've found highly correlate with various personality traits and other preferences. In other words, people shouldn't make claims that one pattern is objectively more readable than another. Ever.
I've reached the point where anyone who claims some style is "more readable" without adding "to me" I just start to tune out. There's very little objective truth to be had here.
What should one do? If on a team and you're the outlier, suck it up and conform. If you're on a team and someone else is the outlier? Try to convince them to suck it up and conform. If you're on a new team? Work empathetically with your teammates to understand what the happy medium style should be.
However I'd defend the notion that on the bad end of things you can have such a thing as (objectively?) hard to read code. With "hard to read" I do not mean "nobody can figure it out with time", what I mean is, that figuring it out takes 99% of programmers longer than the equivalent in another language or style. As you rightly point out, it is important to realize that this is a statistical realization and not an universal law of nature, so it really matters on which cohort of people you look at and recommendations stemming from such observations should be taken with a grain of salt.
Yet, underneath all of that isn't there such a thing as truly objectively hard to read style? Brainfuck for example is an objectively hard to read language – they put that even into the name of the language. Does that mean there is not a single person who can read it fluently? Probably not, but that doesn't invalidate the point. A double black diamond ski track is objectively harder to ski, exactly because there are less people that are able to ski it.
If you see programming as working with language and symbols to achieve some behavior, it is clear that there are patterns who match more with the known (familiar) of most people. If you ask non-programmers or non-mathematicians how to describe some action in any way they like they will probably use their daily language. That means code that looks somewhat similar to how a regular person would write it down is surely on the "very familiar"-end of things. Now I did argue that maximum familiarity to regular people is not in itself a desireable goal, we need to make a tradeoff between familiarity and suitability to express program structures and operations. The latter is not how regular people think at all, so using them as an absolute guide isn't a good idea. However thinking about how to write things that they are both expressive in terms of programming behavior and easy to reason about is a desireable goal. There just isn't a single right way of doing that and sometimes good enough is just that.
And yes, the maximal familiarity is a huge aspect. Many years ago I got into an argument with a colleague regarding FP patterns. He contended that they were objectively harder to reason about and cited the difficulty of new hires in ramping up. I was contending that this is untrue, but rather those new hires had existed in a world where they didn't often encounter FP patterns. For practical purposes the end result is the same, I'll admit. But to your point: if we accept that the missing ingredient is familiarity vs it being objectively bad it changes how one might approach the problem.
Maybe you haven't seen the sort of code I've seen.
Consider this specimen, for example:
// Call a function.
// Function call syntax isn't very visible (only a couple parens tacked onto an identifier);
// this is a major win for readability.
function callFunction(func, ...args) {
return func(...args);
}
// Perform math.
// Math symbols are arcane and confusing, so this makes it much more clear.
// I'm trying to program here, not transmute lead into gold!
function doMath(left, op, right) {
const OPS = {
"less than or equal to": (l, r) => l <= r,
"minus": (l, r) => l - r,
};
// TODO: add support for other operators.
return callFunction(OPS[op], left, right));
}
const MEMO_TABLE = {}
function fibonacci(n) {
if (callFunction(doMath, n, "less than or equal to", 1)) {
const result = n;
// Memoization makes this fibonacci implementation go brrrrrrrrr!
MEMO_TABLE[n] = result;
return result;
} else {
const result = callFunction(fibonacci, callFunction(doMath, n, "minus", 1)) + callFunction(fibonacci, callFunction(doMath, n, "minus", 2));
// Memoization makes this fibonacci implementation go brrrrrrrrr!
MEMO_TABLE[n] = result;
return result;
}
}
Yes, it was an intentional choice to never actually make use of MEMO_TABLE -- this (reproduced from memory) is a submission from an applicant many, many years ago (the actual code was much worse, it was somehow 3 to 4 times as many lines of code).Compared to:
function fibonacci(n) {
if (n <= 1) {
return n;
} else {
return fibonacci(n - 1) + fibonacci(n - 2);
}
}
> In other words, people shouldn't make claims that one pattern is objectively more readable than another. Ever.Taken literally, I can't argue with this. But if you amend that to "people shouldn't make claims that one pattern is objectively more readable than another to competent individuals" (which I suspect is intended to be implied by the original quote), then I'd have to disagree. The choice between functional vs procedural may mostly be subjective, but a convoluted pattern is objectively worse than a non-convoluted one (when the reader is competent).
Code that interleaves high and low level concerns, that has a lot of variables used all over the place, is deeply nested, etc. vs code that is modular, keeps the nesting shallow, splits up the functionality so that concerns are clearly separated and variables have the smallest scopes possible.
One of these styles is better than the other and it's not subjective in the least, so long as you're a human. We all have roughly the same limitations in terms of e.g. memory capacity so ways of programming that reduce the need to keep a lot of stuff in memory at a time will be objectively better.
> I've reached the point where anyone who claims some style is "more readable" without adding "to me" I just start to tune out. There's very little objective truth to be had here.
Adding qualifiers all over the place is just a defensive writing style, best fitting online forums like this one where people will nitpick everything apart. However, it's not reasonable to pay so much attention to someone's particular word choices.
Objective truth is very easy to find actually. The problem you're solving is objective and the desired outcome is too. It's just a matter of analysing the problem space to find the correct design. The feeling of subjectivity largely just comes from the high complexity of the problem space.
> If on a team and you're the outlier, suck it up and conform.
It's way more complicated than you make it sound. It's entirely possible the entire team is wrong.
A general strategy when joining a new team is to follow what the team does exactly and not attempt to make changes until you learn why the team does things the way they do (Chesterton's Fence). Once you understand, you can start suggesting improvements.
seeinglogic's article made me think of a 3rd option:
1. Sorta long functional chain where the type changes partway through 2. Use temp variables 3. (New option) Use comments
(Here's funcA from seeinglogic's article, but I added 3 comments)
function funcC(graph) {
return
// target node
graph.nodes(`node[name = ${name}]`)
// neighbor nodes
.connected()
.nodes()
// visible names
.not('.hidden')
.data('name');
}
Compare to funcB which uses temp variables: function funcB(graph) {
const targetNode = graph.nodes(`node[name = ${name}]`)
const neighborNodes = targetNode.connected().nodes();
const visibleNames = neighborNodes.not('.hidden').data('name')
return visibleNames;
}
For me the commented version is easier to read and audit and it also feels safer for some reason, but I'm not how subjective that is.In this case, I'd lean towards intermediate variables, but sometimes I'll use functions or methods to group things.
I prefer function/method/variable names over comments because the are actual first class parts of the code. In my experience, people are a bit more likely to update them when they stop being true. YMMV.
In a procedural loop, you can assign an intermediate result to a variable. By giving it a name, you can forget the processing you have done so far and focus on the next steps.
The standard approach to complexity like that is to invent useful concepts and give them descriptive names. Then you can reason about the concepts themselves, without having to consider the steps you used to reach them.
If your entire objection is that you might want intermediate-named variables… you can just do that?
var longBooks = books.filter(book => book.pageCount > 1000)
var authors = longBooks.map(book => book.author)
var distinctAuthors = authors.distinct()
For short chains (95%+ of cases), this is far more mental overhead. For the remaining cases, you can just name the parts? I'm just completely failing to see your problem here.With procedural code, it's widely accepted that you should not do too many things in a single statement. But in functional code, the entire chain is a single statement. There are no natural breakpoints where the reader could expect to find justifications for the code.
Welcome to all features of every programming language?
Sacrificing readability, optimization, and simplicity for the 95% case because some un-principled developers might overdo it in the 5% case (when the cost of fixing it is trivially just inserting variable assignments) is… not a good trade-off.
Besides, programming language evolution is mostly driven by the fact that everyone is lazy and unprincipled at least occasionally. If you need to be disciplined to avoid footguns, you'll trigger them sooner or later.
Not necessarily. You can use intermediate variables when necessary.
How are we deciding what's "functional code", here? Because functional languages also provide means like `let` and `where` bindings to break up statements. The example might in pseudo-Haskell be broken up like
distinctAuthors = distinct authors
where
authors = map (\book -> book.author) longBooks
longBooks = filter (\book -> book.pageCount > 1000) books
IMO the code here is also simple enough that I don't see it needing much in the way of comments, but it is also possible and common to intersperse comments in the dot style, e.g. distinctAuthors = books // TODO: Where does this collection come from anyway?
// books are officially considered long if they're over 1000 pages, c.f. the Council of Chalcedon (451)
.filter(book => book.pageCount > 1000)
// All books have exactly one author for some reason. Why? Shouldn't this be a flatmap or something?
.map(book => book.author)
// We obviously actually want a set[author] here, rather than a pruned list[author],
// but in this imaginary DinkyLang we'd have to implement that as map[author, null]
// and that's just too annoying to deal with
.distinct()I know that intermediate states are generally easier to comprehend, because I never have to explain them in code reviews. To avoid having to explain chains to others, I end up having to add descriptive comments to the intermediate steps, far exceeding the number of characters the descriptive intermediate variables would take. That's why I avoid them, or break them up: time spent in code reviews has proven to me that people have trouble with chains.
I've found that the constraint of thinking in chains forces me to think of the recipe that I need for my data. Of course, not everything can be done in a stepwise manner (.pipe helps with that), but often, this constraint forces you to think about what you are doing.
Every good Pandas user I know uses it this way. I've taught hundreds more. Generally, it feels weird at first (kind of like whitespace in Python), but after a day, you get used to it.
Do you store intermediate results of SQL?
> Just run the code and verify that the current step works. Then, proceed to the next.
Yes, it's not debuggable/"viewable" without cut/paste/commenting out lines, once it's constructed.
Giving names to things makes it easier to understand the intention of the programmer.
And that also allows you to create a TREE of dataflow-code not just a CHAIN. For instance 'longBooks' could be used as the starting point of multiple different chains.
It gets complicated at some point but I think other approaches result in code that is even harder to understand.
Filter to the books with >1000 pages
Then their authors.
Finally, distinguish those authors.
If you're familiar, you don't mentally represent each link in the chain as the totality of everything that came before it _plus_ whatever operation you're doing now. You consider each link in the chain in isolation, as its inputs are the prior link and its outputs will be used in the next link. Giving a name to each one of those links in the chain isn't always necessary, and depending on how trivial the operations are, can really hurt readability.
I think its very much a personal preference.
You don't have to give a name to every intermediate state, just like you don't have to comment every single line of code. But sometimes the names and comments do improve readability.
I have quite literally zero times in my ~25 year career had to deal with some sort of completely inscrutable chain of functional calls on iterators. Zero. I am entirely convinced that the people arguing against this style have never actually worked in a project where people used this style. It's okay! The first time I saw these things I, too, was terribly confused and skeptical.
That said, you didn't address my comment at all. It might be easier, but that doesn't mean it is easy to figure out what that long chain is really done - all too often the algorithm names don't tell you what you are really trying to accomplish in my experience.
A strength of functional idioms is that they expose the structure of the code in a way that a name - even a well chosen name - can only hope to achieve. Often, succinctly and comprehensively. At that point you stop caring so much about variable names. They’re still there but you need them less
var longBooks = books.filter(book => book.pageCount > 1000)
var authors = longBooks.map(book => book.author)
var distinctAuthors = authors.distinct()
could become (in a different language) books
|> Books.filter_by_length(min: 1000)
|> Authors.from_books()
|> Enum.distinct()
and now each step is named and reusable. This example isn't the best, but it can be quite helpful when you have large map() and filter() logic blocks. var authorsOfLongBooks = books
.filter(book => book.pageCount > 1000)
.map(longBooks => longBooks.author)
.distinct()Most professional Pandas users realize that working with chains makes their lives much easier.
By the way, debugging chains isn't hard. I have a chapter in my book that shows you how to do it.
When you're actually analyzing a bug, or need to add a new feature to the code... Then you'll have to keep the whole thing in your mind. No way around it
It gets extra annoying when people have complex maps, reduces, flat maps all chained after the next, and each step moved into a named function.
HF constantly jumping around trying to rationalize why something happens with such code...
It looks good on first glance, but it inevitably becomes a dumpster fire as soon as you need to actually interact with the code.
When the SQL and Pandas examples are isomorphic except for shallow syntactic differences, the root cause of the complaint must either be:
* that the judgment was emotional rather than substantive * or that the syntactic differences (dots and parens) actually matter
SELECT DISTINCT authors.some_field FROM books JOIN authors ON books.author_id = authors.author_id WHERE books.pageCount > 1000
And if you wanted to grab the entire authors record (like the code does) you'd probably need some more complexity in there:
SELECT * FROM authors WHERE author_id IN ( SELECT DISTINCT authors.author_id FROM books JOIN authors ON books.author_id = authors.author_id WHERE books.pageCount > 1000 )
SELECT * FROM authors WHERE author_id IN (SELECT author_id FROM books WHERE pageCount > 1000);
But I think you're missing the point. The functional/procedural style of writing is sequentialized and potentially slow. It's not transactional, doesn't handle partial failure, isn't parallelizable (without heavy lifting from the language--maybe LINQ can do this? but definitely not in Java).
With SQL, you push the entire query down into the database engine and expose it to the query optimizer. And SQL is actually supported by many, many systems. And it's what people have been writing for 40+ years.
But I don't think I missed the point, the original text talks about measuring complexity as a function of operators, operands, and nested code. The true one to one mapping is more complex than the original comment I replied to
for (index, node) in nodes
.expect("Error: No blocks for the Body")
.children()
.expect("Error: blocks node has no children")
.nodes()
.iter()
.enumerate()
{
let block = Block::new(node, index);
self.blocks.push(block);
} authors_of_long_books = set()
for book in books:
if len(book.pages) > 1000:
authors_of_long_books.add(book.author)
return authors_of_long_books
You are told explicitly at the beginning what the type of the result will be, you see that it's a single pass over books and that we're matching based on page count. There are no intermediate results to think about and no function call overhead.When you read it out loud it's also it's natural, clear, and in the right order— "for each book if the book has more than 1000 pages add it to the set."
That isn't natural to anyone who is not intimately familiar with procedural programming. The language-natural phrasing would be "which of these books have more than thousand pages? Can you give me their authors?" -- which maps much closer to the parent's linq query than to your code.
This is not about "procedural programming" - this is exactly how this works mentally. For kicks I just asked me 11-year old kid to write down names of all the books behind her desk (20-ish) of them and give me names of authors of books that are 200 pages or more. She "procedurally"
1. took a book
2. flipped to last page to see page count
3. wrote the name of the author if page count was more than 20
The procedural is natural, it is clear and it is in the right order
1. (Take the books)->(that have 200 pages or more)->(and mark down the name of the authors)->(only once)
Even the people arguing against functional style are able to understand it.
Strangely, this argument is quite similar to arguments I encounter when someone wants to rid the codebase of all SQL and replace it with ORM calls.
we must be in completely different worlds cause I have yet (close to 30 years now hacking) to see/hear someone trying to introduce ORM on a project which did not start with the ORM to begin with. the opposite though is a constant, “how do we get rid of ORM” :)
I haven't really encountered software engineers who really struggle with functional style in almost 20 years. It's just another tool that one has to learn.
I recall vividly when Java 8 came out (not the greatest example but also perhaps not too bad) having to explain over and over concept of flatMap (wut is that fucking thing?) or even zipping two collections. even to this day I see a whole lot of devs (across several teams I work with) doing “procedural” handling of collections in for loops etc…
The argument is always that "junior developers won't know SQL".
But yeah I've also seen the opposite happening once. People going gung-ho on deleting all ORM code "because there's so much SQL already, why do we need an ORM then".
And then the argument is that "everyone knows SQL, the ORM is niche".
I guess it's a phase that all devs go through in the middle of their careers. They see a hammer and a screwdriver in a toolbox, and feel the need for throwing one away because "who needs more than one tool"...
My personal take is that "how to execute" is more useful for lower level and finer grained control, which "what the results should be" is better for wrangling complex logic
authors_of_long_books: set[Author] = {book.author for book in books if book.page_count > 1000}
These are somewhat contentious as they can get overly complex, but for this case it should be small & clear enough for any Python programmer. # Functional approach
var favoriteFoodsOfFurryPetsOfFamousAuthorsOfLongChineseBooksAboutHistory = books
.filter(book => book.pageCount > 100 and book.language == "Chinese" and book.subject == "History")
.map(book => book.author)
.filter(author => author.mentions > 10_000)
.map(author => author.pets)
.reduce(acc, curr => acc.append(curr), [])
.filter(pet => pet.is_furry == True)
.map(pet => pet.favoriteFood)
.distinct()
# Procedural approach
var favoriteFoodsOfFurryPetsOfFamousAuthorsOfLongChineseBooksAboutHistory = set()
for book in books:
if len(book.pageCount > 100) and book.language == "Chinese" and book.subject == "History" and book.author.mentions > 10_000:
for pet in book.author.pets:
if pet.is_furry:
favoriteFoodsOfFurryPetsOfFamousAuthorsOfLongChineseBooksAboutHistory.add(pet.favoriteFood)
# Comprehension approach
var favoriteFoodsOfFurryPetsOfFamousAuthorsOfLongChineseBooksAboutHistory = {
pet.favoriteFood for pet in
pets for pets in
[author.pets for author in
[book.author for book in
books if len(book.pageCount > 100) and book.language == "Chinese" and book.subject == "History"]
if author.mentions > 10_000]
if pet.is_furry
}
FWIW, for more complex problems, I think the second one is the most readable.Depending on language you might also have some `.flat_map` option available to drop the `.reduce`.
I think I like the second approach because the loop behavior seems clearest, which helps me analyze the time complexity or when I want to skim the code quickly.
A syntax like something below would be perfect for me if it existed:
var favoriteFoodsOfFurryPetsOfFamousAuthorsOfLongChineseBooksAboutHistory = books[i].author.pets[j].favoriteFood.distinct()
where i = pagecount > 100,
language == "Chinese",
subject == "History",
author.mentions > 10_000
where j = is_furry == True fn bookFilter(book: Book) -> bool {
return book.pageCount > 100 and
book.language == "Chinese" and
book.subject == "History" and
book.author.mentions > 10_000
}
var favoriteFoodsOfFurryPetsOfFamousAuthorsOfLongChineseBooksAboutHistory = books
.filter(bookFilter)
.flatMap(book => book.author.pets)
.filter(pet => pet.is_furry)
.map(pet => pet.favoriteFood)
.distinct() (from book in books
where book.pagecount > 100
&& book.language == "Chinese"
&& book.subject == "History"
&& book.author.mentions > 10_000
from pet in book.author.pets
where pet.is_furry == true
select pet.favoriteFood)
.Distinct()
But it also demonstrates the...erm, chronic "halfassedness" of LINQ's query syntax form with distinct() not available there and having to fall back to method syntax form anyway... var favoriteFoodsOfFurryPetsOfFamousAuthorsOfLongChineseBooksAboutHistory = books
.filter(book =>
book.pageCount > 100 and
book.language == "Chinese" and
book.subject == "History" and
book.author.mentions > 10_000
)
.flatMap(book => book.author.pets)
.filter(pet => pet.is_furry)
.map(pet => pet.favoriteFood)
.distinct()
Or in Scala: val favoriteFoodsOfFurryPetsOfFamousAuthorsOfLongChineseBooksAboutHistory = (for {
book <- books if
book.pageCount > 100 &&
book.language == "Chinese" &&
book.subject == "History &&
book.author.metnions > 10_000
pet <- book.author.pets if
pet.is_furry
} yield pet.favoriteFood).distinct
Though, most Scala programmers would prefer higher-order functions over for-comprehensions for this.An alternative, which in FP-friendly languages would have almost identical performance, would be to make the shift in objects more explicit:
var favoriteFoodsOfFurryPetsOfFamousAuthorsOfLongChineseBooksAboutHistory =
books
.filter(book => isLongChineseBookAboutHistory(book))
.map(book => book.author)
.filter(author => isFamous(author))
.flatMap(author => author.pets)
.filter(pet => pet.isFurry)
.map(pet => pet.favouriteFood)
.distinct()
I slightly prefer this style with such a long pipeline, because to me it’s now built from standard patterns with relatively simple and semantically meaningful descriptions of what fills their holes. Obviously there’s some subjective judgement involved with anything like this; for example, if the concept of an author being famous was a recurring one then I’d probably want it defined in one place like an `isFamous` function, but if this were the only place in the code that needed to make that decision, I might inline the comparison.I just improved the comprehension code as well using the same idea as your code, eliminating an entire list!
{*map(_.author, filter(_.page_count > 1000, books))}
It uses lambdas package.Syntax highlighting in print is more limited because of technological and economic constraints, which might leave just bold, italics and underlines on the table, while dropping color. On screens and especially in our editors where we see the most code, a lack of color is often a self-imposed limitation.
authors_of_long_books: set[Author] = {
book.author
for book in books
if book.page_count > 1000
}They're just a tad more verbose in Python than mathematics because it uses words like 'for' and 'in' instead of symbols.
I would argue that's a downside: you have to pick the appropriate data structure beforehand here, whereas .distinct() picks the data structure for you. If, in the future, someone comes up with a better way of producing a distinct set of things, the functional code gets that for free, but this code is locked into a particular way of doing things. Also, .distinct() tells you explicitly what you want, whereas the intention of set() is not as immediately obvious.
> There are no intermediate results to think about
I could argue that there aren't really intermediate results in my example either, depending on how you think about it. Are there intermediate results in the SQL query "SELECT DISTINCT Author FROM Books WHERE Books.PageCount > 1000"? Because that's very similar to how I mentally model the functional chain.
There are also intermediate results, or at least intermediate state, in your code: at any point in the loop, your set is in an intermediate state. It's not a big deal there either though: I'd argue you don't really think about that state either.
> and no function call overhead
That's entirely a language-specific thing, and volatile: new versions of a language may change how any of this stuff is implemented under the hood. It could be that "for ... in" happens to be a relatively expensive construct in some languages. You're probably right that the imperative code is slightly faster in most languages today, and if it has been shown via performance analysis that this particular code is a bottleneck, it makes sense to sacrifice readability in favor of performance. But it is a sacrifice in readability, and the current debate is over which is more readable in the first place.
> a single pass over books
Another detail that may or may not be true, and probably doesn't matter. The overhead of different forms of loops is just not what's determining the performance of almost any modern application. Also, my example could be a single pass if those methods were implemented in a lazy, "query builder" form instead of an immediately-evaluated form.
In fact, whether this query should be immediately evaluated is not necessarily this function's decision. It's nice to be able to write code that doesn't care about that. My example works the same for a wide variety of things that "books" could be, and the strategy to get the answer can be different depending on what it is. It's possible the result of this code is exactly the SQL I mentioned earlier, rather than an in-memory set. There are lots of benefits to saying what you want, instead of specifying exactly how you want it.
Procedural code in JS doesn't say how you want something done any more closely than the functional style variant. for-of is far more generic than .map/.filter() since .map() only works on Array shaped objects, and for-of works on all iterables, even generators, async generators, etc. In any case you're not saying how the iteration will happen with for-of, you're just saying that you want it. Implementation of Set is also the choice of a language runtime. You're just stating what type of container you want.
Sometimes functional style may be more readable, sometimes procedural style may.
> ... no function call overhead.
This code has more function calls. O(n) vs 3 for the original
I'm going to take my own advice and go back to work :)
If I run this in the JS console I get two anonymous stack frames. The first being the console itself.
[1, 2, 3].filter(x => [][0]()) authors_of_long_books = set()
Now I know that authors_of_long_books is the empty set. Do I need to bother reading the rest?For example, use all the prime numbers as an expression in your chain.
import Data.Function
import Data.Numbers.Primes
main = do
let result :: [Int] = primes
& filter (startingWithDigit '5')
& asPairs
& map pairSum
& drop 100000
& take 10
print result
asPairs xs = zip xs (tail xs)
pairSum (a, b) = a + b
startingWithDigit d x = d == head (show x)
> [100960734,100960764,100960792,100960800,100960812]> 3 MiB total memory in use (0 MB lost due to fragmentation)
Which require the author to actually have an idea how big the numbers are, but that is very often the case regardless of how you write your code.
SELECT DISTINCT authors FROM books WHERE page_count > 1000;
?- setof(Author, Book^Pages^(book_author(Book, Author), book_pages(Book, Pages), Pages > 1000), Authors).
Depending on the structure of the Prolog database, it could be shorter: ?- setof(Author, Pages^(book(_, Author, Pages), Pages > 1000), Authors).Three dots is just a random Tuesday.
To know the return type of the chain, I have to read through it and get to the end of each line.
A longBooks array, and map(longBooks, ‘author’) wouldn’t be much longer, but would involve more distinct and meaningful phrases.
I used to love doing chains! I used lodash all the time for things like this. It’s fun to write code this way. But now I see that it’s just a one-liner with line breaks.
select distinct author from book where pageCount > 1000;This functional style is what I call write only code, because the only person who can understand it is the one who wrote it. Pandas loves this kind of method chaining, and it's one of the chief reasons pandas code is hard to read.
this could be something more like
distinct
filter books .pageCount >1000
.author
i think fp looks pretty terrible in js, rust, python, etcIt actively hurts maintainability, please stop using it.
> But you should be familiar enough that you stop randomly badmouthing map and filter like you have some sort of anti-functional-programming Tourette's syndrome.
I've been moderated for saying much tamer, FYI.
In my book, keeping simple things simple and the not simple things not simple beats simplicity everywhere. This is actually what I consider a big drawback of functional style: often the not-simple parts are way too condensed, almost indistinguishable from trivialities. But in the loop scenario it's often the reverse.
My happy place, when writing, would be an environment that has enough AST-level understanding to transform between both styles.
(other advantages of functional style: skills and habits transfer to both async and parallel, the imperative loop: not so much)
let authorsOfLongBooks =
books
|> Seq.filter (fun book -> book.pageCount > 1000)
|> Seq.map (fun book -> book.author)
|> Seq.distinct
...you can set breakpoints anywhere in the pipeline! var longBooks = books.filter(book => book.pageCount > 1000)
var authorsOfLongBooks = longBooks.map(book => book.author).distinct()Three things typically happens:
1. people who like these chains really like them. And I've seen multiple "one liner expressions" that was composed of several statements anded or ored together needing one or two line breaks.
2. when it breaks (and it does in the real world), debugging is a mess. At least last time I had to deal with it was no good way to put breakpoints in there. Maybe things have changed recently, but typically one had to rewrite it to classic programming and then debug it.
3. It trips up a lot of otherwise good programmers who hasn't seen it before.
ISet<Author> authorsOfLongBooks =
books
.filter(book => book.pageCount > 1000)
.map(book => book.author)
.distinct()
.toHashset()
Or whatever the equivalent for ISet<Author> is in the respective language. Or IReadonlySet<Author> if the set should be immutable.Specifically around type-safety, that is knowing that the chained type is what you expect and communicating that expectation to the person who is reading the code without them needing to know the wider context of both the chained-API nor the function the chain resides in. In the context of this article, that means more complexity, and therefore less readability.
I feel this is important because I have worked on many legacy code bases where bugs were found where chains were not behaving as expected, normally after attrition in some other part of the code base, and then you have to become a detective to work out the original intent.
For readability chains are bad, because they can lie about their intent, especially if there’s various semantics that can be swapped. But, like any industry or code base, if their use is consistent, and the api mature/stable, they can be powerful and fast, if.
This is not at all implied by anything else in the article. This feels like a common "I'm unfamiliar with it so it's bad" gripe that the author just sneaked in. Once you become a little familiar with it, it's usually far easier to both read and write than any of the alternatives. I challenge anyone to come up with a more readable example of this:
By almost any complexity metric, including his, this code is going to beat the snot out of any other way of doing this. Please, learn just the basics of functional programming. You don't need to be able to explain what a Monad is (I barely can). But you should be familiar enough that you stop randomly badmouthing map and filter like you have some sort of anti-functional-programming Tourette's syndrome.