News for hackers. Stuff that matters.
Re:What Makes Code Hard to Read: Visual Patterns of Complexi... by stouset on Tuesday March 11, 2025 @03:38PM (#43336238)
You have literally just described the set of objects asked for: the unique authors of the books with more than 1,000 pages. I don't understand how you expect to get any simpler than that. The functional style isn't even requiring you to describe how to accomplish it, it almost verbatim simply describes the answer you're trying to get.

If your entire objection is that you might want intermediate-named variables… you can just do that?

    var longBooks       = books.filter(book => book.pageCount > 1000)
    var authors         = longBooks.map(book => book.author)
    var distinctAuthors = authors.distinct()
For short chains (95%+ of cases), this is far more mental overhead. For the remaining cases, you can just name the parts? I'm just completely failing to see your problem here.
Re:What Makes Code Hard to Read: Visual Patterns of Complexi... by jltsiren on Tuesday March 11, 2025 @03:56PM (#43336412)
The problem is that it's easy to overdo it. When you are writing the code, you already know what it's supposed to do, and adding a few more things to the chain is convenient and attractive. But when you are reading unfamiliar code, you often wish that the author was more explicit with their code. Not just with what the code is actually doing, but what it's trying to do and what are the key waypoints to get there.

With procedural code, it's widely accepted that you should not do too many things in a single statement. But in functional code, the entire chain is a single statement. There are no natural breakpoints where the reader could expect to find justifications for the code.

Re:What Makes Code Hard to Read: Visual Patterns of Complexi... by stouset on Tuesday March 11, 2025 @04:40PM (#43336793)
> The problem is that it's easy to overdo it.

Welcome to all features of every programming language?

Sacrificing readability, optimization, and simplicity for the 95% case because some un-principled developers might overdo it in the 5% case (when the cost of fixing it is trivially just inserting variable assignments) is… not a good trade-off.

Re:What Makes Code Hard to Read: Visual Patterns of Complexi... by jltsiren on Tuesday March 11, 2025 @05:31PM (#43337222)
5% is common enough that you'll encounter it almost every time you read code. And fixing it is not easy, because you first need to understand the code before you can add useful variable names.

Besides, programming language evolution is mostly driven by the fact that everyone is lazy and unprincipled at least occasionally. If you need to be disciplined to avoid footguns, you'll trigger them sooner or later.

Re:What Makes Code Hard to Read: Visual Patterns of Complexi... by stouset on Tuesday March 11, 2025 @05:32PM (#43337241)
The cost of this "footgun" is basically zero. Every step in a functional pipeline is isolated and wholly independent. If you want to split such a pipeline in two, doing so is trivial.
Re:What Makes Code Hard to Read: Visual Patterns of Complexi... by whstl on Tuesday March 11, 2025 @06:34PM (#43337843)
5% is also low enough that you can just use another technique for the exceptions.
Re:What Makes Code Hard to Read: Visual Patterns of Complexi... by vendiddy on Thursday March 13, 2025 @03:17AM (#43350984)
It's not clear what point you are trying to make because so far you are describing problems common to all programming languages.
Re:What Makes Code Hard to Read: Visual Patterns of Complexi... by whstl on Tuesday March 11, 2025 @06:35PM (#43337845)
> But in functional code, the entire chain is a single statement

Not necessarily. You can use intermediate variables when necessary.

Re:What Makes Code Hard to Read: Visual Patterns of Complexi... by syklemil on Tuesday March 11, 2025 @07:05PM (#43338076)
> But in functional code, the entire chain is a single statement. There are no natural breakpoints where the reader could expect to find justifications for the code.

How are we deciding what's "functional code", here? Because functional languages also provide means like `let` and `where` bindings to break up statements. The example might in pseudo-Haskell be broken up like

    distinctAuthors = distinct authors
      where
        authors = map (\book -> book.author) longBooks
        longBooks = filter (\book -> book.pageCount > 1000) books

IMO the code here is also simple enough that I don't see it needing much in the way of comments, but it is also possible and common to intersperse comments in the dot style, e.g.

    distinctAuthors = books // TODO: Where does this collection come from anyway?
        // books are officially considered long if they're over 1000 pages, c.f. the Council of Chalcedon (451)
        .filter(book => book.pageCount > 1000)
        // All books have exactly one author for some reason. Why? Shouldn't this be a flatmap or something?
        .map(book => book.author)
        // We obviously actually want a set[author] here, rather than a pruned list[author],
        // but in this imaginary DinkyLang we'd have to implement that as map[author, null]
        // and that's just too annoying to deal with
        .distinct()
Re:What Makes Code Hard to Read: Visual Patterns of Complexi... by bonoboTP on Tuesday March 11, 2025 @08:26PM (#43338657)
If you're used to it, then it doesn't read like a single statement, even though technically it is. You put each call of the chain on its own line and it feels like reading the lines of regular imperative code. Except better because I can be sure that each line strictly only uses the result of the previous line, not two or three lines before so the logic flows nicely linearly.
Re:What Makes Code Hard to Read: Visual Patterns of Complexi... by __mharrison__ on Tuesday March 11, 2025 @04:03PM (#43336457)
It's also harder to write and debug with the intermediate steps.
Re:What Makes Code Hard to Read: Visual Patterns of Complexi... by nomel on Tuesday March 11, 2025 @06:10PM (#43337624)
How so? The states of the intermediate steps are logically and easily exposed in a debugger. You can also easily set conditional breakpoints relative to the intermediate states.

I know that intermediate states are generally easier to comprehend, because I never have to explain them in code reviews. To avoid having to explain chains to others, I end up having to add descriptive comments to the intermediate steps, far exceeding the number of characters the descriptive intermediate variables would take. That's why I avoid them, or break them up: time spent in code reviews has proven to me that people have trouble with chains.

Re:What Makes Code Hard to Read: Visual Patterns of Complexi... by __mharrison__ on Tuesday March 11, 2025 @06:40PM (#43337882)
Build up and debug the chain as you work in an environment like Jupyter. No need to create variables. Just run the code and verify that the current step works. Then, proceed to the next. Then, put the chain in a function. If you want to be nice, put a .loc as the first step to explicitly list all of the input columns. Drop another .loc as the last step to validate the output columns. (This also serves as a test and documentation to future you about what needs to come in and out.) Create a simple unit test with a sample of the data if you desire.

I've found that the constraint of thinking in chains forces me to think of the recipe that I need for my data. Of course, not everything can be done in a stepwise manner (.pipe helps with that), but often, this constraint forces you to think about what you are doing.

Every good Pandas user I know uses it this way. I've taught hundreds more. Generally, it feels weird at first (kind of like whitespace in Python), but after a day, you get used to it.

Do you store intermediate results of SQL?

Re:What Makes Code Hard to Read: Visual Patterns of Complexi... by nomel on Tuesday March 11, 2025 @08:51PM (#43338834)
> Build up and debug the chain as you work in an environment like Jupyter.

> Just run the code and verify that the current step works. Then, proceed to the next.

Yes, it's not debuggable/"viewable" without cut/paste/commenting out lines, once it's constructed.

Re:What Makes Code Hard to Read: Visual Patterns of Complexi... by __mharrison__ on Wednesday March 12, 2025 @12:13AM (#43339894)
If it is breakpoints you are concerned about, you can set a breakpoint on a method in the chain and inspect `self`.
Re:What Makes Code Hard to Read: Visual Patterns of Complexi... by nomel on Wednesday March 12, 2025 @02:19PM (#43346151)
Most languages don't expose the internal of map to set a breakpoint, so you're left with individual entities. But yes, there are tricks you can use to make it work, although most require more complex conditional/sequential breakpoints. In your method breakpoint example, you would need to set a chained breakpoint, as in "don't break until this other breakpoint above the chain has been hit", otherwise the breakpoint in the method won't be "spatially" relevant to the code you're debugging.
Re:What Makes Code Hard to Read: Visual Patterns of Complexi... by galaxyLogic on Wednesday March 12, 2025 @01:50AM (#43340297)
Your example shows that it is possible to give NAMES to the intermediate results of a long chain.

Giving names to things makes it easier to understand the intention of the programmer.

And that also allows you to create a TREE of dataflow-code not just a CHAIN. For instance 'longBooks' could be used as the starting point of multiple different chains.

It gets complicated at some point but I think other approaches result in code that is even harder to understand.