Why does any code start failing?

That’s not the point they were making. The point is that when you start chaining together so many calls, it can become difficult to debug. Prettier to write, but more difficult to debug. But that’s okay, you need to find the balance that’s appropriate for the particular project.

> but more difficult to debug.

I've found the stream debugger in IntelliJ to be very useful when looking at debugging streams.

https://www.jetbrains.com/help/idea/analyze-java-stream-oper...

The old plugin version of it has the same functionality (and better pictures) - https://plugins.jetbrains.com/plugin/9696-java-stream-debugg... (click the 'more' link)

Selecting a piece of data within the stream shows you how it moved through the stream.

The other part of streams, for me, is that with the additional "each line does one, and only one thing" and you're not putting too much complexity in a single map, I feel that it forces you to write simpler code that doesn't need much debugging. The question of "how did that data get into the stream is where most of the debugging comes from.

First.

If getPrice fails, the stack trace will start there. If it returns null (which it should not) and trigger a npe, then the line:

   map(e -> e.getKey().getPrice().multiply(new BigDecimal(e.getValue()))).

Is just as dense as the original:

   sum = sum.add(entry.getKey().getPrice().multiply(new BigInteger(entry.getValue())

And the stack trace would be just as confusing.

Second.

The key thing with streams is to borrow from the functional programming paradigm: Split data and functions, avoid or isolate side-effects.

Do this correctly and there is a quite real plus in productivity.

Could I trouble you for an example or a reference on what this would look like refactored with "Split data and functions, avoid or isolate side-effects"? I would like to understand better but I don't know enough about functional programming to grasp your meaning just from the comment.

You might read up on pure functions [1] which are unable to produce side-effects when run.

Imagine you have two methods in Java:

  int add(int a, int b) {
    log.info("Adding two numbers {} {}", a, b);
    return a + b;
  }

  void doStuff() {
    add(1, 2);
    add(2, 3);
    add(3, 4);
  }

The result of add in doStuff is unused. However, add has a log statement which someone might be relying on elsewhere. The log line makes understanding the usefulness of this code much harder. ie: Can you delete this call? It's impossible to know without understanding everything that might consume the log line. The log-line is a side-effect in these methods.

In languages that understand "pure functions" there are optimizations that can be done by the toolchain (think automatic memoization, deferred computation, and much more) when only pure functions are called.

[1] https://en.wikipedia.org/wiki/Pure_function

I don't know any good resource. And not all functional programmers are good programmers; but try to understand what they are trying to do.

There as a bit of zen to it, less is more, in sense that a language gets more powerful if it is more constrained. For example if you know (by the type system or just coding conventions) that p.getPrice() nevers returns null, it is easier to reason about (proof, test, read) the code.

Like wise if you know that if p1 == p2 then p1.getPrice() == p2.getPrice() (that would be no side effects).

If you as some one suggested, need to support some crazy localization, then don't put it into p.getPrice(). If you must, change the name to something telling and make its input explicit: p.calculateLocalizedPrice(locale). Or better make it an explicit function (static method, or maybe something sitting in a service) calculateLocalizedPrice(product, locale) and again have it be without side effects.

I think definitions vary slightly, but the quality you mentioned (a == b => f(a) == f(b)) would be called a deterministic function [1] - very useful, often applicable or even type check enforced in functional programming languages.

Having no side effects is a different very useful quality - function doing exactly as specified and no more (I/O, setting variables ..). I'm not sure if it means not accessing global state, though it is usually better if both inputs and outputs are explicit.

All in, it is usually easier to reason about functions that are explicit, deterministic and side-effect free, yet I find it profoundly more valuable if it can actually be relied on (a known subset of) functions having those qualities.

[1] https://maksimivanov.com/posts/pure-functions-and-side-effec...

Because stuff fails?

Maybe getPrice used to be a static lookup from a map that couldn't fail, but then the Sales Team wanted to go multi-national and now it's a database lookup with multiple dependencies, that can fail.

"But why wasn't it caught in a code review" etc...

Have you actually worked with a big team ever? Why would (or how could) anyone (outside of Google) go through every dependency of the getPrice function and check that every use case is handling errors/exceptions properly?

Stuff breaks, code is read and debugged more often than it's written. Optimising stuff to be easy and fast to write is the wrong way to make maintainable code. Unroll your loops, add toggleable debug logging and add comments why stuff is done the way it is.

The method getPrice is not really correctly named if it did the things you describe. Naming things carefully will give you far more productivity than "unrolled loops, toggleable debug logging and comments on why stuff is done the way it is".

Functions aren't always refactored to their perfect names, for multiple reasons.

getPrice might start up as something that just gets the price, after 5-10 years it might be a complex process accessing some ERP systems.

> Stuff breaks, code is read and debugged more often than it's written. Optimising stuff to be easy and fast to write is the wrong way to make maintainable code.

This is exactly why people like collection functions. For loops can do anything, you have to spend more time reading and understanding the loop to build your mental model of what is happening. Mapping does one thing, transforms a collection into another collection. Same with filter, etc. If you are optimizing for readability, collection functions give way more information to the reader. Your approach is to optimize for debugging, which I'm not saying is wrong, but it's not optimizing for readability.

What map returns greatly depends on lambda inside. So yoir collection of beans is changing into collection of good knows what and you have to keep while chaining in mind - because it is nowhere visible.

Whatever exists inside the map lambda would have to exist inside the for loop as well. So if you're dealing with a confusing transformation, a loop doesnt offer you any extra tools for making that transformation more apparent to the reader. Loops have plenty of advantages (computer execution is more obvious, stack traces can be cleaner, easier concept for beginners to grasp), but I have never seen a loop be more readable than a well written functional composition.

Yes and it tends to be more apparent what is its type. It also tends to have a name that helps understanding a lot . It is not even primary loop vs stream difference. This particular frustration is the fluent api vs procedural difference.

It is chaining that obfuscates in this case. Through, I really don't find functional style more readable in general.