With a little experience, functional programming is quite straightforward. Practically everything comes down to map, filter, and fold, or whatever they're called in a given language.

So the first example looks to me like two loops (hopefully the compiler does a better job on that), while the second one is obviously one loop.

I am already trying to guess how a compiler would parallelize that...

The first example is simply more abstract. It might be a single loop in case the implementation is lazy, it might be two loops in case it is strict. However, look closer. Those filter and map operations can be applied to basically anything you want, including asynchronous / infinite streams.

Your second loop on the other hand would have a hard time being translated to anything else than something that works on finite lists and that builds a finite buffer and not doing anything else while doing this.

Or in other words, your knowledge about this manual loop is not transferable ;-)

But a "smart enough compiler" is what compiler people say when they are talking about their version of the abstract anthropogenic equivalent AI. The fact is that the first one is much easier to parallelize, so that real implementations actually do it, while the second one is not done on practice. And it's easier to parallelize because there are some extra guarantees on the signature of those functions that a for loop does not give.

But in the second example, you are telling it to iterate over each element, sequentially. Since you are telling step by step what needs to be done, it's not parallelizable unless you implement it to be so.

The second example is about control flow. The required behaviour is specified imperatively, in the form of a statements to be executed in a defined order, and the combined effect of those statements is to update the list until it has the required value.

In the second case, the programmer specifies more details explicitly. In this case, those extra details probably aren’t helping, but the optimiser still has to prove that transforming the control flow (for example, into a parallelised form) really will give the same observable behaviour. In the first case, the optimiser doesn’t have to prove the things the programmer never specified anyway, so it has more latitude in how it implements the underlying computation.

There are guarantees about the way a map function works. It doesn't mutate its input, it only has access to one element at a time, you can't access the data structure you're building, etc.

All of these traits are true of the imperative version as well, but it's a lot harder to write a program which understands that. Meanwhile, you know that's how map and filter work, so it's trivially easy to know that those guarantees hold.

Can't be done.

For a simple example, code that performs regular array-based accesses, where index computations are simple affine functions (a*i + b) of a single iteration variable 'i', is a case that's received a lot of attention. See e.g. Polly for LLVM.

We could talk about why the first example could be parallelized of course, but that's not what I wanted to talk about and you missed the point ;-)

So what do you actually have to do to make this actually run in parallel? Or do you truly get it automatically?

In contrast, in the imperative form, you are specifying how items are appended to a list, which means that any attempt to do it in parallel could change the order of the operations and therefore the order of the result. Sure, a sufficiently smart compiler could in theory figure out what "should" happen and see how to optimize it, but in practice today's smartest compilers can barely handle the functional case.

The second one takes a lot more work. First, I have to read through the code and recognize that this is a loop that accumulates values into a new array. Then I have to pick it apart and see exactly where the accumulation takes place, and how that value is derived. It's not hard by any means, but the first one is far more straightforward.

But the second one is more debuggable than the first, which I think is even more important than readability.

In the first case, you need to rewrite the control structure to even be able to inspect anything:

    let allPersons = names.map(Person.init)
    {log allPersons[0].name}
    {breakpoint}
    let persons = allPersons.filter { $0.isValid }

It's also totally unclear what the order of operations is. Are Person.init and $0.isValid alternated? Is the `map` run in full before the `filter` starts? No way to know.

People talk shit about procedural programming, as if it's antiquated. But the core promise of functional programming—that you can stop thinking about the underlying procedures—never seems to fully pan out. So when you inevitably need to start digging under the hood to figure out what your declarative code actually means in practice, you end up having to think procedurally anyway. Now you're thinking procedurally, but your code is declarative, and the runtime is trying as hard as it can to prevent you from knowing what's exactly happening moment to moment.

I think there are specific cases where a declarative interface is the right abstraction. CSS is a good example. But these are narrowly defined domains with relatively clear semantics that get frequent use, so the time it takes to learn the semantics will pay off.

The idea of littering your entire codebase thickly with declarative APIs, each of which has unique control structures that must be understood in order to read code, is not a good approach in my opinion.

This is what Rails is, and it creates a situation where you are captive to your tools: you can do a lot very easily, but you cannot stray far beyond the declarations that your library author overlords have chosen for you, or you quickly find yourself in a space where in order to not shoot yourself in the foot you need to have a huge body of internals in your head.

The first is less likely to require debugging in the first place.

> There are lots of data structures in this style of programming that don't have any names.

So you can only reason about things that have names? Now we know where idiomatic Java comes from.

> Who knows what kind of data structures map and filter create in order to do their work.

In most reasonable implementations, the only data structure being created is the final result (a functorial value in map's case, a sequence in filter's case). For example, in SML:

    fun map _ nil = nil
      | map f (x :: xs) = f x :: map f xs

    fun filter _ nil = nil
      | filter p (x :: xs) =
        if p x then x :: filter p xs
        else filter p xs

Functional programming doesn't promise freedom from procedures. It promises (and delivers) freedom from physical object identities when you only care about logical values.

---

@banachtarski:

Code that's likely to require debugging (say, because it implements tricky algorithms) should be isolated from the rest anyway, regardless of whether your program is written in a functional style or not. Say, in Haskell:

Bad:

    filter (\x -> tricky_logic_1) $
    map    (\x -> tricky_logic_2) $ xs

    -- Now trickyFunction1 and trickyFunction2 can be
    -- tested in isolation. Or whatever.
    trickyFunction1 x = ...
    trickyFunction2 x = ...
    
    filter trickyFunction1 (map trickyFunction2 xs)

I'm all for functional languages but this scares me a bit. What do you do when you need to debug something and everything ends up being harder to debug but "less likely to need debugging." I've actually run into this situation a number of times and faced with a sea of linked compound expressions, debugging can be a daunting proposition.

It's kind of strange to me. We generally acknowledge that not repeating yourself and dividing responsibilities sensibly leads to better code that has fewer bugs and is easier to reason about. And yet when we consider doing the same thing with iteration, we say, "Whoa, hang on. Why can't we just write out the whole thing every time instead of factoring the common bits into a function?"

You have a ton of those statements, that's why you organize them in functions, and go debugging high level functions before you go into lower level ones, and then into atomic statements.

Anyway, you'll almost certainly want to do that in an repl. I don't know if swift supports one, if not that would be a real drawback.

I think the point was that you can debug things that have names because they are separately watchable.

But apart from that breaking things down and naming them can make for easier comprehension. This is true in written English: Naming actors when explaining something and using an active voice is generally recommended. e.g "The user enters a password and the program encrypts it and stores it in the database" Rather than: "Passwords will be stored encrypted"

But also in mathematics. When working something out it's better to name intermediate values (x,y) and then use them in new equations rather than use the equivalent of a point free style that you sometimes see in functional programs.

Note I said “reason”, not “debug”. When manipulating algebraic expressions, I don't need to give every subexpression a name - that would be torture!

    fs = [f(x) for x in xs]

Kinda reminds me of microservices, actually. Tough to debug, but good in other ways.

Five years ago, I would have found the second form easier to read. These days, not only do I find the first form much clearer but I find the second one a bit smelly because it mutates data.

It's actually surprising how quickly you get used to the first form of code once the language you use supports it.

In this case, as long as append is O(1), i think the imperative version has a big benefit, it avoids building the name size list of persons. If you've got a billion names and 2 valid person objects, the imperative version is a big win. Of course that predicate i mentioned is the right way to go though.

I think the right way to do it is with a fold. But that's not built in, so hard to expect of novices.

I see what you're saying about mutation, i guess i have a higher tolerance for mutating stuff that you have the only reference to. I'm not really sure doing a += [validPerson] is much of a win. (but i think would be the right answer in a fold)

No "object" is made if the name can't create a "valid person object", it'll return a stack-allocated null/nothing value. That pattern is also much better for concurrency issues.

> much much better to filter the names, then make the objects.

You're just duplicating the validation logic (or worse, the "objects creation" assumes it's given valid names and does no checks)

I think the right way to do it is with a fold.

[0] http://hackage.haskell.org/package/base-4.9.0.0/docs/Data-Ma...

[1] https://doc.rust-lang.org/std/iter/trait.Iterator.html#metho...

   validPerson : thunk

   validPerson : invalid : invalid : invalid : ... : thunk

  validPerson : invalid : ... : validPerson

getting that first answer is cheap and quick. getting the second answer requires evaluating those billion - 1 thunks.

maybe you're thinking of list fusion? AFAIK, only haskell does that.

Perhaps i'm misunderstanding your point. But even then, lisp, scheme, smalltalk, python and perl don't implement map and filter in a lazy way. Of course, i'm thinking in terms of haskell lazy, perhaps, again, i'm misunderstanding and you mean something else.

In retrospect, i think i'll revise my opinion and say map.filter is probably a bigger code smell than referentially transparent mutation.

The use laziness will avoid the need to construct a large intermediate collection of Person instances. Obviously you'll need to iterate the entire source collection to find all valid Persons but the same applies to the imperative version. The lazy version composes better however and avoids extra work if some downstream caller only requires the first n valid Persons.

But, there's already a good clean functional composable answer to this kind of thing,

fold takes an operation (like map does) a list (like map does) and an accumulator for building up results, and it returns the resulting accumulator.

   foldl(op, names, acc){
       if(names.empty)
           return acc
       else
           foldl(op, names.tail, op(names.head, acc)
   }

the op would look something like

   op(name, acc){
      let p = Person.init(name)
      if(p.isValid)
          return acc += [p]
      else
          return acc
      }

Map forces you to build the intermediate representation, which the imperative version avoids.

I'm just sort of mystified that people are defending map.filter over filter.map (which isn't crazy, as name.isValidPerson isn't provided.) but, you know, fold is a thing.

Not quite. Lazy sequences make it so that thinking about fusion/deforestation makes sense. This is an optimization the compiler can make. Naively, you still end up with as many allocations as before (just in a different order).

In Haskell, this sort of optimization is actually user-customizable using REWRITE RULES. And you get it for lists out of the box.

It helps, for me, that I have more of a math background (academically) than CS (dual majors, but strongly preferred the math coursework).

For me, I see three ways to write, as an example, a summation:

  n
  Σ g(i)
  i=1

  vs.

  seq(1,n).map(g).sum() // or something similar

  vs.

  for(i = 1; i < n; i++)
    sum = g(i)

Or a reason to go back to checking all code I try to type in.

  (1...i).reduce(0) { $0 + g($1) }

    val persons = names
        .map { name => Person(name = name) }
        .filter { person => person.isValid }

    val persons =
      names.map(Person.apply).
      filter(_.isValid)

  let persons = [ p | n <- names
                    , p <- peopleNamed n
                    , isValid p ]

  filter isValid (map peopleNamed names)

In any case, all you'd need to add is something like concat.

At the moment I'm working in Objective-C, and I literally just wrote a loop to filter an array and form a new array with the results like the one in the example - and god I wish there was an as straight-forward, easily understandable functional way in Obj-C to do it like in the Swift example.

I miss these 'nice' things - I don't need the fully functional Haskell package, but I like having some of the nice things Swift has, just because I got used to them and can actually write and read them better, and it is definitely more elegant!

  NSArray *filteredArray = [array filteredArrayUsingPredicate:[NSPredicate predicateWithBlock:^BOOL(id object, NSDictionary *bindings) {
      return [object test];
  }]];

    a  := #(1 2 3 4 5).
    Transcript show: (a printString); cr.
    b := a select: [:x | x \\ 2 == 0].
    Transcript show: (b printString); cr.

For that reason, I really wish Swift had list comprehensions, just because it's the first "slightly functional" exposure most non-developers get if they come from Python.

In the first example, because I know what "map" means, I know that `Person.init` is applied to each name. And then I know that only the valid `Person`s are returned by the `filter` call.

In the second example, I have to understanding the unique logic of the loop block to get to the same conclusion.

I've been shepherding a bunch of front end Javascript tests, and recently had to go through fixing a systemic problem with how we were handling multiple promises.

The broken code didn't look too different from your second example, but the same three people made the same mistake repeatedly, leading to tests that generated empty lists and thus didn't verify anything.

Now, I'm not claiming this is a good pattern of testing. Indeed in all of the straightforward cases I removed the array entirely, and with great relish (especially since it also sped up the tests dramatically). And there are obviously some gaps in their theory of testing that they didn't notice the problem until I pointed it out.

But it did illustrate to me again that there are (alarmingly) a lot of people struggling with basic data manipulation, and if your language supports anything like list comprehensions, I think you should probably get used to using them. It keeps those gaps out, and makes people decompose the problem instead of mashing together a block of conditional code that reads like a Choose Your Own Adventure.

It also depends on the language obviously. In this example, the first example has special tokens for the filter. It doesn't have to be so.

    persons = filter isValid (map init names)

The other thing is that experience brings the ability to track what's going on. The formulation of the answer here is probably new to you. As you get use to this, or Streams for Java, or threading for Clojure, etc, you'll understand it by default.

When it's optional, well, sadness ensues.

https://h4labs.wordpress.com/2016/09/30/functional-swift-usi...

And in cases where a data-deriving loop has so much going on directly in the loop body that it makes map/filter/reduce hard to read, there's very often some refactoring that would improve either version.

In the cases where x gets also transformed or you do some unpacking the syntax is very efficient and easy to read: [f(y) for x, y in items if g(x)]. Basically it's a poor man's relations programming (think databases). It's brilliant. (And definitely easier to read and write than the Haskell list comprehension syntax).

Funnily enough, I find exactly the opposite: for me, the functional style is significantly easier to work with when things get crazy. I think this is mostly because you tend to be composing recognisable patterns, which in turn means the only custom code you’re writing is the “interesting” parts, like deciding exactly which data to select or exactly how to combine each pair of elements. With lots of loops and conditionals and early exits, I also have to work out whether the code is really doing what it looks like or whether there are edge cases that work differently, and even the “what it looks like” part can wind up scattered across several places in the code that are some distance apart.

Some of the projects I work on do a lot of quite intricate manipulations of complicated data structures. Earlier incarnations were written in Python, but even there I found myself using a functional style for most of these situations as the code base grew in size and complexity. More recently, for various reasons including that one, I’ve been writing this sort of code in Haskell, a language designed for that programming style and therefore cleaner in both syntax and semantics. IMHO, it would be hard to overstate how much easier the newer code is both to write originally and to read, fix and extend later. Possibly the most striking thing is how much shorter the code is: the functional style combined with a language and libraries designed to support it really is remarkably expressive for data crunching work compared to the “longhand” form of writing out all of the loops and conditionals manually.

    persons = [person for person in (Person(name) for name in names) if person.isValid()]

    [person

It's almost like a select query in (pseudo) SQL.

    select person from people where person.isvalid = true

    [person for person in people if person.isvalid]

  (filter (fn[p] (p :valid?)) 
    (map (fn[n] (Person n) names))

And incidentally, this would probably be something like (EDIT: made example more realistic):

  filter personIsValid (map initPerson names)

I'll admit Lisp's myriad nesting of brackets is not ideal either. But surely there are more elegant and intuitive notations for functional scoping than is seen here.

Admittedly, it might be written with $ in practice, but that's a fairly simple idiom, and I tend to avoid it.

Very simple example: once upon a time, in the early 1960's, Lisp only had the COND operator. There was no IF. Programmers often had to make two-way decisions using COND, writing things like (COND (condition then-expr) (T else-expr)). Too many parentheses. So they came up with the IF macro allowing (IF condition then-expr else-expr). This simply expanded to the COND. Six parentheses are reduced to two.

Like most other programmers, Lisp programmers care about not writing mountains of distracting, irrelevant code that is hard to understand. That's why the backquote syntax was invented for instance. Before the backquote syntax, macros were difficult to write.

Say you wanted to transform the syntax (foo bar) into (let ((bar (whatever))) (do-something bar)).

You had to write a macro function which took the object (foo bar) as a single argument, analyzed it, and then constructed the output. Suppose we already have the BAR part of the (foo bar) from in a variable called SYM. Then we have to do something like:

   (list 'let (list (list sym '(whatever))) (list 'do-something bar))
   ;; I'm not going to bother to get this right

   `(let ((,sym (whatever))) (do-something ,sym))

Obviously, backquote templates have parentheses. But the notation itself isn't parentheses; it consists of the backtick syntax, and the commma operator for interpolating values. Also a ,@ operator for splicing lists. In some Lisp dialects, the backtick is transformed into a nested list object. For instance `(a b ,c) might turn into (quasiquote a b (unquote c)) "under the hood".

Lispers also invented destructuring: being able to write the macro with a kind of pattern match for the syntax, so that the elements of the to-be-transformed-form are pulled apart into separate variables.

Lisp is not a finished language. New ideas continue, and new surface syntax like the backquote is not off the table. Usually, Lisp programmers would, I think, prefer that such new syntax integrate into Lisp by not "disturbing" surrounding syntax by involving it in ambiguity. Something tidy and simple that has a big payoff is best.

Lisp programmers are not tone-deaf to notational advantages, and do not regard macros as the one and only way to reduce verbosity.

I'm conducting my own one-man research program into improving Lisp and have come up with some great new ideas.

I have a Lisp dialect which is quite ergonomic, leading to terse programs for everyday "data munging" tasks (and continuing to get better).

The direct translation of yours' to Clojure would be:

    (filter valid-person? (map init-person names))

    filter (\p -> isValid p) (map (\n -> Person n) names)

However you're right that the function names would probably be a little longer in practice, and I've edited my example to reflect that. (But they're not convenience functions, they just have longer names due to Haskell's more limited namespacing)

Often people assume the relatively simple syntax for s-expressions is the syntax of the programming language Lisp. It isn't. Lisp syntax is defined on top of s-expression.

In fact, while I still have issue reasoning about some aspects of functional programming when writing the code - it is magnitudes easier for me to read, maintain, modify, or extend the code.

    [person for person in (Person(name) for name in names) if person.isValid()]

    filter(lambda p: p.isValid, map(Person, names))

    persons = []
    for name in names:
        person = Person(name)
        if person.isValid():
            persons.append(person)

    persons = [Person(name) for name in names]
    valid_persons = [person for person in persons if person.isValid()]