News for hackers. Stuff that matters.
Re:Stop writing lambda expressions in Python by another-cuppa on Thursday September 27, 2018 @05:37PM (#18088991)
Why does it need that, though? What's wrong with just naming the function?
Re:Stop writing lambda expressions in Python by zmmmmm on Thursday September 27, 2018 @08:43PM (#18090121)
It's not so much naming the function, but that doing so moves it outside the flow of the logic it is participating in. Especially when it's going to reference local context that is particularly confusing. There are definitely times when extracting out a function and naming it is the right thing to do. But there are an equal number of times when keeping all the local context together is better. It's probably hard to appreciate this unless you spend some time in languages where closures are seamlessly integrated like Groovy.
Re:Stop writing lambda expressions in Python by another-cuppa on Friday September 28, 2018 @02:00AM (#18091366)
I've used Lisp extensively and I've used closures extensively. In Python you can simply define the function on the line above where you plan to use it. I really don't see the need. Lisp needs lambda because you can't just randomly introduce new variables everywhere (you have to use flet, which actually is a lambda).
Re:Stop writing lambda expressions in Python by mlthoughts2018 on Thursday September 27, 2018 @11:09PM (#18090689)
I find it tremendously helpful to move the function out of the flow of logic so it is modularized. When I read scala or haskell code where as soon as you see map or filter or something you know you’re gonna get some crazy anonymous function to follow, I get sad because it’s a miserable and confusing way to write code instead of pulling the function out into a separate definition with documentation, and then having the map or filter part be extremely concise using only predefined functions.
Re:Stop writing lambda expressions in Python by jcelerier on Friday September 28, 2018 @02:37AM (#18091455)
> I get sad because it’s a miserable and confusing way to write code

you just have to visualize it as a tree

Re:Stop writing lambda expressions in Python by mlthoughts2018 on Friday September 28, 2018 @06:59AM (#18092408)
Yes, that is a confusing way to write and read code, rather than a linear flow, like flattening the tree by extracting functions into separate definitions.
Re:Stop writing lambda expressions in Python by jcelerier on Saturday September 29, 2018 @03:27AM (#18099141)
> Yes, that is a confusing way to write and read code, rather than a linear flow, like flattening the tree by extracting functions into separate definitions.

I don't understand. It's harder to flatten the tree: you have to unflatten it in your mind afterwards to understand what's happening. It's easier to just visualize, say, this lisp function as a tree directly.

    (defun good-enough-p (guess x)
      (format t "~% Guess =~7,4f     Guess^2 = ~7,4f    Error= ~7,4f" guess 
              (* guess guess) (abs (- (* guess guess) x)))
      (< (abs (- (* guess guess) x)) .001))
Re:Stop writing lambda expressions in Python by andrewstuart on Thursday September 27, 2018 @05:40PM (#18089015)
There's real readability and clarity to positioning an anonymous function right at the point in the code that it will be used.

When you name and define a function some other place and use it elsewhere, you have increased the overall complexity. What is this named function? Where is it used? Is it accessed by other bits of code? Should it be? All these questions come up when you see a named function. It is in fact less readable and requires more code to define a named function elsewhere and then call it. Inline terse anonymous functions are more straightforward, self explanatory and simple.

It is often the case that you know an anonymous function will only ever be used once, in this particular bit of code so it is much more clean, readable and understandable if the anonymous function lives right there.

Especially valuable in reducing the complexity of async programming.

It's hard to make a strong case just in words, but once you really understand the power of the fat arrow syntax for anonymous inline functions then you use it more and more and it becomes second nature and the programs you write have a completely different style, oriented towards neatly positioned inline anonymous functions everywhere that typically can be read and understood at a glance as you skip through the code.

There is simply no other way to write a function that absolutely cannot be called by some other bit of code. That's the problem with functions that are defined outside their usage context - you may know for sure that the function will only ever be called from this one point in your code, but if you have to declare it as a named function elsewhere then there is always the possibility that in the future someone will come along and make use of that function - it is more complex in the immediate term and leaves open the door for increasingly code complexity in the longer term. Terse inline anonymous functions such as the fat arrow syntax solve this perfectly. They are so clear and easy to understand that it's actually rather beautiful.

This is how terse you can make a JavaScript fat arrow function:

  x => x
It is a function that take a param of x and returns x. A more useful example might be:

  message => console.log('new message: ', message)

My primary languages are Python and JavaScript .... and coming first and primarily from Python, I know that the Python community feels it has a monopoly on readability. But having gained a fair amount of experience in JavaScript too, I can now say that things like inline anonymous functions and the extremely terse fat arrow syntax greatly increase readability even further. If Python had both then it would be even more readable, terse and powerful. Add destructuring on top of that and programming in Python would be almost a new experience.
Re:Stop writing lambda expressions in Python by mlthoughts2018 on Thursday September 27, 2018 @11:18PM (#18090732)
I actually think that positioning an anonymous function at the site where it’s used is extremely unreadable and unintuitive. I work in Scala and Haskell a lot so I see it all the time and I’m usually forced to write code that way to stick to local conventions, and I despise it.

It makes no sense at all to break up the wonderful conceptual flow of functional components like map, fmap, filters, folds, monadic operations, with sudden harsh definition of a function that takes your brain out of the context of what was flowing and into the context of what is the function.

I want to see something like

    val someStat = myData.groupBy(thingExtractor)
      .foldLeft(baselineValue)(statAccumulator)
I want the mental task of grokking thingExtractor, baselineValue or statAccumulator to be a wholly separate task from seeing what flows into the calculation of someStat. I want to look elsewhere in the code for those things, kind of like expanding or collapsing a block of text: keep it out of mind when it doesn’t matter.
Re:Stop writing lambda expressions in Python by danShumway on Friday September 28, 2018 @02:05PM (#18095563)
It comes down to a preference thing.

I want to read code like an essay. I don't want to have to jump to a function definition to figure out what `thingExtractor` does. To me, that's like taking a book and rearranging the chapters in alphabetical order. No, put them in chronological order. Don't define a function three "chapters" ago in your code and then expect me to remember it if you're only going to use it once. I wouldn't read Ikea instructions that way, why should I read code that way?

But again, personal preference. And you don't have to choose either/or -- I love that Javascript's anonymous and named functions are so similar because it means you can switch between the two styles with basically zero cognitive overhead.

I will say, having worked in large enterprise before, I have seen bugs come out of people not inlining code and instead attaching it to classes or leaving it otherwise accessible. Specifically, people will use functions that are intended to be method-specific in other places. Even if those functions are well-written and don't have side-effects (which is often not the case), the original author still doesn't know that other people are now depending on them as an API.

So later on the code gets refactored and those methods get changed or removed, and suddenly you have a broken build. That's something you can partially mitigate by making a function private (although not entirely, because I've also seen it happen in code within the same class/scope). Python doesn't have true privates, but you can also mitigate that problem by just using code reviews to force people to respect `_`. In practice, I often found that it was easier to make the function anonymous, so everyone knew 100% that it was only being used in exactly one place.

Re:Stop writing lambda expressions in Python by mlthoughts2018 on Friday September 28, 2018 @05:42PM (#18097294)
I also work in an enterprise situation and very many bugs that I deal with from legacy code and others’ code comes directly from in-lining anonymous functions.

Many bugs have to do with utilizing closures to access variables needed in the function body, which then make refactoring harder and make modifying unit tests harder.

This can be even worse in languages like Python where there is a distinction between early binding and late binding and you have to be aware if by closure you’re using a reference name that may be associated with different data during its lifetime, or a name whose value won’t change, because a change to the underlying value could make a difference in what is bound inside the anonymous function at different times when it’s called. The classic example is trying to define functions in a loop where functions use the loop variables by closure. Then being surprised when every one of the functions has a reference to only the final value of the loop variable.

Even in statically typed languages, this makes things much harder to reason about than they should be. On the other hand, making an anonymous function that accepts many arguments for all the data it would try to access by closure is stupid: those arguments need to be documented, and it’s just so much cleaner and maintainable to do that with a regular function definition, which also makes it much clearer what all the conditions are for calling the function.

What’s worse is that these things can be deeply welded into some coding context, like using a flatMap over some TypedPipe in Scala / Scalding, and can result in needing to make in-line functions that are hundreds of lines long with arbitrarily complex function bodies, which then become tied into assumptions about the runtime context you’re embedded in, and then nobody can figure out how to refactor it into a standalone function, so it just grows by attrition to an inline lambda over years and is extremely fragile. Change something seemingly unrelated about the outer context it’s defined in, and suddenly you get unexpected, cryptic compiler errors complaining something’s wrong with the TypedPipe, and you have to dig deeper to understand why it’s related to the anonymous function.

I would say many of the most serious closure-related bugs and bugs related to unrefactorable yet undocumented dependence on an enclosing context that I’ve seen have been largely a direct result of the programming style of in-lining anonymous functions inside functional programming constructs.

I sympathize with your claim of “not reading an essay” especially because people can be prone to try to use functional programming or overloading the Python data model and operator syntax with cutesy bullshit that they try to pass off as expressiveness.

But I think there’s a middle ground where you think about it not like an essay, but just basic modularity and separation of concerns, and write functions separately except when they are very short and really trivial.

Re:Stop writing lambda expressions in Python by danShumway on Friday September 28, 2018 @07:08PM (#18097747)
Hrm. I wonder if the differing bugs just comes from which one people are doing more.

I often have to fight to get people not to expose huge amounts of state whenever they build a module, so by far the most common bugs that I see are people being too loose with turning things that really shouldn't be modules into very fragile, cumbersome modules that depend on being used only in specific (undocumented) places with specific (undocumented) setup.

If I was in the opposite situation, and everyone I worked with already inlined code all the time, then probably most of the bugs I'd see would be related to people reusing variables, abusing hoisting by making spaghetti references to variables that are defined later in the function, etc... and in that case I could definitely see myself agreeing with you.

I have on occasion wanted the ability to define an anonymous function that didn't inherit variables from the scope that it was defined in. So I'll give you that - I would love for the ability to make an anonymous function that only has access to variables that are explicitly passed in. If I could isolate variables going into a closure as easily as I can isolate variables going out, I suspect a lot of the problems you're talking about would be easy to solve.

Re:Stop writing lambda expressions in Python by mlthoughts2018 on Friday September 28, 2018 @08:40PM (#18098070)
That is a good point that whichever approach displays the most bugs in a given team is likely to just be whatever is the most common approach for that team, by simple base rates.

I’m not sure how we could objectively decide if either of these two approaches is definitively better, but in the specific, restricted case of a framework like Scalding, I’d strongly wager that avoiding in-lining ends up better in the long run. Those cases also have little connection to the weak module design issue you brought up, since it’s usually a module with de facto map reduce boiler plate and then just a few isolated places with any actual implementation, and when those parts are expressed as huge in-lined anonymous functions inside Scalding data type wrappers, I know right away it’s a bad code smell.

Re:Stop writing lambda expressions in Python by danShumway on Friday September 28, 2018 @10:24PM (#18098375)
> I’m not sure how we could objectively decide if either of these two approaches is definitively better

The correct approach might just be the opposite of whatever you and your team's predilection is. In your case, you're saying that the teams you work with are using inline functions instead of following a restricted framework, in part because they're using languages that encourage them to just slap a bunch of nested code in instead of writing out the extra boilerplate.

Well, they probably already know to be careful about module design -- so if you encourage them to inline less code, odds are pretty good you won't suddenly wake up in the morning with a codebase with a hundred classes and a bunch of obscure private/public methods named `setupEntityExtractorForCollisionPart3`.

On the other hand, if your team is coming from a Java background and half of them are starting from the position that lambdas are just witchcraft, then it's probably not a bad idea to get them over that fear.

In a setting where everyone is inlining most of their code, probably the tests that are coming out are all integration tests, so... yeah, bias towards creating units so you can unit test. In the opposite situation, I'm actually just trying to get people to stop testing private methods and leaking implementation details into their tests. So I would love if people were testing with a little less granularity.

I know that my initial reaction to you listing off the problems you've run into with people building anonymous functions that couldn't be refactored was just, "yeah, but why the heck would anyone make that mistake? How hard is it to organize the variables in one function?" So I assume that other people might listen to my complaints about improper code reuse and think, "yeah, but why the heck would anyone ever just reuse a method in a class without checking the documentation first?" So my takeaway from that is, "different people struggle with different things."

Re:Stop writing lambda expressions in Python by y7 on Thursday September 27, 2018 @06:10PM (#18089249)
Python has a convention for "private" functions, which is to prepend "_" to the name.

> It's hard to make a strong case just in words, but once you really understand the power of the fat arrow syntax for anonymous inline functions then you use it more and more and it becomes second nature and the programs you write have a completely different style, oriented towards neatly positioned inline anonymous functions everywhere that typically can be read and understood at a glance as you skip through the code.

This is exactly why I think Python has such limited anonymous functions: because it does not want to promote that style.

> There is simply no other way to write a function that absolutely cannot be called by some other bit of code.

Python also believes that you do not need to defend against this possibility, as long as you make it clear to other developers that they should not call your "private function". This is also why the language allows monkey patching anything at will. If someone wants to ignore good sense, let them.

Re:Stop writing lambda expressions in Python by andrewstuart on Thursday September 27, 2018 @06:12PM (#18089259)
Private functions do not restrict the usage of that function to one point in the code. There's a world of difference between a coding convention - essentially a comment to say "please don't access this function from outside the current class" versus it simply being impossible to do so. And the "private" convention does not say "don't use this function at any other point except one", it says "please don't access this function from outside the current class", which is completely different to the point I make. Anonymous inline functions reduce lines of code and complexity.

>>because it does not want to promote that style.

Why would Python not want to promote a more readable, simple, powerful, reliable and maintainable programming style?

Can you reference anything to back this assertion?

Re:Stop writing lambda expressions in Python by anentropic on Friday September 28, 2018 @04:41PM (#18096879)
The parent comment about private naming convention is a red herring.

99% of the time in places where you might use a multi-line anonymous function you're already inside another function

So when you define your named function in Python, it is only available within the scope of the current method call and there's no danger of it being abused by other developers.

The lack of true private vars in python is a separate issue, and one of the best things about the language (because library authors inevitably get carried away and mark things as private which don't need to be - then you're stuck in copy & paste land... I did loads of ActionScript back in the day and this was super frustrating)

Re:Stop writing lambda expressions in Python by joshuamorton on Thursday September 27, 2018 @08:29PM (#18090057)
As a user of JavaScript and es6, fat arrows are only necessary if you haven't started using async and await syntax, and I found myself almost never using fat arrows after moving to async/await.

Indeed naming my functions resulted in more testable and clearer code.

Re:Stop writing lambda expressions in Python by andrewstuart on Thursday September 27, 2018 @09:10PM (#18090243)
Hmmmm. I don't see the relationship between async/await and fat arrows.

Can you explain more about how async/await obviates the need for fat arrows?

Re:Stop writing lambda expressions in Python by joshuamorton on Thursday September 27, 2018 @09:18PM (#18090279)
Just async is callback based.

You don't need callbacks with async/await. No callbacks, no need for fat arrow.

Re:Stop writing lambda expressions in Python by andrewstuart on Friday September 28, 2018 @01:03AM (#18091215)
Interesting idea.
Re:Stop writing lambda expressions in Python by danShumway on Friday September 28, 2018 @02:09PM (#18095598)
Getting rid of fat arrows is one thing, but I use async/await pretty much all the time, and I still use anonymous functions in my code.

  await (async function () {
    //Do a thing
  }());
I don't think async changes anything about how useful or problematic an anonymous function is.
Re:Stop writing lambda expressions in Python by joshuamorton on Friday September 28, 2018 @02:45PM (#18095920)
That feels like a weird pattern (for one because you can have async fat arrows), but also because in general I find

    await request_some_data()
much clearer than

    await (async () => {
        // just make the async request inline
    })()
Re:Stop writing lambda expressions in Python by danShumway on Friday September 28, 2018 @03:15PM (#18096206)
When arrow functions were first released, they didn't support the `async` keyword. I guess that wasn't fixed until ES2018? Maybe earlier - I think Babel would compile them pretty early on. Regardless, some people who adopted async patterns early got into the habit of avoiding arrow functions entirely, so it wouldn't be crazy to me to hear someone say, "I used arrow functions before I learned async."

Otherwise, this is just restating the same arguments people already made above.

There's no practical reason why

  (() => {
  
  }());

would be useful but

  await (async () => {
  
  }());
wouldn't be. The scenario you're talking about where async code removes the need to have an anonymous function only makes sense if you were only using anonymous functions for callbacks. In practice, that's not the only (or even primary) use case for people who advocate inlining code in Javascript. They're specifically advising against the pattern where single-use functions are given names and removed from program flow.
Re:Stop writing lambda expressions in Python by joshuamorton on Friday September 28, 2018 @03:21PM (#18096250)
Correct, and I fundamentally disagree with that pattern. If it's more than a single expression, it's doing something complex enough to deserve a name or to be mocked in tests, etc.
Re:Stop writing lambda expressions in Python by dragonwriter on Friday September 28, 2018 @10:07AM (#18093452)
While I can see async/await replacing a lot of the most common use in JS, anonymous functions have a lot more uses than that (passing a single use function to generic algorithm, like an ordering function to a sorting routine, is one of the more common other uses, across languages.)
Re:Stop writing lambda expressions in Python by another-cuppa on Friday September 28, 2018 @02:08AM (#18091386)
Come on. You've defined a function inside another function. It's very clear that you've done it so you can pass it to some higher level function. If someone comes along and starts using your inner function willy nilly in the outer function without anyone noticing then you've got much bigger problems that multiline lambdas are not going to help you with.
Re:Stop writing lambda expressions in Python by danShumway on Friday September 28, 2018 @02:18PM (#18095671)
> Python also believes that you do not need to defend against this possibility, as long as you make it clear to other developers that they should not call your "private function".

Python has apparently never worked in a large, corporate environment with a multinational team.

Yes, in theory you can just not call private methods. And in my own personal code I don't care much about restricting variable access, because I trust myself. However, it is a tremendous amount of work to get a company culture to start respecting privates if they aren't already. I've had so many phone calls trying to explain why "yes, you can technically call this function, but in reality you can't, and I don't care that your deadline is tomorrow, you still can't. And yes, I know that technically you're on a separate team and not under my jurisdiction, but you can't just remove me from the code review and call all of my library's private functions anyway."

It's taking the path of most resistance. True privates are very helpful on large teams.

Re:Stop writing lambda expressions in Python by another-cuppa on Friday September 28, 2018 @02:03AM (#18091378)
You do realise you can define a function inside another function in python, right?
Re:Stop writing lambda expressions in Python by BerislavLopac on Friday September 28, 2018 @07:25AM (#18092523)
So if I understand correctly, in Javascript a fat arrow is just the syntax sugar for:

    var x = function(x){ return x };
Re:Stop writing lambda expressions in Python by mlcdf on Friday September 28, 2018 @09:14AM (#18093019)
It also changes the behavior of this. [0]

[0] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

Re:Stop writing lambda expressions in Python by wwright on Friday September 28, 2018 @09:02AM (#18092949)
The two biggest differences iirc:

- it can never be named

- it inherits `this` lexically from its parent scope (which imo is the sane default)