News for hackers. Stuff that matters.
Re:Examples of Incorrect Abstractions in Other Languages by tines on Wednesday May 20, 2020 @01:46AM (#23243641)
Special cases like this make generic code very hard to write. If I'm writing a library that needs to handle user-provided data agnostic of type and I want to do a sum operation, now I have to add checks all over the place for whatever special cases python has.
Re:Examples of Incorrect Abstractions in Other Languages by wirrbel on Wednesday May 20, 2020 @03:12AM (#23244143)
I think never have I written generic code for summing up numbers AND concatenating strings. Even the Haskell people would agree I think. So the design problem is in overloading `+` for string and list concatenation in Python in the first place.

But I guess in terms of strings let's take a moment and remember the one truly good design decision that Python devs made about strings (which isn't usual for an imperative language). Strings are immutable by default.

Re:Examples of Incorrect Abstractions in Other Languages by lmm on Wednesday May 20, 2020 @05:51AM (#23245024)
> I think never have I written generic code for summing up numbers AND concatenating strings. Even the Haskell people would agree I think.

Erm, Haskell people write code that can do this all the time - you write stuff in terms of generic monoid and then it just does. I can even think of a concrete case where I used those two examples: I had essentially a report that looped over a bunch of rows and accumulated something as a secondary effect, and in one client's case that was a string explaining what had happened, and in another client's case that was a count of how many things had been processed.

Re:Examples of Incorrect Abstractions in Other Languages by still_grokking on Wednesday May 20, 2020 @08:33AM (#23246117)
So you've written (of course type safe) generic code that was in principle able to achieve the same effect as described in https://stackoverflow.com/questions/9032856/what-is-the-expl... ? ;-)

Batman!

Re:Examples of Incorrect Abstractions in Other Languages by rowanG077 on Wednesday May 20, 2020 @08:56AM (#23246294)
No. He was talking about monoids. The content in the link you are talking about aren't monoids.
Re:Examples of Incorrect Abstractions in Other Languages by still_grokking on Wednesday May 20, 2020 @09:17AM (#23246459)
Do the implementation details matter if you could achieve the same result?
Re:Examples of Incorrect Abstractions in Other Languages by rowanG077 on Wednesday May 20, 2020 @10:01AM (#23246855)
It's not about the implementation details. It's about that the things that are mentioned in the link make no logical sense. The addition operator allows values of different types to be added together and even worse the resulting values type isn't required to have the type of either operands.

It's not possible to write the generic code since there are basically no constraints. Generic code is all about constructs with certain invariants and use those invariants to create something new. Javascript addition basically has no invariants.

Re:Examples of Incorrect Abstractions in Other Languages by still_grokking on Wednesday May 20, 2020 @12:01PM (#23248449)
>It's not about the implementation details. It's about that the things that are mentioned in the link make no logical sense. The addition operator allows values of different types to be added together and even worse the resulting values type isn't required to have the type of either operands.

I would argue that that's not true.

JS is a "mono-typed" language and there is therefore only one type of things you can operate on and return as a result.

>It's not possible to write the generic code since there are basically no constraints. Generic code is all about constructs with certain invariants and use those invariants to create something new. Javascript addition basically has no invariants.

That's backwards to my point. I tried to point out that having generic code which can operate on (almost) arbitrary data structures mixing for example natural numbers or lists of arbitrary things will lead in the end to the same puzzling results as the "classical JS-Batman" example.

Don't get me wrong, generics are a great tool! But imho it's easy to "overstretch" their usage until a point where things become "so generic" that your functions can operate on "almost everything" but the results become as surprising as the alleged "JS insanities".

Something like the Monoid abstraction is incredibly powerful (most languages can't express such an abstraction in a sensible way even). Using such a powerful tool to abstract over some very concrete and banal things (that would be handled better separately as they're "not the same", no matter there may be a deep mathematical connection) is not a good architecture imo. If your code handling "concrete things" is written in a very generic and abstract way you won't have any "normal" tools left on your belt in case you would need to abstract over that stuff you've just written (at this point you could enter type-level programming or something that will achieve something similar but you will than end up with laughable complex code to handle in the end banal business logic).

https://en.wikipedia.org/wiki/Rule_of_least_power

Re:Examples of Incorrect Abstractions in Other Languages by lmm on Thursday May 21, 2020 @09:33AM (#23258084)
> JS is a "mono-typed" language and there is therefore only one type of things you can operate on and return as a result.

Well, in that case programmers should not be surprised when they subtract an integer from a string and get NaN. Apparently they are are surprised though. I'd prefer my language not let me subtract an integer from a string, since if I ever attempt to subtract an integer from a string that's almost certainly an error.

> That's backwards to my point. I tried to point out that having generic code which can operate on (almost) arbitrary data structures mixing for example natural numbers or lists of arbitrary things will lead in the end to the same puzzling results as the "classical JS-Batman" example.

But that's not true. Nothing you do with a monoid will create such a puzzling result, because summing a list via a monoid is - by definition - equivalent to summing it by calling + pairwise. (Of course, if you have a type for which + behaves in a confusing way, then this might be confusing - but it's not the monoid that's being confusing there).

If you want to claim monoid-based code can be confusing, try using some actual examples of monoid-based code, rather than code in a different language that isn't using monoids and wouldn't compile if it did.

> If your code handling "concrete things" is written in a very generic and abstract way you won't have any "normal" tools left on your belt in case you would need to abstract over that stuff you've just written (at this point you could enter type-level programming or something that will achieve something similar but you will than end up with laughable complex code to handle in the end banal business logic).

This is completely wrong. If you write your code generically it becomes easier to abstract over, not harder. For example, if you write a sort function that operates generically from lists and separate that from your comparison function instead of writing separate functions for sorting lists of integers and lists of strings, it's much easier to then make further abstractions over your sorting.

> https://en.wikipedia.org/wiki/Rule_of_least_power

Again you're getting it backwards. Code written in terms of monoids is less powerful than code written in terms of a specific datatype, which means there are far fewer ways to get it wrong.

Re:Examples of Incorrect Abstractions in Other Languages by rowanG077 on Wednesday May 20, 2020 @02:47PM (#23250325)
> JS is a "mono-typed" language and there is therefore only one type of things you can operate on and return as a result.

Javascript has a type system that is enforced at run-time.

> That's backwards to my point. I tried to point out that having generic code which can operate on (almost) arbitrary data structures mixing for example natural numbers or lists of arbitrary things will lead in the end to the same puzzling results as the "classical JS-Batman" example.

I don't agree. The entire point of the link you posted was the insanity of the semantic of the addition operations in Javascript. Monoids are very simple and well defined. The javascript insanity would need pages upon pages of explanations because basically everything is a special case. In contrast monoids can be fully explained in a two sentences.

Re:Examples of Incorrect Abstractions in Other Languages by still_grokking on Wednesday May 20, 2020 @06:59PM (#23253007)
>> JS is a "mono-typed" language and there is therefore only one type of things you can operate on and return as a result.

>Javascript has a type system that is enforced at run-time.

Nobody questions that JS has "runtime type-checks". But that's irrelevant to my point. I've referenced of course this here:

https://existentialtype.wordpress.com/2011/03/19/dynamic-lan...

>Monoids are very simple and well defined.

Sure. Just define an instance for a type that models a JS runtime-object, let's see what happens… ;-)

But I see, my point is still not explained well: What I'm trying to say is that using abstractions that are so powerful that the resulting code can operate on (almost) arbitrary "unrelated" things using that code will lead to the same kind of puzzling outcomes as "JS Watman".

(And like I said, a deep mathematical relation doesn't make things "related" in a practical sense automatically. For example when it comes to "correct" domain modeling of business cases where you explicitly want to treat different parts of your system separately; as this results in a less "entangled" architecture. Gluing together parts that are unrelated through clever code on the other hands side creates maintenance hell where everything depends on everything, but you can't even see what depends on what as everything is generic; that's the opposite of good software engineering, imho. To throw a few phrases: "Divide and conquer!" "Use the right tool for the job!" "Use the least powerful tool that gets the job done well!").

Re:Examples of Incorrect Abstractions in Other Languages by Avi-D-coder on Wednesday May 20, 2020 @04:44AM (#23244698)
We don't just write generics over types of strings ([Char], Text, etc..) we write over Semigroups and Monoids. Concatenation with <> is standard practice.
Re:Examples of Incorrect Abstractions in Other Languages by noelwelsh on Wednesday May 20, 2020 @04:36AM (#23244651)
It's very common in data processing to sum up stuff (e.g. unique visitors, total visitors, total revenue, etc.) To handle this you typically want to write a generic library that makes summing up stuff easy. I can't think of an example where you'd want to add up strings (more likely lists of strings) but it is definitely the case for other data structures (sets, vectors, lists, numbers, etc.)
Re:Examples of Incorrect Abstractions in Other Languages by kragen on Wednesday May 20, 2020 @05:19AM (#23244852)
Python sum() can handle summing up other data structures, although, as with foldl, you may have to give it an initial accumulator argument:

    >>> import numpy as np
    >>> xs = np.arange(9).reshape((3,3))
    >>> xs
    array([[0, 1, 2],
           [3, 4, 5],
           [6, 7, 8]])
    >>> sum(xs)  # builtin sum, not numpy sum
    array([ 9, 12, 15])
    >>> sum([[3, 11], [5]], [])
    [3, 11, 5]
    >>> sum([(3, 11), (5,)], ())
    (3, 11, 5)
However, that doesn't work for strings; there's a special-case check for that.
Re:Examples of Incorrect Abstractions in Other Languages by tgv on Wednesday May 20, 2020 @02:43AM (#23243962)
There's no language in which you would be able to write a function for arbitrary data structures using sum(), and get a reasonable outcome.

The problem with Python is that it doesn't have typing that can express "I can sum".

Re:Examples of Incorrect Abstractions in Other Languages by anentropic on Wednesday May 20, 2020 @05:44AM (#23244981)
Actually, I believe this is expressible in Python's type system:

    # Python 3.8
    from typing import Protocol, TypeVar

    T = TypeVar('T')

    class Summable(Protocol[T]):
        def __add__(self, other: T) -> T: ...

In Python 3.4-3.7 you'd need to install typing-extensions library:

    from typing_extensions import Protocol
Re:Examples of Incorrect Abstractions in Other Languages by still_grokking on Wednesday May 20, 2020 @09:28AM (#23246551)
Interesting. It seems that "every second language" gets "type-classes" these days…

C++ added such feature, this above looks like a Python version, Rust has it, Scala had them as one of the first "OOP" languages.

It's nice to see that Haskell pioneered a very useful and now spreading language feature!

Re:Examples of Incorrect Abstractions in Other Languages by sacado2 on Wednesday May 20, 2020 @06:05PM (#23252504)
Ada had them before Haskell, and I'm not even sure it was the first language with them.
Re:Examples of Incorrect Abstractions in Other Languages by naasking on Wednesday May 20, 2020 @08:01AM (#23245817)
> There's no language in which you would be able to write a function for arbitrary data structures using sum(), and get a reasonable outcome.

Computing the number of bytes used by a data type seems perfectly reasonable use of sum in that context. This is also achievable in many languages. All 'sum' needs is a map from a value to a number, which can have many sensible meanings even for arbitrary data types.

Re:Examples of Incorrect Abstractions in Other Languages by dragonwriter on Wednesday May 20, 2020 @11:07AM (#23247740)
And that works in python to, even if the source type is strings:

  sum(map(mappingFunction, iterable))
works fine as long as mappingFunction maps the elements of an iterable to numbers.
Re:Examples of Incorrect Abstractions in Other Languages by throwaway_pdp09 on Wednesday May 20, 2020 @04:41AM (#23244680)
I recently got bitten by trying to do exactly this, concat strings with sum, and spent a while trying to find out if I was being stupid (in this case, no). It damn well should work.

This works

  from functools import reduce
  nums = [1,2,3]
  def add(p, q): return p + q 
  print(reduce(add, nums))
  strs = ["how", "are", "you"]
  print(reduce(add, strs))
so why not that?
Re:Examples of Incorrect Abstractions in Other Languages by rozab on Wednesday May 20, 2020 @05:53AM (#23245039)
Python is a language that seems to make a lot of sense at first but gets a bit weird when you dig into things. The + operator particularly is a bit funny; I recently came across a case where x+=y was not equivalent to x=x+y... Though I can't figure out what it was now.
Re:Examples of Incorrect Abstractions in Other Languages by pansa2 on Wednesday May 20, 2020 @07:47AM (#23245726)
> a case where x+=y was not equivalent to x=x+y

This is true for lists in Python. `a = a + b` creates a new list, but `a += b` works in-place:

    >>> a = [1, 2]
    >>> b = a
    >>> a = a + [3]
    >>> print(a, b)
    [1, 2, 3] [1, 2]

    >>> a = [1, 2]
    >>> b = a
    >>> a += [3]
    >>> print(a, b)
    [1, 2, 3] [1, 2, 3]
There are other issues with the `+=`-style operators as well - IMO it was a mistake to have `a += b` be anything other than a shorthand for `a = a + b`.
Re:Examples of Incorrect Abstractions in Other Languages by tsegratis on Wednesday May 20, 2020 @07:24AM (#23245578)
Agreed, but seems haskell also fails to keep commutativity here tho?

Concatenation is not addition in my opinion

I've often wanted something like bits++bits concatenation to work in the same way for numbers as for strings

Re:Examples of Incorrect Abstractions in Other Languages by throwaway_pdp09 on Wednesday May 20, 2020 @08:08AM (#23245871)
> Concatenation is not addition in my opinion

For python, syntactically it is. In a general sense, an operation is what you define it to be. Just my opinion.

Re:Examples of Incorrect Abstractions in Other Languages by theelous3 on Wednesday May 20, 2020 @07:28AM (#23245598)
You were likely doing something with threads and failed to lock `x`, which I would describe as an issue that should be expected. It doesn't really have to do with `+=` so much as it has to do with being aware of race conditions on variable unprotected by a mutex.
Re:Examples of Incorrect Abstractions in Other Languages by G4E on Wednesday May 20, 2020 @07:40AM (#23245686)
Python's list sum are funny like that.

x+=y is equivalent to x.append(y)

x = x + y creates a new list, then assign it to x (highly inefficient in a loop)

Re:Examples of Incorrect Abstractions in Other Languages by dragonwriter on Wednesday May 20, 2020 @11:21AM (#23247948)
> I recently got bitten by trying to do exactly this, concat strings with sum, and spent a while trying to find out if I was being stupid (

Perhaps not stupid, but ignorant, and not just of the special casing in sum which exist because of a footgun.

sum is basically a readability alternative for functools.reduce for adding numbers, and it's special cased for strings because though it would superficially work for strings otherwise, the performance of repeated string concatenations is pathological in python (as in many garbage-collected languages where it forces many allocations), so a special cases was added which points people to the method to produce exactly the same result without the pathological performance issue.

If you are doing something more complex than numbers with heterogenous data that includes strings in a reduction by the addition operator so that the join you'd use with homogeneous collections of strings isn't what you want (I'm not sure that hypothetical is ever going to happen), use functools.reduce, which for readability you probably should be using in preference to sum() if you aren't doing a mathematical sum anyway.

Re:Examples of Incorrect Abstractions in Other Languages by throwaway_pdp09 on Wednesday May 20, 2020 @11:45AM (#23248249)
Like you, I know the O(n^2) bahaviour that would occur with a naive repeated concatenation, and like you I would know how to deal with it. You'd use "".join(...) for the special case it was strings and therefore so would I. This needn't be a footgun of any sort, and don't assume I (or others) are ignorant.
Re:Examples of Incorrect Abstractions in Other Languages by dragonwriter on Wednesday May 20, 2020 @11:03AM (#23247701)
> If I'm writing a library that needs to handle user-provided data agnostic of type and I want to do a sum operation, now I have to add checks all over the place for whatever special cases python has.

No, you don't, because: (1) if your idea of a sum is just reducing with “+”, you use functools.reduce; and (2) if your idea of sum is a mathematical sum, you use the very slight shortcut of the sum() built-in function, which is (barring things like the runtime type checking) equivalent in result to:

  def sum(iterable, start=0): 
    return functools.reduce(lambda x,y: x+y, iterable, start)
Re:Examples of Incorrect Abstractions in Other Languages by sacado2 on Wednesday May 20, 2020 @06:02PM (#23252470)
What's a sum operation though? Is the sum of a, b and c the same thing as the sum of b, c and a?
Re:Examples of Incorrect Abstractions in Other Languages by Veedrac on Wednesday May 20, 2020 @08:46AM (#23246222)
You needed those checks anyway. Python is just making sure you don't forget.