News for hackers. Stuff that matters.
Re:Proposal: Go should have generics by NateDad on Thursday April 14, 2016 @12:08PM (#11497843)
I work on juju (https://github.com/juju/juju), which all told is about 1M LOC. In my almost 3 years on the project, I have not been bothered by lack of generics, basically at all (and I worked for 10 years in C# on projects that used a lot of generics, so it's not like I don't know what I'm missing).

Do we have 67 implementations of sort.Interface? Sure. Is that, by any stretch of the imagination, a significantly difficult part of my job? No.

Juju is a distributed application that supports running across thousands of machines on all the major clouds, on OSes including CentOS, Ubuntu, Windows, and OSX, on architectures including amd64, x86, PPC64EL, s390x... and stores data in a replicated mongoDB and uses RPC over websockets to talk between machines.

The difficult problems are all either intrinsic to the solution space (e.g. supporting different storage back ends for each cloud), or problems we brought on ourselves (what do you mean the unit tests have to spin up a full mongodb instance?).

Generics would not make our codebase significantly better, more maintainable, or easier to understand.

Re:Proposal: Go should have generics by atombender on Friday April 15, 2016 @01:18AM (#11502361)
A lot of people are focusing on your sorting comment, but I want to point out that your 1M LOC is already benefiting from generics. Slices, map and chan are polymorphic types, and functions like make(), len(), append(), copy(), etc. are generic.

The only distinction is that they're built in. But they are demonstrably useful, and your codebase would be worse without them.

It's hard to argue the hypothetical effect, good or worse, of generics on a codebase. You can't claim to know it for certain, seeing as the definition of generics isn't even completely defined. Maybe it wouldn't be hugely better. I don't think anyone is claiming that generics would be magical.

But well-designed generics (Haskell, Ocaml) can improve in other ways, some more or less subtle than others: You get less boilerplate, more code reuse, expressiveness. Go is pretty easy to read, but I find that the lack of expressiveness tends to obscure the meaning more than it reveals it. For example, you tend to end up with clunky C-style loops where a map would be much more concise, expressive and readable.

Another example: map types. I frequently find myself writing awkward code to copy maps, merge them, apply a common access pattern (such as [1], which coincidentally horrific and probably not performant) and so on. The ergonomics are terrible.

I disagree that it's not a "bother". It probably depend on what the project is; I also work on distributed apps, though.

[1] https://gist.github.com/atombender/01a07926115a17d3a56145adc...

Re:Proposal: Go should have generics by jdcarter on Thursday April 14, 2016 @12:31PM (#11498100)
I've had a very similar experience from about 3 years of writing Go. The lack of generics hardly affected me. When it did, it was dead simple to write the type-specific code and move on.

I've been working on a Java project recently, and by contrast, this code base abuses generics to an almost pathological level. I've also been burned by Java's runtime type erasure, and wow that leads to some nasty bugs. (That's not the fault of generics in principle, but Java's poor implementation of them.)

While I appreciate generics in theory, in practice I think Go is better as-is. Too often I've seen generics lead to complexity and abuse which greatly outweigh their utility.

Re:Proposal: Go should have generics by stcredzero on Thursday April 14, 2016 @01:13PM (#11498554)
Too often I've seen [X] lead to complexity and abuse which greatly outweigh their utility.

Programmer hubris is a problem. There was a widely acknowledged problem in Smalltalk with the overuse of #doesNotUnderstand: and other esoterica to do "clever" stuff which then makes it difficult for new programmers to debug and understand the system.

There is a reason why certain methodologies emphasize "the simplest thing that could possibly work." As a group, we programmers sometimes waste our own and other's time being too clever by half...an order of magnitude. (Is it any wonder why our estimates are often off by that much?)

Re:Proposal: Go should have generics by NateDad on Thursday April 14, 2016 @01:49PM (#11498910)
Yep. I remember spending a day trying to genericize a Rules Engine type system in C# and finally realized it was idiotic and made the code more complicated, and we'd probably only ever use the code exactly how it was now (i.e. not generic), and so I left it as-is (and to my knowledge, yes, it stayed exactly the same for forever).

I see the same tendencies in many programmers - "hey, I want to write this once and cover every single case that could ever come up"... even when they really only need to solve one specific problem, and making the solution more generic makes the code a lot more complicated than it needs to be for the specific problem you're solving.

Re:Proposal: Go should have generics by stcredzero on Thursday April 14, 2016 @02:40PM (#11499318)
One place where I worked had a rule that you couldn't invoke DRY until something had been repeated at least 3 times. Empiricism wins in the end when the goal is to seek truth.
Re:Proposal: Go should have generics by jdcarter on Thursday April 14, 2016 @04:19PM (#11500014)
Yes, thank you. When you use generics for a sortable collection class, that makes sense--it's a perfectly reasonable application of the technology. Problem is that "clever" programmers go hog-wild with generics and create APIs that resemble Greek tragedies where everybody dies in the end.

I think that the Go core team acknowledges this tendency, and that's why they resist adding features which tend towards complexity/abuse when those "clever" programmers get ahold of them. Sure, it means some minor pain when implementing the Sort API for your class. The unseen benefit is that you don't get the horrific abuse of generics that I'm seeing in my current Java job.

Re:Proposal: Go should have generics by d4rkph1b3r on Thursday April 14, 2016 @01:12PM (#11498552)
>Do we have 67 implementations of sort.Interface?

Hahaha. This has to be satire right?

>Generics would not make our codebase significantly better, more maintainable, or easier to understand.

Generics are literally a form of abstraction. You might as well be arguing that abstraction doesn't help. Why do you even have subtype polymorphism then? Why not just reimplement everything? That's not a significantly difficult part of your job as you said.

One of the best things about Go is it seems to be a strong signaler of the type of engineering team I avoided.

Re:Proposal: Go should have generics by stcredzero on Thursday April 14, 2016 @01:18PM (#11498612)
Generics are literally a form of abstraction

Is your unstated assumption then that all forms of abstraction must be used? If you've done substantive projects, you'll come to realize that abstractions have a cost, and that everything should be considered on a cost/benefit basis.

You might as well be arguing that abstraction doesn't help.

This is a black and white binary fallacy invoked to then create a straw man, which also seems to suggest that you haven't learned the importance of considering cost/benefit.

One of the best things about Go is it seems to be a strong signaler of the type of engineering team I avoided.

I would agree, this would seem to be a good signaler.

Re:Proposal: Go should have generics by d4rkph1b3r on Thursday April 14, 2016 @01:36PM (#11498774)
can you articulate the exact cost of adding generics? The benefits are profound, and the PL community has been doing research on it for the last forty some odd years.

Some of the benefits are opportunities for

* specialization

* reduction in boilerplate

* parametricity

* free theorems

* type classes

Objectively, a collections library written with generics and no subtyping will be much better and cleaner than a subtype based one.

The problem is, I've never heard generics argued against by someone who really understands generics. It's usually folks who got confused by them in java or really didn't dig into the theory behind them. An argument from ignorance isn't much of an argument.

Re:Proposal: Go should have generics by stcredzero on Thursday April 14, 2016 @02:34PM (#11499266)
Your comment is illustrative of a lack of awareness of context. You don't specify the context, but it seems like you're stuck in this academic/language theory mindset. From that standpoint, I rather like generics. It's clear to see how they can enable DRY if used judiciously. (Clear from even a freshman CS undergrad perspective.)

However, as a professional who gets paid to wrangle C++, I find the "Tragedy of the Commons" that results from every bright-eyed recent grad wanting to leave their mark on a system...tiresome. I recently fixed a bug caused by a small find-replace mistake, where a static_cast<int> was left out, resulting in an int() operator being generated by a confluence of preprocessor macros, inlined functions, and composed templates, where the call breaking in the stack trace was expressed nowhere in the code-base. It's one thing to DRY, but taking it one step too far to "Don't State Yourself In The First Place" is way too implicit. Abstractions have a cost, and sometimes the cost is epiphenomenal and gets paid years afterwards.

An argument from ignorance isn't much of an argument.

The decision of the Go team to not include generics is conservative and pragmatic. The context they consider is across an entire language community, and their decision is informed by observations made on code-bases at Google and elsewhere. How many 500k+ line code bases that have been around more than a decade have you worked on? I'm on something like my 4th. My conclusion from that is that we programmers as a group are mostly too anxious to be "clever" and biased towards doing too much when they evaluate the cost-benefit of "clever."

Do you have good data/experience on the epiphenomenal harm done by many, many "clever" programmers over years?

Re:Proposal: Go should have generics by d4rkph1b3r on Thursday April 14, 2016 @04:37PM (#11500140)
Ah, so you can't argue with me, so you're going to try the ol' appeal to authority ("i've worked on such big code bases that you'll never see").

You seem to assume I'm some naive recent grad. I've worked on more than a few 500k LOC applications. I'm a lead at a very large tech company (Fortune 50). The idea that '"clever"' programmers is a thing is incorrect.

There are good programmers and bad programmers. It doesn't matter if the lack of abstraction used by one type creates a monolithic mess of spaghetti code or if they use overly obscure attempts at abstraction. Bad code causes technical debt either way, and I've seen both done quite often.

Re:Proposal: Go should have generics by stcredzero on Thursday April 14, 2016 @04:53PM (#11500255)
Ah, so you can't argue with me, so you're going to try the ol' appeal to authority ("i've worked on such big code bases that you'll never see").

Uh, no. I asked if you understand such a context and if you have such data. Going by what you state, you do. A simple "yes" would have sufficed, and you could have left out the projection. Thank you for including the projection, as it is another valuable "signal."

The idea that '"clever"' programmers is a thing is incorrect.

There are good programmers and bad programmers.

Bad code causes technical debt either way, and I've seen both done quite often.

You do understand the use of quotes, then? "Clever" programming is thought to be clever by the perpetrator, but is actually bad programming and comprises technical debt. So either you are contradicting yourself above, or you are implying that "bad programmers" know they are bad, but do bad things anyways? This doesn't fit my experience.

Re:Proposal: Go should have generics by zimbatm on Thursday April 14, 2016 @02:51PM (#11499398)
Generics introduce more complexity in the type system which in turn makes the compiler slower.

Generics introduce more complexity for the reader of the code because it's another abstraction to understand.

It's debatable but when your brain is thinking about generics or context-switching because it has to wait on the compiler to finish, it's less time making progress on the actual thing that needs to be done.

Re:Proposal: Go should have generics by d4rkph1b3r on Thursday April 14, 2016 @04:52PM (#11500250)
The whole point of abstractions is that you don't have to worry about as much. Generics take away complexity, that's the whole point.
Re:Proposal: Go should have generics by zimbatm on Sunday April 17, 2016 @10:54AM (#11514693)
I suppose it depends on the level of understanding that you want from your code. Generics introduce another dimension which you have to think about when you want a good level of control on allocations for example. Different data types also have different optimizations available which could be missed when blindly relying on the generic algorithm (think sorting on a fixed set for example)
Re:Proposal: Go should have generics by stcredzero on Thursday April 14, 2016 @05:33PM (#11500512)
The whole point of abstractions is that you don't have to worry about as much. Generics take away complexity, that's the whole point.

Sure. Abstractions are perfect, and you never have to think about their costs. Generics always take away complexity. Gotcha.

Re:Proposal: Go should have generics by NateDad on Thursday April 14, 2016 @01:43PM (#11498846)
> >Do we have 67 implementations of sort.Interface?

> Hahaha. This has to be satire right?

Nope.

    /home/nate/src/github.com/juju/juju$ grep -r ") Less(" . | wc -l
    67
(granted, 10 are under the .git directory, so I guess 57)

But in any other language, we'd still have the same 57 definitions of how to sort a type.... we'd just have 3 fewer lines of boilerplate for each of those (which live off in the bottom of a file somewhere and will never ever need to change).

Re:Proposal: Go should have generics by pklausler on Thursday April 14, 2016 @01:53PM (#11498958)
> But in any other language, we'd still have the same 57 definitions of how to sort a type...

That claim turns out to not be the case.

Re:Proposal: Go should have generics by NateDad on Thursday April 14, 2016 @02:10PM (#11499103)
Aside from trivial types, like strings or integers, how does the language know how to sort a list of values, if you don't tell it how to?

Translate this into whatever language you like:

    Machine {
        Name string
        OS  string
        RAM int
    } 
You have 3 places that want to sort a list of machines, one by name, one by OS, and one by RAM. You're telling me there's a language that can do that without having to write some kind of code like this for each?

   sort(machines, key: Name)
I don't understand how that's possible, but I welcome your explanation.
Re:Proposal: Go should have generics by pklausler on Thursday April 14, 2016 @02:27PM (#11499233)
Sorting on all three fields in priority order is what I had in mind, and that's trivial in Haskell by adding "deriving(Ord)" to the data type definition and then just using the standard "sort :: Ord a => [a] -> [a]".

If you're always going to sort them based on some (other) relation between the fields, make your type a custom instance of Ord, e.g. "instance Ord Machine where compare = compare `on` name".

To sort the same type with distinct comparators, you'll obviously need to distinguish them, as in e.g. "osSort = sortBy (compare `on` os)".

Re:Proposal: Go should have generics by NateDad on Thursday April 14, 2016 @02:47PM (#11499372)
So... you will still need 57 spots in the code where you define how to sort a type.

Maybe my reference to sort.Interface is confusing people. When I say we have 57 implementations of sort.Interface, that's 57 different types and/or different ways of sorting one of those types. So, like, sorting Machine by Name would be one implementation, sorting Machine by Name then OS then RAM would be another implementation. You write an implementation of sort.Interface for every type, and for each way you would like to be able to sort it.

An implementation of sort.Interface just requires three methods:

    Len() int // return the length of the list
    Swap(i, j int) // swap items at indices i and j
    Less(i, j int) bool  // return true if list[i] is less than list[j]
It's the implementation in Less that determines the order.

That's not really so different than what you're describing in Haskell, it's just not part of the type, it's a new type that you convert the original type into, to pass into the sort.Sort() function (and because the underlying type is a slice, which is a glorified struct with a pointer to an array, that also sorts the original value).

Re:Proposal: Go should have generics by infogulch on Monday April 18, 2016 @06:11PM (#11523505)
It's possible to make one implementation for a type that supports multiple orderings, at the cost of another indirection [0]. This turns O(N*M) implementations for N types and M sorting orders into just O(N). (I'm not counting an inline anonymous function as a new implementation.)

In practice, it's rare to need to sort a slice more than one way.

[0]: https://gist.github.com/infogulch/5db15e5ae5cf073f1088033ba4...

Re:Proposal: Go should have generics by parenthephobia on Friday April 15, 2016 @11:04AM (#11504878)
There are no languages where you can sort machines by either name, OS, or RAM without in some way saying which to sort by.

That's a straw man: nobody is telling you that.

But, you are being told there are multiple languages where you can sort machines by their fields without having to write

    type byName []Machine
    type byOS []Machine
    type byRAM []Machine

    func (a byName) Len () int {
      return len(a)
    }

    func (a byOS) Len () int {
      return len(a)
    }

    func (a byRAM) Len () int {
      return len(a)
    }

    func (a byName) Swap(i, j int) {
      a[i], a[j] = a[j], a[i]
    }

    func (a byOS) Swap(i, j int) {
      a[i], a[j] = a[i], a[i]
    }

    func (a byRAM) Swap(i, j int) {
      a[i], a[j] = a[j], a[i]
    }

    func (a byName) Less(i, j int) bool {
      return a[i].Name < a[j].Name
    }

    func (a byOS) Less(i, j int) bool {
      return a[i].RAM < a[i].RAM
    }

    func (a byRAM) Less(i, j int) bool {
      return a[i].OS < a[j].OS
    }

    sort.Sort(byName(machines))
    sort.Sort(byOS(machines))
    sort.Sort(byRAM(machines))
For example, in Julia, you can just do

    sort(machines, by=m->m.Name)
    sort(machines, by=m->m.OS)
    sort(machines, by=m->m.RAM)
The equivalent is possible in any almost modern language. E.g. in C#

    machines.OrderBy(m=>m.Name)
    machines.OrderBy(m=>m.OS)
    machines.OrderBy(m=>m.RAM)
In Haskell

    sortBy (comparing Name) machines
    sortBy (comparing OS) machines
    sortBy (comparing RAM) machines
This is an extensively solved problem in modern programming languages.

As a bonus, the opportunity for making an error each time you want to sort a new type of array in a new way is reduced if you only have to write one line of code each time.

Re:Proposal: Go should have generics by jimbokun on Thursday April 14, 2016 @02:47PM (#11499373)
The canonical solution to this problem is to provide a function to perform the comparison, or to require the types implement a "Sortable" or "Comparable" interface.
Re:Proposal: Go should have generics by NateDad on Thursday April 14, 2016 @03:34PM (#11499697)
yes, which for 57 different types and/or comparison methods requires 57 different functions... which is basically the exact same thing you do in Go. It's just in go, you define a new type based on the original value, rather than just a function.
Re:Proposal: Go should have generics by junke on Thursday April 14, 2016 @02:50PM (#11499390)
Just to nitpick, that is 4 lines because you add a type too. Also, I noticed this in controller.go:

        // Unreachable based on the rules of there not being duplicate
	// environments of the same name for the same owner, but return false
	// instead of panicing.
	return false
Guess what, I worked with a sort function with the same kind of assumptions, but the implicit rules was broken: the "should never happen" path happened (names were not unique, after all). I found about that only after I wrote my own sort which was careful enough to check that the order was indeed total and when results diverged for some tests. I really disliked that because sorting was an important part in that tool (maybe it is not in yours).
Re:Proposal: Go should have generics by AnimalMuppet on Thursday April 14, 2016 @02:39PM (#11499310)
> >Generics would not make our codebase significantly better, more maintainable, or easier to understand.

> Generics are literally a form of abstraction. You might as well be arguing that abstraction doesn't help.

You missed one word: "significantly". Sure, abstractions help. That wasn't the claim. The claim was that, in a million lines, the lack of that particular way of doing abstractions did not significantly hurt.

Would it have helped? Sure. Would it have helped enough to matter "very much"? No (by NateDad's standards, which may differ from yours).

67 implementations of sort.Interface? Sure, I don't like it, but in a million lines, you've got much bigger things to worry about.

Re:Proposal: Go should have generics by eternalban on Thursday April 14, 2016 @01:29PM (#11498705)

   package pleasePutMeInYourVendorsDir

   import (
      "io"
       ...
   )
   ...

   func sureYouCan() {
      // ...
      io.EOF = io.ErrShortWrite
      // ...
   }