News for hackers. Stuff that matters.
Re:Records Come to Java by djsumdog on Thursday January 16, 2020 @04:51PM (#22069382)
Java finally gets what's been available in Scala (case classes), C# (structs or maybe properties), Kotlin (data classes) and others for a very long time .. so long we already have tools like Immutables and Lombok to get past this really dumb limitation in Java.

I was really surprised to learn that the Java compiler/VM, when it sees the patterns "somevar" and functions "getSomeVar" and "setSomeVar" doing purely getter/setter stuff, removes the actual function calls and silently changes them in the background to field lookups for speed.

If you attach a debugger or memory profiler, is changes them back to functions and slows them down. I'm not sure if this is still true; saw an example of this at a meetup back in the Java 8 days.

Re:Records Come to Java by pron on Thursday January 16, 2020 @05:16PM (#22069614)
> Java finally gets what's been available in ... and others for a very long time

You seem to think that the goal is to get as many features as possible, as quickly as possible. Maybe some languages have that goal, but Java's design philosophy -- from its inception [1] -- has been to be a conservative language that only adopts features that have become familiar to programmers and stood the test of time in providing a good cost/benefit. Java hasn't always done this, but that's the aspiration. Worked out pretty well, I think.

Given Java's design philosophy, the far more interesting question is which features of those more adventurous languages Java does not intend to adopt.

> Java compiler/VM, when it sees the patterns "somevar" and functions "getSomeVar" and "setSomeVar" doing purely getter/setter stuff, removes the actual function calls and silently changes them in the background to field lookups for speed. If you attach a debugger or memory profiler, is changes them back to functions and slows them down.

It's true, and the optimizing JIT compiler does much more than just that. It performs deep speculative optimizations, like inlining virtual calls and removing untaken branches, reverting them (using a process called deoptimization) either for debugging or when it detects its assumptions are false. What you're referring to is just an instance of inlining.

[1]: https://www.win.tue.nl/~evink/education/avp/pdf/feel-of-java...

Re:Records Come to Java by Cladode on Thursday January 16, 2020 @05:33PM (#22069743)

   Worked out pretty well, I think.
Au contraire mon ami!

ML was better than most of its successors, including Java, which held programming back. Milner's ML avoided some of Java's glaring mistakes (e.g. exception specifications, lack of generics, non-uniform treatment of types, lack of higher-order functions, lack of pattern matching) from the start. Xavier Leroy showed how efficiently to compile ML in the 1990s.

The main thing ML lacked (w.r.t sequential computing), was a good integration with OO and HKTs. That was solved by Scala.

Re:Records Come to Java by pron on Thursday January 16, 2020 @05:38PM (#22069790)
I meant that it's worked out pretty well for Java. ML -- one of my favorite languages, alongside Java and a couple of others -- has also managed to avoid being used by anyone, really. But look at Java now: it has generics and higher-order functions, and it's getting a uniform treatment of types (Project Valhalla), and pattern matching [1]. And so while ML introduced those features in a research context, Java is the language that has made and will make the world actually use them. Being ahead of your time is often just as much of a mistake as being behind (as far as product design goes).

As to the question of "holding programming back," it is yet to be seen how much of an impact these features have in practice. Same goes for what ML "lacked." I personally like some features more and some less, as do you, but I don't think anyone knows which programming language features really push programming forward.

[1]: https://cr.openjdk.java.net/~briangoetz/amber/pattern-match....

Re:Records Come to Java by AnimalMuppet on Thursday January 16, 2020 @07:03PM (#22070475)
"Holding programming back" means that it would have advanced in the absence of Java. What language would have advanced it? Cladote seems to assume that ML would have, which would have been actually used if Java hadn't taken over the world. That seems unsupported by evidence. Overly optimistic, as well.

And if Cladote's answer, ML, would not have taken over, then did Java hold back anything? No. Was it better than, say, C++? For many uses, yes.

"We would have ruled the world if it wasn't for you meddling kids!" is wishful thinking. If ML was so good, it should have won anyway. Instead, "has managed to avoid being used by anyone" is a pretty fair assessment.

Re:Records Come to Java by psychoslave on Friday January 17, 2020 @12:10AM (#22072064)
>If ML was so good, it should have won anyway.

Experience proves otherwise, to the best of my knoledge. For gaining wide propagation in a niche, whether of habitat or market, being the paragon of what can be conceived is rarely the way to optimize success. There are many other factors, and technical merit is lightly pondered.

That's true for life, where there are numerous examples where a function is filled by an organ that evolved in a counter manner of what an engineer sense of elegance/parsimony would provide.

That's also something that happens for technical products, where early proliferation in the market seems to offer a far more successful advantage than excellence of the product.

That said, I never had the opportunity to touch ML, and only already heard about it through my personal research on the roots of Coq that I studied during my CS courses in University. I didn't touched much of Java since I left University. Last year I worked on an SDK provided for several languages, including Java, so it gave me the opportunity to look at how it evolved, and play with its integration of functional paradigm. That still requires far too much boilerplate code to my gust, but is bit less frustrating in how you can structure your code.

Re:Records Come to Java by pron on Friday January 17, 2020 @06:40AM (#22073825)
> Experience proves otherwise, to the best of my knowledge.

It's very hard to say that it does in programming, at least. Even though we could easily come up with value metrics for programming languages, we don't really know what they are for any particular language. However, that, in itself tells us something. The software industry is an environment with strong selective pressures. That big difference in "quality" (i.e. some metric with a bottom-line impact) will go unnoticed is hard to believe. It doesn't make sense from a theoretical perspective -- adaptive traits should be detected in a selective environment, and it also doesn't fit with observed reality. We observe that technologies that truly provide an adaptive benefit are adopted at a pace commensurate with their relative adaptability; often practically overnight.

So we have reasons to believe that it is not true that "better" languages lose, at least not ones that have some measure of adoption at all.

Re:Records Come to Java by ernst_klim on Friday January 17, 2020 @08:16AM (#22074302)
> strong selective pressures

The opposite is true. Unlike in electronics, construction, shipbuilding, you can write a shitty software and patch the hell out of it after release most of the time with the few exceptions.

Hence the only thing that matters is how cheap and replaceable the developers are. Unlike pharma or architecture, where errors are always costly.

Java was successful because is was 1) pushed by big enterprise company, 2) familiar to people with C,C++,perl background and 3) overall easy to write at the expense of verbosity and bloated codebases (who cares if you could just hire too many of the cheap devs who would write as many classes as needed).

Re:Records Come to Java by pron on Friday January 17, 2020 @10:27AM (#22075382)
> The opposite is true. Unlike in electronics, construction, shipbuilding, you can write a shitty software and patch the hell out of it after release most of the time with the few exceptions.

What happens in software is what the market wants. You cannot claim that something is better if it is better at doing something that the industry doesn't want (and even that you can't show). It's like the Betamax vs. VHS: Betamax was better at some metric that few if anyone cared about, but VHS was better at what people actually cared about.

> Java was successful because

Java was (and is) successful because it was (and is) a very good product with good backing. At the time, other companies, like Microsoft, Borland, IBM and others that few now remember also heavily pushed other products, and they didn't do as well. I am not claiming that Java is the best possible programming environment, but it is pretty funny to claim that some specific platform/language is better with absolutely nothing to support that claim.

Re:Records Come to Java by ernst_klim on Friday January 17, 2020 @11:30AM (#22076071)
> What happens in software is what the market wants.

Sure thing, I'm just emphasizing that it's a more of a social-enterprise-management issue, than a software quality.

> Java was (and is) successful because it was (and is) a very good product with good backing.

Or just because it's Sun/Oracle/baked into android/symbian. Who knows? IMHO it's more of a "nobody was fired for using java" than "java is a good language". It's a language for managers, not for programmers.

> Microsoft, Borland, IBM

MS does pretty well with CLR, for another reason, though. There were no product at the moment which could compete with Java in terms of marketing, money, support etc.

Borland had nothing for serious enterprise, MS had C++ and VB only back then, regarding IBM I don't even know what you mean, Smalltalk?

Re:Records Come to Java by pron on Friday January 17, 2020 @12:15PM (#22076669)
> Or just because it's Sun/Oracle/baked into android/symbian.

Fact is, in the 25 years since its introduction virtually no one has built a platform with a similar combination of performance, productivity and observability. Java has very little competition in the "serious software" space, although platforms/languages like Node, Python and Go -- that suffer in some or all of the factors I mentioned -- get used in microservices, that are eating into "serious software".

> Borland had nothing for serious enterprise, MS had C++ and VB only back then, regarding IBM I don't even know what you mean, Smalltalk?

I meant Borland with Delphi (which was quite heavily used for a while, even after Java came out), MS with Visual C++, VB and Fox Pro, and IBM with Smalltalk.

Re:Records Come to Java by Cladode on Friday January 17, 2020 @02:15PM (#22078076)
I agree with what you write.

But I also believe the counterfactual that, had Sun/Oracle been based their language upon (core) ML rather than Java (with suitable pragmatic additions like code loading and been able to convince programmers to use it), then that language would be even more dominant and performant today. Many of Java's mistakes and detours would have been avoided from Day 1.

Can I prove this? No. But the fact that Java and essentially all other successful languages have gradually been adapting essentially all of ML (sans modules), is evidence. It means that language designers like G Bierman, A Hejlsberg, Odersky et al, too have gone Milner's way.

Re:Records Come to Java by pron on Friday January 17, 2020 @02:42PM (#22078390)
> then that language would be even more dominant and performant today

I think you should watch the first 20 minutes of this talk: https://youtu.be/Dq2WQuWVrgQ

Java's early designers realised (or believed) that the features with most bang-for-the buck weren't linguistic, but were: GC, dynamic linking, and performance with observability. Linguistic features, overall, were a detractor from those, because people were hesitant to use them, and so they missed out on the big benefits. Java decided to provide these most important features wrapped in a language that was non-threatening at the time; James Gosling called it "a wolf in sheep's clothing." In other words, a familiar language was a very central part of the strategy. You say you believe it would have worked with a non-familiar one, well, we can never know, but if it hadn't, then much none of the other, more important features would have mattered.

> But the fact that Java and essentially all other successful languages have gradually been adapting essentially all of ML (sans modules), is evidence

As I said in another comment, it's not that all successful languages are adopting ML's features but some, and those ones do it because they influence one another and they knowingly borrow from ML because they like it. In other words, the designers of some influential languages like ML. Is it a coincidence? I don't know, but I'm hesitant to say that it's evidence that ML was "right." I, too, personally like ML, but I wouldn't claim its design leads to better outcomes than others. Some successful language clearly do not adopt ML's features -- e.g. all the untyped ones and Go.

BTW, Java's objects are a different take on modules. If you've seen 1ML, that's almost an OO language.

Re:Records Come to Java by Cladode on Friday January 17, 2020 @03:35PM (#22078953)
I understand the network effect, I understand that most decisions on PL are not based on technical reasons. Most non-experts mostly worry about syntax, as per P Wadler:

   "The simplest thing to do is a significant 
   change to the semantics. People won't argue 
   the way they do about syntax."
ML had GC, it would have been easy to add dynamic linking, and X Leroy made Ocaml fast, with a tiny fraction of the resources that have been poured into the JVM. Admittedly, this was a major breakthrough that was unexpected. I reckon the main problem was getting OO to work properly with generics: type inference becomes a problem and you have to worry about co/contravariance.

A Hejlsberg certainly wasn't on the ML train in the past. For a start he didn't think generics are a good idea (he told me personaly). Typescript shows that he changed his mind! Go explicitly rejected generics at the start, but Go-2 is scheduled to get them.

Module systems are not such a big deal in my exerience, I don't think ML's was a full success.

Re:Records Come to Java by Cladode on Thursday January 16, 2020 @07:30PM (#22070630)
If we collapse the two concepts of

- best possible X

- most popular X

are we loosing possibly interesting perspectives on X?

Re:Records Come to Java by pron on Thursday January 16, 2020 @07:38PM (#22070688)
I don't think AnimalMuppet is confusing most popular with best. But as you don't have the data required to conclude which language is better by any meaningful metric, anyway, then your claim is just wishful thinking. If ML had won and Java lost, one could have just as easily said that ML was holding programming back from some imagined benefit Java would have brought. I think that it is you who are confusing "an interesting perspective" with a value judgment. Yes, ML had pattern matching before Java, while Java had pervasive dynamic linking and dynamic code loading. Those are simple facts and perhaps "an interesting perspective," but how do you decide which is better?
Re:Records Come to Java by Cladode on Thursday January 16, 2020 @07:55PM (#22070788)
Nobody has credible data enabling reproducible comparison of programming languages. The big lacuna of PL research ...

If popularity mattered ... PHP, JavaScript etc.

I find the evolution of all successful programming languages eventually to evolve into an ML-like core fascinating.

Re:Records Come to Java by pron on Thursday January 16, 2020 @08:26PM (#22070993)
> Nobody has credible data enabling reproducible comparison of programming languages.

Right, so as "better" and "best" are unknown, we should stop using those terms. I like Java and I like ML, but I don't know that one of them is better than the other, and if so which.

> I find the evolution of all successful programming languages eventually to evolve into an ML-like core fascinating.

Some of the successful typed languages, sure: Java, C++, C#, but the three heavily influence one another, and at least Java's and C#'s designers were aware of ML. So I don't think that, to the extent this is true, it's some "natural" draw, but rather a direct or indirect influence. On the other hand, you have Go, which is also quite successful (admittedly in a lower tier of success), and it doesn't seem to be converging on something ML-like, though (although it is influenced by another Milner idea), and neither is C.

Re:Records Come to Java by AnimalMuppet on Thursday January 16, 2020 @10:39PM (#22071649)
Absolutely. Popularity is absolutely fallible.

But I don't take claims of "X language was better and should have won" claims very seriously, partly because it's claimed by advocates of so many different languages. Should ML have won, or should Pascal/Modula2/Oberon? I've seen that claim, here, just from different people and for different reasons.

But I also don't take such claims very seriously because, while popularity can be wrong, so can unpopularity. That one guy spouting off about how everyone else is wrong and he's right? Sometimes he is right - but not often. More often, that person who thinks they have the right answer is seeing less than the full picture. I mean, how many people use Java? Several million? Would ML really have been a better answer for the majority of those several million people?

I react the same in government, by the way. When someone says, "I know what you should want, and I'm going to make it happen, even if it isn't what you actually want", well, I trust the people to know what fits them better than someone who isn't in their shoes.

Re:Records Come to Java by agumonkey on Thursday January 16, 2020 @06:52PM (#22070389)
Java inertia was also a side effect of its market. It was big corps which were used to slow and bloat. But Oracle seems to have changed to follow the more hectic pace of web 2.11 era.
Re:Records Come to Java by tomcam on Thursday January 16, 2020 @05:25PM (#22069680)
> deep speculative optimizations

my new blog title

Re:Records Come to Java by anthonybsd on Thursday January 16, 2020 @05:51PM (#22069922)
>Java finally gets what's been available in Scala (case classes), C# (structs or maybe properties), Kotlin (data classes) and others for a very long time .. so long we already have tools like Immutables and Lombok to get past this really dumb limitation in Java.

All the ones you named are fairly new languages, and C# does not have anything even remotely like records (structs are value types, and properties are fields) so what's your point? Java historically has been very conservative with implementing things because they usually try to get it right (generics non-withstanding). For that matter C# rushed into a few things too and screwed the pooch badly on them.

Re:Records Come to Java by nightski on Thursday January 16, 2020 @06:01PM (#22070012)
I'd be curious which features you think C# messed up on. I have been using C# professionally since 2000 in it's first beta and can't think of a newer feature that would meet this criteria?

That said you are correct about records. Although it is getting them also.

Re:Records Come to Java by anthonybsd on Thursday January 16, 2020 @06:13PM (#22070105)
The one that annoys me the most in C# on a day-to-day basis is Nullable<T>. Java took their time with Optional<T> but once it was released it was great. Can be used in lambdas, all libraries are modified and aware of it etc. C# Nullable<T> is near useless because it only works on value types. I.e., you know, the types that are least likely to generate NullReferenceException for you.
Re:Records Come to Java by wvenable on Thursday January 16, 2020 @06:32PM (#22070254)
In both C# and Java, reference types are nullable. Nullable<T> exists in C# to provide for nullable value types. It's not comparable to Optional<T> as that's an entirely different feature.
Re:Records Come to Java by anthonybsd on Thursday January 16, 2020 @07:03PM (#22070473)
They aren’t “entirely different feature”. Why do you think C#8 introduced Nullable reference types that work exactly like Java Optional ? https://docs.microsoft.com/en-us/dotnet/csharp/nullable-refe...
Re:Records Come to Java by elfexec on Thursday January 16, 2020 @07:06PM (#22070487)
> C# Nullable<T> is near useless because it only works on value types. I.e., you know, the types that are least likely to generate NullReferenceException for you.

You are completely missing the point of Nullable. It was introduced because databases allowed int, float, etc ( value types ) to be null. It was never meant to be an Option type. Reference types are already "nullable" so the database returning null for strings wasn't a problem. But databases returning nulls for int was a problem. It led to performance issues ( boxing and/or adding additional code to marshal data from one source to another ). If you find Nullable to be annoying then you either don't code in C# much and you especially don't deal with data/databases.

For what it was intended for, Nullable did it's job. And if Nullable is your only complaint, given the extraordinary changes that C# has gone through, I think it speaks well for C#.

Re:Records Come to Java by anthonybsd on Thursday January 16, 2020 @07:19PM (#22070577)
> You are completely missing the point of Nullable

Am I? Apparently so is everyone else because Microsoft has re-worked Nullable<T> in C# 8 to be almost exactly like Java Optional! Silly me right?

Re:Records Come to Java by wvenable on Thursday January 16, 2020 @08:02PM (#22070837)
Nullable/Non-nullable reference types in C# 8.0 having nothing to do with Nullable<T> and aren't even implemented using it.

C# 8.0 implementation of something like Optional<T> for reference types uses static flow analysis to ensure correct use of potentially null values. That's vastly superior in every way to Optional<T>.

[dead]
Re:Records Come to Java by wvenable on Friday January 17, 2020 @12:29AM (#22072152)
Yes, because it's checked by the compiler using normal constructs.

    someNullableInstance.DoSomething();  // compiler error
    if (someNullableInstance != null) 
        someNullableInstance.DoSomething();  // not an error
Option<T> is fine but it's a lot of syntax to use correctly.
Re:Records Come to Java by GordonS on Thursday January 16, 2020 @06:16PM (#22070120)
> C# does not have anything even remotely like records (structs are value types, and properties are fields) so what's your point?

How would you compare records to c# language features then? Are records value or ref types? Do they live in the stack or the heap?

Re:Records Come to Java by anthonybsd on Thursday January 16, 2020 @07:06PM (#22070489)
Reference; heap;

They are data containers, not control structures.

Re:Records Come to Java by GordonS on Friday January 17, 2020 @04:03PM (#22079296)
Can't you use structs in C# as data containers?
Re:Records Come to Java by anthonybsd on Friday January 17, 2020 @04:33PM (#22079607)
Ughh, you can provided that they are

1) Small 2) Short lived.

Because they live on the stack and follow the rules of value types you really shouldn't be using them for anything major. I.e. you want to model a SQL query result for a table with more than 3 columns? Probably struct is NOT the way to go, use a POCO class.

Re:Records Come to Java by GordonS on Friday January 17, 2020 @04:38PM (#22079667)
C# structs don't always live on the stack (unless you use a ref struct)[0]

[0] https://kalapos.net/Blog/ShowPost/DotNetConceptOfTheWeek16-R...

Re:Records Come to Java by anthonybsd on Friday January 17, 2020 @04:51PM (#22079830)
Well of course. Just like an int doesn't always live on the stuck. I.e. when it's declared as a field for example. If you are piping data from SQL DB into a List<YourStruct> the list is a reference type and will of course be on the heap.

Another issue with struct being big is really when it's being passed around and value copying starts to occur. You want to minimize this if the struct is large. My point being is that if in doubt, you probably don't want to use a struct and hence it's not a general purpose data container.

Re:Records Come to Java by shawnz on Thursday January 16, 2020 @06:05PM (#22070051)
C# does not have records but they have been "proposed" and subsequently delayed for some time now.
Re:Records Come to Java by apta on Thursday January 16, 2020 @05:01PM (#22069483)
> C# (structs or maybe properties)

Those are not directly analogous to records. As a matter of fact, C# is getting records as well: https://blog.cdemi.io/whats-coming-in-c-8-0-records/

Re:Records Come to Java by to11mtm on Thursday January 16, 2020 @06:07PM (#22070064)
C# has had records slated for years without delivery, alas. Thankfully we do finally have a version in testing.

In any case to answer GP, structs should not be considered a valid substitute for records, they are 'value types' so passing them around means you copy all the data every time you pass (i.e. lots of allocations if the record is larger than 1 or 2 fields)

Properties get you 'closer'. You can do some code Which is a good bit less than Java by default (i.e. not Lombok or similar)

F# does have Records, and technically there's nothing stopping you from declaring your records in F# and importing them into C#... but that may not be ideal. I don't think you get the full benefits (like copying a record with specific properties changed)

Re:Records Come to Java by statictype on Thursday January 16, 2020 @06:03PM (#22070035)
This looks like syntactic sugar for comcepts that already exist?

I think in Java’s case, its only mow getting struct-like semantics

edit I may be wrong. It could be a purely syntax level feature in Java as well.

Re:Records Come to Java by paulddraper on Thursday January 16, 2020 @06:31PM (#22070246)
Java records are syntax sugar.

Value semantics are proposed in JEP 169 [1] and are part of Project Valhalla. [2]

[1] https://openjdk.java.net/jeps/169

[2] https://wiki.openjdk.java.net/display/valhalla/Main

Re:Records Come to Java by nitwit005 on Thursday January 16, 2020 @05:52PM (#22069931)
I believe it's not explicitly looking for get/set strings in method names, but generally inlining trivial methods. There is (or was, last time I checked) a -XX:MaxTrivialSize option to set the size.

Debugging optimized code awkward as compilers will inline and eliminate many functions. I assume the debugger is being nice to you and undoing the optimizations to make things more intuitive.

Re:Records Come to Java by crystaldev on Thursday January 16, 2020 @05:05PM (#22069522)
In Java years, Kotlin hasn't had anything for a "long time," the language itself is still a baby. Most Java shops are still living in the "Java 8 days."
Re:Records Come to Java by dfox on Thursday January 16, 2020 @05:45PM (#22069860)
I believe that the JIT does not remove get/setFoo by matching the naming pattern, but simply because these methods are trivial and usually not polymorphic and thus prime candidates for speculative inlining.
Re:Records Come to Java by lennydizzy on Thursday January 16, 2020 @05:06PM (#22069530)
C# struct is a value type, the Java Record is not a value type, right?
Re:Records Come to Java by pjmlp on Thursday January 16, 2020 @05:12PM (#22069572)
Right, inline classes are the value types (as part of Valhalla).
Re:Records Come to Java by hinkley on Thursday January 16, 2020 @06:36PM (#22070271)
Function inlining has been in Hotspot forever. Possibly day 1?

In fact there was a point in time where polymorphic inlining was an active field of research and a number of people used the JVM to prototype their theories. Pretty sure a later version of Hotspot had polymorphic inlining as a feature.

Re:Records Come to Java by bitL on Thursday January 16, 2020 @06:22PM (#22070180)
LOL, record was an intentional omission during designing Java, similar to output parameters in function calls. It's not like James didn't know about them or couldn't implement them.

The only truly problematic thing in the original Java design was type erasure, leading to mess in all JVM-based languages like Scala, Kotlin or Clojure, but it saved some memory, so it was a design compromise. C#/CLR chose the other option, making their VM more flexible.

Re:Records Come to Java by paulddraper on Thursday January 16, 2020 @07:23PM (#22070593)
> leading to mess in all JVM-based languages like Scala, Kotlin or Clojure

!!!

Type-erasure of generics is what gives the other languages flexibility.

Scala eventually gave up C# precisely because CLR reification proved a huge challenge with Scala semantics.

You may not like JVM type-erasure but Scala would literally not be possible without it.

---

If anything, I would argue that Java should have done full type-erasure, not just type-erasure of generic types. (That does come with downsides though.)

Re:Records Come to Java by bitL on Thursday January 16, 2020 @10:00PM (#22071466)
> Scala eventually gave up C# precisely because CLR reification proved a huge challenge with Scala semantics.

In other words, Scala was designed on a hack involving JVM's type erasure, then authors found out the hack can't be ported to CLR. That's another way to view it. Type erasure is a half-baked mess in original Java, it was causing headaches right with the first version of generics...

Re:Records Come to Java by pron on Thursday January 16, 2020 @10:07PM (#22071499)
Type erasure is not a hack. It's the most common technique in typed languages. It causes what is, at worst, a minor inconvenience in the Java language in exchange for making the Java platform a very attractive and very convenient target for languages. Reification -- and I believe .NET is the only platform with variance that does it -- makes a very inconvenient compilation target for any language not designed around it.
Re:Records Come to Java by bitL on Friday January 17, 2020 @12:39PM (#22076998)
That sounds as ex post facto justification. IIRC the designers hit the issue once they started implementing generics and the gravity of the situation became apparent just at that time. Then another set of hackers figured out a way to utilize it for language interoperability. I am not saying CLR's final implementation is great either, but it addresses some points JVM missed as they had time to learn from SUN's mistakes, while making their own errors in the process as well. Language/platform design is notoriously difficult and unpredictable.
Re:Records Come to Java by paulddraper on Friday January 17, 2020 @03:29PM (#22078868)
> That sounds as ex post facto justification.

The initial reason was indeed backwards compatibility. As it turns out, compatibility with Java 1 unintentionally yielded compatibility with future languages too.

Because of this flexibility (ex post facto notwithstanding), the JVM became the most popular VM target ever.

Note that GraalVM -- which from day 1 intended to be a multilingual VM -- also uses type erasure.

---

Know that there is more than conceptualization of a type system, and in particular a parameterized type system. C vs C++ vs Java vs Haskell. The more of this is baked into the runtime, the more reflection can be done at runtime, but the less interop can be done with other type systems.

(There is a reason why C not C++ is the de facto FFI standard and its not just age.)

Re:Records Come to Java by pron on Friday January 17, 2020 @01:04PM (#22077322)
The original issue was compatibility and interop between two languages with different variance strategies. It's just that those two languages were two versions of the Java language. Calling it a "justification" is also misplaced: baking a variance strategy into the runtime makes the runtime very over-fitted to a particular language; AFAIK this is only done by .NET. So if it's a "justification" it is a very reasonable one. I don't think either of the approaches is a black-and-white mistake, but I personally very much prefer Java's. The loss is a very minor inconvenience (mostly the inability to overload methods with different generic arguments, e.g. foo(List<String> x) and foo(List<Integer> x)), and the gain is that the Java platform is a great compilation target for many different languages. Whether or not those who made this decision had other languages in mind is irrelevant; it's a very good decision, even though some would prefer the opposite one. It's certainly not a mistake and not a hack. It's not a hack because it is a well-known, well-understood, common compilation technique; it's not a mistake because there is no alternative that's clearly better, and many (myself included) think it's significantly better than the alternative chosen by .NET.
Re:Records Come to Java by pron on Thursday January 16, 2020 @06:32PM (#22070255)
Type erasure of generic parameters is what allows the Java language, Kotlin, Clojure, JavaScript and other Java platform languages to interoperate so well and share data types with no runtime conversions despite having different variance strategies. By baking variance into the VM, .NET is much less flexible, as can be seen by how cumbersome it is to share data structures among different languages.
Re:Records Come to Java by dwohnitmok on Thursday January 16, 2020 @07:35PM (#22070661)
How is that? I don't have as much experience with .NET languages so I'm not sure of what difficulties lie in .NET interop, but from my very cursory experience with F# And C# I didn't find anything particularly difficult about using one's data structures in the other (granted they don't have different variance strategies).

What about Java's type erasure lets languages play better with variance strategies and data structures? It's not apparent to me that e.g. IronPython has a hard time sharing data structures with C# (although I haven't actually used IronPython FWIW). Why doesn't the usual strategy of upcast everything to `Collection<Object>` within a language where collections are covariant and then downcast as needed in a language that doesn't care about variance (e.g. a dynamic language) work?

Re:Records Come to Java by pron on Thursday January 16, 2020 @07:47PM (#22070732)
> granted they don't have different variance strategies

... because it's baked into the platform.

> It's not apparent to me that e.g. IronPython has a hard time sharing data structures with C#

https://ironpython.net/documentation/dotnet/dotnet.html#acce...

> Why doesn't the usual strategy of upcast everything to `Collection<Object>` within a language where collections are covariant and then downcast as needed in a language that doesn't care about variance (e.g. a dynamic language) work?

Because that does not work for mutable collections, or even for immutable collections created by, say, an untyped language. Clojure can create a list of strings and pass it to a Java method expecting a List<String>.

Re:Records Come to Java by dwohnitmok on Thursday January 16, 2020 @08:05PM (#22070855)
Right but my impression was that that was basically a performance optimization for IronPython, similar to type hints for Clojure and that they aren't actually necessary. IronPython was mainly what I was thinking when it comes to alternate variance strategies. I mean most typed languages I know of with stricter typing regimes than Java either all have the same variance schemes or eschew subtyping altogether, JVM, CLR, or otherwise.

Why doesn't that work for mutable collections (are you referring to the need to be invariant here?) or immutable collections? More specifically, why can't you always have `Collection<A>`, upcast to `Collection<Object>`, downcast to `Collection<B>` as an escape hatch? You break type safety but erased generics already break type safety (this is especially clear with C#'s choice to follow Java's convention on array variance and the pain that this causes when it's inconsistent with everything else in C#).

For passing back to a typed language, you add a downcast of `Collection<Object>` to `Collection<String>` and have no runtime conversion cost. Why doesn't that work?

EDIT: Ah I see, is the basic problem you don't get free (more accurately cheap) casts, especially with generics? I'm not familiar with how casting works in the CLR since the vast majority of my time is on the JVM.

Re:Records Come to Java by pron on Thursday January 16, 2020 @08:20PM (#22070947)
> are you referring to the need to be invariant here

Yes, or contravariant.

> erased generics already break type safety

They don't, they just don't enforce type safety across languages. BTW, most languages erase types (including Haskell).

> Why doesn't that work?

Because platforms with safe casts do a runtime instanceof test, and if the platform reifies generics, then it is not true that the runtime type of Collection<Object> can be cast to Collection<String>. But if the generic parameter is erased, they both have the same runtime type.

Re:Records Come to Java by dwohnitmok on Thursday January 16, 2020 @08:41PM (#22071081)
Sure, Haskell relies on parametric polymorphism and typeclasses to get away with full type erasure with opt-in reification via Typeable. C++ does something similar with templates and has opt-in reification with RTTI. The JVM is just weird because it has some type reification, but not all the way.

RE type safety, it doesn't really make sense to talk about type safety across languages since type safety is tied to a particular language's semantics. I would argue that every ClassCastException is a violation of type safety in Java the language even if it's reasonable at the JVM level but whatever that's just definitions diving.

I see. Basically what you're saying is that you don't really see examples of reified types with unsafe runtime casts? Presumably because that kind of negates the whole point of reifying the types in the first place?

Re:Records Come to Java by pron on Thursday January 16, 2020 @08:55PM (#22071150)
> The JVM is just weird because it has some type reification, but not all the way.

I don't know if it's "weird" given that there are only two examples, and they're different from one another. It's not like there's a norm here.

> I would argue that every ClassCastException is a violation of type safety in Java the language even if it's reasonable at the JVM level but whatever that's just definitions diving.

But you can't get a CCE in Java without explicit casts (or a compiler bug), just as in C#.

> Basically what you're saying is that you don't really see examples of reified types with unsafe runtime casts?

I'm not sure what you're asking.

Re:Records Come to Java by dwohnitmok on Friday January 17, 2020 @01:45AM (#22072433)
You can get CCEs without explicit casts or compiler bugs though. javac will catch the most glaring examples of generic type erasure with a warning.

  List<String> myStuff0 = new LinkedList<>();
  List eraseMyStuff = myStuff0;
  // Warning here
  eraseMyStuff.add(1);
  // CCE here
  String myString = myStuff0.get(0);
However, if you obfuscate it a little javac doesn't know about it anymore.

  List<String> myStrings = new LinkedList<>();
  List<Integer> myIntegers = new LinkedList<>();
  myIntegers.add(1);
  Stream.of(myStrings, myIntegers)
    // Wildcard capture
    .map(x -> x)
    .reduce((x1, x2) -> {
      // Would expect this line to blow up either at compile time or at runtime with reified generics
      list1.addAll(x2);
      return x1;
  });
  // Blows up with a CCE
  String result = myStrings.get(0);
The above is trick that was featured in some HN post from a while ago (maybe a year or so ago? Need to dig it up...) and is not an original idea of my own. It's essentially combining wildcard capture conversion with type erasure to delay a CCE away from the point of the problem. As I remember I think this isn't a compiler bug in the sense that this follows the Java spec.

I thought, however, there was an easier trick to get a CCE using type erasure of generics without warnings and without having to involve capture conversion, but it looks like I may have just been wrong about that as my efforts all ended up getting warnings from javac.

Of course whether any of this actually matters is a separate issue altogether. Java is still a workhorse language that powers a ton of software.

As for weirdness, I can't think of any other language with generics that preserves runtime type information without a way of preserving that for generics as well. Most as you point out either do full erasure (most stuff in the ML-inspired family) or either fully preserve all the information (of course all the .NET languages)/let you opt-in to preserving generics (Scala to some extent with TypeTags) or just forgo generics (Go).

For example in OCaml if you try to do an unsafe and incorrect cast you segfault. IIRC in GHC you end up with a crazy incorrect value (e.g. unsafeCoerce [1, 2, 3] :: Int results in some messed up integer).

My question was me thinking out loud; I was guessing that CIL bytecode has a instruction analogous to checkcast in JVM bytecode that you can just omit and then you can freely cast at will, potentially wreaking havoc if you cast incorrectly. I don't know if that's true. Basically I was separating the verification of type information from the storage of it at runtime. If the CIL separates them then the downcast trick works. If it doesn't the trick doesn't work.

Also in a way this unsafe cast is really just a run around reified types to begin with. I guess the only difference is that you still have storage of the types so you can do runtime reflection. On reflection, I can certainly see how this complicates implementing things like higher-kinded types (because you sort of end up implementing half of type erasure yourself) but I'm not sure it complicates marshalling ordinary data.

Re:Records Come to Java by pron on Friday January 17, 2020 @06:22AM (#22073745)
> You can get CCEs without explicit casts or compiler bugs though

Using raw types is cheating. It's essentially using a different language and, as you say, the compiler will tell you that you are.

> However, if you obfuscate it a little javac doesn't know about it anymore.

It's a bug (rather, a few bugs): http://wouter.coekaerts.be/2018/java-type-system-broken (I assume that's where you got the example from)

You shouldn't get a CCE without explicit casts, and if you do, that's a (soundness) bug in the Java compiler and/or spec. Those bugs aren't always fixed quickly when they're not very important in practice (if it's determined that it's hard to hit them by mistake, which is why it's taken 15 years to find them).

> Would expect this line to blow up either at compile-time or at runtime with reified generics

Reified generics have no impact at compile time. Everything a language with reified generics knows at compile-time, a language with erased generics also knows. However, a runtime with reified generics and a similar compiler bug would, indeed, throw at this line at runtime.

> As for weirdness, I can't think of any other language with generics that preserves runtime type information without a way of preserving that for generics as well.

There are good reasons for doing it the Java way: so that you don't bake a language's particular variance strategy into the VM. There are also good reasons for doing it the C#: it's slightly more convenient for that one language. I can't think of any other language with variance and a runtime type system that reifies generics for reference types. When all you have are two instances, it's hard to say which of them is weird.

> If the CIL separates them then the downcast trick works. If it doesn't the trick doesn't work.

I don't know CIL, but looking at how languages behave on .NET to interop, I wouldn't think that can work. It would totally break the safety of the runtime type system if you could cast without a typecheck.

> I guess the only difference is that you still have storage of the types so you can do runtime reflection.

BTW, you can reflect on generic type arguments in Java, as well (https://docs.oracle.com/en/java/javase/13/docs/api/java.base...). When you reflect on a method that returns a List<String>, it will tell you it returns List<String>, but you can't get such a Type from an instance of that list; you can only get a Class.