News for hackers. Stuff that matters.
Re:Overhead of Returning Optional Values in Java and Rust by _old_dude_ on Saturday October 16, 2021 @01:05PM (#28889326)
I expect Java to be slower, Optional was introduced as an afterthought not as a native construct like in Rust.

Sadly, the two benchmarks are not equivalent, the Rust version loop over ints, the Java version loop over longs.

Looping over longs is notoriously slow in Java, because there is no vectorisation and the JIT insert insert a GC checks because looping over longs may take a long of time.

Also one of the codes uses Optional<Long> instead of OptionalLong, so the code does a double boxing long -> Long -> OptionalLong.

Re:Overhead of Returning Optional Values in Java and Rust by tialaramex on Saturday October 16, 2021 @02:31PM (#28890197)
Rust doesn't have types named 'int' and 'long' out of the box. It does have types named e.g. i32, i64, u32 and u64.

Because the function get_int() takes a u64, Rust will conclude that the variable i must be a u64 so as to fit in that parameter.

It seems to me that "Java is bad at 64-bit integers" is also useful information but not pertinent to this benchmark, unless you assume it's trying to compare Java to Rust, at which of course Java will be far slower - but I don't think that's the point of the benchmark at all.

Re:Overhead of Returning Optional Values in Java and Rust by comex on Saturday October 16, 2021 @01:58PM (#28889895)
The Rust version iterates over u64, which is the same size as Java long, though it’s unsigned whereas Java long is signed.
Re:Overhead of Returning Optional Values in Java and Rust by Gadiguibou on Saturday October 16, 2021 @01:59PM (#28889908)
> Sadly, the two benchmarks are not equivalent, the Rust version loop over ints, the Java version loop over longs.

I thought longs were 64 bit signed integers in Java, what's the difference with a u64 in Rust? Is it the signedness that affects performance?

> OptionalLong

Thanks for the info! I didn't know there were Optional primitive types in Java!

Re:Overhead of Returning Optional Values in Java and Rust by sushsjsuauahab on Saturday October 16, 2021 @02:32PM (#28890204)
Well, it's not exactly a primitive (long) and instead it is an Object (Long), but I understand what you mean :)
Re:Overhead of Returning Optional Values in Java and Rust by shawnz on Saturday October 16, 2021 @11:37PM (#28893733)
OptionalLong is an object which optionally contains a value type (lowercase L) long. Optional<Long> is an object Optional which optionally contains an object Long.
Re:Overhead of Returning Optional Values in Java and Rust by sushsjsuauahab on Sunday October 17, 2021 @03:35PM (#28899112)
Oh nice! I should have checked the docs or Intellij to see that it returns the primitive type ;)
Re:Overhead of Returning Optional Values in Java and Rust by hn_throwaway_99 on Saturday October 16, 2021 @02:01PM (#28889922)
This is somewhat tangential, but in my opinion implementing Optionals as a "normal" wrapper type, as is done in Java, doesn't really help that much, and can in fact actively hurt, because (a) there is nothing that prevents an Optional variable from actually being null (I've seen a method with an Optional return type that returned null in some cases - obviously a horrible decision, but not prevented by the compiler nonetheless), (b) you don't get compiler support that knows, for example, that if you preface an empty check on the variable that it's then safe to get the value and (c) it makes code super-verbose and hurts readability.

You really need first class support for optionals, a la TypeScript and Kotlin, to get a benefit IMO.

Re:Overhead of Returning Optional Values in Java and Rust by hashmash on Saturday October 16, 2021 @02:43PM (#28890300)
> nothing that prevents an Optional variable from actually being null

That's why you're supposed to use Optional<Optional<X>>. ;-)

Re:Overhead of Returning Optional Values in Java and Rust by herbstein on Saturday October 16, 2021 @02:54PM (#28890381)
> This is somewhat tangential, but in my opinion implementing Optionals as a "normal" wrapper type, as is done in Java, doesn't really help that much

The great thing about Rust is that `Option` is a normal, generic type. You could very easily implement your own equivalent type.

Re:Overhead of Returning Optional Values in Java and Rust by mqus on Saturday October 16, 2021 @03:48PM (#28890855)
Not quite. The compiler does have the knowledge that it can use \0 for representing None in Option<Pointer> types. I'm not sure it can infer that for home-grown Option enums.

So it can optimize the memory overhead of Option<> away completely.

Re:Overhead of Returning Optional Values in Java and Rust by umanwizard on Saturday October 16, 2021 @03:59PM (#28890948)
That is called “niche optimization” and the compiler will in fact still do it for a hand-rolled Option type.
Re:Overhead of Returning Optional Values in Java and Rust by tialaramex on Saturday October 16, 2021 @07:11PM (#28892408)
Although, Rust only knows about certain obvious niches. It can reason that it can fit None in the same space as NonZeroU64 or a reference &T but it doesn't see how to do this with a hypothetical EvenU64 as I understand it because that "niche" is in the wrong place.
Re:Overhead of Returning Optional Values in Java and Rust by herbstein on Saturday October 16, 2021 @05:51PM (#28891918)
For anyone not sure, here is the proof.

https://godbolt.org/z/4zhWM9hTP

Re:Overhead of Returning Optional Values in Java and Rust by Jach on Saturday October 16, 2021 @03:21PM (#28890621)
My experience is they helped a lot. a) basically never happened (I have a vague recollection of catching one in code review, but usually my reviews would instead be "instead of returning null here, use Optional please" or figuring out a way to avoid nulls entirely (e.g. empty list when returning collections)) b) I think both Eclipse and IntelliJ will issue a warning if you perform an Optional.get() without an isPresent() check c) it seems less verbose and more readable than null checks, especially if you have a map chain/are using Streams. It could be even nicer, like what Kotlin has, but it's an improvement.
Re:Overhead of Returning Optional Values in Java and Rust by heurisko on Saturday October 16, 2021 @03:10PM (#28890526)
Java Optionals are useful in Streams.

I've found I agree with the principle that Optionals only make sense for return values (and if the method returns null, that should be treated as a bug).

I've seen Optionals used for member variables, and they might as well just be null. (And are highlighted as such by IDEs like IntelliJ).

Re:Overhead of Returning Optional Values in Java and Rust by Zababa on Saturday October 16, 2021 @02:08PM (#28889975)
It works very well in ML-like languages. The problem is when you have a language where everything can be null. TS and Kotlin need first class support for optional because the underlying language has first class support for null.
Re:Overhead of Returning Optional Values in Java and Rust by foolfoolz on Saturday October 16, 2021 @02:06PM (#28889956)
- if you don’t use null this isn’t an issue

- it’s verbose because handling optional values safely is complex

Re:Overhead of Returning Optional Values in Java and Rust by hn_throwaway_99 on Saturday October 16, 2021 @02:16PM (#28890055)
> if you don’t use null this isn’t an issue

I mean, no shit. But the problem is that tons of existing Java APIs do use nulls, so you can't just pretend they don't exist, and importantly a major point of the optional construct is to get compile-time guarantees, and the Optional class doesn't give you that.

> it’s verbose because handling optional values safely is complex

No, it's not. Kotlin has optionals built into the language, and it's safer and much less verbose.

Re:Overhead of Returning Optional Values in Java and Rust by tadfisher on Saturday October 16, 2021 @11:53PM (#28893805)
Nit: Kotlin doesn't exactly have optionals, it has essentially a type modifier (?) that represents a "nullable" type, i.e. the sum of a type T and the value 'null'. This is indeed safer and terser, therefore strictly more powerful than Java's Optional type, but it begs a question: Why not allow users to create similar higher-kinded types instead of baking this one very specific type kind into the compiler?

Their recent decision to axe typeclasses in favor of multiple function receivers, which solves a small subset of problems that HKTs do in Scala and Haskell, make me look for greener pastures whenever I need them.

Re:Overhead of Returning Optional Values in Java and Rust by Zababa on Saturday October 16, 2021 @02:09PM (#28889978)
How is "handling optional values safely" complex? Do you have any examples of that?
Re:Overhead of Returning Optional Values in Java and Rust by imoverclocked on Saturday October 16, 2021 @03:57PM (#28890925)
First class support for nullable is more basic and needed.

Java On its own it doesn’t support this however there are ways of enforcing this in a Java codebase. Eg: Errorprone and NullAway comes to mind here.

Re:Overhead of Returning Optional Values in Java and Rust by mastax on Saturday October 16, 2021 @01:17PM (#28889443)
The most equivalent Rust type to `Long` would be `Option<Box<u64>>`, which is represented as a nullable pointer to a u64. `Box<Option<u64>>` is represented as a non-nullable pointer to a struct containing a u64 number and an aligned-to-u64 discriminant.

Does it change the benchmarks? Not if you let the inner function get inlined: https://rust.godbolt.org/z/Ejc5c7sE8

Re:Overhead of Returning Optional Values in Java and Rust by alkonaut on Saturday October 16, 2021 @01:35PM (#28889626)
Comparing C# (.NET 5) with Rust would perhaps be interesting since at least then it’s languages on the same footing wrt heap allocating Option<> (neither does). But on the other hand then it should mostly be equal for a microbenchmark and that’s not very interesting.

Allocating anything like a “Long” or “Optional” on the heap seems like an extremely bad idea. Even if allocation and GC is almost free, the pointer chasing might not be.

Re:Overhead of Returning Optional Values in Java and Rust by munchler on Saturday October 16, 2021 @02:13PM (#28890019)
I would throw F# into that test as well.
Re:Overhead of Returning Optional Values in Java and Rust by vips7L on Saturday October 16, 2021 @12:18PM (#28888906)
I’m pretty sure returning Optionals from a method always makes them “escape” and become heap allocated. Hopefully if Valhalla ever lands Optional will be a primitive class.

Here’s a fascinating read on the current state of HotSpot escape analysis:

https://gist.github.com/JohnTortugo/c2607821202634a6509ec3c3...

Re:Overhead of Returning Optional Values in Java and Rust by mwcampbell on Saturday October 16, 2021 @01:03PM (#28889307)
And this is one reason why I think Kotlin's approach to nullability is better than Scala's Option or Java 8's Optional. Also, the Kotlin solution has better interop with legacy Java APIs.
Re:Overhead of Returning Optional Values in Java and Rust by vbezhenar on Saturday October 16, 2021 @01:50PM (#28889812)
Java Optional is not a general replacement for nulls (although people might use it this way). It's a way to replace nulls in method return value, when those nulls indicate missing value. It's a subtle philosophical distinction, but I think that it's worth to remember it.

If you want to improve nulls handling in Java, you should use @NotNull annotation and corresponding linters.

https://stackoverflow.com/a/26328555/315129

Re:Overhead of Returning Optional Values in Java and Rust by vips7L on Saturday October 16, 2021 @01:40PM (#28889676)
Yeah I’m hoping Java takes the C# approach with a compile option that can enable that type of null safety.
Re:Overhead of Returning Optional Values in Java and Rust by pjmlp on Sunday October 17, 2021 @02:15AM (#28894366)
The C# approach will take decades for .NET shops to wrap their minds around it.

2021 was the first year I deploy anything related to .NET Core in production, and it was still .NET Core 3.1.

Will this velocity no one is going to their .NET repos and rewrite the libraries with the required annotations to keep the compiler happy.

If you want null safety in Java today, just make use of @Nullable, a good IDE with static analysis, and deploy PMD, FindBugs or SonarQube into the CI/CD pipeline.

Re:Overhead of Returning Optional Values in Java and Rust by bonzini on Saturday October 16, 2021 @01:20PM (#28889481)
Do you have a pointer?
Re:Overhead of Returning Optional Values in Java and Rust by ptx on Saturday October 16, 2021 @01:36PM (#28889636)
Re:Overhead of Returning Optional Values in Java and Rust by comex on Saturday October 16, 2021 @02:28PM (#28890171)
That post says that escape analysis is "heavily dependent on methods pertaining to candidate objects being inlined", implying that the check for whether something is returned from a method happens after inlining. The post then complains that the inliner's decision of whether to inline isn't informed by whether doing so would be beneficial for escape analysis. But that shouldn't matter for this particular microbenchmark where, according to the post, the function returning the optional does get inlined.
Re:Overhead of Returning Optional Values in Java and Rust by ridiculous_fish on Saturday October 16, 2021 @01:55PM (#28889850)
This is the first I've seen `num::NonZeroU64` which is nice, intrusively using a zero value to mean none.

Is there any analog for a const pointer? I want to point into or just beyond a const array, intrusively using null as a None value.

There is `ptr::NonNull` but this always converts to a *mut T, and one is just supposed to "be careful" to not use the mutable features if your pointee isn't actually mutable.

Re:Overhead of Returning Optional Values in Java and Rust by returningfory2 on Saturday October 16, 2021 @03:02PM (#28890450)
Anything of type `&T` has the same sizing property as `num::NonZeroU64`: the size of `Option<&T>` is the same as the size of `&T`. So I think you would just use a plan old Option?
Re:Overhead of Returning Optional Values in Java and Rust by ridiculous_fish on Saturday October 16, 2021 @03:54PM (#28890901)
The problem with &T is that you can't make one to just-past-the-end of an array.

If you look at the implementation of slice::Iter, it stores a (cursor, end) pointer pair, but there's no way to say "this is a non-null const pointer" so it casts the cursor pointer to *mut.

https://doc.rust-lang.org/src/core/slice/iter.rs.html#67

Re:Overhead of Returning Optional Values in Java and Rust by nqzero on Saturday October 16, 2021 @01:42PM (#28889700)
JMH intentionally disables many optimizations that hotspot would otherwise perform, so comparing the (optimized) rust performance with the (de-optimized) jmh performance doesn't make any sense
Re:Overhead of Returning Optional Values in Java and Rust by aw1621107 on Saturday October 16, 2021 @01:50PM (#28889811)
Really? That's a surprise to me, considering the supposed purpose of JMH, though to be fair I'm not hugely experienced in it.

Do you know what optimizations are disabled? Are they disabled automatically, or only if certain parameters/functions are used (e.g., BlackHole)?

Re:Overhead of Returning Optional Values in Java and Rust by nqzero on Sunday October 17, 2021 @07:38PM (#28900721)
for the official oracle view, see: https://www.oracle.com/technical-resources/articles/java/arc...

"JMH provides a [snip] foundation for writing and running benchmarks whose results are not erroneous due to unwanted virtual machine optimizations"

unsolicited personal opinion: they're almost always erroneous for other reasons

i'm not sure which optimizations are currently disabled

Re:Overhead of Returning Optional Values in Java and Rust by aw1621107 on Wednesday October 20, 2021 @10:19PM (#28939508)
Hmmm. Wonderfully vague. Suppose I'd need to dig into the source to figure it out.

To be fair, now that I think about it I suppose some deoptimizations make sense (e.g., preventing replacement of loops).

Thanks for pointing me to that article!

Re:Overhead of Returning Optional Values in Java and Rust by Zababa on Saturday October 16, 2021 @02:06PM (#28889951)
I find the focus on Rust vs Java a bit weird. Yes Rust is faster than Java, I sure hope it is. But the interesting thing is that Rust is better at optimizing the overhead of Option.

> The most ugly and error-prone solution turned out to be the fastest: primitive types and magic values.

That's often the case in "older" languages. ES6 iterators on arrays are usually twice as slow as a simple "for" loop. Which usually means that by targetting ES5 or lower, you get a free optimization by Babel/tsc!

Re:Overhead of Returning Optional Values in Java and Rust by tialaramex on Saturday October 16, 2021 @03:56PM (#28890920)
This is a failure of optimisation though.

In the NonZeroU64 case Rust can literally do the same trick with magic values ... in the machine code, the magic value is zero, a NonZeroU64 can't be zero and so Rust will squeeze None into that value of the register.

The programmer and their colleagues both reading and writing the code don't need to understand how this trick works, don't need to be aware of magic values, and if some day the code needs a full u64 and a NonZeroU64 isn't good enough, they just change it and sure, it might be a little slower because the optimisation wasn't available, but it still works as expected because magic values were just an optimisation not part of the program logic.

Re:Overhead of Returning Optional Values in Java and Rust by Zababa on Saturday October 16, 2021 @04:05PM (#28890989)
It is, but Java has been having trouble optimizing things like that since forever. Project Valhalla is still in development. Pointer chasing and boxing is the reality of Java today.
Re:Overhead of Returning Optional Values in Java and Rust by pornel on Saturday October 16, 2021 @07:57PM (#28892693)
Both languages are used in high-performance servers (where you know Ruby or Python won’t cut it).

HotSpot has a reputation of being close to a Sufficiently Smart Compiler, so it’s interesting to see where its limits are, compared to lower-level language constructs.

Re:Overhead of Returning Optional Values in Java and Rust by Zababa on Saturday October 16, 2021 @08:50PM (#28892997)
My mental model would be, from lowest to highest peformance, something like Ruby/Python < JavaScript < Java/C# < Go < C++/Rust. From what I understand, Java is a huge improvement over Python, Ruby and even JS, but is still not the same as Rust, C++ or even Go. Still, those are rough indicators, and not absolute truth. Yon can usually go really far before being stopped by the absolute strict limit of a language.

For example, here's a post about Discord switching from Go to Rust with great success https://discord.com/blog/why-discord-is-switching-from-go-to..., and here's another about how CockroachDB optimizes Go https://www.cockroachlabs.com/blog/how-to-optimize-garbage-c.... In the same vein, there's Shopify AOTing Ruby using Sorbet and LLVM https://sorbet.org/blog/2021/07/30/open-sourcing-sorbet-comp....

All of that to say that I'm not surprised that Rust has better defaults compared to Java, as Rust is supposed to be faster, and option types are a code Rust language feature, while they are a relatively recent addition to Java.

Re:Overhead of Returning Optional Values in Java and Rust by pjmlp on Sunday October 17, 2021 @02:19AM (#28894383)
With the small difference that Rust lacks 25 years of libraries for every kind of enterprise use case, battle tested in production.

Beating Java with optional values is nice, but meaningless.

Re:Overhead of Returning Optional Values in Java and Rust by exabrial on Saturday October 16, 2021 @08:03PM (#28892734)
I would certainly hope Rust is much faster! But Java is a very quick language after it's optimizing native compiler kicks in.

Speaking of, one thing that bothers me about nearly every Java benchmark ever, is nobody checks to see what was _actually_ compiled to native code. To really compare Apples and Oranges, was the code being executed in interpreted mode? What about compiling with Graal? These questions are important for any long-lived program (which most Java programs are).

Here, the warmup iteration of "2" and limit of "5" test runs doesn't really accomplish anything. Depending on the JDK the default compilation threshold is 1.5k or 10k IIRC.

Re:Overhead of Returning Optional Values in Java and Rust by Zababa on Saturday October 16, 2021 @08:57PM (#28893036)
> These questions are important for any long-lived program (which most Java programs are).

Is this still true in the world of CI/CD? If you're following "best practices" and doing multiple deployments a day, how much will your performance suffer? I don't have any answers to that, though I suspect that it may be one of the reasons why Go is used so much for "cloud stuff".

Re:Overhead of Returning Optional Values in Java and Rust by exabrial on Sunday October 17, 2021 @02:09PM (#28898415)
> Long running

1500 or 10k in production for hot methods should only take a few mins in most production code I'd hope. Workload dependent, of course!

And, you can always tune the JDK for your workload. An optimizing compiler is a very powerful tool

Re:Overhead of Returning Optional Values in Java and Rust by pjmlp on Sunday October 17, 2021 @02:18AM (#28894378)
If you are doing it properly, JIT caches should be used.
Re:Overhead of Returning Optional Values in Java and Rust by spullara on Saturday October 16, 2021 @02:18PM (#28890065)
Kind of silly compare this before value types in Vahalla. Obviously Java optionals will be slower than Rust ones as they are just wrapper objects. However, they should at least rerun it using OptionalLong rather than Optional<Long>.
Re:Overhead of Returning Optional Values in Java and Rust by thinkharderdev on Saturday October 16, 2021 @02:37PM (#28890248)
I think the point is not really to compare Java to Rust but highlight that in Java you pay a significant performance penalty (relative to representing Optional.empty() as null).
Re:Overhead of Returning Optional Values in Java and Rust by tialaramex on Saturday October 16, 2021 @03:34PM (#28890741)
And without the Rust code, you just know people would say see, Optional is too expensive, this is why I use null. So the Rust here shows that actually no, this style doesn't need to come with a performance cost.

It's been more of a surprise to me to see people defending null as somehow preferable style in these comments.

Re:Overhead of Returning Optional Values in Java and Rust by spullara on Sunday October 17, 2021 @01:00AM (#28894081)
I think what they are saying is that it is the preferrable style in Java for now. As soon as Vahalla lands in the VM, then Optional in Java will be closer to Rust.
Re:Overhead of Returning Optional Values in Java and Rust by rakingleaves on Sunday October 17, 2021 @12:34PM (#28897652)
Mike Ernst wrote about the downsides of Optional in Java some time back https://homes.cs.washington.edu/~mernst/advice/nothing-is-be... with pluggable type checking it is possible to use `null` safely in Java
Re:Overhead of Returning Optional Values in Java and Rust by WmyEE0UsWAwC2i on Saturday October 16, 2021 @04:15PM (#28891071)
I don't think that's the best way to use Optional in java. In particular the use of isPresent & get kind of defeats the purpose. Should be something along the lines :

   ...
   sum = 0
   opt = getOptional(n)
   sum = sum + opt.orElse(0);
   ...
Also maybe use something like OptionalInt (but for long).
Re:Overhead of Returning Optional Values in Java and Rust by esprehn on Saturday October 16, 2021 @08:18PM (#28892808)
This code should use OptionalLong instead of Optional.of() which doesn't have primitive overloads and caused wrapping of the long into a Long and extra heap indirections. It still probably can't beat Rust, but what's presented is probably not optimal.
Re:Overhead of Returning Optional Values in Java and Rust by layer8 on Saturday October 16, 2021 @07:24PM (#28892483)
Optional was added to Java in anticipation of Valhalla, which still hasn't materialized. So no suprise here at the overhead.
Re:Overhead of Returning Optional Values in Java and Rust by Traubenfuchs on Saturday October 16, 2021 @12:44PM (#28889142)
Using null is fine. You probably don‘t need Optional. Using Optional is often „clever“ (=bad) overengineering.

Every NullPointerExceptipn my code ever caused was good, because it helped find an error state that had no acceptable default non error state alternative.

Re:Overhead of Returning Optional Values in Java and Rust by setr on Saturday October 16, 2021 @12:49PM (#28889184)
> Every NullPointerExceptipn my code ever caused was good, because it helped find an error state that had no acceptable default non error state alternative.

Isn’t that the point of Optional? It just moves that identification of those error states to compile-time rather than at runtime.

Re:Overhead of Returning Optional Values in Java and Rust by xxs on Saturday October 16, 2021 @04:05PM (#28890990)
My takes is that Option is (next to) useless - the most common pattern is ...orElseNull() - yikes.

The only sane approach is removing all nulls as early as they come and declaring/initializing all member variable in the c-tors.

Re:Overhead of Returning Optional Values in Java and Rust by setr on Saturday October 16, 2021 @09:00PM (#28893057)
The problem I think is that if your dependent apis aren’t also consuming Optionals, and relying on type = null || type logic, then you end up constantly translating back and forth between the two endlessly, defeating any ergonomic gains.

The ideal state is that anything that isn’t Optional is guaranteed to be non-null. So when you go ahead and remove all nulls as early as possible, you can statically guarantee that remains true forever-more.

But if all your libraries just go ahead and drop this knowledge anyways, and when they return an object without meaning non-nullable, then you’re back to where you started.

Whereas in an ML, I know every library is already adhering to this, so something being not-optional actually means something. Bolting it into a language ad-hoc, you go from everything can be null to this weird tristate — a type might be nullable, or might not be nullable and if it’s in an optional, it definitely is nullable… so your best bet is just checking for null every time anyways

Re:Overhead of Returning Optional Values in Java and Rust by valenterry on Saturday October 16, 2021 @12:47PM (#28889163)
Not sure if satire. But if not:

> Every NullPointerExceptipn my code ever caused was good, because it helped find an error state that had no acceptable default non error state alternative.

Yeah, and with Optional you would have found it at compile-time.

Re:Overhead of Returning Optional Values in Java and Rust by shawnz on Saturday October 16, 2021 @01:09PM (#28889350)
Using "Optional" doesn't provide any extra compile-time guarantees except that the function could return Optional.empty(). It's already possible to see that it could return null, so there's no extra guarantees there. The only advantage of "Optional" is that it lets you deal with the empty/null case with a nice functional API instead of if statements. It doesn't do anything to solve the reverse problem of guaranteeing that non-Optional functions can't return null.

If you just want to provide a hint to the caller at compile time about whether you intend for the possibility of returning null, you could use @Nullable/@NotNull annotations to achieve that with no overhead.

Re:Overhead of Returning Optional Values in Java and Rust by ubercow13 on Saturday October 16, 2021 @01:10PM (#28889368)
Yes, in a language like Java where it's retrofitted in years after the decision was already made to allow literally everything to be null. In other languages, that doesn't apply.
Re:Overhead of Returning Optional Values in Java and Rust by brabel on Saturday October 16, 2021 @01:44PM (#28889742)
It doesn't apply in other languages either as long as their nulls are "safe", as in Kotlin or TypeScript.
Re:Overhead of Returning Optional Values in Java and Rust by Zababa on Saturday October 16, 2021 @01:58PM (#28889897)
The thing is that people usually don't have a lot of code to handle the null case, so having optional where you can easily chain the happy case often offers a better experience. It also allows you to differentiate when people will return null for nothing, or when they will return an empty object. I know that in the codebase I work with, people use both. With Optional, you don't have that problem.
Re:Overhead of Returning Optional Values in Java and Rust by shawnz on Saturday October 16, 2021 @05:03PM (#28891485)
Why not just chain the happy case by ignoring nulls, and let any potential NullPointerException bubble up to the point in the code where you would have had an "Optional.orElse" or whatever? Isn't that often just as easy? If you want it to be explicit then you could just add "@throws NullPointerException" to your javadoc. And of course having Optional in your type signature doesn't guarantee you'll never return empty objects (or even null).
Re:Overhead of Returning Optional Values in Java and Rust by valenterry on Saturday October 16, 2021 @09:36PM (#28893217)
Because then you cannot differentiate anymore between "NullPointerException as a bug" or "NullPointerException as a feature".

E.g. you might want to return the streetnumber of a user's address. When a NPE is thrown, was it because there is a bug somewhere in the code or because the user's address has no street number?

With Optional, it is clear: there was no street number.

Re:Overhead of Returning Optional Values in Java and Rust by hashmash on Saturday October 16, 2021 @01:04PM (#28889310)
Only if you never have the case where Optional is empty but you expected it not to be. How often is Optional.get or orElseThrow called blindly? How often is boilerplate code written which checks if the optional is empty and then throws "should not happen" exception or just logs a message? Once NullPointerException got "helpful" messages, I don't see that much advantage to Optional, considering that it provides less information out of the box.
Re:Overhead of Returning Optional Values in Java and Rust by thinkharderdev on Saturday October 16, 2021 @02:51PM (#28890353)
The advantage is if you are explicitly modelling a value that could be empty. You can encode that into the type system by making git an Option<T> instead of just a T (that could be null). At it's most basic level it can just help reduce the possibility of NullPointerExceptions but the real benefit and reduction of boilerplate comes when you use Option as a monad. For instance, in Scala if you have a something like

case class Address(street: String, city: String, state, String) case class User(id: Int, name: String, address: Option[Address])

def filterCAUsers(users: List[User]): List[User] = users.filter(_.address.exists(_.state != "CA"))

Re:Overhead of Returning Optional Values in Java and Rust by esrauch on Saturday October 16, 2021 @01:59PM (#28889903)
> Only if you never have the case where Optional is empty but you expected it not to be

I kind of think that sentence is contrary to the point of Optional, and if you start from that mindset then Optionals do end up no better than nulls.

Instead you enforce good hygiene where you unwrap early and pass non-Optionals for anything that you "know" is non-null. If you call a function which returns optional, and you're find yourself thinking "nah, I _know_ this is definietly non-empty" then you're incorrect; the API explicitly is telling you that it definitely can be empty.

Re:Overhead of Returning Optional Values in Java and Rust by hashmash on Saturday October 16, 2021 @02:27PM (#28890160)
There are certainly cases where Optional is better than a "this can be null" comment, but consider the possibility that the Map interface was designed to always return Optional. If I'm using a Map that I have complete control over, that I know what's in it (I think), then I'll call Map.get(x).orElseThrow(). This isn't an improvement over a Map.get(x) call that can return null. It's also uglier, and as shown by the Rust benchmark, much slower. I'd be happier with a Map interface that had a "tryGet" and "get" pair, where only the latter threw an exception. No need for Optional.
Re:Overhead of Returning Optional Values in Java and Rust by valenterry on Saturday October 16, 2021 @09:44PM (#28893252)
It _is_ an improvement.

If you just do "Map.get(x)" and you get null because your thinking was wrong, then the NPE will pop up potentially 10 layers later or an hour later, when the null-value was tried to be used in your program.

On the other hand, with "Map.get(x).orElseThrow()" you will have an exception thrown immediately! Even better, you should customize that as "Map.get(x).orElseThrow(NoSuchElementException(x))" so that you know the value that was unexpectedly not in the list.

That will make debugging much easier and will potentially fail a test case while a "Map.get(x)" might not cause the test to fail.

Re:Overhead of Returning Optional Values in Java and Rust by convolvatron on Saturday October 16, 2021 @01:13PM (#28889401)
for me this is one of the bigger festering wounds in programming. error handling doubles the size of the code. error handling is complicated.

so we put on our big-dev pants and build all sorts of structures around handling errors to make it more sound/ergonomic.

but at the end of the day:

   o its still very difficult to respond in a semantically meaningful way to errors as code

   o they still almost always just get dumped into a log or ignored, or just result in a panic

   o even if we have managed to surface them - they still aren't really that actionable by the end user
so we haven't helped the situation much, but some of these answer like catch/throw have really unpleasant consequences - we may have made it worse
Re:Overhead of Returning Optional Values in Java and Rust by valenterry on Saturday October 16, 2021 @09:40PM (#28893233)
It's pretty much a solved problem. Just use a language that has better ergonomics for errors.

Scala's ZIO is one of the better examples if you are interested to look into an alternative.

Re:Overhead of Returning Optional Values in Java and Rust by Zababa on Saturday October 16, 2021 @01:57PM (#28889883)
> Using null is fine. You probably don‘t need Optional. Using Optional is often „clever“ (=bad) overengineering.

Option types are 8 years older than null (ML: 1973, null: 1965). There's nothing clever about it. It's simple to understand, and is an explicit wrapper around an object, while a pointer is an "implicit" pointer around an object. Option<Obj> is clear, you know that it can be something or nothing. Plain Obj is deceiving.

> Every NullPointerExceptipn my code ever caused was good, because it helped find an error state that had no acceptable default non error state alternative.

That's the point of option types and pattern matching, it can force you to handle both cases.

Re:Overhead of Returning Optional Values in Java and Rust by plmpsu on Saturday October 16, 2021 @01:18PM (#28889449)
I agree. Add null checking at compile time with something like checkerframework to get the best of both worlds.
Re:Overhead of Returning Optional Values in Java and Rust by brabel on Saturday October 16, 2021 @01:50PM (#28889813)
Exactly. I don't get why people complain about null-pointers in Java when all it takes to get rid of them is enabling static analysis that enforces checking for null at compile time... as that's exactly as safe as using Optional, and as Kotlin has shown, it's also possible to have a nicer interface (e.g. `something?.let { it.isNotNullHere() }`) with bare nulls... without incurring the cost of allocation or of declaring generic types everywhere.

Since I started using "safe" nulls with Ceylon, nearly a decade ago, later with Kotlin and TypeScript, I've been firmly in the camp of supporting nulls in the language instead of `Optional` or `Maybe`. Rust should've used them IMO... using `?` for error propagation instead is such a weird choice.

Re:Overhead of Returning Optional Values in Java and Rust by Zababa on Saturday October 16, 2021 @02:13PM (#28890016)
? is used for Result, and not for Option. For Option, you use .unwrap(). Supporting nulls in the language also means that everything else is going to be second-class, while sum types allows you to do pretty much everything you want.
Re:Overhead of Returning Optional Values in Java and Rust by brabel on Saturday October 16, 2021 @02:55PM (#28890399)
Looks like you have much to learn about Rust. Option can also use `?`

Example (from [1]):

    // Assume
    // fn halves_if_even(i: i32) -> Option<i32>

    fn do_the_thing(i: i32) -> Option<i32> {
        let i = halves_if_even(i)?;

        // use `i`
    }
Besides, the `?` is syntax sugar for the `try!` macro as explained in the `try!` macro docs [2], and has nothing to do with sum types, at least not directly.

Ceylon would be a closer case where nullable types are represented as sum types and not a special-case in the language (`String | Null` is the same as `String?`)... but Ceylon, as Rust, special-cased the `?` operator for "something"... in Ceylon, they are used for null-checks, and in Rust, for propagating errors via the `try!` macro, which is what my previous comment was about.

[1] https://stackoverflow.com/questions/42917566/what-is-this-qu...

[2] https://doc.rust-lang.org/std/macro.try.html

Re:Overhead of Returning Optional Values in Java and Rust by Zababa on Saturday October 16, 2021 @03:35PM (#28890750)
You're right, I'm not sure how I thought that it worked only with Result. Thanks for the correction.

> Ceylon would be a closer case where nullable types are represented as sum types and not a special-case in the language (`String | Null` is the same as `String?`).

Aren't those union types? My understanding is that they are different from sum types, as you usually have to declare the sum types beforehand, and sum types are collections of values, while union types are collections of types, and are often declared "on the spot". At least that's how they work in Typescript, I don't know much about Ceylon.

To go back to option types, people usually like them because there are well-known patterns to program with them (like map). Of course you can get the same safety with static flow analysis (Typescript does this), but I think part of it is ergonomics, influenced by functional programming. Though if there is an overhead like in Java, I'm not sure about their value.

Re:Overhead of Returning Optional Values in Java and Rust by brabel on Monday October 18, 2021 @07:35AM (#28904357)
I don't know if there's a meaningful difference between sum types and union types as both end up allowing a variable to be of one of a set of types.

Ceylon:

    alias StringOrInt = String | Integer;
    StringOrInt myValue = "foo";
    StringOrInt otherValue = 10;
Rust:

    enum StringOrInt {
        Str(String),
        Int(usize),
    }
    let my_value: StringOrInt = StringOrInt::Str("foo");
    let other_value: StringOrInt = StringOrInt::Int(10);
In Ceylon, you're right you don't need to name a "union", which IMO makes it much nicer, as I was able to use adhoc unions everywhere, e.g. inside streaming operations:

    [ for x in foos if (x.something()) "foo" else 10 ]
The above "automatically" has type `[String | Integer]`. In Rust (and Kotlin) I would need to name a type just for this.

Another disadvantage of Rust's approach is that the type variants are not types themselves. That's really annoying sometimes, e.g. you want to have a method that only handles one of the variants, but you can't (you need to pass the values the variant wraps separately, but that may be undesirable semantically as that would allow something that has no instance of the enum to also use the function).

Re:Overhead of Returning Optional Values in Java and Rust by Zababa on Tuesday October 19, 2021 @05:40AM (#28916391)
I think there is a difference. Sum types allows you to choose between values, union types allows you to choose between types. The values of the sum types can contain themselves types, but they don't have to. On the other hand, you have to declare sum types beforehand, and you don't have that with union types. The equivalent of sum types in Ceylon seems to be enumerated types, which from what I understand you also have to declare beforehand. The difference with Rust is that Ceylon also has union types (Scala is the same, Typescript has unions but lacks proper sum types).

As an interesting aside, Rust has what seem to be intersection types for trait bounds (fn toto<T: Trait1 + Trait2)(a: T)).

I'm still not sure about the tradeoffs between union/intersection types and sum/product types. I think the difference is that "set types" are structural, while algebraic types are nominal, which comes with the usual tradeoffs, but I'm not sure.

Re:Overhead of Returning Optional Values in Java and Rust by Measter on Sunday October 17, 2021 @08:29AM (#28895978)
The `?` operator doesn't desugar to the the `try!` macro, it uses the `Try` trait.

https://doc.rust-lang.org/std/ops/trait.Try.html

Re:Overhead of Returning Optional Values in Java and Rust by brabel on Monday October 18, 2021 @07:22AM (#28904294)
Ah, right... but as far as I know, it's the exact same behaviour, at least currently... reading the docs for the try macro, however, is what made me finally understand the behaviour of the `?` operator (e.g. how it can "magically" convert error types).
Re:Overhead of Returning Optional Values in Java and Rust by kangalioo on Saturday October 16, 2021 @03:05PM (#28890484)
(? works for both Option and Result, as does unwrap(). ? propagates to the caller, while unwrap() initiates a panic and unwinds the stack)
Re:Overhead of Returning Optional Values in Java and Rust by khuey on Saturday October 16, 2021 @02:54PM (#28890384)
? is used for Option. Just like Result, ? propagates the Err/None to your caller, and unwrap asserts that it's not present.
Re:Overhead of Returning Optional Values in Java and Rust by pjmlp on Sunday October 17, 2021 @02:22AM (#28894396)
Because otherwise how would they sell their favourite guest language?

It is like Google uses their pseudo Java dialect when selling Kotlin features, instead of comparing it with proper Java.

Re:Overhead of Returning Optional Values in Java and Rust by pkolaczk on Saturday October 16, 2021 @02:48PM (#28890339)
With Kotlin based approach to nulls you can't represent Option<Option<T>> values.
Re:Overhead of Returning Optional Values in Java and Rust by brabel on Saturday October 16, 2021 @03:00PM (#28890435)
And that's a good thing, no? I've never met a good use cases for double-wrapping values like this.
Re:Overhead of Returning Optional Values in Java and Rust by Zababa on Saturday October 16, 2021 @03:38PM (#28890764)
I'm not an expert on Kotlin but this reminds me of JS promises. The issue was that by not supporting Promise<Promise<T>>, promises were not monadic. It look like the same issue in Kotlin.
Re:Overhead of Returning Optional Values in Java and Rust by nemetroid on Saturday October 16, 2021 @04:13PM (#28891054)
I thought this was a good example: https://news.ycombinator.com/item?id=28746293