This drives me absolutely bonkers. We had a situation in a machine learning context where we should have been treating the output of a detector as NaN (it was deliberately opting out), but instead it was falling back to zero.

Last I checked (this was years ago) the way to deal with this is to wrap it in a sub-message (which are actually nullable), which is just kinda gross.

I thought proto3 has support for optional scalar fields now (https://github.com/protocolbuffers/protobuf/releases/tag/v3....), and proto2 always had it

Thats why JavaScript has both undefined and null!

Do we really need to differentiate between undef and null?

Not necessarily, but it does allow a level of robustness to be able to distingish between values which have deliberately been assigned null, and values which have not been initialized correctly.

Lua does fine with only a nil, although it gets a tad annoying when you actually want to store a nil value in an array and end up creating a hole instead. (Arguably, the solution to that problem is to use your own marker object and not a nil when you need some sort of unspecified or default value which is not a literal absence of one.)

C# linqs method - first or default has the same problem

The method itself just uses the default value of the corresponding type. If you have a sequence of ints, and you cast it to Nullable<int> first before applying FirstOrDefault, then you can distinguish between null and 0.