You still have == working on all references in this case, but this can be interpreted as reference types having the default value semantics of "every instance is logically different", so referenced values are only equal to themselves.
The annoying part of this approach is that ReferenceEquals() is the one that ends up being used way more often than == in practice for reference types, but it's also much longer to type, and more awkward to use being a function. Python and VB, with their "is" operator for reference equality, handled this one best. Unfortunately, C# wasted "is" as a keyword for something far less common - a type check (and now also pattern matching).
There are languages with a type system that have universal type (and all types are derived from it), in these languages it make sense that "5" == 5 is defined and false.
But C# also tried to improve on Java. One of the things that the latter was commonly criticized for, was the need to use .equals() for strings for value comparisons - which is almost always what you want - and avoiding the more obvious ==. So they overloaded == for strings to mean value equality, while retaining them as reference types.
IMO, the bigger mistake here was to keep strings as reference types - they really ought to be value types (the implementation can still wrap a reference to the actual string data internally, and share it between copies, but the client shouldn't be able to observe that).
My favorite language, C#, has some unfortunate history. The == operator when dealing with references indicates reference equality (Object.ReferenceEquals), but can be overridden to indicate value equality (Object.Equals) even with reference types. Then, when dealing with value types, == suddenly means value equality. It would've been better to define two separate operators for reference and value equality. This would've helped make usages self-documenting.