An interesting question here would be how many of those hundreds of failures were actual bugs and how many were just were just side effect of changing "dynamic is both top and bottom" rule.
So far I have not encountered a single actual issue when migrating users' code to Dart 2 - I encountered "unsoundly" typed code, but it actually worked and did not contain bugs.
1. Is developed at a company with a stronger-than-average culture around quality and tests.
2. Already has copious tests.
3. Is mostly in applications that have been shipped in production.
So most of the bugs were already flushed out before we got to the point where we started tightening things. The more interesting question, which we don't have data for, is how much time it took to flush those bugs out the first time and how much the stricter runtime checking affects that. Hopefully it helps users find these kinds of issues faster.
Sincere question, what am I getting wrong?
But, I think in practice, most languages that we call "sound" do indeed rely on some amount of runtime checking in order to guarantee that. Java and C# are both widely considered sound — and need to be to preserve memory safety! — and they both use runtime checking to ensure that (on cast operators and covariant array usage). I think even Haskell defaults to runtime checking of non-exhaustive pattern matching, which could be considered a form of "type errors".
> Doesn't your definition mean that most type systems (even those of weakly and dynamically-typed languages) are sound.
Dynamically-typed languages detect the type error right at the point where you try to perform an invalid operation for that type. They are safe in that that doesn't generally lead you into a hard crash or scary undefined behavior.
C# and Java are similar with respect to array usage — you get the type error right when you attempt to do a covariant store. Dart is similar for covariant generics.
But in most other cases, Dart detects the error earlier, at the point that you attempt to cast a value to the wrong type, not when you later use it as that wrong type.
We do have allocation tracking in some of those, but that doesn't always get you as much as you'd expect. Discovering that, say, "oh, this list was deserialized by the JSON API" still means it could have come from dozens of different regions of the program.
I work on a language, Dart, where all generics are covariant. In Dart 1, invalid uses of covariance were not checked at runtime and were silently ignored. When that leads to wrong behavior down the road, it's profoundly confusing. Not checking this and sacrificing safety, also ties your hands when it comes to compile time optimization because you can longer take any of your type annotations seriously.
In Dart 2, we've moved to a sound type system, which involves checking these misuses at runtime. I have personally gone through and fixed hundreds of these runtime failures. Others at Google have done the same. It's grueling, difficult work. Figuring out where an object was constructed before it eventually wound up somewhere in a covariant position is not easy.
I would much rather have static control over variance even with the additional complexity if causes, and I've heard similar requests from many users.