Re:How UTF-8 Works
by andrepd on Tuesday February 08, 2022 @01:01PM
(#30261948)
That's in general a problem with dynamic languages with weak type systems. How "Your code runs without crashing" is really really != "your code works". How do people even manage production python! A bug could be lurking anywhere, undetected until it's actually run. Whereas in a compiled language with a strong type system, "your code compiles" is much closer to "your code is correct".
[ Parent
| on HN ]
Re:How UTF-8 Works
by BiteCode_dev on Tuesday February 08, 2022 @02:42PM
(#30263483)
I don't think a type system can help you with decoding a file with the wrong charset.
A type system can refuse to turn a `bytes` into a `utf8str` until it's been appropriately parsed.
(It doesn't even need to be a very good or strongly-enforced type system - Go makes it dangerously easy to convert between `[]byte` and `string` by other-type-system standards, and yet everything works pretty well. It's enough to hitch your thinking and make you realize you need another step.)
Re:How UTF-8 Works
by BiteCode_dev on Wednesday February 09, 2022 @09:01AM
(#30272686)
How does the type system knows the bytes are utf8 and not CP850?
The same way you can parse a string into an integer and it knows you got an either[integer, err].
Re:How UTF-8 Works
by BiteCode_dev on Wednesday February 09, 2022 @12:04PM
(#30275008)
So it doesn't, and it's not about the type system. Any decode operation in python 3 raise an exception or return string.
It's about the runtime error handling making it mandatory to deal with the error and won't let you panic at run time for this specific error.
I'm sorry you don't see exceptions as part of a type system yet.
The compiler can raise a warning about bad characters. It can't detect all problems, but it certainly can help with some.
It's not the type system, but many dynamic languages are interpreted instead of compiled.
Re:How UTF-8 Works
by dotancohen on Wednesday February 09, 2022 @03:09AM
(#30270201)
Those bad characters are fed into the program long long after the compiler has done its job.
Not necessarily? I'm talking about constant string literals in source code which can be validated at compile time.
But this is not a matter of UTF-8 in the code, rather a matter of UTF-8 in the input or output. How does compiling a program ensure that it is robust on a range of inputs?
> How does compiling a program ensure that it is robust on a range of inputs?
This is quite literally the job of a type system: to impose a semantic interpretation on sequences of "raw bits" and let you specify legal (and only legal) operations in terms of the semantic interpretation rather than the bits.
There are a number of mitigations, so those kind of bugs are quite rare. In our large code base, about 98% of bugs we find are of the "we need to handle another case" variety. Pyflakes quickly finds typos which eliminates most of the rest.
This is the difference between people who embrace static typing and everyone else. A static type lover hears that 98% of your bugs are of the "we need to handle another case" variety and says, "well, that means you could have gotten rid of 98% of your bugs with better typing".
No, what I mean is that an additional key comes in (with the json or similar hash) and we now need to do some thing with it, or something different than we thought we were supposed to with it. Typing is not going to fix it because the full cases were unknown at development time.
> "your code compiles" is much closer to "your code is correct".
That is so far from the truth.
Re:How UTF-8 Works
by ben-schaaf on Wednesday February 09, 2022 @01:05AM
(#30269481)
How is it anything but the truth? The express purpose of static analysis, like a type system, is to catch bugs before running your code. That pretty clearly means that code that successfully compiles is closer to being correct than code that doesn't.
I mean, correct as in logically correct and that's just a different matter than being grammatically correct.
The parser assures your code is grammatically correct; the type system assures your code is semantically consistent, which is usually a much stronger guarantee, and by most practical measures will be closer - often much closer, and for total functions on total types, sometimes all the way - to "logically correct".