edit: fixed formatting problem

> sentinel values

Using in-band signaling always involves the risk of misinterpreting types.

> This is part of more general problem

DWIM ("Do What I Mean") was a terrible way to handle typos and spelling errors when Warren Teitelman tried it at Xerox PARC[1] over 50 years ago. From[2]:

>> In one notorious incident, Warren added a DWIM feature to the command interpreter used at Xerox PARC. One day another hacker there typed

    delete *$

>> to free up some disk space. (The editor there named backup files by appending $ to the original file name, so he was trying to delete any backup files left over from old editing sessions.) It happened that there weren't any editor backup files, so DWIM helpfully reported

    *$ not found, assuming you meant 'delete *'

>> [...] The disgruntled victim later said he had been sorely tempted to go to Warren's office, tie Warren down in his chair in front of his workstation, and then type 'delete *$' twice.

Trying to "automagically" interpret or fix input is always a terrible idea because you cannot discover the actual intent of an author from the text they wrote. In literary criticism they call this problem "Death of the Author"[3].

[1] https://en.wikipedia.org/wiki/DWIM

[2] http://www.catb.org/jargon/html/D/DWIM.html

[3] https://tvtropes.org/pmwiki/pmwiki.php/Main/DeathOfTheAuthor

>> [...] The disgruntled victim later said he had been sorely tempted to go to Warren's office, tie Warren down in his chair in front of his workstation, and then type 'delete $' twice.

Ironically, this did not render the way you intended because HN interpreted the asterisk as an emphasis marker in this line.

It works here:

    ... type 'delete *$' twice.

because the line is indented and so renders as code, but not here:

> ... type 'delete $' twice.

because the subsequent line has emphasized text*. So the scoping of the asterisks is all screwed up.

Eh. "Death of the Author" is a reaction to the text not being dispositive as to what the author meant. It's deciding you don't care what the author meant, no longer considering it a problem that the text doesn't reveal that. Instead the text means whatever you can argue it means.

Which can be a fun game, but is ultimately pointless.

It gets more complicated when the author himself changes their mind about that.

That’s a shrewd observation. Static types help with this somewhat. E.g. in Inflex, if I import some CSV and the string “00.10” as 0.1, then later when you try to do work on it like

x == “00.10”

You’ll get a type error that x is a decimal and the string literal is a string. So then you know you have to reimport it in the right way. So the type system told you that an assumption was violated.

This won’t always happen, though. E.g. sort by this field will happily do a decimal sort instead of the string 00.10.

The best approach is to ask the user at import time “here is my guess, feel free to correct me”. Excel/Inflex have this opportunity, but YAML doesn’t.

That is, aside from explicit schemas. Mostly, we don’t have a schema.

If we're talking about general problems, then I don't think we can be satisfied with "sometimes it's a problem with types and sometimes it's a UI bug." That's not general.

> E.g. sort by this field will happily do a decimal sort instead of the string 00.10.

So that system is not consistent with type checking? How is this not considered a bug?

I mean if the value is imported as a decimal, then a sort by that field will sort as decimal. This might not be obvious if a system imports 23.53, 53.98 etc - a user would think it looks good. It only becomes clear that it was an error to import as a decimal when we consider cases like “00.10”. E.g, package versions: 10.10 is a newer version than 10.1.

Types only help if you pick the right ones.

Sure. In most static type systems though, you would be importing the data into structures that you defined, with defined types. So you wouldn’t suddenly get a Decimal in place of a String just because the data was different. You’d get a type error on import.

And of course the plague that is CSV when your decimal delimiter is ,

Basically, autoimmune disease, but for software.