But it's completely unusable for the Real World (tm) because strings cannot contain the '[' and ']' characters AND YET there is no mechanism for escape sequences. What if my data legitimately contains those two characters???
That's just the default string type. I started out with escaping but noticed it's a lot of complexity that is rarely needed (not for my own use case, which is accounting, inventory, and some web apps which don't need it).
The advantage of not having escaping is easier seds and greps which don't miss the field boundaries.
The important concept though is that arbitrary datatypes elements can be added by the user of an API implementation (the python library already offers that). The datatypes define their lexical syntax, like in perl6. I will also declare more "default" datatypes and might include a C-like string after enough consideration.
ASCII control characters are forbidden in the entire WSL file. Then apart from [] everything is allowed.
For practical applications, by far the most important requirement of strings is being able to include space characters to make a short sequence of words, like [Trinidad and Tobago].
I don't know a better word for "sequence of words" than "String". Maybe "Words", but technically it really is a string (containing an arbitrary sequence of the allowed characters). Even approaching the enforcement of more structure would be a lot of work with little returns. And you can't include a literal newline in a C string literal, and you can't even have a NUL character in the interpretation (memory layout) of it, right?
I actually started out with a C-like string as default type (so named it "String") but noticed
- A big problem with string culture is that "" strings use identical start and end markers. A problem which Joe Armstrong mentioned as well.
- Escaping means significantly higher complexity of parsing out the interpretation from the literal, while it's not really needed for most applications.
Both these problems make data unnecessarily hard to process with dirty one-off scripts. So after considering some other options I'm now with [this style] because [] are not too often needed or when needed can often be substituted with (), and are very pleasing on the eye in most fonts.
In conclusion I guess it will stay "String", and other less frequently needed types will be called "CString", "Base64", and "BinaryString". Or optional parameterization will be created for "String" to declare the escaping style without needing a separate metatype.
The question is what you do when you're converting some data from some other format (for example, dumping a database) and there are strings that actually contain these special characters. Even if it's a bit ugly, it's good to have some way to represent the data so that it can be read back in again without any loss. In this kind of tool, you can't just say "don't do that" because the data has already been saved - you're just converting it. (So my idea of not calling it a string type probably doesn't make sense, on second thought, if you want to be able to interoperate.)
Square brackets are an interesting choice. If you just want to do simple lossless escaping, it doesn't seem that hard:
\\ means a literal backslash \] means a literal ']'
Anything else gets written as-is. (But, what if the database actually does contain control characters in some of its strings?)
What you implemented there isn't a serialization format, it's a SQL DSL. Any time you implement a DSL, the first question should always be: "Have you considered S-Exprs?". If s-exprs do the job, then you just saved yourself having to implement a ton of stuff, including 90% of a spec.
The conventional wisdom is "All problems in computer science can be solved with another level of indirection". Nevertheless, as stated I will very likely add C-style string literals to the required set of datatypes.
As to "it's a SQL DSL". It's very much not. You missed the point. And S-Exprs is completely out of scope (no schema support, no nice and canonical syntax, etc.)
[1] http://jstimpfle.de/projects/wsl/main.html