Re:Cicada – Unix shell written in Rust
by panic on Saturday July 01, 2017 @03:38AM
(#14675912)
If you use bytes that are invalid in UTF-8 (e.g. 0xF5-0xFF) as delimiters / structure characters, you can use text-only tools to do structured operations on UTF-8 strings without ever having to escape anything -- the structural "characters" you would need to escape can never appear in the encoded bytes.
[ Parent
| on HN ]
> If you use bytes that are invalid in UTF-8 (e.g. 0xF5-0xFF) as delimiters / structure characters
No need for that. UTF-8 is a superset of ASCII, and ASCII already includes a handful of control codes that could be used here.
The Unix shell operates on binary streams, not on text.
Ah, I thought this was about structured text (like JSON) vs. plain text -- binary content is a bigger issue!