Escape characters are one of the most stupid things in the computing world.

The formats could be so simple: first the length of the data, then raw data of that length

The better solution is allow control over what the terminating characters are. Rust raw strings allow N ‘#’ characters around them, like r”x”, r#”x”, r#####”x with “quotes””#####. So no escapes are ever needed, just increase the number of hashes to outstrip the maximum consecutive hashes appearing in the string after a quote character.

But then we would never be able to treat text as a stream. All of your one pass algorithms would become two pass.

I’m curious what you mean by this — why does coding for length prevent streaming? The receiving end can certainly treat the text as a stream still. Do you mean that the sender cannot “stream” if they are generating the contents on the fly? That seems trivial to solve - just break it up into chunks, assuming a little bit of buffering is OK.

>How would it allow you to write a regex that matches a dot?

You could write

    1:.

>Or text in an XML element that includes an ampersand?

The point is to not use XML, because it has escape characters. You could replace it with ASN.1 BER

>Even for quotes contained in string literals it's a shitty solution because string literals are part of programming languages and having to count characters every time you change a String literal is just about the last thing you want in your language (and it would be horrible to read as well).

The IDE could update the length automatically

> You could write 1:.

So separate wildcards and literals, each prepended by length? That would be far, far less readable than using escape sequences.

> The point is to not use XML, because it has escape characters. You could replace it with ASN.1 BER

Fair enough, but most people would consider that a solution worse than the problem.

> The IDE could update the length automatically

Making you completely dependent on an IDE to support whatever format you're editing, including nesting. You can't even read your code now without an IDE. Making up some kind of DSL on the fly? Start counting string lengths again.

Sorry, but no. Escape sequences are bad, this would so much worse.

>So separate wildcards and literals, each prepended by length? That would be far, far less readable than using escape sequences.

Wildcards do not need a prefix, when they always are a single character.

Here is a ordinary regex for a path:

    C:\\foo\\bar\\.*\.txt

And here the regex in C:

    "C:\\\\foo\\\\bar\\\\.*\\.txt"

But with lengths in a hypothetical syntax:

    11:C:\foo\bar\.*4:.txt


    22:11:C:\foo\bar\.*4:.txt

That is quite readable

> You can't even read your code now without an IDE. Making up some kind of DSL on the fly? Start counting string lengths again.

I like binary formats. The lengths are easy to see in a hex editor. 16 bytes in each line or something

You always depend on an IDE to decode the file. You could not read an UTF-8 file, if it would not decode the UTF-8 characters and show individual bytes. Or read ASCII if it were showing individual bits.

Irrespective of who's right or wrong it's pretty unnecessary to call each other names just because you don't agree, particularly when the question is theoretical and has almost no real world impact.

Commenter above, correct me if I’m misinterpreting, but — I imagine they meant for when programs talk to other programs. Regexes and string literals are written by humans; it would be easy for the compiler/program to calculate the length for you.

The general solution is parametric construction. When we're building SQL queries, regular expression matchers, HTML files, etc., we're actually attempting to build a richer data structure but representing it as a flat string. Instead of concatenating strings, we can turn our original string into one where some leaf nodes are placeholders for future literal entities, which can be substituted for their actual values without fear of potential metacharacters that could be misinterpreted.

While the solution may be fairly clear, not everyone provides easy interfaces for parametric construction--regular expression engines being the one where the lack is the most distressing for me (although this is less because I need to substitute in user-generated strings and more because I want to combine smaller regexes into larger ones to make the regex easier to follow).

Which would, again, only help with delimiters, but not with other characters that have special meaning but also need to be used literally.