That, plus you'd be a fool to store a huge string in this way in any case.
Not necessarily. A 64-bit system could give processes an address space that’s significantly larger than half the full 64-bit address space and have an allocator that allows you to allocate a block of more than SIZE_MAX bytes (malloc takes a size_t, but you can use calloc)
The calloc() [1] function mentioned above takes two values of type size_t, and allocates their product bytes.
I'm on mobile without (!) the C99 draft spec but at least the man page gives no such restriction.
However, the standard says (https://en.cppreference.com/w/c/types/size_t):
“size_t can store the maximum size of a theoretically possible object of any type (including array).”
and (https://en.cppreference.com/w/c/language/pointer):
“Pointer is a type of an object that refers to a function or an object of another type, possibly adding qualifiers. Pointer may also refer to nothing, which is indicated by the special null pointer value.”
⇒ pointers must either be null or point to an object, and objects aren’t larger than SIZE_MAX, so I think having a pointer pointing to a block larger than SIZE_MAX violates the standard.
it's simply not possible, by definition.
calloc returns NULL when it's not possible to satisfy the request.
On a quick test on my 64-bit system, a C program doing `printf("%zu\n", SIZE_MAX);` outputs 18446744073709551615, which looks like (2^64)-1 to me.
Or is there a thing in the standard that says this isn't always the case?
> The type ssize_t shall be capable of storing values at least in the range [-1, {SSIZE_MAX}].
[1] https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/sy...
Note that neither is ptrdiff_t guaranteed to be that signed version, or to hold any possible value in the domain of size_t or (strictly speaking) any possible object size. Both GCC and Clang assume the latter, though, and can miscompile[3] code that relies on (e.g.) malloc() succeeding for sizes > 2^31 on a 32-bit system.
[1] https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/sy...
[2] https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/li...
[3] https://trust-in-soft.com/blog/2016/05/20/objects-larger-tha...
In the same way that since we identify unicode code points with a 16-bit value, it's impossible to include U+1D460 in a string?
In the same way that since Matroska files encode the length of their segments, there's a hard upper limit on the length of a segment?
Of course none of those things is actually true. Storing the string size has no implications for how long the string can be. It requires an amount of space, to store the string size, that is logarithmic in the length of the string, and completely insignificant.
Not a single 16-bit value. Some codepoints are two 16-bit values.
What do you gain in handling short strings from an inability to handle long ones?
Flexible width encoding is more complicated compared to simple member access to get at the first character. And how do you handle construction of a string whose size you don't know yet? You might have to move the string away to make space for a bigger string length field. I don't like it.
I don't think this is true either. It's almost true. But what happens if the string length is 0?
If you make the assumption that you can access the first character of a zero-length string by just grabbing whatever is in memory after the string header, you're going to make the exact mistake the length field is there to stop you from making, a memory access violation. You have to process the length field in order to do any access at all; many strings don't have a first character.
> And how do you handle construction of a string whose size you don't know yet? You might have to move the string away to make space for a bigger string length field.
That's true; you'll either need to be willing to store the character data and the length metadata in separate locations, or you'll need to be willing to occasionally move the data around.
1. Read the first chunk of the string length.
2. Is it more than 0?
1. Read the first chunk of the string length.
2. Did we get the whole thing?
3. Is the length more than 0?
---
Also, it occurs to me that this whole discussion is talking about how to serialize or deserialize a string, when the original discussion is over how the string should be represented in memory.
c0 (0b0000) -> length is 0xc = 12
1341 (0b0001) -> length is 0x134 = 69,940
239a42 (0b0010) -> length is 0x239a42 = 145,828
deadbeaf239a47 (0b0111) -> length is 0xdeadbeaf239a4 = 3917404957718948.
Maybe reserve 0x1111 for future use.
Maybe the other endian makes more sense here, and maybe 0 should mean zero-length.
It's probably not very performant compared to other solutions, but you can just shift 4 bits, and you're done
I'm curious how many strings are allocated at a particular point in time (across everything, kernel, os, apps, etc)
Other popular approaches to allocate strings dynamically would generally group strings of similar lifetime or by size, amortizing possible overheads over many strings.
If your favorite memory allocator requires a size field to support efficient deallocation, then by all means put that in the allocation record. But this is totally orthogonal to the format of a simple string representation.
In fact, in standard Pascal, string handling is extremely rudimentary; there was no way to express "this variable / parameter / pointer refers to a string with a length not known at compile-time".
https://www.iso.org/standard/18237.html
Additionally, Modula-2 was already available in 1978, sorting out all the issues of original Pascal, with all the features needed for a safe systems programming language in the late 70's.
Do you seriously expect in two years left of 70's for Modula-2 to be available everywhere?
But we digress. The topic at hand was "How does Pascal store strings?" I stand by the pedantic statement that ISO Standard Pascal does not require string lengths to be stored with the string (nor anywhere else); there's no way for the programmer to obtain the length of a string at compile-time, never mind run-time, without resorting to some sort of compiler-specific language extension (most (all?) of which did indeed put the string length in front). Conversely, the Extended Pascal standard pretty much requires strings to instead be implemented as "fat pointers," consisting of a length and a pointer to the actual characters. I say this because substring operations return references to pieces of the original string, thus you can't stuff a length in front of the actual characters of the substring, as this would over-write characters in the original string.
That is how strings use to be stored before C made the choice of using null-terminator. Pascal stored the string size before the string data. The advantage of relying on a terminator symbol is that the string size can be any length where as storing the size at the start forces the string to not exceed certain size.