News for hackers. Stuff that matters.
Re:I'm “still afraid to use spaces in file names” years old by branko_d on Thursday November 11, 2021 @09:09AM (#29187959)
I have an uneasy feeling whenever I see a path parameter declared as string. Path is not a string - it's a sequence of path components and should be treated as such by our APIs. A path should be parsed once - on user input - and then used in its "sequence form" throughout the software stack.

And "path component" is not an arbitrary string either - e.g. appending a path component to the path should first require converting/parsing the string into the path component, and only if that's successful appending it to the path.

Re:I'm “still afraid to use spaces in file names” years old by jerf on Thursday November 11, 2021 @09:14AM (#29188013)
"Path is not a string - it's a sequence of path components and should be treated as such by our APIs."

For maximum correctness, you want to turn it into a file handle as soon as possible, and do all operations through the variations of the file functions that end in "at", like: https://linux.die.net/man/2/openat

The downside of this approach is that you still technically have to carry the path around with you if you ever want to present it back to the user, because once you have a directory handle, you can get back to the root directory easily enough by following parent links and seeing what directories you end up in, but that may not be what the user "thinks" the path is, and they want to see their path, not a canonicalized one. And they're mostly right. And it's not easy to correctly track changes to their intended path from this basis either.

Basically, I don't know of a really solid, 100% correct way to handle this with any reasonable degree of effort.

Re:I'm “still afraid to use spaces in file names” years old by jmull on Thursday November 11, 2021 @11:39AM (#29189942)
> For maximum correctness, you want to turn it into a file handle as soon as possible

That's not right. You want to resolve a file/folder path to a file/folder at the exact point it makes sense.

It's a problem if you're using a path when you wanted the file. The file can be switched/modified out from underneath you.

It's also a problem if you've got the file when you only wanted a reference. Now you can't simply switch/modify the file independent of the reference. E.g., maybe you want config file changes to take effect immediately and transparently.

You can also have the hybrid case, e.g., where you want the folder directly, but have a relative path to a file that is resolved late.

If you're unsure, I'd err on the side of late resolution.

Re:I'm “still afraid to use spaces in file names” years old by Pxtl on Thursday November 11, 2021 @09:48AM (#29188363)
"you want to turn it into a file handle as soon as possible"

But no sooner.

For example, I've run into problems where I'm configuring program A server to talk to file location B... but I don't have access to file location B. But the client-side library for talking to the server tries to convert location B into a file handle and then freaks out because I can't access it. When I don't want to access it. I want that program to serve it.

If it was using simple "path" objects that didn't confirm that I have access to the path, everything would be hunky dory. But because it tried to convert it into a file handle unnecessarily, I get blocked.

Re:I'm “still afraid to use spaces in file names” years old by tmerr on Thursday November 11, 2021 @11:51AM (#29190115)
Another inconvenience with this approach is that you can keep thousands of paths in memory no problem. But thousands of FDs may cause you to exceed per-process limits.
Re:I'm “still afraid to use spaces in file names” years old by globular-toast on Thursday November 11, 2021 @10:56AM (#29189323)
This goes for most instances of user input. Timestamps is the other common one people get wrong. I've even seen programs that pass around timestamps as strings in multiple formats and as integers (Unix time).
Re:I'm “still afraid to use spaces in file names” years old by aqfamnzc on Thursday November 11, 2021 @11:16AM (#29189608)
As a programming noob, I'm wondering what would be the better way to pass or return a unix time value as opposed to an integer?
Re:I'm “still afraid to use spaces in file names” years old by mleonhard on Thursday November 11, 2021 @06:44PM (#29194729)
If you need to keep the timezone with it, then use an ISO8601 [0] string: "2021-11-11T15:32:35-07:00".

Otherwise, use an integer unix timestamp, the number of seconds since 1970-01-01T00:00:00Z: 1636673555. Use an unsigned 32-bit integer or a 64-bit integer to avoid the 2038 problem [1]. JSON's maximum safe integer value is a signed 53-bit integer, so if you're using HTTP JSON RPC, you'll have to check for overflow.

[0] https://en.wikipedia.org/wiki/ISO_8601

[1] https://en.wikipedia.org/wiki/Year_2038_problem

[2] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

Re:I'm “still afraid to use spaces in file names” years old by globular-toast on Friday November 12, 2021 @04:33AM (#29197628)
ISO8601 is a serialisation format. You wouldn't want to use it in internal function calls simply for performance reasons. You also wouldn't want to pass it around as just a "string" type. I think the question was asking about internal function calls. For external data interchange, ISO8601 is the only sane option and deals with all known timezone and leap second bollocks.
Re:I'm “still afraid to use spaces in file names” years old by globular-toast on Thursday November 11, 2021 @11:25AM (#29189741)
Depends on the language but most high-level languages have a timestamp or datetime abstraction which you should be using.
Re:I'm “still afraid to use spaces in file names” years old by joe_guy on Thursday November 11, 2021 @03:23PM (#29192678)
If it's being serialized, consider fully qualified iso8601.
Re:I'm “still afraid to use spaces in file names” years old by BoorishBears on Thursday November 11, 2021 @11:06AM (#29189454)
> For maximum correctness, you want to turn it into a file handle as soon as possible

This is why I get stressed out when I see paths turned into special objects encoding separators and such.

It tells me the path is living for way too long compared to the file handle.

I only want to see path-specific objects if we're modifying the path, and even then I want that to happen as late as possible.

Re:I'm “still afraid to use spaces in file names” years old by aspaceman on Thursday November 11, 2021 @09:59AM (#29188491)
Why not just hold onto both? The users representation and the file handle. Only ever "display" the representation, while you do all operations on the handle. (Not trying to be sarcastic, just curious).
Re:I'm “still afraid to use spaces in file names” years old by cerved on Thursday November 11, 2021 @05:30PM (#29194106)
doesn't this lock the file?
Re:I'm “still afraid to use spaces in file names” years old by dahfizz on Thursday November 11, 2021 @09:55AM (#29188442)
> I have an uneasy feeling whenever I see a path parameter declared as string. Path is not a string

I guess that depends on what you mean by "string". `open` and `fopen` need a char* path to open a file. Whatever fancy Path abstraction you use eventually becomes a char* string, because that's what the kernel needs.

Re:I'm “still afraid to use spaces in file names” years old by VWWHFSfQ on Thursday November 11, 2021 @10:32AM (#29188946)
yeah. it's a string.
Re:I'm “still afraid to use spaces in file names” years old by dwheeler on Thursday November 11, 2021 @10:45AM (#29189157)
On POSIX systems file names are not strings, they are sequences of bytes. They might not be UTF-8 or have any meaning. Python3 had to hack around this, they thought they could force everything to Unicode and discovered that doesn't work.
Re:I'm “still afraid to use spaces in file names” years old by guntars on Thursday November 11, 2021 @10:53AM (#29189281)
Which makes for fun issues like there's no standard way to display a filename in Unix. A system that's, you know, all about files.
Re:I'm “still afraid to use spaces in file names” years old by sipos on Thursday November 11, 2021 @05:07PM (#29193822)
At least for most Linux systems (not sure about other *nix, but I expect the same?), there is a system default encoding, defined by the locale, and I think decoding the filename in that encoding and displaying the resulting string, is probably the correct way to display a filename? That seems as good as you are likely to get on any system really.

I think for any POSIX system, either there is locale support defining the encoding, or it uses the POSIX locale, which defines the encoding (ASCII).

Of course you need to handle cases where filenames cannot be decoded in the system encoding (probably by replacing characters that cannot be decoded), because a filename in a different encoding, or even with no valid encoding, has been used on disk. While systems can say that file names containing bytes that are not valid characters in the system's encoding are not valid file names, that doesn't stop people mounting disks with them, so the problem never goes away if you support opening media from other systems.

What I am saying is that this is no more a Unix problem than it is a problem on any system that supports removable media.

Re:I'm “still afraid to use spaces in file names” years old by warkdarrior on Thursday November 11, 2021 @11:24AM (#29189723)
Unix: everything is a file, including file names!
Re:I'm “still afraid to use spaces in file names” years old by duped on Thursday November 11, 2021 @01:41PM (#29191395)
That's probably because paths aren't properties of the file itself, they're helpers to reference the file.
Re:I'm “still afraid to use spaces in file names” years old by account42 on Friday November 12, 2021 @09:36AM (#29199617)
On POSIX system file paths are C strings, which are sequences of bytes that cannot include the 0 character. UTF-8 or oher meaning is not required for something to be a string.
Re:I'm “still afraid to use spaces in file names” years old by anyfoo on Thursday November 11, 2021 @04:24PM (#29193389)
Strings following certain rules are entirely valid representations of paths, just like sequences of path components in the chosen language/framework are. Similarly, the sequences of bits that make up the sequences of your language/framework in memory are an entirely valid representation of said sequences of components.

Yes, paths have structure, but saying "a path is not a string" is equivalent of saying "C source code is not a string". Both are strings, and both are something else, represented by strings according to rules. Different internal representations have different advantages and disadvantages. I fully agree that for things such as "adding components" an internal sequence/list representation is better, but strings can pass arbitrary IPC or even ABI boundaries much easier for example. (And you wouldn't bat an eye for example when you see FQDNs like "www.google.com" passed as a string instead of as ["www","google","com"] because the string representation works pretty well.)

Re:I'm “still afraid to use spaces in file names” years old by fouric on Thursday November 11, 2021 @04:56PM (#29193701)
C source code and paths are both representable by strings, true, but the fact that they're not actually strings is still important, because most people don't know that, and in the case of paths that leads to a lot of edge cases (in the case of source code it leads to a bunch of inefficient and weak tooling, which isn't quite as bad).

Because neither are strings, their native representation shouldn't be such - it should be something structured, and only when necessary (IPC, FFI, serdes) be serialized into a string representation. This would save people a lot of time and effort.

Re:I'm “still afraid to use spaces in file names” years old by anyfoo on Thursday November 11, 2021 @07:57PM (#29195177)
It really depends. Do you usually keep hostnames as strings? URLs? JPEGs? Why or why not?

Sure, a browser will hopefully quickly parse that URL and break it up, an image viewer will do the same with a JPEG. Will anything that's only interested opening/displaying that URL or JPEG, through a library or external program?

POSIX paths are actually remarkably simple in structure[1]. The only caveat is equality and normalization: Without normalization, a path a might be equal to a path b while their representations differ, e.g. "/etc/foo" and "/etc/bar/../foo". But this is the same whether you have a string or a list of strings, you need to normalize in whatever representation you choose to check for equality.

[1] Almost shocking myself, even Haskell defines its primary FilePath type literally as "String".

Re:I'm “still afraid to use spaces in file names” years old by SAI_Peregrinus on Thursday November 11, 2021 @11:27AM (#29189771)
POSIX "Fully portable filenames" allow all characters except 0x2F (/) and 0x00 (NULL). That means file names can include line feeds, backspaces, EOF, etc.

"This is `a

perfectly vali'd.\010! file name\377, despite the weirdness"

Re:I'm “still afraid to use spaces in file names” years old by naikrovek on Thursday November 11, 2021 @09:12PM (#29195614)
things like this are why the Unix philosophy is so bad.

text processing is hard if you must support Unicode, and that means every Unix command line tool must implement or employ a text processor to handle input. it would be much easier if objects were passed back and forth. PowerShell got this right.