News for hackers. Stuff that matters.
Re:Fixing Unix Filenames (2012) by overgard on Thursday October 31, 2013 @12:51AM (#6645316)
By simple I mean this:

* Printable (space excluded)

* Doesn't change the flow/layout of characters (no weird reverse characters or superscript/subscripts or characters that flow up/down).

Will that make everyone happy? Of course not. But it's pretty much "expressive enough" and it makes dealing with filenames 10x simpler.

Which policy seems more realistic:

A) sensible restrictions on file names that will please the majority of people and keep most programs working, while slightly annoying a lunatic fringe that wants EVERYTHING.

B) Require that every program that handles files in some capacity can handle every single possible weird unicode and control code subtlety in existence? (Keeping in mind that even browsers, some of the most complicated pieces of software in existence, still tend to screw up the corner cases).

Re:Fixing Unix Filenames (2012) by ketralnis on Thursday October 31, 2013 @01:20AM (#6645395)
> Will that make everyone happy? Of course not. But it's pretty much "expressive enough" and it makes dealing with filenames 10x simpler.

Simple enough for a Western language speaker, sure. Pretty inhibiting for a speaker of Kannada that they can't name their résumé file with their own name.

> Require that every program that handles files in some capacity can handle every single possible weird unicode and control code subtlety in existence?

I assert again that once you allow spaces, you're already in the situation that you need to treat filenames as sequences of bytes that need quoting.

Also, I want to point out again that almost all of these problems are only when writing shell scripts. C programs or e.g. GUI applications with file pickers tend (with exceptions of course) to not have these problems since you get the filenames into them.

Re:Fixing Unix Filenames (2012) by overgard on Thursday October 31, 2013 @01:40AM (#6645445)
Ok so fine, we allow people to write filenames top-to-bottom.

So programs that show the filename in the title bar: how do they display it? Really big title bar? Align the title bar on the left or right? What if I mix english and Kannada? What then?

Or what if I start with an english word in the terminal, and then switch to kannada, what's the terminal supposed to do?

I haven't used a lot of asian language systems, but I'm going to guess two things that I'm pretty sure are true:

A) they're already implicitly dealing with these restrictions, whether the file system enforces it or not

B) they probably already have workarounds to deal with these cases anyway. (alternate scripts or whatever).

Regardless of culture, I'm going to make what I think is a pretty obvious statement: good filenames are easy to type and unambiguous. If your filename is neither of those, it's not a good filename. The point of a filename isn't to be as expressive as possible, it's to be universal, descriptive, and easy to work with. None of that requires special characters.

With regard to the space thing, I think it's overblown. Spaces are common enough that good software engineers think to handle it. Newlines in files? I've /never/ seen that, even though it's possible. Also, space doesn't require an escape character. Once you start getting to newlines, it does. Once you start introducing escape characters into it, it's an entirely new layer of complexity.

Re:Fixing Unix Filenames (2012) by josteink on Thursday October 31, 2013 @04:28AM (#6645744)
> So programs that show the filename in the title bar: how do they display it?

That hardly sounds like a file-system issue. That's an application and UI-design issue.

Just because you think some of these issues are hard and you don't know how to deal with them on an application level (beacuse you haven't had to yet), doesn't make that a valid reason into limiting the capabilities of the file-system.

Re:Fixing Unix Filenames (2012) by hdevalence on Thursday October 31, 2013 @02:02AM (#6645492)
Why are ASCII regarded as "normal characters", but Kannada, or Hangul, or Arabic, or whatever are "special characters"?
Re:Fixing Unix Filenames (2012) by jerf on Thursday October 31, 2013 @08:59AM (#6646495)
There is a reasonable, non-"culturist" answer to your question, which is that ASCII admits of a monospace typesetting in which each character is fully isolated in its own box and does not interact with any others across the box, nor are there any combining marks or any of the other vagueries of Unicode, and there's a very limited number of such characters. Possibly Hangul can claim that, I know Arabic and some other scripts can not. Other languages that can claim this include I believe all European languages (the accents are rare enough you can just represent them as characters of their own rather than combining marks) and Cyrillic. None of the ideographic languages can currently claim this, though I heard rumors that Japanese and Chinese are both experiencing some heavy pressure internally to go to an alphabetic subset of their language that actually would meet this criterion. (And not due to authoritative mandate, but rather the natural flow of language.)

It is a problem when you have languages with fundamentally different rules trying to share the screen, and that is also not a culturist observation; indeed one would think that denying this would be the culturist position.

Re:Fixing Unix Filenames (2012) by asdffdsa33 on Thursday October 31, 2013 @12:12PM (#6647657)
For Japanese, in a Unicode context, I've never seen a combining mark actually used.
Re:Fixing Unix Filenames (2012) by duskwuff on Thursday October 31, 2013 @01:11PM (#6648057)
While the precombined versions are much more common, Japanese does have combining characters to add voiced and semi-voiced marks to syllabic characters (e.g, か vs が, は vs. ぱ).
Re:Fixing Unix Filenames (2012) by jerf on Thursday October 31, 2013 @02:40PM (#6648855)
The ideographic languages can't claim "a limited number of characters". (7-bit) ASCII fits on a typewriter ball. (By "limited", I meant small, rather than merely finite.)

Of the various points I mentioned, in modern times this is probably the least important, now that RAM, ROM, and CPU are so dirt cheap. This wasn't always true.

Re:Fixing Unix Filenames (2012) by asdffdsa33 on Thursday October 31, 2013 @01:37PM (#6648277)
It does, but I've never seen them used in a Unicode context.
Re:Fixing Unix Filenames (2012) by einhverfr on Thursday October 31, 2013 @06:52AM (#6646090)
Shouldn't display be a feature of the locale?
Re:Fixing Unix Filenames (2012) by hisham_hm on Thursday October 31, 2013 @01:08PM (#6648029)
> Pretty inhibiting for a speaker of Kannada that they can't name their résumé file with their own name.

That's what transliteration is for. I have no problems writing my name transliterated into the Latin script.

Re:Fixing Unix Filenames (2012) by josteink on Thursday October 31, 2013 @04:24AM (#6645734)
> But it's pretty much "expressive enough"

You mean like ASCII was "expressive enough" until people whose language was not English started using computers?

Your western bias shows.

That "expressive enough" attitude got us code-pages and a million different encodings to cope with the limitations we had put upon ourselves, and the related (and sometimes impossible) compatibility and inter-op issues.

Let's not walk into that one yet again, shall we?

Re:Fixing Unix Filenames (2012) by selmnoo on Thursday October 31, 2013 @05:35AM (#6645898)
A really interesting case is Urdu. In Nastaliq script, the Urdu language word-wise goes right-to-left... diagonally. Here's a picture: http://i.imgur.com/nIHmZJc.png See how every next letter is just a little bit lower than the one preceding it? That's Nastaliq script, it goes left-down-ward.

As you could imagine, it's a special hell supporting this language. What's happened now is... a compromise has basically been made. Nastaliq is how Urdu had predominantly been written for years and years... but with growing usage of computers, even native Urdu speakers are abandoning it, because there's such poor support for it.

This article talks a little more about this: https://medium.com/stories-that-matter/9ce935435d90

Re:Fixing Unix Filenames (2012) by toast0 on Thursday October 31, 2013 @01:08AM (#6645358)
If you're going to whitelist characters from unicode, we're going to have several problems with managing the whitelist:

a) overhead for the whitelist is going to be an issue on some systems

b) I'm going to need a kernel update every time there's a new useful code point for example, the Indian Rupee Sign http://www.fileformat.info/info/unicode/char/20b9/index.htm

Re:Fixing Unix Filenames (2012) by dwheeler on Thursday October 31, 2013 @12:07PM (#6647608)
You can just check "is it UTF-8 encoded?". Then it doesn't matter when a new code point is added, it would still be correctly encoded. You could make this a configurable kernel option; when enabled, filenames can only be created when they encode with UTF-8.

The next big problem is control characters. But really, the only people who need control characters embedded in filenames are people breaking into computers using them. It doesn't matter if you use a GUI or a CUI, people generally don't embed newlines, or escape, or tabs in filenames.

The leading "-" is probably more controversial, but the POSIX standard specifically notes such filenames as nonstandard.