News for hackers. Stuff that matters.
Re:ASCII Delimited Text – Not CSV or TAB delimited text by mbauman on Wednesday March 26, 2014 @01:35PM (#7474944)
It's a shame. The solution is in the charset, but tools never developed to use it so we don't use it. But I'd wager that if tools had historically supported them, then the situation would be no different than it is with tab.

There are representational glyphs for tab, return, and others (⇥, ↵), and editors can show them in 'show whitespace' modes. There could be representational glyphs for these control characters, too. I'm not sure about the history of these symbols, but I imagine they were initially on keyboards. But if these control characters were on keyboards and had a representation in text then they'd be just as useless as the tab character is today.

Precisely what makes them valuable is their difficulty to type or display.

Re:ASCII Delimited Text – Not CSV or TAB delimited text by jarrett on Wednesday March 26, 2014 @04:13PM (#7476261)
> Precisely what makes them valuable is their difficulty to type or display.

Exactly. A naive[1] character-delimited format is only robust when its delimiter isn't on the keyboard. Which means it's practical to read and write only in specialized applications, not in a text editor. Which sort of defeats the point of a character-delimited format anyway. If your use cases are constrained to specialized applications, you may as well just use JSON or something similar, instead of a character-delimited format. (I'm imagining a world where we get to pick our ideal formats, not one where Excel happens to have an almost-working CSV implementation.)

[1] By "naive" I mean one where the format specification describes record and line characters only. Thus the format has no escaping system.

Re:ASCII Delimited Text – Not CSV or TAB delimited text by stingraycharles on Wednesday March 26, 2014 @04:20PM (#7476302)
Is this a problem that's solved with a "movement" ? I'm willing to contribute parsers and perhaps a text editor plugin if other people are willing to chip in. But indeed, this is a problem that's solvable by making the tools support it. Since most of the tools are OSS, this is mostly solvable.
Re:ASCII Delimited Text – Not CSV or TAB delimited text by 3JPLW on Wednesday March 26, 2014 @05:59PM (#7476943)
You're missing my point. If this were a "solved problem," then it would be no better than the status quo with TSV. Tabs are a similar kind of control character, except they can be typed and displayed easily. And TSV files are a mess because tabs end up in the fields themselves. It's a catch-22. (I'm exaggerating; this would still be somewhat better since there will never be a big key on your keyboard dedicated to these ASCII values.)
Re:ASCII Delimited Text – Not CSV or TAB delimited text by gress on Wednesday March 26, 2014 @09:05PM (#7477869)
Tabs are not a record separator. They are a formatting code. That's why a mess is created when they are used to separate records.
Re:ASCII Delimited Text – Not CSV or TAB delimited text by hrabago on Wednesday March 26, 2014 @11:03PM (#7478322)
You just gave further proof that tabs are a catch-22.
Re:ASCII Delimited Text – Not CSV or TAB delimited text by qnaal on Thursday March 27, 2014 @02:13AM (#7478897)
They are if you're saving formatted text in a tsv- thus all the discussion about this rather mediocre attempt to replace them.
Re:ASCII Delimited Text – Not CSV or TAB delimited text by coldtea on Thursday March 27, 2014 @03:49PM (#7482939)
Yes, and he also specified why they are different from record separators in this regard.
Re:ASCII Delimited Text – Not CSV or TAB delimited text by joesb on Friday March 28, 2014 @12:18AM (#7485320)
It's no different, if it can be displayed on the screen and it is on the keyboard, then the user will type it into the text file for how it look, not because they want to really use it as record separator. And if user can type it in the content then you have to escape it, which is back to the same problem with using tabs.

But if it's not on the keyboard, then the user won't type it. So it doesn't get used.

Re:ASCII Delimited Text – Not CSV or TAB delimited text by coldtea on Friday March 28, 2014 @08:19AM (#7486685)
It's very different.

Tab, comma, semicolon are actual characters people DO and need to use inside records. So not only they can be typed into records, they absolutely HAVE to be typed for most textual records.

ASCII record separators are not needed inside records at all. If someone adds them "for how it looks", it's his problem.

One (not using a record separator inside a record) is a matter of choice and doing the right thing. The other (not using comma, semicolon or tab) is a non starter.

Obviously there's a difference.

Re:ASCII Delimited Text – Not CSV or TAB delimited text by kibibu on Wednesday March 26, 2014 @10:16PM (#7478155)
I'd suggest that the most common tool used to generate and manipulate tabular data is Excel, which has terrible CSV export.
Re:ASCII Delimited Text – Not CSV or TAB delimited text by Pxtl on Wednesday March 26, 2014 @01:53PM (#7475064)
> Precisely what makes them valuable is their difficulty to type.

Again, if they'd caught on you'd imagine there would be some eventual convention in text-editors for what keybind would be used to enter them. Too bad the AltGr key (intended for entering rarely-used glyphs) doesn't appear on pure-English keyboards.

Re:ASCII Delimited Text – Not CSV or TAB delimited text by wiredfool on Wednesday March 26, 2014 @02:40PM (#7475420)
I'd bet that they were on the eleventyzillion key IBM terminal keyboards.
Re:ASCII Delimited Text – Not CSV or TAB delimited text by markeganfuller on Wednesday March 26, 2014 @03:37PM (#7475922)
I'm looking at my English keyboard and it has Alt Gr...
Re:ASCII Delimited Text – Not CSV or TAB delimited text by Crito on Wednesday March 26, 2014 @03:56PM (#7476115)
Qwerty keyboards intended specifically for the US market (opposed to the UK for instance) frequently do not have AltGr keys. The thinkpad I'm using right now doesn't have one, and the Das Professional I have next to me doesn't have it either. Come to think of it, I'm not sure if I've ever owned a keyboard with an AltGr key..

To get around this, I have taken to using xmodmap to turn my right alt key into altgr.

Re:ASCII Delimited Text – Not CSV or TAB delimited text by markeganfuller on Wednesday March 26, 2014 @04:45PM (#7476506)
I'm surprised to learn that the Das doesn't have it.
Re:ASCII Delimited Text – Not CSV or TAB delimited text by bruceboughton on Wednesday March 26, 2014 @05:25PM (#7476786)
How would you tell…?
Re:ASCII Delimited Text – Not CSV or TAB delimited text by Crito on Wednesday March 26, 2014 @05:39PM (#7476861)
Ha. Only the Das Ultimate doesn't have printed keys. I got the Professional which has printed keys (http://www.daskeyboard.com/model-s-professional/). I'm not that show-offy about touch-typing. ;)
Re:ASCII Delimited Text – Not CSV or TAB delimited text by StavrosK on Thursday March 27, 2014 @05:51AM (#7479394)
Tip: I've got both, the blank one isn't worth the hassle. I can type fine on it, except when I need to find & or something. Then I have to press all the numeric keys to figure it out.
Re:ASCII Delimited Text – Not CSV or TAB delimited text by flux_w42 on Wednesday March 26, 2014 @06:21PM (#7477063)
Here in Belgium we have to AltGr-the hell out of our keyboard during development. These characters can only be entered using the AltGr key on a Belgian Azerty keyboard (be-latin1): '|' '@' '#' '^' '{' '}' '[' ']' '\' and '~' Its not realy a problem, as long as you're used to it :-)
Re:ASCII Delimited Text – Not CSV or TAB delimited text by Crito on Wednesday March 26, 2014 @06:31PM (#7477125)
Of those, |, @, #, ^, {, } and ~ are all keys that you need shift for on US Qwerty keyboards, so I imagine that the typing experience for those is relatively comparable.
Re:ASCII Delimited Text – Not CSV or TAB delimited text by oneeyedpigeon on Wednesday March 26, 2014 @09:10PM (#7477894)
IF those characters are actually displayed on the corresponding keys. If it's like the # on the Mac keyboard - not displayed anywhere - it's a right pain to learn in the first place without any visual cues.
Re:ASCII Delimited Text – Not CSV or TAB delimited text by egypturnash on Wednesday March 26, 2014 @11:58PM (#7478505)
I'm looking at a couple Macs sitting around the house and they all have a pretty obvious # right above the 3. Perhaps you mean some other key?

(I do really miss the days when Mac keyboards had the weird symbol they use for "alt" in menus on the key.)

Re:ASCII Delimited Text – Not CSV or TAB delimited text by oneeyedpigeon on Thursday March 27, 2014 @04:29AM (#7479183)
Which keyboard do you have? The standard aluminium full size one doesn't label it, at least in the uk
Re:ASCII Delimited Text – Not CSV or TAB delimited text by spolster on Thursday March 27, 2014 @05:45AM (#7479383)
On US Mac keyboards, # is Shift-3. On UK Mac keyboards Shift-3 is the Pound (Sterling) sign. # becomes Alt-3 and is unlabelled on the keyboard.

In the past, I've bought US Mac keyboards just for the #, or switched the keyboard layout in software to US.

Re:ASCII Delimited Text – Not CSV or TAB delimited text by oneeyedpigeon on Thursday March 27, 2014 @06:35AM (#7479511)
I've become used to the alt-3 combo now, but it took a long time. I type # a lot more than I ever type £. $, too.
Re:ASCII Delimited Text – Not CSV or TAB delimited text by egypturnash on Thursday March 27, 2014 @12:33PM (#7481465)
All my machines have US keyboards.
Re:ASCII Delimited Text – Not CSV or TAB delimited text by awwaiid on Thursday March 27, 2014 @12:12AM (#7478558)
I fixed my keyboard -- @,#,^,{,}.... no shift by default. Have to press shift to get 2,3,6,[,],9,0,...

Must be annoying to have to press shift to get those symbol chars while you're coding, eh?

Re:ASCII Delimited Text – Not CSV or TAB delimited text by Crito on Thursday March 27, 2014 @03:52AM (#7479109)
> Must be annoying to have to press shift to get those symbol chars while you're coding, eh?

Honestly no. It's all muscle memory for me, I don't think about it any more than I think about typing capital letters.

Re:ASCII Delimited Text – Not CSV or TAB delimited text by dagw on Thursday March 27, 2014 @04:03AM (#7479138)
I think it's french keyboards that are like that by default. Symbols are the default and you have to press the Shift-key to get numbers.
Re:ASCII Delimited Text – Not CSV or TAB delimited text by seszett on Thursday March 27, 2014 @06:58AM (#7479579)
Yeah, and some people complain that they have to use shift to get numbers.

I just think people will criticise their local layout no matter what. I use both a French AZERTY and a Québec QWERTY everyday (at work/home) and I think they are simply equally good for both typing French and for coding. The Québec keyboard (maybe actually Canadian multilingual or something) might have an edge because it more easily allows typing accented letters in uppercase, but on the other hand it doesn't let me type the € sign, so...

Re:ASCII Delimited Text – Not CSV or TAB delimited text by jablan on Thursday March 27, 2014 @09:10AM (#7480105)
So most Belgian developers are not switching layouts when coding? I thought most of us non-US developers are "bi-lingual" when it comes to keyboard layouts.
Re:ASCII Delimited Text – Not CSV or TAB delimited text by srg0 on Thursday March 27, 2014 @09:55AM (#7480348)
That's nothing. Standard Italian layout lacks tilde and backtick completely… and one needs three fingers to type “{” or “}”. I use only US layout for programming.

Actually, I switch between three layouts on my machine (the third one is for my mother tongue), and I also use AltGr for typographic characters.

Re:ASCII Delimited Text – Not CSV or TAB delimited text by hnal943 on Wednesday March 26, 2014 @04:15PM (#7476271)
Living in the US, I've never heard of an AltGr key until this discussion.
Re:ASCII Delimited Text – Not CSV or TAB delimited text by nnq on Thursday March 27, 2014 @10:51AM (#7480708)
Almost nobody calls it AltGr, it's just the goddamn "right Alt key" :) ...and yes, it has a different key code from the left one, even if in most software it works just like another Alt.

The clever thing is how some international keyboard layouts use it like some kind of "second shift" for typing character with accents/decorations: like AltGr+a => "ă", AltGr+q => "â", AltGr+s => "ș" etc. ...but not even these keyboard layouts are popular, and usually marked as "alternative" or "programmers' layout for language XYZ", because people are stupid and refuse to learn how to use this and prefer instead a funky layout national language keyboard instead of an US English keyboard with an AltGr that would just solve 99% of special characters problems.

If all the keyboards in the world would just be US English Standard keyboards with an AltGr (most US English keyboards I've seen do have an AltGr!), all latin-alphabet languages with special characters would be easy to type, we polyglots could easily use the same keyboard for typing in multiple languages without having to remember what keys' positions have radically changed on each layout... but people are stupid and refuse to learn even simple key combinations.

Oh, and somebody should shoot the British (and French) for adding that annoying extra key to the right of the left Shift that I always have to disable (and making the Shift much smaller), and for creating extra confusion by branding them as "british international" or "us english business" keyboards.

Re:ASCII Delimited Text – Not CSV or TAB delimited text by spoiler on Wednesday March 26, 2014 @06:08PM (#7476998)
I'm from Croatia, and we have the AltGr key.

However, I discovered that Alt + Control = AltGr when I needed to use it at work[1], so it's simply a shortcut, I think.

[1]: We have (I recently switched to an UK keyboard, because it suits me better) keyboards at work, because the Croatian (all slavic languages, to be honest) is horrendously counterproductive for programming. Google the layout, and you'll realise why. An example: You need to press AltGr+B for `{` (if I remember correctly).

Re:ASCII Delimited Text – Not CSV or TAB delimited text by chinpokomon on Wednesday March 26, 2014 @10:14PM (#7478150)
Perhaps, but I think it still goes against the original intent. Ctrl-~ or Ctrl-^ should give you a record separator (RS) and Ctrl-Del or Ctrl-_ should give you a unit separator (US). For the same reason Ctrl-m or Ctrl-M should give you carriage return (CR). This is because ASCII values from 00-1F are control characters and effectively grounded the most significant bits 7 and 6. Shift similarly would toggle or ground bit 6, depending on the implementation.

What happened was that the Ctrl key became synonymous with "command" after Teletype, so it became more about doing something. Think about Ctrl-x, Ctrl-c, and Ctrl-v as an example, but you still see some relics like Ctrl-d as End of Transmission (EOT) to close a shell or terminal. Alt is like a shift, but it is actually closer to the Fn key on most laptop keyboards. It was an alternative function of that particular key, so where the shift key provided you with an alternate case, Alt was more akin to an entirely different key... it isn't Alt plus an 'a' key, it is Alt-a.

AltGr was like another Alt key. It was originally there to allow you to enter an alternate glyph, especially line drawing characters available in extended ASCII, B0-DF. I thought it was a mapping closer to flipping the most significant bit to 1, but it doesn't exactly overlay the lower ASCII range, so that might be another change that evolved on the way to the modern keyboard.

To your original point, Microsoft Windows will now usually treat the chord Ctrl-Alt as AltGr. I don't know if that is with all layouts, or just those keyboards that lack AltGr. I find that most Linux distributions tend to follow Microsoft's lead and provide similar mappings but now they even repurposed the Win key as Meta or sometimes called Super. So it is likely that Ctrl-Alt is commonly the equivalent of AltGr.

For the propose of this discussion, I think it'd be better if Ctrl could be used to type these text separators, but the way modern operating systems map their modern keyboards, it might be difficult to ever reach consensus on how this should be done.

Re:ASCII Delimited Text – Not CSV or TAB delimited text by gpvos on Thursday March 27, 2014 @04:49AM (#7479244)
This hasn't been true since IBM keyboards became popular. For example, on older keyboards shift+number would simply toggle a bit, so shift+2 would be a double quote, etc., but this hasn't been common for decades now. Unfortunately.
Re:ASCII Delimited Text – Not CSV or TAB delimited text by chinpokomon on Friday March 28, 2014 @04:59PM (#7489949)
Probably on the path to scan codes. By using scan codes, they could abstract what a particular key meant and thereby remap the keys so that they didn't have to match the ASCII table layout. I still don't understand why we evolved scan codes the way we did. This requires the OS to be in sync to be able to map them back.
Re:ASCII Delimited Text – Not CSV or TAB delimited text by reidrac on Thursday March 27, 2014 @04:55AM (#7479259)
I'm a Spanish living in UK and I love using UK keyboards for the same reason: the layout is way better for programming!
Re:ASCII Delimited Text – Not CSV or TAB delimited text by EmielMols on Thursday March 27, 2014 @07:07AM (#7479603)
Windows used to have a "Czech (Programmers)" keyboard layout. As I remember, it was one of the few layouts that _just worked_. First thing I would do on new installs is make it the only available layout.
Re:ASCII Delimited Text – Not CSV or TAB delimited text by louthy on Wednesday March 26, 2014 @05:40PM (#7476868)
Living in the UK, I've always seen it and for some reason never questioned what the Gr meant!
Re:ASCII Delimited Text – Not CSV or TAB delimited text by Nilzor on Wednesday March 26, 2014 @06:01PM (#7476959)
so.. what does it mean?
Re:ASCII Delimited Text – Not CSV or TAB delimited text by flux_w42 on Wednesday March 26, 2014 @06:44PM (#7477196)
According to wikipedia:

The meaning of the key's abbreviation is not explicitly given in many IBM PC compatible technical reference manuals. However, IBM states that AltGr is an abbreviation for alternate graphic, and Sun keyboards label the key as Alt Graph.

Apparently, AltGr was originally introduced as a means to produce box-drawing characters, also known as pseudographics, in text user interfaces. These characters are, however, much less useful in graphical user interfaces, and rather than alternate graphic the key is today used to produce alternate graphemes.

Re:ASCII Delimited Text – Not CSV or TAB delimited text by Sami_Lehtinen on Thursday March 27, 2014 @03:38AM (#7479090)
If there isn't a glyph, you can always use text representation for control codes. Here's an example using Notepad++. http://i60.tinypic.com/nps3d1.png
Re:ASCII Delimited Text – Not CSV or TAB delimited text by jrochkind1 on Wednesday March 26, 2014 @05:02PM (#7476636)
> but I imagine they were initially on keyboards

I wouldn't be surprised if the last time they were on a keyboard, it was a teletype keyboard or a keyboard that punched cards!

Although, actually, were they just typed using the control key from the start?