News for hackers. Stuff that matters.
Re:Dangers of CSV Injection by Sir_Cmpwn on Tuesday October 10, 2017 @09:44AM (#15441683)
Let's send patches to text editors so they render fancy ASCII characters? I also find it amusing that "fancy ASCII characters" is even a statement that can make sense, there's only 127 ASCII characters!
Re:Dangers of CSV Injection by burntsushi on Tuesday October 10, 2017 @11:53AM (#15442802)
Sure. Send patches. Until it becomes easy to read and write fancy ASCII characters in text editors, my criticism stands.

I predict this will never happen under the principle of "good enough to use, bad enough to complain on HN." My point is that people keep suggesting ASCII delimiters as if it's some obvious solution that solves the entire problem, but it doesn't. People should instead be suggesting that text editors better support ASCII delimiters so that we can use them roughly as easily as we use commas today.

Re:Dangers of CSV Injection by pavel_lishin on Tuesday October 10, 2017 @10:26AM (#15442011)
This might solve the "readable" part of the problem, but not the "writable" part of the problem.
Re:Dangers of CSV Injection by EGreg on Tuesday October 10, 2017 @10:35AM (#15442107)
Explain to me why humans have to be the ones to write proper syntax and escaping, instead of their editors. Do you do this for other formats, like RTF? What is this obsession with hand-coding complicated formats :)
Re:Dangers of CSV Injection by pavel_lishin on Tuesday October 10, 2017 @12:27PM (#15443080)
> Do you do this for other formats, like RTF?

No, but I do it for formats like HTML, CSS, JSON. My editor can assist me, but I don't need it. To a lesser degree, the same is true of Java, though I do admit to leaning much more heavily on my IDE for that.

> What is this obsession with hand-coding complicated formats :)

Well, part of it is that they're not all that complicated unless you're doing something fancy. Another part is almost certainly our (we as in developers) pride in being able to use nothing but vi installed on a decade-old operating system to get things done.

Re:Dangers of CSV Injection by EGreg on Tuesday October 10, 2017 @02:06PM (#15443953)
Why? What benefit do you get from coding it by hand? The only one I can think of is being able to type some esoteric incantations in a textbox.

Downsides include:

XSS and malicious injection - users should be using markdown or provided an actual contentEditable HTML editor instead of a textarea. Like in GMail.

Syntax errors. How many times have you typed some JSON by hand and realized you forgot to balance some braces or add a comma, or remove a comma at the end of an object?

Complexity not that complicated, really? Basic C, HTML, CSS is not complicated. Advanced stuff is complicated. Forgetting a brace or a semicolon torpedoes the whole document. SQL is not complicated. Does that mean you want to write SQL by hand for production code?

Why don't you install a simple extension to vi like an html format adapter? I'm a developer too. But let me tell you what it sounds like when you say you want to use nothing but vi: that's like saying you want to be able to build robots using nothing but a hammer and some nails, and nothing else can be built that requires further abstraction or advancement where you can't do the same with a hammer.

And also keep in mind that not everyone is a developer. The fact that you like to keep esoteric syntax rules in your head for HTML, XML, CSS, Javascript, C++ and so on doesn't mean EVERYONE should have to. When it comes to CSV, it's not even a real standard. You have to keep in your head all the cross platform quirks, like \r\n garbage similar to how web developers need to keep in their head all te Browser quirks and workarounds.

All this... for some pride thing of being able to write text based stuff by hand.

I brought up RTF for a reason. No sane person should want to type .rtf or .docx by hand. So why HTML?

Re:Dangers of CSV Injection by Dylan16807 on Tuesday October 10, 2017 @01:39PM (#15443751)
If you're calling CSV complicated, I really want to see your example of a simple format.
Re:Dangers of CSV Injection by EGreg on Tuesday October 10, 2017 @02:09PM (#15443979)
CSV is complicated in the same way the DOM and Javascript is complicated:

1) there are so many differences between browsers that you have to keep them all in mind when asking people to send you csv files, or generating them, etc. Such as for example \n vs \r vs \r\n and escaping them.

2) You have to keep in your head escape rules and exceptions, and balancing quotes and other delimiters.

3) The whole thing doesn't look human readable or easily navigable for a document of any serious complexity.

And what's the upside? If more people just used Excel or another spreadsheet program to edit these files, you won't face ANY of these issues. They would eventually converge on a standard format, like they did with HTML.

Disclaimer: I wrote a CSV parser

Re:Dangers of CSV Injection by Dylan16807 on Tuesday October 10, 2017 @03:07PM (#15444459)
> there are so many differences between browsers that you have to keep them all in mind

Compatibility horrors from people violating the standard will appear no matter what format you use. That's not fair to blame on the format.

> You have to keep in your head escape rules and exceptions, and balancing quotes and other delimiters.

CSV itself has newlines, commas, and quotations marks for special characters. That's extremely minimal. The only extra thing to keep in your head is "is this field quoted or not".

What set of escapes and delimiters could be simpler? Would you rather reserve certain characters, and abandon the idea of holding "just text" values?

> The whole thing doesn't look human readable or easily navigable for a document of any serious complexity.

> And what's the upside? If more people just used Excel or another spreadsheet program to edit these files, you won't face ANY of these issues. They would eventually converge on a standard format, like they did with HTML.

This sounds like you're arguing for a more complex format! I'm confused.

So again, what is a format that you call simple?

Re:Dangers of CSV Injection by hyperpape on Tuesday October 10, 2017 @04:34PM (#15445101)
> What set of escapes and delimiters could be simpler? Would you rather reserve certain characters, and abandon the idea of holding "just text" values?

There's a lot of design room here.

https://www.lua.org/pil/2.4.html http://prog21.dadgum.com/172.html

I can no longer remember where I saw it (thought it was lua), but I heard the idea to use =[[ ... ]]= and =[[[ ... ]]]= as wrappers (a bit like the qq operator from perl). They can be nested and don't interfere, so =[[[ abc =[[ ]]= ]]]= is a legitimate string.

Re:Dangers of CSV Injection by Piskvorrr on Wednesday October 11, 2017 @09:30AM (#15449543)
Except when the compatibility horror comes from Excel violating the standard, but only if you happen to be located at the wrong hemisphere (hello semicolons). So, even if everyone just used Excel, you will still face this issue. Now what?
Re:Dangers of CSV Injection by EGreg on Tuesday October 10, 2017 @04:24PM (#15445038)
I am not arguing for another format. I am arguing for using a program to edit these files, to avoid syntax errors and other crap that arises when people do manual stuff that doesn't need to be done manually. And sure the format can stand to be slightly more complex, who cares if you're not editing it by hand.

Ascii text format is simple.

Anything where you have arbitrarily complex structure, why not use a program to edit it? What is the downside of using the right tool for the job? Your text editor is a program. Why tunnel through text and manually edit stuff?

Re:Dangers of CSV Injection by efreak on Tuesday October 10, 2017 @06:50PM (#15445963)
Does any system (modern or not) actually use `\r` for anything? Because I'm not aware of it being used alone. I'm not even sure why software today still differentiates between the two instead of treating any of `\r`, `\n`, `\r\n` as a single line break. I can maybe see it being useful eg in word processor to differentiate manual line break from paragraph break, but that's not a plain text format; in the vast majority of cases, treating them as the same character in parsers shouldn't* cause any issues.

* shouldn't ≠ doesn't

Re:Dangers of CSV Injection by burntsushi on Wednesday October 11, 2017 @07:25AM (#15448855)
IIRC, Mac systems used to use `\r` for line endings. It doesn't any more and instead uses `\n`.