Re:Health insurers just published close to a trillion hospit...
by A4ET8a8uTh0 on Tuesday September 06, 2022 @04:33PM
(#32742833)
In their defense, if it was anything but CSV files, they would be accused of making it too complicated/locking into proprietary formats and so on. I can't say CSV would be my first choice, but I don't really want to think what the alternative would be.
[ Parent
| on HN ]
Re:Health insurers just published close to a trillion hospit...
by easrng on Tuesday September 06, 2022 @05:12PM
(#32743268)
NDJSON? Sqlite?
Re:Health insurers just published close to a trillion hospit...
by ironick09 on Tuesday September 06, 2022 @08:03PM
(#32744742)
csv is many times over again a simpler format than sqlite and easier to understand by anyone across the world.
Never heard of ndjson, can’t see one publishing this data in a format that isn’t nearly as common as something like csv (or regular json which some of the data is published in).
Re:Health insurers just published close to a trillion hospit...
by easrng on Tuesday September 06, 2022 @08:18PM
(#32744840)
CSV only seems simple. Lots of parsing edge cases. Sqlite isn't readable by hand but it's basically bulletproof. NDJSON is literally just newline seperated JSON, it's just easier to process as a stream without a special parser.
Re:Health insurers just published close to a trillion hospit...
by irrelative on Tuesday September 06, 2022 @08:45PM
(#32745055)
Sure, could be simpler, but there's a spec and multiple implementations: https://www.rfc-editor.org/rfc/rfc4180.html
And the spec is ~5 pages.
Not sure why NDJSON is considered simpler, as json objects can be arbitrarily nested. Breaking into records is easier, but parsing is harder.
Re:Health insurers just published close to a trillion hospit...
by easrng on Tuesday September 06, 2022 @09:25PM
(#32745335)
(ND)JSON is simpler because people actually follow the spec, unlike with CSV.
Re:Health insurers just published close to a trillion hospit...
by Dylan16807 on Wednesday September 07, 2022 @03:18AM
(#32747500)
People violate the JSON spec all the time.
And following the CSV spec is much easier than following the JSON spec. And there's only like three edge cases.
Re:Health insurers just published close to a trillion hospit...
by mschuster91 on Wednesday September 07, 2022 @04:52AM
(#32748211)
> And there's only like three edge cases.
Unless you involve MS Excel or, worse, MS Excel on macOS. OTOH, pitfalls are: UTF-8 BOM, comma vs semicolon, single vs double quote, multiline cell content and escaping, escaping in general...
Re:Health insurers just published close to a trillion hospit...
by mattewong on Thursday September 08, 2022 @01:05AM
(#32761394)
All this banter arguing over CSV, JSON, sqlite seems unnecessary when you can just push format X through a pipe and get whichever format Y you want back out: https://github.com/liquidaty/zsv/blob/main/docs/csv_json_sql...
(disclaimer: I'm one of the zsv authors)
Re:Health insurers just published close to a trillion hospit...
by mr_toad on Tuesday September 06, 2022 @09:36PM
(#32745407)
Then you’d have to get your hands on a parser. Who wants to write up the business case and spend months going through change control (they’d probably just say no).
JSON and NDJSON can also be much larger than CSV files if the wrong structure is used.