News for hackers. Stuff that matters.
Re:Tables: Tracking work for teams by forgotmypw17 on Tuesday September 22, 2020 @11:53AM (#24555989)
This is often claimed, but more often than not, data provided by export features, and not just Google's, are in a format which makes it useless to non-programmers.
Re:Tables: Tracking work for teams by megous on Tuesday September 22, 2020 @02:31PM (#24558155)
Good. I still remember when Facebook export was some shitty useless html garbage file, that didn't even include post types, or names of other people, or sensible structure (just a soup of <br> tags), or entity IDs, and was basically useless for about anything.

These days exports are at least usable, so that you can hire a programmer to make something useful out of it, if you need.

I've already helped people make tools to create automated summaries for counting billable hours for language lessons taught over skype or messanger, which was only possible thanks to these exports finally being in some processable format, with enough metadata included.

It's not either or though. Services can export in both nice usable formats, and html garbage that is nice to look at but useless for anything else.

Re:Tables: Tracking work for teams by shadowgovt on Tuesday September 22, 2020 @11:56AM (#24556045)
That's going to be the common case; without knowing the end-target application, most data formats we would pick at random are useless to non-programmers.

Even JSON is too complex for non-programmers.

Re:Tables: Tracking work for teams by forgotmypw17 on Tuesday September 22, 2020 @02:23PM (#24558086)
Plaintext would not be too complex for non-programmers.

Coupled with in-file tokens, it makes for an easy format to export into anything.

That's why I am using it as the base format for my forum application.

At any time, the entire forum content can be exported into plaintext, which is readable by most users on most platforms (except iOS)

And also can be re-imported into just about anything with a quick Perl script.

Re:Tables: Tracking work for teams by shadowgovt on Tuesday September 22, 2020 @02:42PM (#24558264)
Plaintext is too complex for programmers and likely drops semantic information that is needed to reconstruct the relations in the original data.

There's a reason plaintext is rarely used as the preferred method of semantic data exchange. What's our preferred method of representing images with plaintext? Audio files? Rich text, such as Google Docs? Spreadsheets (complete with formulae and attached macros)? Databases that contain bin-blob fields?

> And also can be re-imported into just about anything with a quick Perl script.

In your specific use case, I expect that works well. In the general case, my experience has been that plaintext can be imported into just about anything incorrectly with a quick Perl script. And when the data is MBs / GBs, the odds that such an error goes unnoticed until the data is needed and cannot be reconstituted are high.

Re:Tables: Tracking work for teams by forgotmypw17 on Tuesday September 22, 2020 @07:44PM (#24561363)
I don't agree at all.

For instance, consider email or newsgroups formats, with key-value attribute headers, with the difference of putting them at the bottom of the file.

With each "item" being a text file, you could easily fit the average person's Facebook profile into a zip file of (folder-arranged) text files with the content at the top and the extra attributes (e.g. timestamp, reply-to, etc.) below.

Even for someone very active, this would only be several hundred text files per day, so several hundred thousand per year, and several million for an entire profile. With that many, you'd probably have to split them up, but for the average person with maybe several hundred thousand, that's a manageable number of textfiles to put into a zip file.

These textfiles could be almost trivially imported into a relational database with a simple script.

At the same time, the user can also browse the contents of the zip file if the folder structure is arranged thoughtfully.

Of course, I am talking about text only, not images or videos. For these, you would have to either use metadata or have a matching text-file to go with the file.