News for hackers. Stuff that matters.
Re:Google Takeout: Download your data from Picasa, Contacts,... by kylec on Tuesday June 28, 2011 @05:59PM (#2707532)
There are some serious problems with their Picasa export (possibly others, I only tried Picasa). First, a good number of my photos simply didn't download - I only got 1328 of my 2283 photos. Second, I uploaded all my photos at their native sizes to Picasa, but the photos I got from Takeout were resized to around 1200x1600. However, my originals are kept by Google and I can download them individually through the Picasa interface, so they must be resizing them specifically for the download.

The first strikes me as a bug that will eventually be fixed, but the second seems to be an unfortunate design choice - Google can save money on bandwidth if they downsize your photos, and if you know that what you get when you export is downsized you're less likely to do it. I feel sorry, though, for people whose only copy of some photos is in Picasa.

Re:Google Takeout: Download your data from Picasa, Contacts,... by brown9-2 on Tuesday June 28, 2011 @07:43PM (#2707848)
Is the amount of bandwidth saved by Google in doing this really going to be all that cost efficient? I would chalk this up to a bug or poor design choice as well rather than something with the bottom-line in mind.
Re:Google Takeout: Download your data from Picasa, Contacts,... by hebejebelus on Tuesday June 28, 2011 @07:41PM (#2707841)
There are things I would _really_ like from google - where I fit in to their adsense categories, for example. (I'm not sure whether this data is anonymised or not)

I just downloaded this data, all the data they would give me. I got my email address and my name back. I also got a handful of contacts. I consider myself a reasonably heavy google[-owned projects] user.

This, for the most part, is useless information.

What I would like is exactly what they would give to the government (obviously, after confirming fifty times that it's actually me they're giving it to).

Regardless of all that, I think this should boost Google's image. Very smart to focus on social privacy, when that's Facebook's one downfall.

Re:Google Takeout: Download your data from Picasa, Contacts,... by wmf on Tuesday June 28, 2011 @08:21PM (#2707927)
Re:Google Takeout: Download your data from Picasa, Contacts,... by WalterGR on Wednesday June 29, 2011 @03:47AM (#2708719)
There are things I would _really_ like from google - where I fit in to their adsense categories, for example.

Absolutely. I'd love to be able to get all of my Analytics data, for example. Retrieving some subset of the data report by report is nowhere near the same thing.

Re:Google Takeout: Download your data from Picasa, Contacts,... by jncraton on Tuesday June 28, 2011 @04:43PM (#2707228)
Why not give me a nice big mbox file with all my emails in it as well? That's honestly the main thing that Google has that I care about preserving.
Re:Google Takeout: Download your data from Picasa, Contacts,... by sdfjkl on Wednesday June 29, 2011 @01:47AM (#2708550)
This is possible via the Google Apps Email Audit API[1], which is accessibly only by paid Google Apps accounts. The API submitted export request takes usually several hours to complete and results in a series of large mbox files (1 GB chunks I think) containing the entire mailbox, including trash (if requested). However some meta information such as labels/folders is not present, but the results can easily be concatenated into a single mbox file containing the entire contents of a Google Apps email account.

[1] http://code.google.com/googleapps/domain/audit/docs/1.0/audi...

Re:Google Takeout: Download your data from Picasa, Contacts,... by eli on Wednesday June 29, 2011 @08:31AM (#2709217)
Thanks for that. Weird that it's a buried feature.

Seems like "easy self-directed backups" would be a nice selling point.

Re:Google Takeout: Download your data from Picasa, Contacts,... by ch0wn on Tuesday June 28, 2011 @04:50PM (#2707264)
Shouldn't that be possible via IMAP?
Re:Google Takeout: Download your data from Picasa, Contacts,... by joejohnson on Tuesday June 28, 2011 @07:54PM (#2707869)
Yes, it is possible
Re:Google Takeout: Download your data from Picasa, Contacts,... by netrus on Tuesday June 28, 2011 @05:00PM (#2707308)
It should be easier than that.
Re:Google Takeout: Download your data from Picasa, Contacts,... by Groxx on Tuesday June 28, 2011 @05:14PM (#2707356)
POP? It's leaner.

Seriously, it's an absolutely perfect match for exporting everything, and it's easy to set up, and you get it in the format of your choice. Infinitely easier than having them support only formats they have time to write exporters for. It's also easily resumable, unlike the 8GB zip file I'd otherwise have to download.

Re:Google Takeout: Download your data from Picasa, Contacts,... by pyre on Tuesday June 28, 2011 @07:29PM (#2707809)
You do realize that POP only supports your Inbox, right? The protocol has no concept of 'folders' or 'tags.'
Re:Google Takeout: Download your data from Picasa, Contacts,... by Groxx on Tuesday June 28, 2011 @07:35PM (#2707828)
I think if you switch it to "all mail", it downloads everything. Though you might be right.
Re:Google Takeout: Download your data from Picasa, Contacts,... by lukeschlather on Tuesday June 28, 2011 @06:54PM (#2707692)
POP might be a start, though for a proper backup tags need to be included somehow, maybe in a custom header field.
Re:Google Takeout: Download your data from Picasa, Contacts,... by albertzeyer on Tuesday June 28, 2011 @05:36PM (#2707440)
Afaik, not really. Mails in GMail are not represented in a hierarchical tree structure and thus don't directly match the IMAP protocol.
Re:Google Takeout: Download your data from Picasa, Contacts,... by ammmir on Tuesday June 28, 2011 @07:43PM (#2707846)
IMAP can be used to download the raw message bodies. hierarchical structure (aka threading) comes from the parsing of Message-ID, References, In-Reply-To MIME headers contained in the bodies.
Re:Google Takeout: Download your data from Picasa, Contacts,... by albertzeyer on Tuesday June 28, 2011 @08:00PM (#2707883)
I mean directories. GMails IMAP interface maps each tag to a directory but that is not exactly the same. Esp. when your extracting algorithm doesn't know about this, you end up with a lot of duplicates.
Re:Google Takeout: Download your data from Picasa, Contacts,... by thwarted on Wednesday June 29, 2011 @12:28AM (#2708394)
So only download the All Mail folder into a single mbox.
Re:Google Takeout: Download your data from Picasa, Contacts,... by albertzeyer on Wednesday June 29, 2011 @07:16AM (#2709056)
Then you miss the tagging information.
Re:Google Takeout: Download your data from Picasa, Contacts,... by pyre on Tuesday June 28, 2011 @07:29PM (#2707807)
I'm confused. How does a single large mbox file of all emails in a GMail account map directly to GMail's structure? That's the the OP was asking for.
Re:Google Takeout: Download your data from Picasa, Contacts,... by afhof on Tuesday June 28, 2011 @05:05PM (#2707331)
Unfortunately it is still pretty hard to get any of your Google Talk chat logs out. Most of the solutions are from years ago and no longer work. I ended up being able to download an sqlite database of my chat logs using Google Gears.
Re:Google Takeout: Download your data from Picasa, Contacts,... by dmd on Wednesday June 29, 2011 @12:06AM (#2708362)
You can retrieve Google Talk chat logs with this API call: http://code.google.com/googleapps/appsscript/class_gmailapp....
Re:Google Takeout: Download your data from Picasa, Contacts,... by beck5 on Tuesday June 28, 2011 @05:42PM (#2707468)
Do you find them useful for reference or just feel safe having them?
Re:Google Takeout: Download your data from Picasa, Contacts,... by ori_b on Tuesday June 28, 2011 @06:01PM (#2707545)
I log everything because quite often it is useful to refer back to them.
Re:Google Takeout: Download your data from Picasa, Contacts,... by AlecSchueler on Wednesday June 29, 2011 @07:33AM (#2709088)
It might defeat the benefits of using the web interface for you, but you could connect to Talk using Pidgin, or something similar.
Re:Google Takeout: Download your data from Picasa, Contacts,... by ori_b on Wednesday June 29, 2011 @11:45AM (#2709882)
I already do. I prefer Pidgin's UI anyways. (Also, the chats are logged by gmail anyways, so if I want to use Google's search for the logs over grep, I still have that option)
Re:Google Takeout: Download your data from Picasa, Contacts,... by lhnz on Tuesday June 28, 2011 @05:42PM (#2707472)
Releasing this at the same time as Google+ is a very smart PR move. :)
Re:Google Takeout: Download your data from Picasa, Contacts,... by jedc on Tuesday June 28, 2011 @08:24PM (#2707932)
I would guess it's more everyone making launch deadlines before the end of the quarter!
Re:Google Takeout: Download your data from Picasa, Contacts,... by lurker19 on Wednesday June 29, 2011 @10:36AM (#2709605)
Why is this separate from the Data Liberation Front?

Another 20% project destined to be a quick PR hit and never actually become usable enough to fulfill its promise of giving the user control over his own data?

Re:Google Takeout: Download your data from Picasa, Contacts,... by kylemaxwell on Tuesday June 28, 2011 @06:24PM (#2707611)
I gave this a spin as well and was fairly happy with it, at least as far as it goes. Takeout has a fairly limited scope, but of course the DLF (greatest project name EVAR) has done more good than just this.
Re:Google Takeout: Download your data from Picasa, Contacts,... by albertzeyer on Tuesday June 28, 2011 @05:47PM (#2707486)
It doesn't seem to work for me. I tried to create an archive for all my data and it just shows 'Files: 0, Size: 0B' on everything.

Edit: Tried a second time and it works now.

But as it was said, the most important thing I miss is a way to extract the mails (in the way they are stored in GMail with all tags and other meta information).

Re:Google Takeout: Download your data from Picasa, Contacts,... by pyre on Tuesday June 28, 2011 @07:32PM (#2707816)
What meta information does GMail have that you can't get from downloading your emails over POP or IMAP (other than tags)?
Re:Google Takeout: Download your data from Picasa, Contacts,... by albertzeyer on Tuesday June 28, 2011 @08:05PM (#2707895)
Tags are the most important thing for me. If this is not possible, any extracting is really not worth it for me.

To extract tags right now, you could search in every other directory (and assume that this are all available tags) for the same mail and get by that error-prone algorithm all tags of a message.

Maybe also missing is some meta information about the spam level and/or importance level. And meta information why Google thinks that some message is important to me (on the web interface, it sometimes shows a reason like 'because of recent conversation with this person' or so).

And maybe more. I would just like to get everything.

Re:Google Takeout: Download your data from Picasa, Contacts,... by pyre on Tuesday June 28, 2011 @08:30PM (#2707945)

  > et by that error-prone algorithm all tags of a message.
Shouldn't be error-prone. All your emails should have a Message-Id: header. While not impossible, I've never heard of issues due to Message-Id collisions...

You could write something to sync your emails to a database, then when it encounters an email in a 'folder' it can just add it to the email as a tag. I don't know of anything that currently does this, but it's not like the technology (and information) isn't there.

  > And meta information why Google thinks that some
  > message is important to me (on the web interface,
  > it sometimes shows a reason like 'because of recent
  > conversation with this person' or so).
Of what use is this outside of Google, though? IIRC Google is the only one doing something like this. It's not like you could import that meta information into Outlook/Thunderbird/Mail.app.
Re:Google Takeout: Download your data from Picasa, Contacts,... by albertzeyer on Wednesday June 29, 2011 @07:21AM (#2709067)
Error-prone because it adds stupid complexity and many additional steps for just getting some simple information which Google probably has stored already along with the mail. With additional complexity, you always get further things in your algorithm which could go wrong. Such unnecessary complexity should always be avoided.
Re:Google Takeout: Download your data from Picasa, Contacts,... by nodata on Wednesday June 29, 2011 @04:23AM (#2708763)
This is broken for me. I choose "Contacts and Circles", then "Create archive". It then fails. I try again, it fails a bit later. Again: it succeeds with no obvious way to download the file. I try again and get "Download quota exceeded". The file which I haven't downloaded isn't even a megabyte.
Re:Google Takeout: Download your data from Picasa, Contacts,... by beefman on Tuesday June 28, 2011 @08:08PM (#2707901)
Great, now let me get my Reader stuff.

Edit: And put the buzz stuff in some sort of timeline. I guess this raises the question of an archival format for modern web "experiences"...

Re:Google Takeout: Download your data from Picasa, Contacts,... by zbanks on Wednesday June 29, 2011 @01:38AM (#2708530)
Buzz includes shared items/notes in Reader. You can always export (& import) your subscriptions: http://www.google.com/reader/subscriptions/export?hl=en .

Also, although there isn't any manifest or timeline file, each .html file for Buzz has a last modified date that corresponds to when it was created. In addition, in the file itself, it has a timestamp.

This seems more than reasonable. Sure an XML/JSON timeline might be nice, but it wouldn't be human-readable either.

Re:Google Takeout: Download your data from Picasa, Contacts,... by beefman on Wednesday June 29, 2011 @12:58PM (#2710189)
Reader shared items only appear in buzz if you've "connected" reader to buzz. I don't have it connected because people I know hate seeing stuff in buzz they've seen elsewhere.

Plus, I've got hundreds of reader items I shared before buzz existed. I don't think google even keeps them though they imply that they do.

I just used "save web page, complete" in firefox on my buzz feed and got something better than what the takeout buzz download gave me.

The Cloud^TM - because all filesystems should be stochastic. Is my file there? Maybe!! We don't know, honestly. Your query timed out? I guess the system doesn't know either. Try again later!

Re:Google Takeout: Download your data from Picasa, Contacts,... by tluyben2 on Wednesday June 29, 2011 @06:57AM (#2709023)
Voting for mailbox takeout!
Re:Google Takeout: Download your data from Picasa, Contacts,... by barista on Tuesday June 28, 2011 @10:26PM (#2708179)
What use is an exported that that cannot be easily imported to other system. Google should provide connectors to other popular tools e.g. export data from picasa to flickr or smugmug etc. similarly export to hotmail, yahoo mail etc.
Re:Google Takeout: Download your data from Picasa, Contacts,... by hboon on Tuesday June 28, 2011 @10:52PM (#2708227)
I haven't looked at the exported format, but I assume it's readable. But surely other than that, import tools for other services should do their share of the work.
Re:Google Takeout: Download your data from Picasa, Contacts,... by nl on Tuesday June 28, 2011 @11:54PM (#2708345)
Why should they do that? Surely that would be something the other companies should do if they wish to encourage importing?
Re:Google Takeout: Download your data from Picasa, Contacts,... by 1010011010 on Tuesday June 28, 2011 @05:02PM (#2707317)
Anti-data-lockin FTW
Re:Google Takeout: Download your data from Picasa, Contacts,... by intellection on Tuesday June 28, 2011 @07:13PM (#2707763)
+1
Re:Google Takeout: Download your data from Picasa, Contacts,... by intellection on Wednesday June 29, 2011 @04:04PM (#2711012)
Besides not understanding hate/low value towards parents' Anti-Data-Lock-in post (On-point with "Avoiding vendor lock-in for computer software": http://en.wikipedia.org/wiki/Vendor_lock-in), I am unsatisfied and frustrated my high-five post gets downvotes. With a certain humility. It's like getting pushed over for waving your hands (or voting).

I passionately support freeing data because of my work and info I want to share freely being archived|trapped away as proprietary property/formats.

I hope a little phrase like Anti-Data-Lock-in can transcend copyright and privacy. Maybe a meme for not losing your data/yourself, by being locked out.