The sad state of personal data and infrastructure

Re:The sad state of personal data and infrastructure by simonw on Friday December 20, 2019 @02:06PM (#21846592)

I've been working on a project along these lines recently. I've called it Dogsheep - the basic idea is to have scripts that export all manner of my personal data (from Google, Apple HealthKit, Twitter, LinkedIn etc) into SQLite database files, then use my Datasette web app to browse them and run interesting queries.

More about that here: https://dogsheep.github.io/

The tools I've built so far are under https://github.com/dogsheep

[ Parent | on HN ]

Re:The sad state of personal data and infrastructure by gpchelkin on Sunday December 22, 2019 @06:56AM (#21856680)

I think that Perkeep https://perkeep.org/ is also worth mentioning here. One of their latest tools is the one for exporting Google Photos: https://github.com/perkeep/gphotos-cdp

[ Parent | on HN ]

Re:The sad state of personal data and infrastructure by ocdtrekkie on Friday December 20, 2019 @05:41PM (#21848016)

Whoa, this is super exciting to me. I've been looking for something that would get some of these types of data into a web-based interface that I could package for Sandstorm. This looks like it would potentially work for that fairly easily.

[ Parent | on HN ]

Re:The sad state of personal data and infrastructure by rasz on Saturday December 21, 2019 @05:19PM (#21853661)

I went similar route after discovering Google Checkout refuses to return my YT comments. I wrote quick&dirty tempermonkey script logging submitted HTML forms (addEventListener "submit").

[ Parent | on HN ]

Re:The sad state of personal data and infrastructure by TurkishPoptart on Saturday December 21, 2019 @05:40PM (#21853740)

Did it work the way a scraper does? Curious to know how you did this.

[ Parent | on HN ]

Re:The sad state of personal data and infrastructure by rasz on Saturday December 21, 2019 @10:36PM (#21855014)

I was only interested in stuff I posted on the internet for indexing/easier search, https://developer.mozilla.org/en-US/docs/Web/API/HTMLFormEle... and http://www.meekostuff.net/blog/Overriding-DOM-Methods/ covered form submissions, result goes to tempermonkey localstorage (GM_setValue).

[ Parent | on HN ]

Re:The sad state of personal data and infrastructure by karlicoss on Friday December 20, 2019 @03:06PM (#21846979)

Looks cool, I'll check it out! Might be good to join forces on scripts for data retrieval/parsing.

[ Parent | on HN ]

Re:The sad state of personal data and infrastructure by austil on Saturday December 21, 2019 @05:44AM (#21850692)

Hey, I'm on this journey too ! https://github.com/austil/datapuller

[ Parent | on HN ]

Re:The sad state of personal data and infrastructure by jakeywankenobi on Friday December 20, 2019 @02:13PM (#21846639)

Nice. Any interesting use cases for the genome one?

[ Parent | on HN ]