News for hackers. Stuff that matters.
Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by toomuchtodo on Friday April 24, 2015 @11:59AM (#9434204)
I guess its time for us to put together "Open Ancestry".
Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by Asparagirl on Friday April 24, 2015 @03:30PM (#9435554)
Waaaay ahead of you: I'm a genealogy nerd and I built an open source archival records management system (Apache Solr + MySQL + PHP) called LeafSeek to do just that. Won second place at the RootsTech Developer Challenge for it.

Website: http://www.leafseek.com Background article about how the project came to be: http://www.leafseek.com/blog/leafseek-gets-published-in-avot...

I currently work with two major non-profit genealogy groups to put their record collections online for free and easy searches, without handing them over to for-profit groups like Ancestry. So far we're at 700,000+ records online using my code, and the newest "big" addition is the Israeli State Archives agreeing to put their British Mandate period marriage and divorce certificates online via one of the non-profits using the code.

I'm planning on launching the system as a multi-tenant SaaS one of these days, once my kids are a little older. :-)

Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by toomuchtodo on Friday April 24, 2015 @03:51PM (#9435646)
Two questions:

Business plan?

Data license? (Public domain, creative commons, etc)

I make everything I retrieve publicly available with a copy thrown in the Internet Archive for good measure.

Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by Asparagirl on Friday April 24, 2015 @04:04PM (#9435717)
Three or four tiered plan, based on number of records uploaded and desire for additional bells and whistles (personalized subdomain, whitelabel for better known libraries and archives, ability to keep some part of records private except to users who are affiliated with your group through offline membership, etc.).

No freemium, only paying clients. BUT a price discount if users allow (1) auto backup of datasets in CSV format to Internet Archive through their S3 system, even if the data is "darked" and not visible (I'm on Archive Team and I also created their "Antecedents" collection of genealogy-related web crawls: https://archive.org/details/archiveteam_antecedents) and (2) they explicitly put their data in the public domain.

Pie in the sky idea: create API for higher tier paying users to allow them to license their data directly to Ancestry, MyHeritage, FindMyPast, etc. Which would flip the pay-for-data model on its ear, allowing users to regain control of their own datasets, and for-profit data brokers to pay them for access.

Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by mwest on Friday April 24, 2015 @04:14PM (#9435771)
Looks like Archive Team managed to make a backup of ancestry.com late last year. Good work!

https://twitter.com/archiveteam/status/591693476556840961

https://archive.org/details/archiveteam_ancestry

Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by Asparagirl on Friday April 24, 2015 @04:17PM (#9435780)
Who do you think told them about the shutdowns? :-)
Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by toomuchtodo on Friday April 24, 2015 @04:06PM (#9435728)
Nicely done.
Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by Asparagirl on Friday April 24, 2015 @04:14PM (#9435765)
Thank you! In a perfect world there could be a free plan for small time data publishers, individual researchers without a lot of funds, etc. But you have to be very wary of recreating the RootsWeb scenario from 12 years ago, where a giant collection of genealogy records created through volunteer-run goodwill gets in over its head financially and needs a white knight to come bail them out. In that case, the white knight was Ancestry, who (amazingly) haven't shut them down all these years. But I am aware that any new project to put archival data online MUST be fiscally responsible and prudent because it is such a big responsibility. This is customer data unlike most others.

To that end, I will also be rejecting VC and bootstrapping the venture. Ain't nobody going to be flipping this site or acqui-hiring that data. This is going to be done for the long-term.

Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by toomuchtodo on Friday April 24, 2015 @05:29PM (#9436119)
Have you thought about incorporating as either a non-profit or a social B corp? Neither prevents you from drawing income from the project, but might increase the chances of using grant money or kickstarter/indiegogo funds to further the cause.
Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by Asparagirl on Friday April 24, 2015 @06:59PM (#9436441)
I have thought about, and rejected, the non-profit idea, as they have so much overhead to run (tons of paperwork) and might limit any political advocacy I might want to do in connection with the desire for more open data/records policies. But I don't know much about the social B corp. Would need to do more study...
Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by Someone1234 on Friday April 24, 2015 @02:30PM (#9435188)
That's fine, but someone has to physically go to the records office and scan in thousands of documents to be digitised. That's the problem with Ancestry, they have a massive digital archive and the funds to keep growing it.

Some copies of the census are digitised already, but the further you go back the less likely that is. And a lot of the information people want originates abroad (e.g. UK, Germany, Italy, France, etc).

Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by toomuchtodo on Friday April 24, 2015 @03:09PM (#9435428)
So, I might have some insight into this.

I'm working with an organization to collect real estate parcel data for the entire US and open source it. I track every county or district that has parcel data, I built a workflow to get the data from them, manage the ingest of the data, and a separate workflow if they're difficult and I have to FOIA them for it (via muckrock.com). There is a cost, but as always, everything has a cost.

Same deal, but closer to what Distributed Proofreaders for Project Gutenberg does with scanning.

It can be done. Funds and organization would be necessary, but let's not think its some herculean task.

Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by Asparagirl on Friday April 24, 2015 @03:34PM (#9435572)
You don't actually need to scan all the images, just transcribe the main points of the data (names, dates, places), make that data searchable, and then provide information about how a user could go about ordering a copy of the original data if needed.
Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by gsnedders on Friday April 24, 2015 @04:42PM (#9435893)
Transcription errors are very much a problem, though, especially when you're dealing with faded or smudged ink.

Scanning the images allows people to verify the transcription and to distribute the work of doing the transcription.