News for hackers. Stuff that matters.
Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by basseq on Friday April 24, 2015 @08:51AM (#9432956)
Ancestry has a borderline monopoly on web-based genealogy, and unfortunately doesn't seem to be a very good steward for that data.

Generally, the genealogy community is very open and sharing. Distant relatives (n cousins nth removed) will spend hours helping you out. Random people will go to local libraries or town halls to pull hard copies of vital records. LDS has one of the largest free genealogy libraries in the world.

Then here comes Ancestry. $240/yr. to use their software, which puts everything you do behind a paywall. Want to share your family tree with others? They have to pay Ancestry for the privilege.

There's serious value in digitized records that might well be worth the subscription fee alone. But these are public records that Ancestry has bought the exclusive rights to digitize. They're aggressively monetizing public data while shutting out competition.

It's great that family history is so easy for the masses, but I'm worried the long-term future is going to be silo'd and less effective in the long run.

Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by suninwinter on Friday April 24, 2015 @11:03AM (#9433793)
You can get the subscription fee waived if you join the LDS Church.

https://familysearch.org/blog/en/create-free-account-familys...

Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by Someone1234 on Friday April 24, 2015 @02:28PM (#9435170)
How do they verify you're LDS?
Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by gliese1337 on Friday April 24, 2015 @03:29PM (#9435548)
They ask the Church's servers.

The Church is meticulous about good record keeping (which is what makes them so good at genealogy in the first place), and every official member of the LDS church has a unique member identification number, used for things like tracking donations, ordinance records, and, of course, genealogy. Nowadays, the vast majority of these records are digitized and you can access them online through Church-owned-and-operated websites. If you happen not to be in the system yet (maybe you were baptized in the jungles of New Guinea and have only just made it back to civilization), it's pretty simple to meet up with a bishop or clerk who can verify your paper records and tell you what your membership number is, which will let you set up an account on Church websites.

So, to summarize, the Church knows who its members are, and knows how to associate them with online identities. When one of them wants free access to Ancestry.com, it's basically the same process as "Sign in with Google" and such- you tell Ancestry that you're LDS, Ancestry asks the Church's servers if you're telling the truth, and the Church's authentication system teels Ancestry to let you in.

Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by suninwinter on Friday April 24, 2015 @03:11PM (#9435437)
You ask the membership clerk of your ward for your membership number, which you use to upgrade your familysearch.org account to an LDS account. Then ancestry.com uses OAuth to get access to your familysearch account.
Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by dguaraglia on Friday April 24, 2015 @03:18PM (#9435471)
Sounds like a pretty steep price, eternal damnation for $240 :P
Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by ra88it on Friday April 24, 2015 @09:55AM (#9433265)
> But these are public records that Ancestry has bought the exclusive rights to digitize.

What exclusive rights are you referring to? I'm trying to learn more about Ancestry.com.

Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by Asparagirl on Friday April 24, 2015 @03:36PM (#9435582)
Philadelphia BMD. New York State Census records (State off-year, not Federal). Swedish BMD (previously available online, then pulled down). Czech BMD (ditto). The list goes on... Check the genealogy blogs, of which there are many, for details.
Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by toomuchtodo on Friday April 24, 2015 @11:59AM (#9434204)
I guess its time for us to put together "Open Ancestry".
Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by Asparagirl on Friday April 24, 2015 @03:30PM (#9435554)
Waaaay ahead of you: I'm a genealogy nerd and I built an open source archival records management system (Apache Solr + MySQL + PHP) called LeafSeek to do just that. Won second place at the RootsTech Developer Challenge for it.

Website: http://www.leafseek.com Background article about how the project came to be: http://www.leafseek.com/blog/leafseek-gets-published-in-avot...

I currently work with two major non-profit genealogy groups to put their record collections online for free and easy searches, without handing them over to for-profit groups like Ancestry. So far we're at 700,000+ records online using my code, and the newest "big" addition is the Israeli State Archives agreeing to put their British Mandate period marriage and divorce certificates online via one of the non-profits using the code.

I'm planning on launching the system as a multi-tenant SaaS one of these days, once my kids are a little older. :-)

Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by toomuchtodo on Friday April 24, 2015 @03:51PM (#9435646)
Two questions:

Business plan?

Data license? (Public domain, creative commons, etc)

I make everything I retrieve publicly available with a copy thrown in the Internet Archive for good measure.

Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by Asparagirl on Friday April 24, 2015 @04:04PM (#9435717)
Three or four tiered plan, based on number of records uploaded and desire for additional bells and whistles (personalized subdomain, whitelabel for better known libraries and archives, ability to keep some part of records private except to users who are affiliated with your group through offline membership, etc.).

No freemium, only paying clients. BUT a price discount if users allow (1) auto backup of datasets in CSV format to Internet Archive through their S3 system, even if the data is "darked" and not visible (I'm on Archive Team and I also created their "Antecedents" collection of genealogy-related web crawls: https://archive.org/details/archiveteam_antecedents) and (2) they explicitly put their data in the public domain.

Pie in the sky idea: create API for higher tier paying users to allow them to license their data directly to Ancestry, MyHeritage, FindMyPast, etc. Which would flip the pay-for-data model on its ear, allowing users to regain control of their own datasets, and for-profit data brokers to pay them for access.

Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by mwest on Friday April 24, 2015 @04:14PM (#9435771)
Looks like Archive Team managed to make a backup of ancestry.com late last year. Good work!

https://twitter.com/archiveteam/status/591693476556840961

https://archive.org/details/archiveteam_ancestry

Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by Asparagirl on Friday April 24, 2015 @04:17PM (#9435780)
Who do you think told them about the shutdowns? :-)
Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by toomuchtodo on Friday April 24, 2015 @04:06PM (#9435728)
Nicely done.
Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by Asparagirl on Friday April 24, 2015 @04:14PM (#9435765)
Thank you! In a perfect world there could be a free plan for small time data publishers, individual researchers without a lot of funds, etc. But you have to be very wary of recreating the RootsWeb scenario from 12 years ago, where a giant collection of genealogy records created through volunteer-run goodwill gets in over its head financially and needs a white knight to come bail them out. In that case, the white knight was Ancestry, who (amazingly) haven't shut them down all these years. But I am aware that any new project to put archival data online MUST be fiscally responsible and prudent because it is such a big responsibility. This is customer data unlike most others.

To that end, I will also be rejecting VC and bootstrapping the venture. Ain't nobody going to be flipping this site or acqui-hiring that data. This is going to be done for the long-term.

Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by toomuchtodo on Friday April 24, 2015 @05:29PM (#9436119)
Have you thought about incorporating as either a non-profit or a social B corp? Neither prevents you from drawing income from the project, but might increase the chances of using grant money or kickstarter/indiegogo funds to further the cause.
Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by Asparagirl on Friday April 24, 2015 @06:59PM (#9436441)
I have thought about, and rejected, the non-profit idea, as they have so much overhead to run (tons of paperwork) and might limit any political advocacy I might want to do in connection with the desire for more open data/records policies. But I don't know much about the social B corp. Would need to do more study...
Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by Someone1234 on Friday April 24, 2015 @02:30PM (#9435188)
That's fine, but someone has to physically go to the records office and scan in thousands of documents to be digitised. That's the problem with Ancestry, they have a massive digital archive and the funds to keep growing it.

Some copies of the census are digitised already, but the further you go back the less likely that is. And a lot of the information people want originates abroad (e.g. UK, Germany, Italy, France, etc).

Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by toomuchtodo on Friday April 24, 2015 @03:09PM (#9435428)
So, I might have some insight into this.

I'm working with an organization to collect real estate parcel data for the entire US and open source it. I track every county or district that has parcel data, I built a workflow to get the data from them, manage the ingest of the data, and a separate workflow if they're difficult and I have to FOIA them for it (via muckrock.com). There is a cost, but as always, everything has a cost.

Same deal, but closer to what Distributed Proofreaders for Project Gutenberg does with scanning.

It can be done. Funds and organization would be necessary, but let's not think its some herculean task.

Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by Asparagirl on Friday April 24, 2015 @03:34PM (#9435572)
You don't actually need to scan all the images, just transcribe the main points of the data (names, dates, places), make that data searchable, and then provide information about how a user could go about ordering a copy of the original data if needed.
Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by gsnedders on Friday April 24, 2015 @04:42PM (#9435893)
Transcription errors are very much a problem, though, especially when you're dealing with faded or smudged ink.

Scanning the images allows people to verify the transcription and to distribute the work of doing the transcription.

Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by Lawtonfogle on Friday April 24, 2015 @10:39AM (#9433596)
Ancestry was given that monopoly by their users. Served up to them on a silver platter. Without Ancestry, some other business would have come along as long as the users are so willing to put their work behind a paywall. The core fault lies with the users who are willing to do such.
Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by spacemanmatt on Friday April 24, 2015 @11:09AM (#9433842)
Victim-blaming, eh.
Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by Lawtonfogle on Friday April 24, 2015 @01:38PM (#9434825)
How in the world is being the victim of a crime against your will similar to choosing to buy and use a pay-walled product? Especially for a service that isn't fulfilling some need?
Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by VienneseCPA on Friday April 24, 2015 @02:00PM (#9434968)
It's not literally. But the HN ethos is that users lack agency and are thus not responsible for their actions.
Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by spacemanmatt on Friday April 24, 2015 @02:29PM (#9435173)
I don't agree with your characterization of the HN ethos. I would suggest the ethos is a little closer to users lack collectivization, and we're talking about how a service provider has taken advantage of that. It's a social question, not a legal one.
Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by jacquesm on Friday April 24, 2015 @03:52PM (#9435648)
In the case of ancestry it is not just their users, but also the users extended family. One of my relatives found it necessary to put me in there and I've had a really hard time to get them to delete me.
Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by Lawtonfogle on Friday April 24, 2015 @03:26PM (#9435523)
They are consenting adults, no? That they optimize for short term gain ignoring long term costs should be their own responsibility.
Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by spacemanmatt on Friday April 24, 2015 @02:25PM (#9435147)
It's not the same. That is your straw-man, not mine.
Re:Deleting the Family Tree – Ancestry.com erased 10 years o... by Lawtonfogle on Friday April 24, 2015 @03:24PM (#9435512)
I asked a question; I never made any claim that could even be a strawman.

(If we are going to go by exactly what we said and not the implications of the words we used, though hopefully we can agree that would actually be dishonest.)