Re:Show HN: Semantic programmatic search for computable data
by PabloOsinaga on Tuesday January 24, 2012 @01:21PM
(#3505936)
Can you elaborate a bit further on the data you have? Is it a flexible architecture where you plugin data from multiple sources? How do you solve ETL for that? Or is it a fixed data set? If so, what is it?
Re:Show HN: Semantic programmatic search for computable data
by kemvi on Tuesday January 24, 2012 @01:33PM
(#3505996)
Currently, it's great for socioeconomic data on countries, as well as for high-res socioeconomic data for the US (county resolution). There's data from the World Bank, Dept of Health, Dept of Labor, Dept of Justice, and several other public sources, with all entities reconciled.
We're building a pipeline to make it so that ETL involves as little human time as possible, but it's in its early stages now.
Re:Show HN: Semantic programmatic search for computable data
by Stwerner on Tuesday January 24, 2012 @03:12PM
(#3506492)
This is awesome. Are you planning on adding more diverse data sets beyond government and economic? Something like professional sports stats, college sports stats, etc?
Re:Show HN: Semantic programmatic search for computable data
by kemvi on Tuesday January 24, 2012 @03:53PM
(#3506713)
If there were canonical sports stats data sources to include, it wouldn't be difficult. Email us at [email protected] if there's something specific you have in mind.
Re:Show HN: Semantic programmatic search for computable data
by kemvi on Tuesday January 24, 2012 @01:09PM
(#3505876)
The idea is that you'd use this via the python shell (and soon, from your R code or Javascript code), and not through the website. Looks like we need to make that clearer. The website stack is Postgres+Django+Python.
We're building a pipeline to make it so that ETL involves as little human time as possible, but it's in its early stages now.
The data backend is indeed NoSQL, but we didn't choose a graph db because there are no really good graph solutions that are easily parallelized.