Re:Precog is being open-sourced
by alexatkeplar on Tuesday September 10, 2013 @03:54AM
(#6358754)
Thanks for the explanation. What do you actually upload the JSON into - is it a proprietary NoSQL technology, something built on top of Akka, a wrapper around HDFS or something else?
Re:Precog is being open-sourced
by JulianMorrison on Tuesday September 10, 2013 @06:40AM
(#6359211)
How does this compare to rethinkdb (sharding, map-reduce, indexes, queries and joins on JSON data)?
columnar storage, compression etc
Re:Precog is being open-sourced
by JulianMorrison on Tuesday September 10, 2013 @04:47PM
(#6363373)
Interesting, so precog breaks JSON objects into columns dot-named by their path. I wonder how rethink does it, internally?
How is this different/better than using something like Elastic Search? Does it handle real time upload/searching or do you have to bulk upload, process the data and then run your searches?
No processing necessary. Just upload JSON.
The Quirrel language is much more high level than the API for ElasticSearch. It comes with things like machine learning (e.g. clustering and linear regression):
http://precog.com/developers/how-tos/machine-learning/
Even without libraries like machine learning, you can write somewhat readable queries which do some impressive things:
It's an analytics platform for big data. Basically, you upload lots of JSON, use the custom language (http://quirrel-lang.org/) for analytical queries and the platform will handle efficient storage and querying of your data.
There are also has extras like a library for charting and reporting query responses using JavaScript and an IDE for writing Quirrel and uploading data.