Re:What Happened When We Entered Every Music CD in Existence...
by jmount on Tuesday April 17, 2012 @11:51AM
(#3853157)
Isn't "3.1 million disks and 43 million tracks" only a small amount of data and "0.1 seconds" slow (assuming this is only metadata, not the music itself) at current machine scales? The article mentions use of a 300 GB 7200 RPM disk drive but I would assume (couldn't find it in the article) they would easily have more than 8GB ram- more than enough to hold everything plus indexing structures.
[ Parent
| on HN ]
Re:What Happened When We Entered Every Music CD in Existence...
by endersshadow on Tuesday April 17, 2012 @12:07PM
(#3853260)
As somebody that works in data warehousing, this strikes me as a very small data set. To have a full text index and return a simple search on "Adele" in 0.8 seconds on only 3.1 million rows (or even 43 million) is not particularly impressive.
Was this supposed to be something that surprised me? I regularly deal with billions of rows of very wide data, so I admit that my sense of scale is a little...skewed.
Re:What Happened When We Entered Every Music CD in Existence...
by jbigelow76 on Tuesday April 17, 2012 @01:16PM
(#3853741)
Maybe it's because RavenDB is a document database as opposed to an OLAP database (I'm assuming that's what you refer to when you say billions of rows with wide data and not another NoSQL like Mongo). Indexes and data arrangement between the two types are not an apples to apples comparison.
Re:What Happened When We Entered Every Music CD in Existence...
by endersshadow on Tuesday April 17, 2012 @01:38PM
(#3853919)
I was mostly referring to a relational database, but I do a lot of work with multidimensional databases (mostly SQL Server Analysis Services). You don't typically do full text searching on a cube. In warehousing, you often get wide tables because they're optimized for the reads, not the writes, so are typically second normal form (2NF) rather than the third normal form (3NF) that most people are familiar with.
Re:What Happened When We Entered Every Music CD in Existence...
by aaronblohowiak on Tuesday April 17, 2012 @12:20PM
(#3853338)
The article says:
>Querying for “Query:Adele” again? 32 milliseconds.
That would suggest that the 0.1 is for a cache miss to disk, and that would be hinted at by the phrase "cold boot" in an earlier paragraph.