News for hackers. Stuff that matters.

A comment on Loading half a billion rows into MySQL.

Re:Loading half a billion rows into MySQL by advocate1 on Tuesday June 05, 2012 @06:15PM (#4071259)
It's more work for the computer to process data stored as "rows" than data stored as columns.

Think of how a disk is organized.

If you abandon MySQL and store data as columns you can load a trillion rows.

But forget I mentioned this.

[ Parent | on HN ]
Re:Loading half a billion rows into MySQL by dfragnito on Tuesday June 05, 2012 @10:13PM (#4072060)
We have been doing some tests with http://www.monetdb.org/Home. We saw inserts per second in the range of 50k/sec on a laptop.
[ Parent | on HN ]
Re:Loading half a billion rows into MySQL by advocate1 on Tuesday June 05, 2012 @10:37PM (#4072126)
mmap. +1. 10MB source tarball. -1. Looks like it's worth a test. Thanks for the reference.
[ Parent | on HN ]
Re:Loading half a billion rows into MySQL by aw3c2 on Wednesday June 06, 2012 @03:40AM (#4072809)
what is the technical difference? I would have thought the dbms takes care of a performant storage/organization system for me.
[ Parent | on HN ]