Re:Architecture of a Database System
by adamnemecek on Saturday January 24, 2015 @02:31PM
(#8940660)
Those look awesome, thanks for posting them. Quick question, how do you learn about all these papers? They all seem very recent.
I'm working on join algorithms at the moment, so I spent the last few months getting up to speed on the latest research.
The rest is just from general interest. I spend around ten hours a week reading papers or textbooks. Whenever I find something really mind blowing I follow up citations, set up google scholar alerts for the authors, subscribe to their rss feed etc.
A lot of my favourite papers are linked on the OP site - it looks like a good place to start.
@jamil, thanks for the pointers! I will store some of these away for future editions of The Morning Paper :)
Awesome :)
Great work on the blog by the way, it's going into my morning reading list.
Do you know if HyPer is open source? The name makes searching a bit of a challenge.
http://hyper-db.de/index.html
I believe it's closed source and they plan to commercialise it, although I can't figure out where I got that idea from.
http://arxiv.org/abs/1310.3314 - Understanding the core difficulty in answering relational queries and examples of problems for which current query optimisers always produce plans which are asymptotically suboptimal
http://arxiv.org/abs/1404.0703 - The first theoretical analysis which can relate the choice of indexes to worst-case bounds. Presents a single join algorithm which is asymptotically optimal on every problem without even using cardinality estimates.
http://arxiv.org/abs/1210.0481 - A join algorithm that meets some of the bounds of the above paper and is also fast in practice and can be incrementally maintained.
https://infosys.uni-saarland.de/projects/octopusdb.php - Treating index creation, query optimising, view materialisation, incremental maintenance etc as one large optimisation problem.
http://www.vldb.org/pvldb/vol4/p539-neumann.pdf - Compiling query plans to LLVM because the query plans / indexes are good enough to make the plan cpu-bound instead of memory-bound
http://hyper-db.de/HyperTechReport.pdf - Running OLAP and OLTP workloads on the same database without interference
It seems possible that in the future, far from having a Cambrian explosion of specialised databases, we will be able to store everything in a single db and treat questions of data layout, partitioning, indexing etc as a direct optimisation problem.