Re:How Does the SQL Server Query Optimizer Work?
by needusername on Friday January 20, 2017 @02:42AM
(#13442292)
> At around a billion records, you start getting into distributed/scale-out systems, and distributed indexes are performance death whether for OLTP or analytics. Even before you cross into distributed query processing, the random access nature of indexes levies a heavy tax.
I assume you mean once process/aggregate a billion records. Having a billion records in a single table on a single, non-distributed system is not an issue at all.
Re:How Does the SQL Server Query Optimizer Work?
by gaius on Friday January 20, 2017 @09:55AM
(#13444184)
At around a billion records, you start getting into distributed/scale-out systems
Uhh, a billion-row table you can handle on a laptop easily. Probably in SQLite.
Re:How Does the SQL Server Query Optimizer Work?
by orf on Friday January 20, 2017 @12:01PM
(#13445120)
> At around a billion records, you start getting into distributed/scale-out systems
Yeahhh.... no. I remember running a 250-300,000,000 row postgres table of Wikipedia links on my 2010 Dell XPS and it functioning fine, if not a bit slow (it had a huge index on an array column). Good machine, but ancient now.
What about different compression schemes for different sets of "column" values - with the idea that you execute against the compressed values, rarely ever decompressing. Even column store systems that allow you to specify a compression scheme aren't adaptive. Why the hell not? This is almost 2020.