Breaking the trillion-rows-per-second barrier with MemSQL

Re:Breaking the trillion-rows-per-second barrier with MemSQL by ddorian43 on Monday March 19, 2018 @06:22AM (#16617317)

So some questions::

1. Isn't (3)vectorization and (4)SIMD the same thing ?

2. I don't see the data-size before-after compression ?

3. How much RAM has each server ?

4. How do all cores work for all queries ? Is the data sharded by core on each machine or each core can work on whatever data ?

5. What's a comparison open-source tool to this ? Only I can think about is snappydata.

[ Parent | on HN ]

Re:Breaking the trillion-rows-per-second barrier with MemSQL by plamb on Monday March 19, 2018 @11:52AM (#16619340)

SnappyData employee here. In general this is called the "HTAP" industry (Gartner's phrase: Hybrid Transactional/Analytical Processing).

SnappyData: https://www.snappydata.io, MemSQL: https://www.memsql.com/, Splice Machine: https://www.splicemachine.com/, SAP Hana: https://www.sap.com/products/hana.html, GridGain: https://www.gridgain.com/

are some of the technologies within it

[ Parent | on HN ]

Re:Breaking the trillion-rows-per-second barrier with MemSQL by ryanworl on Monday March 19, 2018 @07:00AM (#16617473)

For 5, check out Clickhouse. It isn’t identical, but scanning a trillion rows a second is just a matter of sharding the data into enough nodes.

https://clickhouse.yandex

[ Parent | on HN ]

Re:Breaking the trillion-rows-per-second barrier with MemSQL by mehrdadn on Monday March 19, 2018 @06:38AM (#16617387)

(1) I don't know if this is what they're thinking, but not all single instructions that operate on "multiple data" are vector operations... e.g. I wouldn't call summing all items in an __m128 to be a vectorized operation. Similarly I suppose not all vectorized operations are SIMD, since they might just use normal (SISD) instructions on entire arrays. But again, I'm not sure this is really what they mean; it wouldn't seem to be a very useful distinction to make.

[ Parent | on HN ]