Using SIMD to aggregate billions of values per second

Re:Using SIMD to aggregate billions of values per second by bluestreak on Wednesday April 08, 2020 @05:39AM (#22811441)

these are against the latest 4.0 KDB+, after 26 March. KDB before that could not aggregate in parallel implicitly.

[ Parent | on HN ]

Re:Using SIMD to aggregate billions of values per second by binomiq on Wednesday April 08, 2020 @09:37AM (#22812813)

In your benchmarks, KDB's max on longs takes twice as long as sum on longs. I am not able to replicate this with the most recent version of KDB (2020.03.30).

    $QHOME/l64/q -s 4
    KDB+ 4.0 2020.03.30 Copyright (C) 1993-2020 Kx Systems
    l64/ 4()core 516718MB <snip>

    q)zz:1000000000?1000j
    q)0.01*system"t do[100;max zz]"
    257.52
    q)0.01*system"t do[100;sum zz]"
    251.95

There's a marginal difference here rather than double.

For the version of KDB with the regression bug:

    q)0.01*system"t do[100;max zz]"
    512.21
    q)0.01*system"t do[100;sum zz]"
    254.24

Which starts to look more like your numbers.

[ Parent | on HN ]

Re:Using SIMD to aggregate billions of values per second by bluestreak on Wednesday April 08, 2020 @10:13AM (#22813097)

sorry, i think we tested with this one:

    KDB+ 4.0 2020.03.17 Copyright (C) 1993-2020 Kx Systems
    l64/ 12(16)core 63960MB
    EXPIRE 2021.03.26

[ Parent | on HN ]

Re:Using SIMD to aggregate billions of values per second by dintech on Wednesday April 08, 2020 @10:20AM (#22813160)

Even without that 2x, all benchmarks are still in the same ballpark which is impressive in its own right. It will be interesting to see where this goes.

[ Parent | on HN ]