News for hackers. Stuff that matters.

A comment on Using SIMD to aggregate billions of values per second.

Re:Using SIMD to aggregate billions of values per second by binomiq on Wednesday April 08, 2020 @05:35AM (#22811418)
Were these benchmarks before or after 2020.03.26? There was a bug that caused max operations to take twice as long.

From the KDB+ 4.0 release notes:

2020.03.26 FIX fixed performance regression for max. e.g. q)x:100000000?100;system"ts:10 max x"

[ Parent | on HN ]
Re:Using SIMD to aggregate billions of values per second by bluestreak on Wednesday April 08, 2020 @05:39AM (#22811441)
these are against the latest 4.0 KDB+, after 26 March. KDB before that could not aggregate in parallel implicitly.
[ Parent | on HN ]
Re:Using SIMD to aggregate billions of values per second by binomiq on Wednesday April 08, 2020 @09:37AM (#22812813)
In your benchmarks, KDB's max on longs takes twice as long as sum on longs. I am not able to replicate this with the most recent version of KDB (2020.03.30).

    $QHOME/l64/q -s 4
    KDB+ 4.0 2020.03.30 Copyright (C) 1993-2020 Kx Systems
    l64/ 4()core 516718MB <snip>

    q)zz:1000000000?1000j
    q)0.01*system"t do[100;max zz]"
    257.52
    q)0.01*system"t do[100;sum zz]"
    251.95
There's a marginal difference here rather than double.

For the version of KDB with the regression bug:

    q)0.01*system"t do[100;max zz]"
    512.21
    q)0.01*system"t do[100;sum zz]"
    254.24
Which starts to look more like your numbers.
[ Parent | on HN ]
Re:Using SIMD to aggregate billions of values per second by bluestreak on Wednesday April 08, 2020 @10:13AM (#22813097)
sorry, i think we tested with this one:

    KDB+ 4.0 2020.03.17 Copyright (C) 1993-2020 Kx Systems
    l64/ 12(16)core 63960MB
    EXPIRE 2021.03.26
[ Parent | on HN ]
Re:Using SIMD to aggregate billions of values per second by dintech on Wednesday April 08, 2020 @10:20AM (#22813160)
Even without that 2x, all benchmarks are still in the same ballpark which is impressive in its own right. It will be interesting to see where this goes.
[ Parent | on HN ]