binomiq
Re:Using SIMD to aggregate billions of values per second
on Wednesday April 08, 2020 @09:37AM (#22812813)
In your benchmarks, KDB's max on longs takes twice as long as sum on longs. I am not able to replicate this with the most recent version of KDB (2020.03.30).
$QHOME/l64/q -s 4
KDB+ 4.0 2020.03.30 Copyright (C) 1993-2020 Kx Systems
l64/ 4()core 516718MB <snip>
q)zz:1000000000?1000j
q)0.01*system"t do[100;max zz]"
257.52
q)0.01*system"t do[100;sum zz]"
251.95
There's a marginal difference here rather than double.For the version of KDB with the regression bug:
q)0.01*system"t do[100;max zz]"
512.21
q)0.01*system"t do[100;sum zz]"
254.24
Which starts to look more like your numbers.
Re:Using SIMD to aggregate billions of values per second
on Wednesday April 08, 2020 @07:40AM (#22811985)
KDB works around this by storing the nanos in 64 bits but only for a particular time range.
1707.09.22D00:12:43.145224194 (max negative) to 2292.04.10D23:47:16.854775806 (max positive)
With 0 at 2000.01.01D00:00:00.000000000
Re:Using SIMD to aggregate billions of values per second
on Wednesday April 08, 2020 @05:35AM (#22811418)
Were these benchmarks before or after 2020.03.26? There was a bug that caused max operations to take twice as long.
From the KDB+ 4.0 release notes:
2020.03.26 FIX fixed performance regression for max. e.g. q)x:100000000?100;system"ts:10 max x"