jloveless

We’ve used versions of L in production for approaching year 10

More niche than many yes - but there’s something about learning an array language that changes how you look at programming in general. And for the better. Of course, the places where these things are used are generally very well paying positions. They tend to be areas with a very high amount of data that is incredibly valuable.

intentional. I consider kdb/kdb+ to be understood as the KX products - but L runs the same database style functionality (the qSQL is essentially just sugar atop the q/k code. Most of which is written in k [or C in L for a number of components])

I’d encourage running the master bench yourself! Super easy

https://github.com/l-labs/master-benchmark

Or on the database side of the house try the h2o.ai base that duckdb excels at

https://github.com/l-labs/db-benchmark/tree/main/l

Even deeper is the full test suite ala

https://github.com/l-labs/rust_ipc/blob/master/tests/ipc_tes...

Which is ~6k tests (Rust is actually what I use to drive release testing)

For QC this is a pretty solid tester (run on L or other compatible runtime) - focused on performance as it pushes multiple edges.

https://github.com/l-labs/master-benchmark

Which can also be run with minimal modifications on ngn and others.

For database the h2o.ai for duckDB really is a solid hard bench IMHO:

https://github.com/l-labs/db-benchmark/tree/main/l

fair point. I am not a front end designer and I did want something less spartan than proverbial q.txt [1] . To be honest - many folks in the community have had access for some time , and a website was just put up this week as a 'larger community access'.

That said the mail group has more details and the blog posts are effectively internal posts turned into shorter (less code) versions. The binary itself is years of effort - with small improvements over many years and shaving a few kb each time ... 'built the old way' like amish furniture :)

[1] https://web.archive.org/web/20160306154854/http://kx.com/q/d...

relative to CPU speed ... no. see [1][2].

the early versions of L (circa ... 2011/2012) couldn't really deliver any substantial performance improvement over what is out there (BQN/ngn/Kona). The memory bandwidth was the limit - I simply couldn't keep the cores active. ~2018/2019 I started from scratch with arrays (vectors) using compression by default. That was a massive unlock - and I could not keep most of the CPU doing actual compute! Then it was years of working on compression native operations - some of which were obvious[3] like sum/reductions ... others not so much!

[1] https://www.emergentmind.com/topics/memory-wall [2] https://www.cse.iitd.ac.in/~rijurekha/col216/quantitative_ap... [3] https://lv1.sh/blog/compute-on-compressed/

Definitely not a vibe coded language - I really wish the ai models were more helpful - but I should do a write up of the assembly analysis (and asm2vec). ATW is working on even more impressive things at the moment! Well beyond the scope of an interpreter itself.

KlongPy and KlongPy + duckDB are wonderful. GREAT job. I think there's a whole world of backprop / ML work that can be done in that style! Given the heritage not surprising that most Klong code is ~= L in k mode! Same with ngn/k . E.g. sigmoid is almost the same (:: for assign in klongpy , : for l/q/k). The autograd work is great!

Sorry about that. Basically you can now write and run code in the K/q/or qsql languages on your laptop for free. For a very long time (20 years+) this language has not been accessible to most - it’s almost exclusively used on Wall Street and has a very high price barrier to entry. Q programmers are still some of the most highly paid engineers consistently - as the workforce is small and the use cases are (generally) extremely lucrative. If you are a programmer wondering what comes next… q might be for you. Over the decades I’ve taught a few dozen people Q- and I’m happy to say they’ve gone on to have wonderful and stable careers in high paying jobs with deep stability. Mostly hedge funds and banks (and exchanges). It’s a wonderful and often times world changing language and now you can play with it yourself. It’s also very very fast. As a database it’s still insanely great for many usecases.

duckdb is great and its h2o.ai benchmark was difficult to beat!

there's one in the footer and agree.

Unfortunately the code itself is in a style of C many find difficult to read. I blame my upbringing. ATW open sourced examples and it was not really helpful. More recently others are doing a step by step in more standard C https://github.com/ardentsia-cgs/kparser/

there's even a secret -17!`name that will show the details. e.g.

  //100k random 32b ints ... 
  l>v:100000?255
  l>v
  196 124 18 216 63 169 151 173 126 99 90 133 92 158 217 169 201 191 138 105 13..
  // but actually they are 1/4 the size e.g. int8
  l>-17!`v
  1b        // is compressed?
  100032j   // compressed bytes
  400000j   // original bytes

simple example at https://lv1.sh/blog/compute-on-compressed/ But in general compression is reducing the bit width of the input through an encoder (FOR or Frame of Reference is an old and good example). So we store the base in an offset location then the large payload is a much smaller size. E.g. i64 can goto i16. Then simd gets more #'s per cycle on the smaller, and the base is added to the scratch in stack (for sum). avg is similar (since it is just sum / count)

give it a spin! download is ~500kb for mac. It is however targeting folks who come from that world - but K/Q are absolutely worth exploring!

yup website is Claude Design for prototype. For the core ... AI has been less helpful than I hoped - I believe largely because array style languages have so little source in training? But where Claude was especially helpful (other than the web design w/ Claude Design for the prototype) was analyzing ASM output of functions and optimizing those (although it was hard to go ASM>C or ASM>rust). E.g. lots of small mistakes would have been missed by not having restrict/const in places. Claude was great at compile all functions, analyze ASM, suggest optimized ASM (and ASM2vec was helpful as well for finding any "similar" code paths that could be combined (e.g. var/dev/cov are just moments)

I think one day we'll do that on https://oxide.computer/ 1: because we love our friends 2: the gear is great (and ROCEv2 supported which L uses)

BQN & CBQN are absolutely wonderful pieces of code. L is mac/lin but linux is avx512 only specifically to try to deal with that problem. The compute on compressed algos helps fit more in those cache lines! https://lv1.sh/blog/compute-on-compressed/

shoutout to bryan @bcantrill who I think explained K best at https://youtu.be/2wZ1pCpJUIM?si=y4ugbFXroTZc22AY&t=471 and https://queue.acm.org/detail.cfm?id=1531242

and fusion (e.g. f g h x has no intermediates e.g. mutates in place) is new as is compute on compressed vectors (very helpful performance unlock) https://lv1.sh/blog/compute-on-compressed/

next letter after k

Not a web dev - but have some experience in benchmarking these types of workloads e.g. https://www.mcobject.com/press/november19-2014/

L used the two open ones that are easy to replicate: H2O.ai (great bench) https://github.com/l-labs/db-benchmark TSBS (less great but useful) https://github.com/l-labs/tsbs

If there are others (will do ClickBench) they'll go there as well

benchmarks at https://github.com/l-labs unlike klong/ngn/bqn et al (which are GREAT) this has the goal of full production database compatibility (and full language compatibility).

There's also a lack of communication with the dev partners in general (e.g. the recent Service Worker issues). Building up a more robust partner program with (perhaps) a different support team could be really helpful.

Agreed. A lot of this is all standard now - even when I wrote about it circa 2013 [1]

[1] https://queue.acm.org/detail.cfm?id=2536492