bladecatcher

Great point re: the “Bespoke” style of programming. It takes some unlearning and getting used to tacit programming [0] and trains [1] to appreciate this.

I’ve worked on fairly complex systems using kdb/q that are a joy to debug because the codebase is a handful of files. q is admittedly a more simplified and “mainstream” cousin of J, but allows a fair bit of expressiveness. The powerful data store (kdb) and q-sql that comes along with it makes it one of the most concise data platforms ever. Too bad it’s not free for commercial use!

[0] https://www.jsoftware.com/help/jforc/tacit_programs.htm

[1] https://www.jsoftware.com/help/learning/09.htm

> Now if it were possible for it to accept an event from Kafka but not guarantee that the event will eventually make it into the materialized view but may be lost, that'd be a problem.

This is usually not a problem these days as it’s possible to guarantee exactly once ingestion using Kafka offsets

You’re right. You don’t get read-your-own-writes guarantees in a typical Kdb tickerplant architecture, as the event streams are propagated to the real-time nodes asynchronously.

I think the usecase for this is streaming queries and aggregations where the query output is materialised asynchronously

In Indian cities like Mumbai, Bangalore, one can easily spend 3 to 4 hours a day commuting

I suppose one can measure the risk of losing messages - I’m not suggesting that it’s always possible, but calling it an unmeasurable risk is probably not true

Typically, one would start off in a quant trading desk which uses kdb+ for their analytics or join First Derivatives [0] that trains graduates

[0] https://www.firstderivatives.com/

https://code.kx.com/q/ has learning material and documentation . Hope it helps

You can start the 32 bit instance with the -s switch (for q to be started with multiple slaves) for parallel execution of a function over data using the "peach" command [0].

But yes, I do agree that the 32 bit instances have limited use, as they can't be used in for-profit projects.

[0] https://code.kx.com/wiki/Reference/peach

As mruts pointed out above, kdb+/q provides what you're imagining - it provides fantastic distributed computing support via its IPC protocol and also has multi-core support.

Related article posted on HN recently:

APL is more French than English [https://news.ycombinator.com/item?id=18640451]

Those who’re interested in lucid dreams should definitely watch the film Waking Life - https://en.m.wikipedia.org/wiki/Waking_Life

>> Which substances are you talking about because I'm talking about Cannabis

I see. That might be true in most cases, but even Cannabis can have adverse effects on people with predisposition to mental illnesses