However, it is not surprising to see every piece in the critical path to be written in-house (feed handlers, for which there are many companies selling solutions) or rebuilt with heavy modifications (kernel).
I had intended to say hash function, but it was very late by the time I posted.
it's the same in the technical startups that I've been involved with (i.e. hardware or science):
- build a quick mvp with off the shelf stuff
- start optimizing.
by the time you're done, most of your parts are no longer generic and have optimizations where you need them.
All the numbers I could give you is what our data engine (which is what we sell) gives on our test environment, I cannot share what our customers actually get once deployed and configured on their system.
Generally, they are happy. :)
We're going to post benchmarks soon as we're going out of stealth mode.
Anyway, even if there are good hash table implementations, sometimes you have to provide a custom hash function. A default hash function will not necessarily give you a table without collisions.
I was under the impression there was a fairly good number of high quality implementations of hashes tables in the wild and that it was a thoroughly studied subject.
"Not invented here" syndrome?