houqp

Very cool and well executed project. Love the sprinkle of Rust in all the other companion projects as well :)

The ROAPI(https://github.com/roapi/roapi) project I built also happened to support a similar feature set, i.e. to expose sqlite through a variety of remote query interfaces including pg wire protocols, rest apis and graphqls.

bottomless looks really nice, thanks for sharing!

Haha, yeah, we should definitely put a little bit more efforts into SEO :) Everyone is so focused on the hard-core engineering at the moment. I think Matthew from the community is actually working on a new comprehensive benchmark for us at the moment, which I hope will be published soon.

Datafusion out performs spark by a large margin. It is on par with photon based on my experiences, see benchmarks at https://github.com/blaze-init/blaze.

Yes, I designed the code base so that the core of the IO and query logic are abstracted into a Rust library called columnq. My plan is to wrap it with pyo3 so the full API can be accessed as a Python package! If you are interested in helping with this, please feel free to submit a PR. The core library is located at https://github.com/roapi/roapi/tree/main/columnq

Most users already have pip installed, so they won't need to install a rust toolchain.

It's manual for now, ROAPI is designed for slowly moving datasets. You just hit the data update API to force a refresh.

I have plan to add automated streaming data update in the background, starting with delta lake tables. It should all be very straight forward to implement.

We cache all the data in memory in Arrow format so queries don't need to go through google api, it will only hit google api when a data refresh is needed.

We are hiring new grads across the board if you have proofs for exceptional skills. Feel free to take a look at all our engineering postings at: https://neuralink.com/careers.

Up to 2x speed up with 20% of the resource consumption, pretty wild performance boost without changing a single line of code!

There are more companies noticing this now and have stopped asking these questions. For example, we at Neuralink[1] only give out practical programming challenges. If are you good at building practical systems, you should be able to ace our coding interviews without any preparation. No leetcode and no whiteboarding. In fact, I prefer to hire those who doesn't waste time practicing leetcode.

Over the past couple years, I have interviewed at a handful of other startups who also have similar coding interview philosophies.

[1]: https://neuralink.com/careers/

I love the product, not only does it have an easy to use web/offline app, I am also able to checkin the diagram source into git for version control, see: https://github.com/roapi/docs/blob/main/src/images/roapi.dra.... Then I can use automations to generate images based off that source file: https://github.com/roapi/docs/blob/main/src/images/roapi.svg.