InfluxDB is betting on Rust and Apache Arrow for next-gen data store

Re:InfluxDB is betting on Rust and Apache Arrow for next-gen... by aldanor on Wednesday November 11, 2020 @04:54AM (#25057024)

Parquet is awesome, but it's sad their Rust parquet crate only works on nightly rustc, which is a no-no in many cases. Not sure why they chose to go that way (well, because of trait specialization, technically speaking - which is not landing and not going to land anytime soon).

[ Parent | on HN ]

Re:InfluxDB is betting on Rust and Apache Arrow for next-gen... by andygrove on Wednesday November 11, 2020 @09:13AM (#25058531)

There are contributors and committers in the Arrow project working to resolve this. We recently removed specialization from the core Arrow crate and we plan on doing the same for the Parquet crate.

[ Parent | on HN ]

Re:InfluxDB is betting on Rust and Apache Arrow for next-gen... by aldanor on Wednesday November 11, 2020 @09:18AM (#25058573)

Yes, thanks a million for doing that! I've read the 'despecialization' of arrow-rust PR which actually seemed to end up being quite simple (but it's a breaking change, so a 3.0 candidate?)

Hope that something similar can be done in parquet so that arrow-parquet-df can all compile on stable in 3.0, that would unlock it for many potential users.

[ Parent | on HN ]