https://github.com/seaweedfs/seaweedfs/wiki/Cloud-Drive-Bene...

https://github.com/seaweedfs/seaweedfs/wiki/Cloud-Tier

https://github.com/seaweedfs/seaweedfs/wiki/Benchmarks

https://github.com/seaweedfs/seaweedfs/wiki/Words-from-Seawe...

https://github.com/seaweedfs/seaweedfs/wiki/Amazon-S3-API

...your true issue is it seems like you're using the filesystem as the "only" storage layer in play, but you also need time and entity querying(!?!).

>> we need to be able to query on a per sensor basis and a timespan

...look at the "Cloud-Tier" wiki page. If you're truly in an "everything's hot all the time" situation, you really should be using a database. If you're pulling "usually recent stuff, occasionally old stuff" then fronting with something like SeaweedFS seems like it might "just" transparently reduce your overall costs.

Really, I'd nudge towards "write .txt ; compact ... ; SELECT ... && cat .txt".

Basically, keep your inbound writes cached to (eg) seaweed as unit files. "Compact them" every hour by appending rows to some appropriate database (I mean: migrate to using litefs, turso, postgres, something like that). When you read, you may need to supplement "tip" data from your incoming files, but the majority should be hitting a "real" remote database, there's plenty to choose from!

A nifty note, sqlite can connect to multiple DB's at once: https://www.sqlite.org/lang_attach.html ... https://stackoverflow.com/posts/10020/revisions

...something like `select * from raw union (select * from one_hour) union (select * from today) union (select * from historical) ...`