News for hackers. Stuff that matters.
Re:HorizonDB, a geocoding engine in Rust that replaces Elast... by dewey on Friday August 08, 2025 @11:12AM (#44837973)
To add another data point: After working with ES for the past 10 years in production I have to say that ES is never giving us any headaches. We've had issues with ScyllaDB, Redis etc. but ES is just chugging along and just works.

The one issue I remember is: On ES 5 we once had an issue early on where it regularly went down, turns out that some _very long_ input was being passed into the search by some scraper and killed the cluster.

Re:HorizonDB, a geocoding engine in Rust that replaces Elast... by itpragmatik on Friday August 08, 2025 @11:46AM (#44838392)
how many clusters, how many indexes and how many documents per index? do you use self hosted es or aws managed opensearch?
Re:HorizonDB, a geocoding engine in Rust that replaces Elast... by binarymax on Friday August 08, 2025 @12:06PM (#44838608)
Lots of other options here if you don't like managing. You can use Elastic cloud, Bonsai.io, and others
Re:HorizonDB, a geocoding engine in Rust that replaces Elast... by lisbbb on Friday August 08, 2025 @07:52PM (#44842891)
A lot of places can't put their data just anywhere.
Re:HorizonDB, a geocoding engine in Rust that replaces Elast... by chatmasta on Friday August 08, 2025 @11:59PM (#44843928)
And they can pay the vendors for "bring your own cloud" or similar. If data sovereignty is important to them, then they can probably afford it. And if cost is an issue, then they wouldn't be looking at hosted solutions in the first place.
Re:HorizonDB, a geocoding engine in Rust that replaces Elast... by dewey on Saturday August 09, 2025 @03:33AM (#44844738)
They manage it in your GCP project, so you can also make use of your commitments etc.
Re:HorizonDB, a geocoding engine in Rust that replaces Elast... by dewey on Friday August 08, 2025 @11:52AM (#44838457)
12 nodes, 200 million documents / node, very high number of searches and indexing operations. Self-hosted ES on GCP managed Kubernetes.
Re:HorizonDB, a geocoding engine in Rust that replaces Elast... by everfrustrated on Friday August 08, 2025 @11:36AM (#44838276)
How big is the team that looks after it?
Re:HorizonDB, a geocoding engine in Rust that replaces Elast... by dewey on Friday August 08, 2025 @11:47AM (#44838412)
Nobody is actively looking after it. Good alerting + monitoring and if there's an alert like a node going down because of some Kubernetes node shuffling or a version upgrade that has to be performed one of our few infra people will do that.

It's really not something that needs much attention in my experience.

Re:HorizonDB, a geocoding engine in Rust that replaces Elast... by heipei on Friday August 08, 2025 @02:36PM (#44840218)
I agree, and I don't get where the claims that ES is hard to operate originate from. Yeah, if you allow arbitrary aggregations that exceed the heap space, or if you allow expensive queries that effectively iterate over everything you're gonna have a bad time. But apart from those, as long as you understand your data model, your searches and how data is indexed, ES is absolutely rock-solid, scales and performs like a beast. We run a 35-node cluster with ~ 240TB of disk, 4.5TB of RAM, and about 100TB of documents and are able to serve hundreds of queries. The whole thing does not require any maintenance apart from replacing nodes that failed from unrelated causes (hardware, hosting). Version upgrades are smooth as well.

The only bigger issue we had was when we initially added 10 nodes to double the initial capacity of the cluster. Performance tanked as a result, and it took us about half a day until we finally figured out that the new nodes were using dmraid (Linux RAID0) and as a result the block devices had a really high default read-ahead value (8192) compared to the existing nodes, which resulted in heavy read amplification. The ES manual specifically documents this, but since we hadn't run into this issue ourselves it took us a while to realise what was at fault.

Re:HorizonDB, a geocoding engine in Rust that replaces Elast... by lisbbb on Friday August 08, 2025 @07:50PM (#44842885)
The thing I like about ES: When the business comes around and adds new requirements out of nowhere, the answer is always: "Yup, we can do it!" Unlike other tools such as Cassandra that force a data design from the get go and make it expensive to change later on.