Show HN: Quanta – A Fast, Memory-Efficient Data Engine for Analytics

Posted by sagari on Sunday February 09, 2025 @10:19AM
from the show-hn dept.

I've built Quanta, a Java-based data engine for fast, memory-efficient analytics on large datasets, all within a single JVM. It uses direct memory management, optimized indexing, and avoids the complexity of distributed systems. Early results are promising, and I'd love to get your feedback.

Key features:

* Millisecond-level queries

* Low memory footprint (~25MB/column for 20M rows)

* Fast ingestion

* Persistence and crash recovery

Example: Querying taxi trip data:

Java

  // Query: Find taxi trips that originated in Manhattan, with Uknown payment
  // and were picked up between 6AM and 11AM (pickup_hour > 5 and < 12).
  Query q = taxiData.newQuery()
          .and("pickup_location", "Manhattan")
          .and("payment_type", "Unknown", "No Charge")
          .gt("pickup_hour", 5)  // pickup hour > 5 (i.e. after 5 AM)
          .lt("pickup_hour", 12) // pickup hour < 12 (i.e. before noon)

Quanta is designed for speed and real-time analytics on structured, columnar data. Ideal for developers needing low-latency lookups without distributed system overhead.

Looking for your feedback!

GitHub: https://github.com/sagaripte/Quanta