News for hackers. Stuff that matters.

A comment on Building an Open, Multi-Engine Data Lakehouse with S3 and Python. Parent comment

Re:Building an Open, Multi-Engine Data Lakehouse with S3 and... by dogman123 on Tuesday February 18, 2025 @05:30PM (#43095938)
sqlmesh execution engine + cloud resource provisioning

provision spark on emr or duckdb on beefy ec2 -> run sqlmesh -> wipe resources.

i'm still in MVP phase of revamping my company's current data platform, so maybe there are better alternatives -- which i'd love to hear about.

[ Parent | on HN ]
Re:Building an Open, Multi-Engine Data Lakehouse with S3 and... by bradhe on Tuesday February 18, 2025 @05:37PM (#43096012)
Cool, would be happy to talk a bit more about it. Want to shoot me an email (in bio) or if you sign up for our waitlist on https://tower.dev I can reach out in the AM (we’re in Germany)!
[ Parent | on HN ]