Enterpise also often wants a full data platform (like Databricks), not a plain data warehouse.
Are there even any "plain data warehouse" vendors left? Even the oldest of the old-school providers seem to have crawled up the modern data stack since the data warehouse glory days, refocused on lakehouse + Open Table Formats either in their core platform or complementary products.
I'm curious - as someone who hasn't been paying super close attention to the space, what does it mean to be a full data platform? Is it just having all the different flavors of DB you might need all from one vendor, or is there also tighter integration than if you cobbled it together from multiple vendors?
Essentially, yes. Different DB’s, federated queries (aka delta sharing, zero copy), definition/semantic layer tools, data engineering/pipelines, model training and notebooks, governance, data lineage, row/column/whatever access control.
It’s basically a luxury minivan. It’s may not be the fastest or prettiest or cheapest, but it’s a safe way for a large family of “data and AI people” to traverse a large organisation.
More seriously, I like to call it an “analytics workbench” in a professional setting.
Exactly, look at what GCP are adding to BigQuery.
> Not commoditising for enterprise. My last gig wouldn’t allow open source software or any company that might not be there in a decade, or which kept data anywhere but our own tenant.
Hence IBM talking up Iceberg: https://www.ibm.com/think/topics/apache-iceberg
> My last gig wouldn’t allow open source software or any company that might not be there in a decade
I bet they had VMware all over the place.
Yup. And OpenShift. And Red Hat for Linux. And SAP. And IBM. But you know what? Tiny group of people relative to the impact, revenue and competitors. If we needed skills we clicked “buy”, 100 consultants would arrive who are experts, sort it and we’d move on. Not scratching around looking for people who know what we use and needing to learn 50 different open source tools. Coming from a much looser universe I learned to appreciate the principles for that context.
Re:Databricks and Neon
by carlhjerpe on Thursday May 15, 2025 @11:00AM
(#43995774)
I've worked at a Swedish streaming company, we bought a filesystem from IBM called "GPFS" or something before it was renamed. Well we had shit performance, reliability and everything else that could be bad. It was running on FC switches recommended by IBM, config from IBM, everything from IBM and it was an absolute fucking joke, metadata timed out regularly and the solution was "deal with it". I think my superiors should've been harder claiming we haven't received what we bought but we're Swedish so we'd rather just fall over and die.
We only used MQ. Against my recommendation, but it was fine if imperfect, and a known entity to all concerned. Did not use that file system.
I managed to add one startup and so far it’s done very well, but it was an exceptional case and the global CEO wanted the functionality. But it used MongoDB and we didn’t have any skills, so rather then learn one tiny thing for an irrelevant data store they added cash to use Atlas with all the support and RBAC etc etc. Kept hiring load down, one number to call, job done.
Startups are from Venus, enterprise are from Jupiter.