News for hackers. Stuff that matters.

A comment on A Data Pipeline Is a Materialized View. Parent comment

Re:A Data Pipeline Is a Materialized View by snidane on Monday February 22, 2021 @02:33AM (#26221802)
Can somebody explain Data Mesh in simple terms? I've watched numerous videos and read the article, but still can't understand what it's supposed to say.

Is it basically to set up one data lake per analytics team, instead of a big centralized data lake?

It seems to me this is already being done from my experience. Each team has their own s3 bucket in which they store their data products and other teams can consume them using their favorite data engines as long as they can read parquet files from s3.

Or am I missing something?

[ Parent | on HN ]
Re:A Data Pipeline Is a Materialized View by benstopford on Monday February 22, 2021 @05:48AM (#26223005)
It's basically a set of 4 principals or best practices: 1. decentralized data ownership. 2. Data as a product. 3. Self-serve data infrastructure. 4. Federated governance. So I don't think of it as some new product thing but rather principals that work well for people with a certain type of use case. It's a bit like Microservices in that regard.
[ Parent | on HN ]