Re:DBT Founder: Is the "Modern Data Stack" Still a Useful Idea?
by jochem9 on Wednesday February 14, 2024 @01:32AM
(#39367133)
The difference is that raw data can come in many shapes (e.g. impossibly nested jsons) and unexpected quality (changing field names...). I cannot easily work with this in SQL and definitely not write tests to cover the ever increasing complexity of dealing with messy data.
After that step the data is a lot more uniform, so then it's easy to use SQL.
After that step the data is a lot more uniform, so then it's easy to use SQL.