News for hackers. Stuff that matters.
Re:The new local data stack: Integrating Dagster, dbt and Du... by dappermanneke on Thursday December 14, 2023 @08:59AM (#38641362)
what stops you from running this on a container and dumping the results with a script to be queried by a reporting solution into bigquery or snowflake?
Re:The new local data stack: Integrating Dagster, dbt and Du... by geoHeil on Thursday December 14, 2023 @12:26PM (#38644084)
Hi,

Georg - one of the authors is here.

In fact, this is exactly what we argue: Keep data the consumer experience high a Paas platform (Fabric, BigLake, Databricks, SF, ...) can make a lot of sense, whereas for the best data development/creator experience a high-code solution based on solid software engineering best practices should be the preferred solution - at least in my/our(authors) opinion.

Re:The new local data stack: Integrating Dagster, dbt and Du... by jakozaur on Thursday December 14, 2023 @12:22PM (#38644021)
I already see companies with BigQuery setuped.

I would like to quickly grab some data, queries and dashboards and run them in my local warehouses.

Re:The new local data stack: Integrating Dagster, dbt and Du... by geoHeil on Friday December 15, 2023 @03:01AM (#38652045)
If you use bigquery in big lake mode - i.e. with parquet or iceberg or delta files on GCS (or other object store) you can easily pull in the data into duckdb as well