News for hackers. Stuff that matters.
Re:SQL should be your default choice for data engineering pi... by billythemaniam on Tuesday January 31, 2023 @06:22AM (#34593798)
> One example is dbt. They started as writing simple SQL models. With the introduction of Jinja, majority of models look nothing like SQL anymore. You need to visualize your model to understand relationships. It took the beauty of SQL and mucked it.

I know you are pitching your startup, but that's just completely untrue.

Re:SQL should be your default choice for data engineering pi... by srcreigh on Tuesday January 31, 2023 @01:03PM (#34598961)
The dbt site describes this as a simple example:

    select * from {{ref('really_big_table')}}
    
    {% if incremental and target.schema == 'prod' %}
      where timestamp >= (select max(timestamp) from {{this}})
     {% else %}
      where timestamp >= dateadd(day, -3, current_date)
    {% endif %}
Re:SQL should be your default choice for data engineering pi... by billythemaniam on Tuesday January 31, 2023 @03:16PM (#34601285)
As someone who has written 100s (maybe 1000s at this point) of DBT models, the amount of Jinja you need is at most 1-2% of the codebase.