News for hackers. Stuff that matters.

A comment on Show HN: Arc, an open-source Databricks alternative.

Re:Show HN: Arc, an open-source Databricks alternative by psing on Thursday March 25, 2021 @08:02AM (#26579160)
Can you specify between complete pulls of the source vs incremental based on a timestamp field?
[ Parent | on HN ]
Re:Show HN: Arc, an open-source Databricks alternative by seddonm1 on Thursday March 25, 2021 @04:08PM (#26584767)
Yes. We usually use a ConfigExecute (https://arc.tripl.ai/execute/#configexecute) stage to dynamically calculate a runtime parameter and pass that into the JDBCExtract query for example. There is an example here: https://arc.tripl.ai/solutions/#delta-processing
[ Parent | on HN ]