Re:Launch HN: Castled Data (YC W22) – Open-Source Reverse ETL
by lmeyerov on Tuesday January 25, 2022 @04:08PM
(#30078040)
"Know" vs "willing to write", esp. OSS for making someone else rich
Nowadays, probably something like python then rust/go, just for community, and especially aligned on apache arrow. OSS async python / HTTP, with arrow dataplane support (fast,typed,standardized), is part of our bar for whether we consider a data proj as a core dep nowadays. A surprising amount of ETL startups are YOLO json for the dataplane, so we've intentionally stayed away due to reliability+perf heart pains. But maybe you can fake it till you make it that way too, and then hire staff to clean it up 2 years later :)
Nowadays, probably something like python then rust/go, just for community, and especially aligned on apache arrow. OSS async python / HTTP, with arrow dataplane support (fast,typed,standardized), is part of our bar for whether we consider a data proj as a core dep nowadays. A surprising amount of ETL startups are YOLO json for the dataplane, so we've intentionally stayed away due to reliability+perf heart pains. But maybe you can fake it till you make it that way too, and then hire staff to clean it up 2 years later :)