What the Heck is a Data Mesh?

Re:What the Heck is a Data Mesh? by brunoqc on Thursday July 29, 2021 @05:53PM (#28001964)

I have a dumb question. Could I use flow to import a text file into a postgresql database? The text file is not append-only.

There's a lot of tools to import logs into stuff like kafka but not to import whole files (that can change) to a database.

[ Parent | on HN ]

Re:What the Heck is a Data Mesh? by jgraettinger1 on Thursday July 29, 2021 @06:03PM (#28002040)

Yep. You can, for example, have it watch file(s) in S3, and every time a file changes it will flow its records through into a table it creates in your DB, keyed on your (arbitrary) primary key.

[ Parent | on HN ]

Re:What the Heck is a Data Mesh? by brunoqc on Thursday July 29, 2021 @06:28PM (#28002251)

Any way to watch local files too? S3 might be fine, but just asking.

[ Parent | on HN ]

Re:What the Heck is a Data Mesh? by ta988 on Thursday July 29, 2021 @11:39PM (#28004180)

inotify?

[ Parent | on HN ]