rabysh
ssh username@server "sqlite3 my_remote_database.db .dump | gzip -c" | gunzip -c | sqlite3 my_local_database.db
I'd try to show how much a single process could scale by writing a simple benchmark, and comparing the performance, cost and latency of the alternative.
The results should speak for themselves at that point.
(But also, spinning up a lambda to wait for a db request? Wat? Must be missing some more context/details)
It can also queue up messages in memory and/or to disk if your remote data sink is having a hiccup.
And for those wondering how to send multiline data, well you don't. If you need to write out a big blurb, you write it out on a single line from the application. If using structured data, you can output lines as separate array items. The current built-in limit for a syslog line is 8096 bytes, but that's tunable. Just make sure the thing that writes to syslog doesn't have a low hardcoded limit like older versions of logger from util-linux (1024 bytes)
edit: the version of rsyslog shipped with the distros might be a bit dated. They're providing packages for their latest stable version, we're using that and it works pretty well.
Would be great to see results for ulib on c3.8xlarge machines.