News for hackers. Stuff that matters.

rabysh

That was an amazing read, thanks for sharing!
gzip/gunzip might also be redundant if using ssh compression with -oCompression=on or -C on the ssh call
I think this could be a single pipeline?

ssh username@server "sqlite3 my_remote_database.db .dump | gzip -c" | gunzip -c | sqlite3 my_local_database.db

Re:Frustrated at Young Programmers on Saturday April 26, 2025 @12:11AM (#43800876)
This looks like a very good opportunity for explaining how fast computers are and how spinning up infra doesn't always make sense.

I'd try to show how much a single process could scale by writing a simple benchmark, and comparing the performance, cost and latency of the alternative.

The results should speak for themselves at that point.

(But also, spinning up a lambda to wait for a db request? Wat? Must be missing some more context/details)

That's for deleting a tag, but is it possible to remove a tag from a note? (aka untag a note)
Don't forget that rsyslog can also parse and generate structured data (json with mmjsonparse for input + templates with json escaping for output).

It can also queue up messages in memory and/or to disk if your remote data sink is having a hiccup.

And for those wondering how to send multiline data, well you don't. If you need to write out a big blurb, you write it out on a single line from the application. If using structured data, you can output lines as separate array items. The current built-in limit for a syslog line is 8096 bytes, but that's tunable. Just make sure the thing that writes to syslog doesn't have a low hardcoded limit like older versions of logger from util-linux (1024 bytes)

edit: the version of rsyslog shipped with the distros might be a bit dated. They're providing packages for their latest stable version, we're using that and it works pretty well.

Re:1M QPS with Nginx and Ubuntu 12.04 on EC2 on Wednesday August 19, 2015 @11:54AM (#10086385)
6M! that's great. But that's not comparing apples to apples, the best test they did was using a 40 core machine with a 10gbe NIC. These cards usually have as many receive/send queues as the number of cores the machine has, while the ec2 instances only have 2 queues. (and most performance gains in packet receive usually come from handling interrupts on multiple CPUs...)

Would be great to see results for ulib on c3.8xlarge machines.