News for hackers. Stuff that matters.
Re:We saved millions in SSD costs by upgrading our filesystem by jhk727 on Tuesday November 09, 2021 @03:46PM (#29167247)
Author here - it's difficult to provide a single number to summarize what we've observed re: CPU, but one data point is that average CPU utilization across our cluster increased from ~40% to ~50%. This effect is more pronounced during NA daylight hours.

Worth noting that part of the reason this is relatively low impact for our read queries is that the hot portion of our dataset is usually in Postgres page cache where the data is already decompressed (we see a 95-98% cache hit rate under normal conditions). We've noticed the impact more for operations that involve large scans - in particular, backups and index builds have become more expensive.

Re:We saved millions in SSD costs by upgrading our filesystem by infogulch on Tuesday November 09, 2021 @04:39PM (#29167849)
Hey thanks for the clarification. That seems like a worthwhile tradeoff in your case.

For backups in particular, are ZFS snapshots alone not suitable to serve as a backup? Is there something else that the pg backup process does that is not covered by a "dumb" snapshot?

Re:We saved millions in SSD costs by upgrading our filesystem by jhk727 on Tuesday November 09, 2021 @05:45PM (#29168567)
We use wal-g and extensively leverage its archive/point-in-time restore capabilities. I think it would be tricky to manage similar functionality with snapshots (and possibly more expensive if archival involved syncing to a remote pool).

That being said, wal-g has worked well enough for us that we haven't put a ton of time into investigating alternatives yet, so I can't say for sure whether snapshots would be a better option.

Re:We saved millions in SSD costs by upgrading our filesystem by derefr on Tuesday November 09, 2021 @06:02PM (#29168756)
I'd personally recommend pgBackRest as a wal-g replacement. We (Covalent) started with wal-g ourselves, but pgBackRest does full backup and restore so much faster. Besides the parallelism (which is great), pgBackRest's backups are manifests, symbolically mapping to the individual objects in storage that may have come from previous backups. Which means that a differential or incremental backup doesn't need to be "replayed after" a full backup, but instead is just like a git commit, pointing to some newer and some older objects.

Also, auto-expiry of no-longer-needed WAL segments (that we use due to our reliance on async hot standbys) along with previous backups is pretty great.

And we haven't even started taking advantage of pgBackRest's ability to do incremental restore — i.e. to converge a dataset already on disk, that may have fallen out of sync, with as few updates as possible. We're thinking we could use this to allow data science use-cases that would involve writing to a replica, by promoting the replica, allowing the writes, and then converging it back to an up-to-date replica after the fact.

Re:We saved millions in SSD costs by upgrading our filesystem by eptcyka on Tuesday November 09, 2021 @06:45PM (#29169105)
Have you looked at postgres row/column compression? Obviously, compressing the same data twice won't be too helpful, but maybe there are more wins to be had.
Re:We saved millions in SSD costs by upgrading our filesystem by TedDoesntTalk on Tuesday November 09, 2021 @05:25PM (#29168345)
how/why did you choose Postgres over MariaDB? I am facing such a decision now.
Re:We saved millions in SSD costs by upgrading our filesystem by anticodon on Tuesday November 09, 2021 @11:43PM (#29171164)
After working for years with both, I'd say that PostgreSQL is much more friendlier to the developer and more pleasant to work with. In any area: documentation, features, error messages, available SQL features, available extensions, available docs, available books.

One tiny example: I prefer to work with databases using CLI interfaces (mysql and psql).

psql CLI is a tool which is pleasant to use, has no bugs in the interface and it even gets improvements from time to time.

mysql CLI is awful to use (e.g. doesn't display long lines properly, has difficulties with history editing, etc) and looks like there wasn't a single improvement since 1996 (I'm sure there were, I just never felt the effect of such improvements).

Re:We saved millions in SSD costs by upgrading our filesystem by throwdbaaway on Wednesday November 10, 2021 @04:32AM (#29172780)
Actually, there was a significant regression. Many many years ago, Oracle decided to drop the support for the gpl-licensed readline altogether, likely because they can't ship it with MySQL Enterprise. To this day, Percona still carries a small patch to add that functionality back, which is great because I wouldn't touch any CLI without readline.
Re:We saved millions in SSD costs by upgrading our filesystem by JonyEpsilon on Wednesday November 10, 2021 @05:28AM (#29173152)
Dropping a link to `rlwrap` in case anyone is not familiar with it:

https://github.com/hanslub42/rlwrap

Note that I've never tried it myself with the mysql/mariadb CLI, but I have used it with other tools, and it's brilliant.

Re:We saved millions in SSD costs by upgrading our filesystem by jamesfinlayson on Wednesday November 10, 2021 @02:14AM (#29171971)
Yep Postgres seems to have more features too for sure. Also, a fun little limitation of the MySQL CLI is that it truncates hostnames to 100 characters - not usually a problem, but AWS database VPC hostnames easily hit that limit, and it just silently truncates rather than failing.
Re:We saved millions in SSD costs by upgrading our filesystem by agent327 on Wednesday November 10, 2021 @06:57AM (#29173694)
"It silently does something unexpected rather than failing" succinctly summarizes all the reasons you shouldn't be using MySQL.
Re:We saved millions in SSD costs by upgrading our filesystem by NegativeLatency on Wednesday November 10, 2021 @01:40AM (#29171790)
I’ve been using a Postgres foreign data wrapper to interact with a MySQL database and it’s much nicer for interactive use.
Re:We saved millions in SSD costs by upgrading our filesystem by jamesfinlayson on Tuesday November 09, 2021 @06:37PM (#29169041)
Not MariaDB but after working with Postgres for six years I'm now using MySQL and... the error messages are useless, and comparing different types works in strange ways rather than failing like in Postgres.
Re:We saved millions in SSD costs by upgrading our filesystem by 1500100900 on Wednesday November 10, 2021 @01:52AM (#29171868)
To add, execution plans presented by MariaDB are also nearly useless.
Re:We saved millions in SSD costs by upgrading our filesystem by TedDoesntTalk on Wednesday November 10, 2021 @10:51AM (#29176018)
User grants in Maria/MySQL drive me nuts. I hope it is better in Postgres
Re:We saved millions in SSD costs by upgrading our filesystem by unixhero on Wednesday November 10, 2021 @05:37AM (#29173203)
Postgres always