News for hackers. Stuff that matters.
Re:Index bloat reduced in PostgreSQL v14 by smt88 on Friday September 10, 2021 @10:21PM (#28488935)
I don't remember specifics, but that Uber article made it clear that they didn't understand how Postgres worked and made some very basic mistakes. The whole thing made them seem surprisingly incompetent, although the general crappiness of Uber apps maybe should have tipped me off sooner.
Re:Index bloat reduced in PostgreSQL v14 by da_chicken on Saturday September 11, 2021 @04:52AM (#28490454)
That's what I remember, too.

That said, Postgres have had a somewhat common problem with poorly selected defaults.

I remember that the performance tests back over 10 years ago between Postgres (v7-v9) and MySQL (~v5.0) always showed MySQL way ahead. For a long time people assumed the reason was because MyISAM (the then-default) isn't transactional. Except InnoDB was still faster. Okay, but by default MySQL didn't `fsync()` after writes. But it was still faster when you enabled that, too.

Turns out that Postgres's default memory configurations were either largely unchanged from v6 initial releases a decade earlier, or perhaps more accurately they're simply set to make the DB not a resource hog out of the box. Well, outside of the development environment, most people set up a dedicated server, and they want the server to be a resource hog! Most RDBMSs do that automatically. It's an odd choice.

It's been 6-7 years since I used the platform daily, but I would not be surprised if it was still that way. And given the complexity of the memory configuration [0] it's hard to NOT make "basic" mistakes.

[0]: https://www.postgresql.org/docs/current/runtime-config-resou...

Re:Index bloat reduced in PostgreSQL v14 by nyanpasu64 on Saturday September 11, 2021 @05:34AM (#28490625)
In production workloads, it makes sense for a database to use up all available system resources. If I self-host Nitter on localhost and it runs Redis, or if I install KDE PIM apps which insist on hosting emails and such on a localhost MySQL database, I don't want them to eat all available resources. In my experience, Redis for Nitter, and a barely-used MySQL WordPress database, are quite lightweight in practice.
Re:Index bloat reduced in PostgreSQL v14 by Zababa on Friday September 10, 2021 @11:18PM (#28489166)
Speaking about mistakes, do you have any good resources on Postgres to ensure that you don't make the same kind of mistakes?
Re:Index bloat reduced in PostgreSQL v14 by smt88 on Saturday September 11, 2021 @12:09AM (#28489364)
Unfortunately database design and database administration are entire fields of expertise unto themselves, and I've never seen a resource that covers 100% of the need-to-know info. Oracle, SQL Server, Postgres, and MySQL are all massive, complex, and old, and they have lots of hidden footguns in them.

You should start by knowing how other people design the type of DB you're building. A reporting DB is going to work very differently from a transactional DB. If you're using PG's excellent JSONB support as an alternative to Mongo, that's also something you need to consider at the design stage.

pgbouncer is very commonly used in PG setups, to the point where I don't understand why it isn't included in the default build.

- https://www.pgbouncer.org

Sensible config generators:

- https://pgtune.leopard.in.ua/

- https://postgresqlco.nf/ (this one is more educational and works as a reference as well)

Security intro:

- https://goteleport.com/blog/securing-postgres-postgresql/

Re:Index bloat reduced in PostgreSQL v14 by Zababa on Saturday September 11, 2021 @01:06AM (#28489566)
Thank you for all of these. I find database-related knowledge usually hard to find compared to programming-related knowledge, so this helps a lot.
Re:Index bloat reduced in PostgreSQL v14 by tomrod on Friday September 10, 2021 @11:27PM (#28489205)
Hear hear!

I am really loving Postgres after a foray with Oracle. But I get concerned about using it for large production loads. I welcome recommendations as well.

Re:Index bloat reduced in PostgreSQL v14 by eloff on Friday September 10, 2021 @10:37PM (#28488995)
In what way did they make basic mistakes?
Re:Index bloat reduced in PostgreSQL v14 by smt88 on Friday September 10, 2021 @11:57PM (#28489320)
My DB knowledge is too patchy to give a good summary, but it was extensively discussed on HN several times[1].

In general it seems like they could have solved all of their problems within Postgres, but someone at Uber seemed to just know MySQL better and want to switch. They also made other major changes (like removing schemas from certain data stores) that likely gave them more of an advantage than switching could have.

1. https://news.ycombinator.com/item?id=12166585

Re:Index bloat reduced in PostgreSQL v14 by eloff on Saturday September 11, 2021 @01:56AM (#28489720)
I remember the article and the discussion, but the fundamental issue seemed to be the difference between MySQL and Postgres indices and the different trade-offs made. Postgres indices are better for read heavy workloads, and MySQL for write heavy. I don't see how this is representative of them not knowing what they are doing. Frankly it's condescending to think you understand their engineering problems better than they do.
Re:Index bloat reduced in PostgreSQL v14 by winrid on Friday September 10, 2021 @11:58PM (#28489323)
Got any links or want to divulge said details? I thought the write amplification points they made were interesting.
Re:Index bloat reduced in PostgreSQL v14 by rattray on Friday September 10, 2021 @11:02PM (#28489094)
Fwiw I personally find their android apps to be among the highest quality of any android app I encounter.