News for hackers. Stuff that matters.
Re:PostgreSQL 13 by kriro on Thursday September 24, 2020 @10:41AM (#24579040)
For me, this project is the biggest FLOSS success story after Linux, xBSD and the GNU stuff. Great quality, amazing team. The best RDBMS that exists today and that includes commercial ones (imo).

Thank you.

Re:PostgreSQL 13 by kumarvvr on Thursday September 24, 2020 @11:44AM (#24579708)
Absolutely. I wish there are case studies about how Pgsql managed to be on the top and innovate so effectively for so long.

I love this db. Its the database for the rest of us and the database for everyone else.

Re:PostgreSQL 13 by mr_overalls on Thursday September 24, 2020 @04:01PM (#24582937)
Postgres is fantastic. But let's be honest about how it compares to commercial offerings.

https://old.reddit.com/r/PostgreSQL/comments/hqnafk/postgres...

"Before I start, a disclaimer: I Postgres, I’ve been a PG Applicative DBA for the better part of the last decade, and as a programmer I absolutely love the PG code base. . . It should also be said that Oracle’s products are insanely expensive, and yes, their business practices have been called into question quite a few times in the past.

Having said all that, and coming from many years of experience as both an Oracle DBA and PG DBA - comparing Oracle Database to Postgresql is almost like comparing a Cessna to an F-22. Only the F-22 can also transform into a Super Hercules and a helicopter.

Putting aside the usual issues of bugs and related incidents (both have had their fair share), the Oracle Database is a technological marvel. I can list off of the top of my head a number of capabilities Oracle has had for almost 20 years that PG either doesn’t have or just might have in the near future:

1. On commit fast-refresh materialized views, 2. Query Rewrite to Materialized Views, 3. Direct and asynchronous I/O, 4. Parallel Query Execution, 5. Oracle RAC - Multi Master without Replication, 6. Zero downtime upgrade, 7. No autovacuums, 8. Flashback, 9. Integrated In-Memory Database, 10. Autonomous Indexing, " etc.

The Reddit post is a good summary and worth a read, imho.

Re:PostgreSQL 13 by sbuttgereit on Thursday September 24, 2020 @05:19PM (#24583799)
I've been working with PostgreSQL professionally for almost 20 years, I also worked professionally with Oracle for around 10 with some overlap between those time frames.

I can vouch for your comment largely. I love PostgreSQL and expect to work with it pretty much exclusively for professional RDBMS needs into a fairly distant future (with perhaps some SQL Server)... but there are features that were in Oracle back in the earlier 00's that PostgreSQL is only within the past few years catching up to. Partitioning, materialized views, etc. even though PostgreSQL is got these things now, the features are still not a feature-full and mature as they were in Oracle back then.

And speaking of SQL Server, the tooling and reporting capabilities there are something that I haven't seen achieved in the PostgreSQL ecosystem yet (to the point where my go-to client interface is still good old psql for lack of anything really better).

So, I think the kudos coming to the PostgreSQL team whenever a post like this surfaces here and elsewhere is definitely earned. But perspective on how far there is left to go is warranted and should be exciting to those that are building the future of PostgreSQL... that not all problems are solved and that you're not just there to oil the cogs laid down by generations past.

Re:PostgreSQL 13 by mathh on Thursday September 24, 2020 @04:48PM (#24583434)
That is an important comment. I really like Postgres, as it is the only free RGDB allowing complex SQL (MySQL is improving), and is also reliable, but some people need to realize that the F22/Cessna comparison is not exagerated.

I professionally use both Postgres and Oracle, for many years, and can confirm that Oracle is so much more advanced.

In addition to the features you listed, I can think of : real-time query monitoring, active session history, inline function definition in SQL, global partitioned indexes, in-memory column store, much more advanced parallel execution capabilities, adaptive plans during query execution, bloom-filters, optimizer baselines, SQL macros, Exadata for tuning at the hardware level, Golden Gate for replication, Enterprise manager for monitoring...and a lot more actually.

And of course : Optimizer hints

Re:PostgreSQL 13 by petersellers on Thursday September 24, 2020 @04:11PM (#24583036)
Sounds like the F-35 might be a better analogy then. Astronomically more expensive, extensive but questionable feature set.
Re:PostgreSQL 13 by mjw1007 on Thursday September 24, 2020 @05:26PM (#24583880)
But a similar list in the other direction would be non-empty, right?

For example, PostgreSQL's SERIALIZABLE isolation level has actually been serializable for nearly a decade now. As I understand it, Oracle's isn't yet.

Re:PostgreSQL 13 by systemvoltage on Sunday September 27, 2020 @08:13PM (#24611450)
This post shows how developers (and people in general) have a difficult time with emotional forbearance when it comes to objective technological critique and being able to put aside (and recognize) personal bias.

The author had to put so much upfront to satiate and calm the crowd before embarking on his review.

We shouldn't have to do that.

We should operate with surgical precision on the way we argue and judge others. Try to double-check when we assume something of others. That does not mean we shouldn't talk about how unethical Oracle is, just that those two discussions can take place simultaneously without drama.

The more senior I become in my career and the older I get, the more I realize that the entire world operates like this. It is grim.

Re:PostgreSQL 13 by antb123 on Friday September 25, 2020 @05:39AM (#24588178)
And to use most of these you need a Oracle consultant for 120 per hour and a 30k per cpu upgrade license.

These guys are professional robbers. They bribed Toronto city hall policitians for a 30 million dollar thing including a well known hockey player.

If you are abstracting the SQL into an ORM you can't take advantage of many of the features anyway.

Was involved in a 10,000 tps payment solution that used Postgresql to grow a 400 million company. And worked at banks where they would pay for oracle and never use half of the features. Relying on hardware based solutions for fail over and replication.

Re:PostgreSQL 13 by castorp on Thursday September 24, 2020 @05:21PM (#24583820)
> 4. Parallel Query Execution

Available since Postgres 9.6 and getting better with every release.

Re:PostgreSQL 13 by namelosw on Friday September 25, 2020 @03:19AM (#24587380)
If it's attempting to leverage many Oracle features, there might be some other good solutions to the problem as well - people don't have to use RDBMS to solve everything.
Re:PostgreSQL 13 by etaioinshrdlu on Thursday September 24, 2020 @05:20PM (#24583808)
But, I was told (by Cantrill and echoed by HN), that Oracle is a rotten company that "ships mediocrity"?

Do they actually have the best RDBMS in the industry in terms of performance and feature set? (I guess this is a hard thing to answer given how these things can be fairly subjective)

Maybe the real problem is that it's better, but still not worth the cost (both money, and the cost of being an Oracle customer).

Re:PostgreSQL 13 by ksec on Friday September 25, 2020 @04:51AM (#24587916)
>"ships mediocrity"

Name me one VM that is remotely as advance as GraalVM.

I dont like Oracle as a business. People hate them for suing Google on open source / Copyright and API. But one Google is not without its fault in the case and two, whether you like or hate a company has nothing to do with its technical capability.

It seems in the tech industry, some company can do no wrong, and some company can do no right.

Re:PostgreSQL 13 by mr_overalls on Friday September 25, 2020 @02:02PM (#24592432)
I have dealt with Oracle in the enterprise space, and endured their audit-turned-blackmail monetization strategies. They have a mixed bag of tech, some excellent, some mediocre - but their business practices are absolutely revolting and predatory.
Re:PostgreSQL 13 by diggan on Thursday September 24, 2020 @12:32PM (#24580404)
> and the database for everyone else

I also love postgresql but lets not get ahead of ourselves! It's great for SQL and as a traditional relational database, but there are plenty of use cases that are better served using other models (especially if you care about _time_ as a concept, then Datomic or crux is probably better suited) [although there are attempts on making bitemporality accessible in postgresql too, like https://github.com/scalegenius/pg_bitemporal]

Re:PostgreSQL 13 by SOLAR_FIELDS on Thursday September 24, 2020 @12:35PM (#24580448)
There is a professionally developed time series extension for Postgres. I haven’t tried it personally but it might be “good enough” for quite a few use cases just like Postgres’ full text search can be “good enough” without having to bring in the machine gun that is ElasticSearch/Solr.
Re:PostgreSQL 13 by jjjensen90 on Thursday September 24, 2020 @02:50PM (#24582119)
I brought timescale (a postgres extension that brings time series to postgres) to my current IoT job, and it has been the biggest engineering win of my career. The tooling, community, extensions, and quality of postgres + very high performance inserts + reads has reduced costs for the business dramatically versus the previous cassandra + sql server setup. We have tens of billions of events in multiple timescale hypertables, with many thousands of inserts per second running on r5.xl nodes in k8s (4 vcpu, 30gb ram) and they don't break a sweat.

So yeah, it's "good enough".

Re:PostgreSQL 13 by paulie_a on Friday September 25, 2020 @03:22PM (#24593240)
Could you send me an email because my company is starting something similar
Re:PostgreSQL 13 by diggan on Thursday September 24, 2020 @05:33PM (#24583951)
I realize now in after hand that we're talking about two different things. You're mentioning time series data, I talked about bitemporal databases.
Re:PostgreSQL 13 by diggan on Thursday September 24, 2020 @12:38PM (#24580502)
Right, and it's probably "good enough" for most use cases. But, as always, it can't cover _everything_. I'm just making an argument against the sweeping argument that "postgresql is for everyone and everything"
Re:PostgreSQL 13 by joseluisq on Thursday September 24, 2020 @01:22PM (#24581028)
PostgreSQL team and community disserve a big thanks for the huge effort during the years.

One of the most amazing FLOSS databases with nothing to envy to others RDBMSs.

Re:PostgreSQL 13 by please-reread on Thursday September 24, 2020 @03:36PM (#24582628)
The typo does a bit of disservice to a well-meant comment.
Re:PostgreSQL 13 by joseluisq on Thursday September 24, 2020 @04:56PM (#24583550)
Sorry for the typo. I was just reading the thread on my mobile while walking, my clear mistake. However it's obvious the well-meant on the thread above.
Re:PostgreSQL 13 by mariushn on Thursday September 24, 2020 @03:39PM (#24582657)
Anyone can share what makes PostgreSQL so successful? Maybe other projects can learn adopt a few of these good practices.

How are the team members funded? I was surprised to write a suggestion a few months ago and got replies from Tom Lane & Bruce Momjian themselves.

Re:PostgreSQL 13 by luhn on Thursday September 24, 2020 @04:24PM (#24583178)
Most of the core team is employed by enterprise Postgres companies (EnterpriseDB, Crunchy Data, 2nd Quadrant, Citus Data). I think they're allowed to put a portion of their workweek towards Postgres.
Re:PostgreSQL 13 by cookiecaper on Thursday September 24, 2020 @05:24PM (#24583853)
> Anyone can share what makes PostgreSQL so successful? Maybe other projects can learn adopt a few of these good practices.

Since honesty is a theme of this thread, let's be real here. PgSQL is popular now only because Oracle killed MySQL.

This is not to denigrate the work of the Postgres team -- it's easily one of the best-managed open-source projects and my admiration for them is immense -- but ~95% of the time, the tech stack is chosen as a fashion statement.

MySQL was fashionable until Oracle showed up. Postgres has been trucking along for decades and has essentially won by default. I don't know if there are many great marketing insights to take away, other than "stay out of the way of the $50B gorilla".

Re:PostgreSQL 13 by koffiezet on Friday September 25, 2020 @09:03AM (#24589240)
Absolutely don't agree. Sure, Mysql mostly became very popular with the rise of PHP in the early 2000's, but Postgres was popular in many less visible industries.

In my 'world', MySQL was seen as an amateur database engine, and never really lost that label. There was some serious hesitation towards Postgres back-then, but eventually it became a good alternative to the commercial RDBMS's. I know we eventually switched everything over. Until the mid 2000's - our server-side software was still sold as a 'blackbox', managed by us, with no mention of it running linux and postgres, but eventually these became more accepted technologies, and towards many of our clients even a selling point. The rise of Redhat in the enterpricy world helped a lot with that.

Re:PostgreSQL 13 by JohnBooty on Thursday September 24, 2020 @09:35PM (#24585716)
I agreed with you until the "fashion" thing.

Yes, Postgres adoption and development really picked up steam once Oracle acquired MySQL and quickly confirmed everybody's fears: they bought it primarily to ensure that MySQL stayed something of a "toy" database.

But the "fashion" thing is needless, insulting, and not even correct in my opinion. As far as mindless fashion trends in the industry go, traditional RDBMS's like Postgres are almost the antithesis of fashion.

Re:PostgreSQL 13 by cookiecaper on Monday September 28, 2020 @01:55PM (#24619412)
I should clarify. I'm not saying that Postgres is unworthy or that its adoption is fashion-driven -- it's part of the 5-10% that is driven by practicality. That is only the case now because, as you said, there is no fashionable conventional RDBMS anymore. It's all about transferring as much money as humanly possible to Jeff Bezos now. :)

The grandparent was asking what we can learn from Postgres in terms of driving adoption, as I understood it. And the reality is that most of Postgres's recent adoption uptick is because there really aren't many other options. It's not a good example of driving adoption in a crowded market, etc.

My apologies for being unclear before, and thanks for pointing it out so I can try to clarify.

Re:PostgreSQL 13 by ruairispain on Thursday September 24, 2020 @11:25AM (#24579470)
Personally, I think MySQL died when Oracle got involved with it. The momentum was sucked right out of the project.

The great think about Postgres is it has huge momentum and the team are delivering new features. Maybe Apache has less momentum because it's matured and doesn't need much momentum.

From a momentum point of view I see Postgres as the biggest FLOSS project. I see no point in choosing another RDBMS right now. Time to short ORA stock

Re:PostgreSQL 13 by pizza234 on Thursday September 24, 2020 @12:13PM (#24580143)
> Personally, I think MySQL died when Oracle got involved with it. The momentum was sucked right out of the project.

This statement is completely baseless, and you clearly haven't followed the MySQL developments of the last 10 years. It's ironic that around the time Sun was acquired, MyISAM was still a thing; MySQL has gone a long way since then.

Independently of a value judgement of "how good PGSQL/MySQL are" (I'm not making any implication in this sense), MySQL's development has been proceeding steadily, with no particular change (on the engineering side, at least). Oracle has actually put considerable effort both on the engineering and marketing side, of the 8.0 release (but again, I don't make any implication about the value of the products per se).

Re:PostgreSQL 13 by jpitz on Thursday September 24, 2020 @11:31AM (#24579557)
As much as I vastly prefer PostgreSQL, I will tell you that MySQL is much more preferred in enterprise settings, probably 8 to 1 in the environments I've seen.
Re:PostgreSQL 13 by dragonwriter on Thursday September 24, 2020 @12:31PM (#24580397)
> As much as I vastly prefer PostgreSQL, I will tell you that MySQL is much more preferred in enterprise settings, probably 8 to 1 in the environments I've seen.

That's largely because enterprises often have big investments in SQL server, OracleDB, and/or DB2, and are only using open source engines for more lightweight purposes, and/or as part of cloud transitions where they are just taking vendor default options or whatever options was supported when they came on or longest.

At least, that's my experience working in enterprise and being literally the single voice urging even considering pros and cons before using MySQL-by-default with no particular rationale in a transition effort (which resulted in us using Postgres.)

Re:PostgreSQL 13 by GordonS on Thursday September 24, 2020 @06:07PM (#24584238)
I work as a consultant in the enterprise space, and I've seen a shift in recent years towards Postgres.

I believe this has largely been driven by cloud - developers are now more able to choose the components they want to work with, instead of being told what to work with. There are of course plenty enterprises that are strict here, mandating cloud hosted SQL Server for example, but the general relaxation of constraints has struck me as a very pleasant surprise.

I've yet to come across MySQL being used in a cloud-based system, but I'm seeing Postgres more and more. When I do see MySQL, it's part of on-prem services that are considered "legacy".

Re:PostgreSQL 13 by jpitz on Thursday September 24, 2020 @07:38PM (#24584945)
One of my contacts is migrating databases between clouds, with MySQL being the most common, followed by sql server, then PostgreSQL.
Re:PostgreSQL 13 by brlewis on Thursday September 24, 2020 @11:43AM (#24579702)
That's likely a holdover from when mysql had a better replication story than postgresql. I think enterprises will come around.
Re:PostgreSQL 13 by throw0101a on Thursday September 24, 2020 @02:24PM (#24581785)
> That's likely a holdover from when mysql had a better replication story than postgresql.

What is the Galera-equivalent in the Postgres world? There's BDR, but the latest versions are closed source.

We use Galera in a bunch of places because it's fairly straight forward to get an HA cluster going, and with keepalived, we can point a front-end to a vIP that fails over automatically if one system goes sideways.

Re:PostgreSQL 13 by cookiecaper on Thursday September 24, 2020 @05:38PM (#24583991)
afaik, MySQL never had a better replication story than PostgreSQL, you've always had to use Percona's add-ons to get something workable. Even then (many years ago), we had multiple data loss incidents that were precipitated by widespread internal confusion over the bizarre intricacies of `binlog_format`.

Like most software that gains adoption, MySQL made some very broad claims about their software's capabilities and never really delivered on them, at least not in a way that would be considered production-ready by Serious Persons(TM).

That's the crux of MySQL v. PgSQL, Linux v. BSD, etc. Good engineers spend their time building good software and are generally too focused on that to spend much time going around and making outlandish promises. Postgres has benefited from Oracle's intentionally-bad stewardship over MySQL, but it doesn't usually work that way.

Re:PostgreSQL 13 by dpcx on Thursday September 24, 2020 @11:35AM (#24579597)
I've experienced this as well, and it's almost always because the tooling for MySQL is "better." I don't necessarily agree, but Sequel Ace (formerly Sequel Pro) is hard to beat.
Re:PostgreSQL 13 by Izkata on Thursday September 24, 2020 @11:54AM (#24579854)
> "better."

Inertia is what I've noticed as well. The few people I've convinced to actually try out postgres have ended up liking it more than mysql.

Re:PostgreSQL 13 by rguldener on Thursday September 24, 2020 @01:25PM (#24581052)
TablePlus is a great alternative for Postgres (and many other DBs) for that on Mac & Linux IMO: https://tableplus.com/
Re:PostgreSQL 13 by apazzolini on Thursday September 24, 2020 @01:09PM (#24580882)
Oh how I wish Sequel Pro/Ace worked on Postgres.
Re:PostgreSQL 13 by mustardo on Friday September 25, 2020 @08:53PM (#24595524)
The DB support in Jetbrains IDEA is awesome, they also have it as a stand alone IDE called Data Grip, could be woth a shot? https://www.jetbrains.com/datagrip/
Re:PostgreSQL 13 by Shorel on Thursday September 24, 2020 @02:36PM (#24581944)
May be in your small circle.

In AWS, Aurora MySQL is almost always the database of choice, for just about everything SQL.

Re:PostgreSQL 13 by cookiecaper on Thursday September 24, 2020 @05:30PM (#24583929)
Do people ever look at how much more they're paying for Aurora? I've saved companies thousands of dollars per month just by switching them to standard RDS instances for systems that see tiny load and were only on Aurora because that's "the super good AWS-optimized version, duh".

The added benefit of this is that you then know what software you're running, instead of Amazon's fly-by-night hackjob. We've hit several real-world bugs that were either incorrect backports of something that had been patched years before upstream, or wholly new issues caused by whatever they're doing to make Aurora auroar.

Re:PostgreSQL 13 by petergeoghegan on Thursday September 24, 2020 @02:47PM (#24582082)
Amazon themselves use Aurora PostgreSQL for Amazon retail fulfillment databases:

https://aws.amazon.com/solutions/case-studies/amazon-fulfill...

Re:PostgreSQL 13 by makoz on Thursday September 24, 2020 @03:06PM (#24582305)
Source?
Re:PostgreSQL 13 by tuatoru on Thursday September 24, 2020 @02:41PM (#24582005)
I first looked at MySQL back in 2000. At the time it couldn't even do row-level locking. I basically wrote it off at that point.

When I looked at Postgres, slightly earlier, I thought the same as you: how can anyone charge money for an RDBMS when this exists?

Re:PostgreSQL 13 by njharman on Thursday September 24, 2020 @12:23PM (#24580266)
ORA is a services/consulting company, now. The DB is important but company will easily survive its death (which will never happen) look at IBM big iron for similar story.
Re:PostgreSQL 13 by arthurcolle on Thursday September 24, 2020 @11:49AM (#24579783)
*ORCL
Re:PostgreSQL 13 by knocte on Friday September 25, 2020 @01:07AM (#24586748)
> after Linux, xBSD and the GNU stuff

Just after Linux IMO.

Re:PostgreSQL 13 by VWWHFSfQ on Thursday September 24, 2020 @11:12AM (#24579349)
I think MySQL is still significantly more widely deployed. And Postresql wouldn't be ahead of things like Firefox, apache, etc in terms of successful FLOSS. The world wouldn't really be that much different if postgres didn't exist. But it would be significantly worse if Firefox or apache never existed
Re:PostgreSQL 13 by spacemanmatt on Thursday September 24, 2020 @11:20AM (#24579431)
I think that is a rather naive take on the scope of PostgreSQL in "invisible" services you depend on regularly
Re:PostgreSQL 13 by VWWHFSfQ on Thursday September 24, 2020 @11:25AM (#24579472)
can be swapped out for another floss database pretty easy without any real impact. internet in general would be significantly worse right now if FF or apache never took off
Re:PostgreSQL 13 by pronik on Thursday September 24, 2020 @12:02PM (#24579974)
I wish you the best of luck replacing PostGIS with just about anything else.
Re:PostgreSQL 13 by jimktrains2 on Thursday September 24, 2020 @11:38AM (#24579639)
Spoken by someone who has never used any db specific functionality to facilitate simplifying logic and constraints.
Re:PostgreSQL 13 by spacemanmatt on Tuesday September 29, 2020 @12:17AM (#24624111)
[laugh react]
Re:PostgreSQL 13 by sosodev on Thursday September 24, 2020 @11:28AM (#24579510)
Measuring the importance of Postgres is tricky. I’ve noticed that a lot of new DB services use Postgres tech at some level. Cockroach DB is a good example and arguably a pretty important project.
Re:PostgreSQL 13 by manigandham on Thursday September 24, 2020 @12:22PM (#24580245)
What do you mean? CockroachDB doesn't use any Postgres tech other than reusing the same wire protocol.
Re:PostgreSQL 13 by sosodev on Thursday September 24, 2020 @12:32PM (#24580400)
But that is Postgres tech and it gave CockroachDB the ability to reuse existing client code for Postgres.

https://www.cockroachlabs.com/blog/why-postgres/

Re:PostgreSQL 13 by developer2 on Thursday September 24, 2020 @05:19PM (#24583804)
I don't know why this comment is downvoted so much. Every company I've worked for uses MySQL (or a MySQL fork, eg. MariaDB or Percona). I've never seen an installation of Postgres used in production at any company I've worked for. I've been waiting for a decade to start using it, but there have always been three issues that made Postgres an unacceptable choice for my coworkers and me, at every single company I've worked for:

1. The lack of built-in replication, requiring third-party plugins based on slow triggers, etc. Postgres finally added, after many years, off-the-shelf replication, resolving this concern.

2. No equivalent to MySQL's "utf8mb4_unicode_ci" or "utf8mb4_unicode_520_ci". Postgres finally added, after many years, the "und-x-icu" collation, resolving this concern.

3. Lack of a multi-threaded model, meaning every new client connection forks a new child process, utterly decimating performance and scaling. Expecting users to throw a connection pool/proxy server in front of every Postgres installation to work around that bottleneck is unacceptable; a proper multi-threaded model needs to be provided off-the-shelf. The developers continue to fight back on this issue, and the fact is they are wrong.

If and when that final point is resolved, I might finally be able to put together a proposal showing all of the benefits and features present in Postgres compared to MySQL; and with those above three points all resolved, I might actually be able to gain traction and support with the other senior developers, and win a plea to management.

Re:PostgreSQL 13 by johnthescott on Thursday October 01, 2020 @05:00PM (#24656105)
> I've never seen an installation of Postgres used in production at any company I've worked for.

many well known companies use postgresql in forward facing and internal systems. my company during 911 led the deployment of critical paging infrastructure run on postgresql.

https://postgresql.us/events/pgopen2015/schedule/speaker/74-...

Re:PostgreSQL 13 by zeroimpl on Thursday September 24, 2020 @07:02PM (#24584680)
Don’t your clients need their own connection pools anyways? Eg if the webserver isn’t pooling connections itself, that’s a problem regardless.
Re:PostgreSQL 13 by spacemanmatt on Thursday September 24, 2020 @07:35PM (#24584921)
Pooler-proxies exist because some applications do not or cannot pool connections. Sometimes these are applications we just don't have the code to.