harisund1990

Meko is the agent-native unified data infrastructure that enables multi-agent systems to learn together, building collective memory, shared knowledge, Conversations and Decision Traces that compounds across the entire system.

This is cool to see! Looks like a compete against pg_mooncake which Databricks acquired. But how is this different from pg_duckdb?

That's a very big difference which makes this hard to believe. I would expect them to be within 20% of each other. I think I know the trick they used. The 48 core Aurora is the worst, it performs just as well as the 32 core one. If they picked and other hardware the numbers would be realistic. Also pg_vaccum is turned off, which is interesting.

Also the article has this note > This is a sponsored report. Google chose the competitors and the test. GigaOm chose the most compatible configurations and ran the testing workloads. Choosing compatible configurations is subject to judgment. We have attempted to describe our decisions fully in this report.

Seems like a perfect use case for a distributed database like YugabyteDB. Have you looked into it?

YugabyteDB a distributed postgres database.

Think of it as a true drop in replacement for postgres that runs on multiple nodes. It internally does replication, sharding and leader election. Just add more nodes and you get to increase both read and write scale.

I personally am working on a few things like online major upgrades, async replication for DR, enhanced backup/restore/pitr/clone capabilities, and more recently supporting DocumentDB extension which provides a true Mongodb API.

Being a startup I also get to talk with large customers, help with marketing content, and participate in database conferences.

This is a fully open source MongoDB alternative that offers high availability and horizontal scalability. It's a joint collaboration between YugabyteDB, FerretDB and the Microsoft DocumentDB teams.

Anyone who has had to do a Postgres major upgrade can relate to this

Is Postgres becoming the new SQL standard? You have the ANSI SQL, but it does not cover a lot of stuff, and everyone has their own variances, and extra features. But, all new DBs like YugabyteDB and Aurora DSQL for ditributed, pg_duckdb and pg_mooncake for OLAP, FerretDB, and Azure DocumentDB for MongoDB are either built on top of Postgres, or are Postgres compatible!

I love headers but I wish you could split them in two so that private functions and variables can line in the c file. This would help reduce a lot of header bloat as well.

Cutis is an extension. That's the best you can get by being outside the core db. If you want true distributed architecture then you need to change the QO, DDL, transaction, even query stat components. At which point it ends up being a fork.

Yes, the merges are hard. But pg12 changed lots of fundamental things making it very challenging. Pg15 to pg17 should be much simpler.

> What we see in cases where someone takes Postgres and replaces the guts (Greenplum, Cloudberry, and of course YDB) is that it becomes a huge effort to keep up with new Postgres versions.

The first upgrade is the hardest, but after that we will have the framework in place to perform consecutive upgrades much sooner. When the pg11 to pg15 upgrade becomes available it will be in-place online without affecting the DMLs, no other pg fork offers this capability today.

Yugabyte does automatic sharding

It's easier to manage 1 database instead of 1000s

The article should be titled "Why Figma HAD TO reinvent the wheel with PostgresSQL". When you have a legacy system and not enough time, or will to move off of it the only option is to get inventive and build with what you have.

There is always a price. In this case the database team did something quick, cheap and easily. But the Application teams now have to deal with handling all the nuaces of the system. Maybe Figma has more people in these Apps teams with time on their hands to handle it.

Sharding is the easy part. Eventually you need to implement distributed transactions, taking a consistent backup across shards, PITR, resharding, load balancing, and the list goes on... That takes exponentially more number of people and time and mainly risk.

It works for Figma(for now), but for it to work as a solution for other companies with different hardware, data schema and access patterns will add even more complexity to the mix.

It's a excellent solution but I don't think it be good enough in the long run.

1% of Visa maybe?

No one needs 10,000 machines. They likely have scale issues and cannot scale their database quick enough so they planned for the worst spike possible which is likely for a hour on cyber Monday.