News for hackers. Stuff that matters.

mthomassen

Re:Ask HN: Do you self-host your database? on Tuesday June 29, 2021 @06:52AM (#27674126)
Not parent, but at {work} we made a similar move, from Azure managed postgres to self-managed postgres.

I elaborated a litle on why here: https://news.ycombinator.com/item?id=24607664

but boils down to; - enormous performance difference - advanced/fine grained user roles (+default permissions, not settable without superuser) - use of extensions (productivity boosts for us come from; Oracle FDW, TDS FDS, TimescaleDB) - private network endpoints - lower cost

Just to provide a counter statement; at $work we definatively do not have the appropriate headcount to even start considering running our own distrubuted datastores. However, we _do_ use Cassandra. And while many persons shared the sentiment of 'it being to hard to maintain' when the first few projects started to incorporate Cassandra, the truth is; there never has been any failure or incident, or even technical hurdle related to the usage/maintenance of the -admittedly smallish- Cassandra cluster (for 2 years now).

The costs of operating the cluster are ~5K/month (that's what our service provider charges us for 24/7 ops). I consider this a scam since averaged maintenance costs are perhaps ~1 hour per month.

(normally peaks at 40.000 writes/sec, 500 reads/sec, 200GB)

Re:Mastering PostgreSQL Administration [pdf] on Sunday September 27, 2020 @03:11PM (#24609324)
I assume you're using ZFS filesystem for the transparant compression, what's your opninion/experience on using ZFS on cloud storage? I mean; the EBS disks are already redundantly stored by AWS and the COW mechanism could lead to a lot of write amplification; negatively impacting the network attached storage?

(I don't use EBS in my day job, but Azure's disk offering don't really offer adequate perforamance when used with any filesystem other then EXT4 in my experience)

Re:Mastering PostgreSQL Administration [pdf] on Sunday September 27, 2020 @02:59PM (#24609234)
We ran a Postgres DBaaS offering of Azure for 2 years before we switched over to a self-managed (IaaS) Postgres installation. Couldn't be happier.

The primary reason was the desire to use extensions; I'm now able to use Foreign Data Wrappers with great effect to turn the Postgres databases in an enterprise data integration platform/data warehouse.

Additional benefits include a staggering 50x performance increase* , a much finer grained user authorization strategy (by being able to (automatically) alter grants and alter default privileges of other users with superuser permissions). Also, my business users can now leverage NFS or sFTP shares to load very large datasets direct from the server and we can actually shut down unused environments outside of business hours (Azure's DBaaS can be scaled down, but not shut down).

The costs of this setup is actually ~20% lower compared to Azure's managed offering (for same core count, could be even lower if you add the enormous performance gains to the comparison). This included cost of developers/operators! (we're finding the operational effort is more or less equal to a managed offering; almost non-existant). The only thing that really caused some discomfort in the beginning was our setup of pgBouncer.

Off course, There're some caveats; we run an OLAP workload, not business critical and therefor haven't really put a lot of effort into a HA setup. Still, the advantages of an "real" Postgres server cannot be overstated! To be able to use extensions alone has brought the time to market of new features from weeks to days.

* Geometric mean of TPC-DS benchmark suite. On Azure's Ls_v2 machines, leveraging the NVMe disk with bcache. Compared to a tuned Azure's "Postres as a Service" with equal core count.