News for hackers. Stuff that matters.
Re:How to Run a Database on AWS with Better Performance and ... by rubiquity on Tuesday November 24, 2020 @10:58PM (#25205905)
Disclosure (I used the right word!): I work at AWS. Opinions are my own.

This is a nice article with some fun comparisons and important details around the specifics of operating TiDB and systems like it on AWS. If I were running a replicated database such as TiDB, hypothetically of course, I'd probably opt for an instance type with locally attached storage. EBS volumes are already replicated themselves for durability so if you're running a replicated DB on EBS volumes then you're going to be replicating while you're replicating. Yo dawg. That's going to be some extra latency, even with a majority quorum replication protocol like Raft.

Historically if you wanted locally attached SSDs the more expensive storage instance types like i2/i3/i3en were all you had but now there's m6gd, c6gd, and r6gd[0]. Lots of options at all sorts of price points for your workload. The Graviton 2 based m6gd instances with local NVMe SSDs are more cost effective than the m5 instances that the blog post is using, which do not have a local SSD.

As the post calls out, if you are going to use EBS then use it for the materialized key-value store portion. Maybe even make a little Step Function to snapshot the EBS volumes to figure out where you can safely trim the Raft log at. It could be nice.

0 - https://aws.amazon.com/about-aws/whats-new/2020/07/announcin...

Re:How to Run a Database on AWS with Better Performance and ... by sudeepj on Wednesday November 25, 2020 @07:04AM (#25208345)
> EBS volumes are already replicated themselves for durability

That is only within the same AZ right? Only snapshots are cross-AZ replicated?

PS: why, oh why ppl use this hard-to-read gray font color? Is there allergy to the black font color on white background?

Re:How to Run a Database on AWS with Better Performance and ... by rubiquity on Wednesday November 25, 2020 @08:44AM (#25209131)
That’s right. But AZs can be big and network paths can vary. Always monitor.
Re:How to Run a Database on AWS with Better Performance and ... by CharlesW on Tuesday November 24, 2020 @10:37PM (#25205800)
A less clickbait-y title would be, "How to Run Our Database on AWS at a Reasonable Cost".

"Reasonable Cost" comes from the conclusion, and I omitted "Better Performance" since they never compare to alternatives (although they do mention Aurora).

Re:How to Run a Database on AWS with Better Performance and ... by gkop on Tuesday November 24, 2020 @10:57PM (#25205903)
For what it’s worth, there’s a good amount of content in the article that I found interesting for my purposes operating Postgres on RDS.
Re:How to Run a Database on AWS with Better Performance and ... by CharlesW on Thursday November 26, 2020 @01:53PM (#25222637)
Interesting. What changes are you making to future Amazon RDS for PostgreSQL instances as a result?
Re:How to Run a Database on AWS with Better Performance and ... by gkop on Thursday December 03, 2020 @12:49PM (#25291937)
I bookmarked the article and will consult it next time I’m reviewing instance types for a cluster.
Re:How to Run a Database on AWS with Better Performance and ... by tabokie on Wednesday November 25, 2020 @04:07AM (#25207290)
Sorry if the title sounds misleading to you. We are simply trying to give our customers (or potentially other cloud database users) a solution that has best performance with limited resources. Competitive analysis is not the focus of this post.
Re:How to Run a Database on AWS with Better Performance and ... by CharlesW on Thursday November 26, 2020 @01:48PM (#25222594)
(This advice is offered in the spirit of improving your content marketing efforts.)

I understand the desire to "reel in" more readers than a post might otherwise warrant, but in my experience you're better off not promising general advice if you're not providing it.

For example, you might say, "How we configure TiDB on AWS to handle XXX transactions per second for XXX/month", or "Squeezing the highest TiDB performance from AWS m5 instances". There are definitely interesting ways to title posts in a way that's supported by the content.

Re:How to Run a Database on AWS with Better Performance and ... by jfim on Tuesday November 24, 2020 @11:03PM (#25205933)
The part that's not mentioned is that there's no real latency guarantee on these IOPS. If you're running a database that has to do a few data dependent reads (eg. walking a B tree), then these can add up quickly if latency spikes to dozens of milliseconds (eg. this datadog blog post [0]).

[0] https://www.datadoghq.com/blog/aws-ebs-latency-and-iops-the-...

Re:How to Run a Database on AWS with Better Performance and ... by tabokie on Wednesday November 25, 2020 @12:46AM (#25206327)
Author here, thanks for the pointer and I actually did read about it before. You are right that cloud vendors don't really offer those guarantees. But from my experience benchmarking on EBS, under normal pressure they are good enough for latency sensitive tasks. I can even say that stability-wise AWS EBS is far better than equivalents from GCP or Azure. Probably they have being improving things since this post came out in 2013? I'm not sure.
Re:How to Run a Database on AWS with Better Performance and ... by jtdev on Tuesday November 24, 2020 @11:22PM (#25206018)
I’m not sure how this could be cheaper than RDS...? Nor do I see how the complexity of this solution could be justified except in the most performance sensitive situations.
Re:How to Run a Database on AWS with Better Performance and ... by gregwebs on Wednesday November 25, 2020 @06:42PM (#25215551)
TiDB is for larger scale data and thus a better comparison would probably be with AWS Aurora. We do see customers saving money by taking on the complexity of operating a database themselves rather than using the AWS offering.
Re:How to Run a Database on AWS with Better Performance and ... by pachico on Tuesday November 24, 2020 @10:59PM (#25205916)
I'm quite surprised, by looking at the post date, they didn't use the new Graviton2 instances. I'm afraid this post is born already obsolete on this matter.