News for hackers. Stuff that matters.
Re:tiDB – Scalable RDBMS Inspired by Google F1 with Support ... by endymi0n on Monday September 07, 2015 @06:24AM (#10180858)
Seems like AWS Aurora just got an open source competitor.

I'm just a little sad that it's all MySQL again, now that we're betting on PostgreSQL and don't really look back.

On the performance side I'm just a litte worried however. While Go is an excellent choice for performant and concurrent app-level code, I'm not too bullish on the language on the database level. InfluxDB sucked hard when we put it under some more load, but let's see what comes out there.

Re:tiDB – Scalable RDBMS Inspired by Google F1 with Support ... by ngaut on Monday September 07, 2015 @06:26AM (#10180864)
PostgreSQL support is on our roadmap.https://github.com/pingcap/tidb/blob/master/ROADMAP.md
Re:tiDB – Scalable RDBMS Inspired by Google F1 with Support ... by sagichmal on Monday September 07, 2015 @09:15AM (#10181303)
InfluxDB's performance problems have very little to do with the implementation language, quite a lot to do with fundamental architectural and team constraints.
Re:tiDB – Scalable RDBMS Inspired by Google F1 with Support ... by ericb on Monday September 07, 2015 @10:12AM (#10181473)
Can you detail, or point me to a blog post which details this stuff? As someone who is about to invest a lot of time in researching use of influxdb at scale, I'm interested in doc of the performance problems, and even moreso if the performance is troubling in the long term due to the architecture or team. Anything you can point me to will save me a lot of time.
Re:tiDB – Scalable RDBMS Inspired by Google F1 with Support ... by c4pt0r on Monday September 07, 2015 @06:47AM (#10180913)
We choose MySQL protocol first because it's widely used and we are more similar with the tool chain and protocol in comparison with PostgresSQL. About the performance, Golang is design for building distributed system in Google, and the development productivity is perfect, InfluxDB's performance problem is partly caused by other reason.
Re:tiDB – Scalable RDBMS Inspired by Google F1 with Support ... by creshal on Monday September 07, 2015 @08:28AM (#10181171)
> I'm just a little sad that it's all MySQL again, now that we're betting on PostgreSQL and don't really look back.

Doesn't Postgres have its own multi-master solution in the works with BDR?

Re:tiDB – Scalable RDBMS Inspired by Google F1 with Support ... by ddorian43 on Monday September 07, 2015 @09:42AM (#10181382)
bdr is async replication and doesn't support sharding
Re:tiDB – Scalable RDBMS Inspired by Google F1 with Support ... by ddorian43 on Monday September 07, 2015 @09:41AM (#10181378)
aws-aurora doesn't support sharding/scaling
Re:tiDB – Scalable RDBMS Inspired by Google F1 with Support ... by acdha on Monday September 07, 2015 @10:36AM (#10181565)
Can you expand on this a bit with some actual technical details about how you think it doesn't scale enough?
Re:tiDB – Scalable RDBMS Inspired by Google F1 with Support ... by ddorian43 on Monday September 07, 2015 @10:40AM (#10181575)
aws-aurora is a fancy storage engine + fancy replication

You can't divide the data on multiple servers (sharding)

It will come a time, when you'll exhaust the biggest machine they have available, and then you're stuck

Re:tiDB – Scalable RDBMS Inspired by Google F1 with Support ... by acdha on Monday September 07, 2015 @10:46AM (#10181591)
Wouldn't the classic sharding approach at that point simply be to use a second instance? I mean, it'd be great if they handled that for you but I'd tend to assume most apps would have hit the wall on the approach of shoving everything into a single database long before maxing out a 32-core/244GB system with 64TB on SSD.
Re:tiDB – Scalable RDBMS Inspired by Google F1 with Support ... by ddorian43 on Monday September 07, 2015 @01:21PM (#10182196)
[meme]You can't simply use a second machine/instance.

While it's true that most companies will be fine in 256GB ram, we were talking about sharding, which it doesn't have.

Re:tiDB – Scalable RDBMS Inspired by Google F1 with Support ... by acdha on Monday September 07, 2015 @02:48PM (#10182529)
> [meme]You can't simply use a second machine/instance.

I'm assuming [meme] is shorthand “overly-broad assertion”? It's nice if your database has horizontal scaling built in but it's not like we don't have an entire generation of successful companies who had application-level sharding logic either by necessity or because they found the control it offered was valuable compared to the built-in generic logic.

> While it's true that most companies will be fine in 256GB ram, we were talking about sharding, which it doesn't have.

You still haven't supported the assertion that it's common for places to have massive, heavily-queried databases like this which would not be better split following natural application-level boundaries. This is particularly relevant when discussing AWS as some of the common reasons for keeping everything in one database are sweet-spots for other services (e.g. migrating data for long-term historical reporting over to RedShift).

Again, I'm not questioning that integrated sharding would have its uses – only your sweeping assertion that this is a likely problem for most people and that it's a dead-end (“you're stuck”) rather than merely one of many growing pains which you'll deal with on a successful product. In particular, it's unlikely that everyone will have the same right answer at that scale since access patterns vary widely.

Re:tiDB – Scalable RDBMS Inspired by Google F1 with Support ... by felixgallo on Monday September 07, 2015 @01:03PM (#10182136)
Wrong, aurora supports read replicas, and of course client side sharding.
Re:tiDB – Scalable RDBMS Inspired by Google F1 with Support ... by ddorian43 on Monday September 07, 2015 @01:19PM (#10182184)
Every database supports client side sharding.