News for hackers. Stuff that matters.
Re:Tuning Your DBMS Automatically with Machine Learning by tkyjonathan on Saturday June 03, 2017 @05:21PM (#14478567)
Technically there is a perl tool to make recommendations for mysql settings - although not with AI. As a DBA, I would probably use this, but 80% of the performance improvements come from indexing, fixing bad data models and archiving - especially with RDS where the options for performance optimization are limited.
Re:Tuning Your DBMS Automatically with Machine Learning by matt4077 on Sunday June 04, 2017 @03:07AM (#14480235)
They use both that script, as well as a performance expert for comparison. Their method is about as good as the DBA, and both beat the simple script by a large margin.
Re:Tuning Your DBMS Automatically with Machine Learning by stdgy on Saturday June 03, 2017 @06:12PM (#14478757)
It would be interesting to see this work expanded to include some of those factors; Namely indexing and archiving. Bad data models seem like a whole other beast, but indexing and archiving should be possible to generate good configuration suggestions for if you had a full history of DB usage trends along with what you were hoping to optimize for.
Re:Tuning Your DBMS Automatically with Machine Learning by gaius on Sunday June 04, 2017 @04:57AM (#14480457)
The most common thing I come across is devs saying the DB is slow but the load on the DB server is 0.1... Always profile before you tune!
Re:Tuning Your DBMS Automatically with Machine Learning by candiodari on Sunday June 04, 2017 @05:11AM (#14480486)
I think AWS, and all other cloudy provider's attitude is more along the lines of :

"Now you can trade not having a DBA, and letting your programmers not get criticized by him for paying us $$$($$$$$$$$$$$$$$). DBA's bad ! Bad ! BAD I tell you".

I fixed a customer's database a few months back. It was a cloud sql instance. They paid cloudy provider $40k for hosting a ~130 MB database (in ~ 8 months) before getting a consultant to look at it. I charged them $2k for an afternoon of hacking onsite and 2 days of followup, and they were ecstatic to pay it, saying that not doing that for 2 more weeks would have been more expensive. They're now paying the minimum charge for that and 5 other databases.

Most of the cloudy database offerings are ridiculously expensive for small, infrequently accessed databases (0.1 to 0.01 qps, let's be honest there's a boatload of these in every company), compared to either hosting it yourself or getting a small shared hosting or dedicated provider to do it.

I often wonder if a lot of the cloud scaling isn't being pushed by these companies for the same reason. Using a distributed publish-subscribe queue will seriously slow down anything that doesn't require, say ~5-10 machines, to execute.

Re:Tuning Your DBMS Automatically with Machine Learning by ris on Sunday June 04, 2017 @07:45AM (#14480894)
This is one of the reasons I get quite nervous when people start talking about these private-implementation, "cloud native" database services (Aurora, Redshift, Spanner...). As wonderful as they sound (and possibly are), not only are you binding yourself to a single vendor by adopting these, but there is nothing to keep the pricing of these services connected to reality.

Whereas as long as you e.g. "keep it postgres", you're able to experiment (at fairly low effort) with different database setups and find what is more efficient or cost effective for you. If Cloud Provider X's pricing model starts to deviate from what people are able to achieve with some custom database setup, people will start flocking to that - and, in fact, Cloud provider X would probably find themselves competing with an easy to use "managed database" service that does just that on their own Cloud Marketplace.

Not that this isn't an extremely interesting project as long as it's possible to wrestle the concept free from Amazon's infrastructure.

Re:Tuning Your DBMS Automatically with Machine Learning by gaius on Sunday June 04, 2017 @05:54AM (#14480595)
I am generally a fan of cloud, but it's important to understand what it replaces:

- Purchasing the kit and waiting for delivery. Probably a few weeks BUT that's a few weeks of no extra work on your part, just waiting. You still need to fill in the PO, get it signed off by your CTO, whatever

- Racking and powering. Except on the biggest jobs, 1-2 days for a DC team of physical labour.

- Installing an OS and software. These days, a few hours, PXE boot then let CFEngine or whatever do it

For everything else, you still need someone to either do it or take responsibility for it being done and verified and documented and reviewed to see if it's still the right solution and so on and so on. As I always say "if you don't have a DBA then YOU are the DBA".

Incidentally "you don't need a DBA anymore" was also the pitch of MongoDB et al... Careful devs, one day it will be your turn :-)

Re:Tuning Your DBMS Automatically with Machine Learning by j_s on Saturday June 03, 2017 @07:24PM (#14479049)
Any techniques, recommendations, and/or open source tools to simplify the archiving step you mentioned?
Re:Tuning Your DBMS Automatically with Machine Learning by seorphates on Saturday June 03, 2017 @08:33PM (#14479279)
That's one of those things where money can go a long way quickly - fat i/o (including network if you're off-loading, sharding, peering, vdbs etc.), good cpu, efficient log sizes. Sticking to the basics is probably the best - if you're running high transaction and data change rates then the collective throughput "off" of your transaction server(s) is every bit as important as the throughput "on" the server(s).
Re:Tuning Your DBMS Automatically with Machine Learning by danieltillett on Saturday June 03, 2017 @06:37PM (#14478861)
I assume you are referring to this project [1].

1. https://github.com/major/MySQLTuner-perl

Re:Tuning Your DBMS Automatically with Machine Learning by tyingq on Sunday June 04, 2017 @09:30AM (#14481476)
I love that project, but it seems like it could use a refresh. For example, it currently still complains if the query cache is turned off...but Mysql does that on purpose, and will remove it entirely soon. Last time I used it, it also seemed like it was still straddling suggestions between MyISAM and InnoDB. It seems safe enough at this point to have the tool strongly suggest converting MyISAM tables and focusing solely on newer storage backends.
Re:Tuning Your DBMS Automatically with Machine Learning by icelancer on Sunday June 04, 2017 @04:42AM (#14480435)
Yep. I use this regularly. I really like it.