News for hackers. Stuff that matters.
Re:Fighting The NoSQL Mindset by steveklabnik on Wednesday March 24, 2010 @05:52PM (#1217000)
I have a feeling that "I made up a random database as big as Digg's, and look, I'm getting 1000x the speed!!!" would kind of tend to imply that you're not accurately replicating their problem.
Re:Fighting The NoSQL Mindset by madair on Thursday March 25, 2010 @01:11AM (#1217790)
Perhaps you missed the point of the whole exercise: To respond to a high-profile yet poor representation of RDBMS performance.

The OP repeated this reasoning more than once.

He showed that the facts cited by Digg do not make sense unless we take into account poor database technology, poor database configuration, or poor database skills, or all three.

Perhaps you disagree with that. However, that's not what you said above. The OP also discussed the nature of this micro-benchmark, and it's relevance despite his own poor knowledge of the actual data characteristics.

So in other words, he has already directly addressed your concern in advance, more than once on that too as a matter of fact. Considering that fact, you haven't actually responded to his article, you just wrote a "tends to" point about micro-benchmarks, I think it's pretty clear that Dennis Forbes knows a thing or two about benchmarks.

Blah blah blah to empty air, this comment page is pretty much a fact-free and nuance-free flame war anyway, so what's the point, sorta embarrassing for the esteemed HN crowd.

Re:Fighting The NoSQL Mindset by regularfry on Thursday March 25, 2010 @04:47AM (#1217986)
It would have helped the author's case to try (at least approximately) replicating Digg's performance problem on a local MySQL install before switching technologies - that would have given some indication that he'd made the right assumptions about the problem and wasn't testing something unrelated. No matter how good the author is (and I've got no reason to suspect that he's anything other than a top-rate engineer), it's awfully easy to be led down a blind alley on this sort of problem.
Re:Fighting The NoSQL Mindset by steveklabnik on Thursday March 25, 2010 @08:36AM (#1218306)
> you just wrote a "tends to" point about micro-benchmarks

I am not a database guy. I also don't know enough about Digg's set up to say with authority if these comments make sense. So I specifically wrote "I feel," "tends to," and "imply" because I'm not comfortable making an absolute statement about the issue.

However... I don't see how this test is in any way relevant. a 30GB database? Running on totally different hardware?

In any case, re-reading the article again, I see that relevance paragraph now. I guess I missed it the first time around between all of the flaming, trollish comments about both NoSQL and MySQL. But I still don't see how we can extrapolate this test in any way to imply anything about Digg's practices at all. Then again, it's 8:30am.

Re:Fighting The NoSQL Mindset by hp1995acer on Thursday March 25, 2010 @12:41PM (#1218784)
How does one "miss a paragraph"?

What a unclever excuse for missing the point.

Re:Fighting The NoSQL Mindset by steveklabnik on Thursday March 25, 2010 @02:59PM (#1219171)
Because it was 8:30am when I read this, and I usually wake up at 10.

It still doesn't change my original point, however. Just because he acknowledges that the benchmark is unrelated to what he's talking about doesn't excuse him from the fact that it's unrelated to what he's talking about.

It also doesn't change the fact that the article is still a troll, regardless of the correctness of his benchmark.

Re:Fighting The NoSQL Mindset by TimothyFitz on Wednesday March 24, 2010 @06:23PM (#1217074)
Agreed. Reading the digg article I'm interpreting their perf claim to mean "on a loaded server in our cluster, if we try to do this query on a cold cache it takes 1.5 seconds to complete", because this is exactly how I would measure the performance.

When I test the performance of a query, I query a live mysql node (or at least a live replicated standby) of some data that's not actively being used (to give a realistic "cold-cache" scenario, even though the caches aren't necessarily cold).

If digg used this method, it would completely account for performance discrepancy. Digg did not release a benchmark, and trying to treat their findings as a repeatable benchmark is wrong.

Re:Fighting The NoSQL Mindset by gill_bates on Wednesday March 24, 2010 @11:10PM (#1217599)
>Digg did not release a benchmark, and trying to treat their findings as a repeatable benchmark is wrong.

Yet they released record counts, schema, and then performance numbers, and then used their results to demonstrate the failure of the RDBMS (which they led into by saying that it, as some given philosophy, optimizes writes at the cost of reads, hence their poor read performance).

Many of the comments in here are baffling. Digg specifically used the hammer of NoSQL to pound the nail of their database needs, replacing MySQL. They've made a big deal about this. So why the noise about "they're different, man?" And now the petty whines about benchmark methodology when Digg made concrete claims about RDBMS systems?

Re:Fighting The NoSQL Mindset by nostrademons on Thursday March 25, 2010 @04:49AM (#1217988)
Because many of us have had the experience of going into a new project where we thought all the existing engineers were idiots, miraculously being given enough latitude to do things our way, and then finding that there were good reasons for those idiotic decisions, and our proposed replacement completely falls down under real-world conditions.

Eventually, you learn not to criticize projects that you aren't actively in the trenches with. There's almost always some subtlety you're missing, and the existing team is too busy fixing it to correct your misconception. I bet the Digg team is looking at these comments (well, if they have time) and thinking to themselves, "We tried that a year ago, and it didn't work. If only they knew..."

Re:Fighting The NoSQL Mindset by jpcx01 on Thursday March 25, 2010 @03:05AM (#1217904)
I think the parent author's main point is they used a defective RDBMS as a representative example of all.

MySQL does not do certain things well. However many mature RDBMS's have solved these problems and its worth pointing that out before you drop the entire class of tools for the next shiny tech.

Re:Fighting The NoSQL Mindset by gill_bates on Wednesday March 24, 2010 @11:05PM (#1217583)
Digg couldn't do a rudimentary join. I'm sorry but if you can read their original entry and not picture an almost comical scene, that you are out of your league.

Now the Joe Plump or whatever guy is telling people that you should sort in PHP. That is the level of expertise of Digg.

Re:Fighting The NoSQL Mindset by steveklabnik on Wednesday March 24, 2010 @11:34PM (#1217635)
I didn't read their original entry. I read an article that was basically "Hurf durf, Digg is retarded, I can do better at home. NoSQL is stupid."

I actually flagged the parent article. It's a troll, and not worth anyone's time. RDBMS is good sometimes. NoSQL is good sometimes. Do we really need _another_ holy war?

Re:Fighting The NoSQL Mindset by houseabsolute on Thursday March 25, 2010 @12:20AM (#1217709)
You seriously flagged the parent article? That's out of line. While it may be wrong (I personally don't think it is) it's chock full of interesting analysis. It's exactly the kind of article that belongs on this site. Please think before you flag.
Re:Fighting The NoSQL Mindset by Confusion on Thursday March 25, 2010 @02:42AM (#1217882)
As tristan_juricek commented elsewhere:

Uh, using language like "bottom-feeder RDBMS" and saying that an engineering team is using it "horribly incorrectly on clearly comically deficient hardware" makes this article highly biased, and trollish. It's pretty clear that this guy things Digg is a bunch of idiot engineers - I mean, why else allude to "rudimentary comp. sci. knowledge"? These articles might have a central point, but when it's surrounded by a bunch of opinion, it like watching TV news (e.g., Fox News). The real information gets drowned out.

That's a pretty good reason to flag a story. The entire argument is based on the premise that others are idiots: hubris of which we are often guilty. After you've been in the trenches for a while, you should realize that there is usually a pretty reasonable explanation for seemingly stupid problems.

Re:Fighting The NoSQL Mindset by steveklabnik on Thursday March 25, 2010 @08:45AM (#1218324)
I _very_ rarely flag articles. But look at this thread. Has it generated good discussion? There's a lot of hot air with no backing up on both sides of yet another holy war that I'm already sick of hearing about.

And a few choice statements from the article:

> I would say Digg's case is an example of a bottom-feeder RDBMS product (apologies for being incendiary, but why does the problem always come down to MySQL? These examples always end up being "we moved from MySQL to NoSQL" rather than "We moved from Sybase ASE to NoSQL"), used arguably suboptimally on unpowered hardware,

> went contrary to the demonstration that even a mediocre machine can beat their results.

> Nonetheless, it is a warning sign of a foundational product issue.

> Decent database products like SQL Server even allow you to include

> So either MySQL is an atrociously bad product at the larger limits, which ample evidence seems to point as a truism,

> Please get away from the compiler and save the world from your monstrosities until you have some knowledge of these basic concepts.

> Alternately you can just clutch onto NoSQL and bleat about how it changes all of the rules anyways, which is the route quite a few have decided to pursue

Okay, I'm done. Point is, dude is straight up trolling about how MySQL sucks. This article does nothing but fuel the fire of yet another flamewar, and so it gets flagged. I'd like discussions to remain sane around here.

Re:Fighting The NoSQL Mindset by houseabsolute on Thursday March 25, 2010 @02:36PM (#1219094)
Looks to me like the problem with this discussion is the dismissiveness and defensiveness of people on Hacker News, not the article.

Maybe MySQL really isn't a decent database product, or not for high performance needs. How many of his statements are troll-ish in that light?

Re:Fighting The NoSQL Mindset by steveklabnik on Thursday March 25, 2010 @03:01PM (#1219183)
They all are. Trolling isn't about what one says, it's about the intention behind the things that are said. Completely factual statements are still trolling if said with the intention to cause disruption.
Re:Fighting The NoSQL Mindset by houseabsolute on Thursday March 25, 2010 @05:22PM (#1219537)
Disruption is on the back burner, but it looks like his primary intention is to inform, so I don't see the problem.
Re:Fighting The NoSQL Mindset by steveklabnik on Thursday March 25, 2010 @08:00PM (#1219882)
I would disagree. His primary intention is to talk shit on MySQL.
Re:Fighting The NoSQL Mindset by gill_bates on Wednesday March 24, 2010 @11:37PM (#1217640)
And you think this article started the holy war? I believe I just saw another article on his about how you should do SQL sorting in PHP until you graduate to MySQL, or so claims Digg's former architect.
Re:Fighting The NoSQL Mindset by ubernostrum on Thursday March 25, 2010 @01:59AM (#1217844)
From what I read of the Digg article, it seemed they were hitting situations where doing some work in PHP rather than the DB was the most efficient solution. I don't find anything unreasonable about that; I occasionally run into situations where it's just not possible to get the performance needed while only using the DB, as does probably anyone who's worked enough with databases.

The specific case -- IIRC -- Digg mentioned was a query which required MySQL to generate a temporary table too large to sit in memory, so it ended up being done on disk. Moving some of the processing out of the DB query and into PHP avoided that, understandably resulting in a huge performance difference (in-memory versus on-disk has a way of doing that...).

Re:Fighting The NoSQL Mindset by steveklabnik on Thursday March 25, 2010 @08:47AM (#1218326)
It certainly didn't start it, but I don't want to see more vapid articles from DBAs about how NoSQL sucks. I want to see actual articles about the real strengths and weaknesses of both technologies. This article just fuels the flames.
Re:Fighting The NoSQL Mindset by ars on Wednesday March 24, 2010 @08:08PM (#1217283)
Or it implies that digg did a bad job.
Re:Fighting The NoSQL Mindset by spamizbad on Wednesday March 24, 2010 @08:51PM (#1217374)
You'd have a pretty low opinion of digg engineers to think they'd optimize that poorly. Which the article author does.

It's actually pretty depressing to see content like this from an alleged Database expert. I've never been able to get 1:1 results between "lab" tests and actual deployments, and I've gone to much greater lengths than the author to simulate workloads.

Re:Fighting The NoSQL Mindset by WorkerBee on Thursday March 25, 2010 @08:52AM (#1218337)
He wasn't aiming for 1:1, he was trying for a ballpark, and he got 1000:1 improvement on modest hardware. On a fricking laptop.

That's all in the article.

Re:Fighting The NoSQL Mindset by gill_bates on Wednesday March 24, 2010 @11:24PM (#1217627)
Did we read the same article? The one I read made ample disclaimers about it being nothing even remotely approximating a serious benchmark.
Re:Fighting The NoSQL Mindset by ahk on Thursday March 25, 2010 @02:46AM (#1217885)
And still uses those same worthless benchmarks to try to prove the point of sqldbs ability to scale.