News for hackers. Stuff that matters.
Re:Programmers Need To Learn Statistics Or I Will Kill Them All by Retric on Saturday January 09, 2010 @04:45PM (#1042023)
Taking more than 100ms is not the same things as "time-out and die". Performance is focused on how long something takes and it normally assumes that the system is not broken. It would take a horribly broken system to timeout 1% of requests. Assuming that 1% of your requests taking long enough the user might notice, but still working is generally acceptable performance for a webapp. So a reasonable goal might be 99% within 150ms and 99.95% don't fail.
Re:Programmers Need To Learn Statistics Or I Will Kill Them All by viraptor on Saturday January 09, 2010 @04:56PM (#1042036)
smanek wrote: "I don't care what your mean or standard deviation are.", which if taken seriously, means exactly that 1/1000th request can take an hour to finish and he doesn't care. Anything over 10s in real-life means a time-out in connection, or user refreshing, or going to the next page, or...

Going to 99.95% not-failing connections doesn't help you much IMHO. It means that you've gone from every 16'th person having a problem to every 32'th person. It's not acceptable at all.

Re:Programmers Need To Learn Statistics Or I Will Kill Them All by Retric on Saturday January 09, 2010 @05:49PM (#1042117)
Your also assuming that he / we measure per element not per page, which is just silly. Also, stating a minimum accepted value is not the same as ignoring the rest. It just places minimum reasonable performance that a context.
Re:Programmers Need To Learn Statistics Or I Will Kill Them All by viraptor on Saturday January 09, 2010 @06:24PM (#1042170)
I read his "Just make sure that no more than 1% [...] of requests" as actual requests. If it meant whole-page-load-time then fair enough.

"stating a minimum accepted value is not the same as ignoring the rest" - then what is it if you don't care about mean and stddev? If you do care about other requests you care about the mean (or stddev, or median, or whatever other parameter) but just don't put it in a scientific way. If you say reasonable, it might mean - it doesn't time-out. That means it's below X seconds. That means with the other maximum load time constraints, you expect a mean time for Y requests to be less than Z. Saying "Assuming that 1% of your requests taking long enough the user might notice" you just put a constraint on that parameter.

(just to explain why I keep arguing this point - because that's what the post is about (kind-of), people say "reasonable time", but don't want to set the actual mean/stddev - which is what they do care about)

Re:Programmers Need To Learn Statistics Or I Will Kill Them All by Retric on Saturday January 09, 2010 @07:02PM (#1042238)
No, because taking less than the threshold is effectively meaningless. So, when you calculate the mean and standard deviation the calculation is altered by meaningless trivia. When you add in the latency and rendering times 1ms and 30ms might as well be the same number. What you really want is to round everything under your thresholds up to that threshold and then look at the breakout as x% take 100ms or less, y% take 1 second or less, and z% take less than 3 seconds etc.

You then look at that breakout based on what the page is, or the time of day ect. But, from a performance standpoint the the only really important number is the first number, once enough of your site is fast enough having a few slow areas, or some issues at peak times is reasonable, and once it's slow enough for a user to notice it's a problem.