I was trying to say something along the lines of "once you've played with rapleaf, you won't care what the stdev is", because rapleaf's data isn't good.
The standard deviation here isn't going to be the standard deviation of programmer salaries -- it will be the standard deviation of rapleafs estimates. Which as I previously mentioned are horrible, and not particularly interesting.
I agree with the author that Haskell is estimated at a lower rate because it's academic. I think the reason we see higher salaries for older languages here is that rapleaf must be using age as an input (with makes sense in some fields).
> The only sort of data you can trust is the data itself.
Agree. Again, here there is no data. It's gibbs sampled noise from shitty models.
> Providing the mean of data is providing a sum value a person thinks is useful to convey the shape of the data, and the mean in particular is unsuitable at conveying shape on its own.
I don't think anyone here is confused about the question of the mean showing the shape of data.
> The easiest assumption about that data is that it follows a normal distribution
Salary most certainly does not follow a normal distribution. I don't understand how you came to write what you wrote without knowing that. It's a classic example of the exponential distribution.
(To be fair, salary estimates from rapleaf do seem to follow a normal distribution :p maybe you can guess why? [hint, it's a phrase I used earlier that starts with "gibbs sampled noise from" and ends with "shitty models"])
Since they show the minimum and maximum we can judge the overall spread, the mean tells us...well...the mean and first and second quartile lets us judge how close data is clustered around the mean.
See http://gnuplot.sourceforge.net/demo/candlesticks.html for gnuplot examples.
It would make sense perhaps to use standard deviations and means for some data, but in cases like this I think quartiles and medians make more sense.
I suppose I am looking for mathematically faithful whilst still Omnigraffle good looks
Cheers
For example, here's a histogram of ages of passengers of the Titanic: https://www.statwing.com/demos/titanic#workspaces/21
Or rents in the bay area: https://www.statwing.com/open/datasets/034b4af16d50a0ad35f1a...
Gives a real nice feel for the data.
These histograms are lacking in that they don't actually have the mean line or std devs on the plot itself; my general point is that if the goal is to get across a distribution, a histogram is the best way to do it, perhaps aided by markers for mean, median, std dev, etc., as the case merits.
Disclosure: I work at Statwing.
They're typically used for confidence intervals or standard error, but you can theoretically use them with any measure of variability as long as you're clear.
What the author tries to get at is a precise quantity, namely the average salary for each language. However, we cannot measure that quantity directly, so taking samples (from a good source) is as good as it gets.
The easiest assumption about that data is that it follows a normal distribution, which does have a mean, but also a standard deviation; both of these are required to talk about any meaning of the result. (If the normal distribution is not a good fit things may get more complicated.)
What I'm saying is that providing just means is not a ballpark way of doing statistics, it is wrong. The statistical ballpark is the normal distribution, and maybe a value ensuring that it is at least somewhat appropriate to use that one.