With data like that, I'd imagine the only sort of data you could trust would be the mean, since I'm guessing its much harder to be wrong about relative income levels between large groups than it is to be right about actual income numbers.
What the author tries to get at is a precise quantity, namely the average salary for each language. However, we cannot measure that quantity directly, so taking samples (from a good source) is as good as it gets.
The easiest assumption about that data is that it follows a normal distribution, which does have a mean, but also a standard deviation; both of these are required to talk about any meaning of the result. (If the normal distribution is not a good fit things may get more complicated.)
What I'm saying is that providing just means is not a ballpark way of doing statistics, it is wrong. The statistical ballpark is the normal distribution, and maybe a value ensuring that it is at least somewhat appropriate to use that one.
I was trying to say something along the lines of "once you've played with rapleaf, you won't care what the stdev is", because rapleaf's data isn't good.
The standard deviation here isn't going to be the standard deviation of programmer salaries -- it will be the standard deviation of rapleafs estimates. Which as I previously mentioned are horrible, and not particularly interesting.
I agree with the author that Haskell is estimated at a lower rate because it's academic. I think the reason we see higher salaries for older languages here is that rapleaf must be using age as an input (with makes sense in some fields).
> The only sort of data you can trust is the data itself.
Agree. Again, here there is no data. It's gibbs sampled noise from shitty models.
> Providing the mean of data is providing a sum value a person thinks is useful to convey the shape of the data, and the mean in particular is unsuitable at conveying shape on its own.
I don't think anyone here is confused about the question of the mean showing the shape of data.
> The easiest assumption about that data is that it follows a normal distribution
Salary most certainly does not follow a normal distribution. I don't understand how you came to write what you wrote without knowing that. It's a classic example of the exponential distribution.
(To be fair, salary estimates from rapleaf do seem to follow a normal distribution :p maybe you can guess why? [hint, it's a phrase I used earlier that starts with "gibbs sampled noise from" and ends with "shitty models"])
Since they show the minimum and maximum we can judge the overall spread, the mean tells us...well...the mean and first and second quartile lets us judge how close data is clustered around the mean.
See http://gnuplot.sourceforge.net/demo/candlesticks.html for gnuplot examples.
It would make sense perhaps to use standard deviations and means for some data, but in cases like this I think quartiles and medians make more sense.
I suppose I am looking for mathematically faithful whilst still Omnigraffle good looks
Cheers
For example, here's a histogram of ages of passengers of the Titanic: https://www.statwing.com/demos/titanic#workspaces/21
Or rents in the bay area: https://www.statwing.com/open/datasets/034b4af16d50a0ad35f1a...
Gives a real nice feel for the data.
These histograms are lacking in that they don't actually have the mean line or std devs on the plot itself; my general point is that if the goal is to get across a distribution, a histogram is the best way to do it, perhaps aided by markers for mean, median, std dev, etc., as the case merits.
Disclosure: I work at Statwing.
They're typically used for confidence intervals or standard error, but you can theoretically use them with any measure of variability as long as you're clear.
When I make charts like this, I always let the X-axis be zero. I find it confusing to look at a chart and not immediately know the percentage difference without using a calculator. With the X-axis at 0 you can immediately tell the proportional differences.
It's like looking at the intra-day variations of a stock's price, and thinking "Whoa! What happened here?!". Then you zoom out and it's a completely straight line.
I don't want to imply maliciousness here, so I'm going to call this an example of unknowingly bad statistics (which, sadly, seems to be the default in most places). Bonus points for the bar (!) chart that starts at 90,000 and looks like ActionScript is twice as profitable as the rest of the languages.