Yes but surely alexa knows how many alexa trackees it has. It knows how much it's scaling that up to make an estimate of the real population.

For example, a very big site, you have enough trackees to make a reasonably statistically valid sample. But smaller sites, you don't see as many trackees, so you set the error bars higher.

Sure, I agree that the system is flawed anyway, seeing as how I don't know a single person who installs the alexa toolbar, and amongst techie sites it's usage is probably 0.

A more useful depiction of the error would be to somehow calculate the bias in the Alexa population (my guess, tech investors are the only segment of the population using the Alexa toolbar, which is incredibly ironic).

Huge error bars on a near zero estimate don't mean a whole lot except that the experiment producing the data is flawed.

The biggest niche population of Alexa toolbar users is probably internet marketers.

Also incredibly ironic. These people are just seeing what they want to see and creating buzz within their own community rather than measuring something useful.

Assuming "calculating the bias" is a well posed question (and I'm not convinced it is) then measuring that bias is a noble goal for alexa engineers. Surely this is a multi-phd-thesis kind of problem though :)

Ok, I'm now being overly-pedantic :)

I don't think alexa "scales up." If they do, then, you're right, there is uncertainty. If they don't, I don't think there's a concept of an error bar here. They track how many people go to what sites. Aside from a few dropped packets here or there, there is no uncertainty in that measurement. Therefor, there is no way to measure that sort of error bar. (Right?)