I think why this happens is this: if you're on a trading desk of 10 people that makes $10m, it's easy to quantify your contribution and use that as leverage to get a big bonus. If you're on one of 10 product teams that each have 10 people and in total generate $100m in revenue, it's hard to quantify your specific contribution and use it to negotiate a large bonus, even though you're bringing in the same revenue on an averaged basis.

I've always wondered why compensation in software engineering is lower compared to revenues than similar professions. A lot of the things that are either facially true or commonly held beliefs: software has low capital costs, good programmers are much more productive than bad ones, a lot of revenue can be brought in with a relatively small team, are the things that make banking, law, consulting, and even accounting more relatively lucrative for workers as opposed to shareholders. But I think there is this holdover perception of treating software engineers like other engineers, even though the dynamics of those fields is very different (massive teams coordinating to build pyramids with huge capital costs).

Commodity programmers have externally limited individual leverage (aforementioned 'teams building pyramids' strategy [1]) and systematically limited measurement (because their measurable output is blended with that of the supporting organization). On top of all that, any commoditized worker is going to have extremely limited leverage in salary negotiations due to the ease of replacing them.

pg's "How to Make Wealth" [2] does a great job of explaining the interplay between leverage, measurement, and wealth:

To get rich you need to get yourself in a situation with two things, measurement and leverage. You need to be in a position where your performance can be measured, or there is no way to get paid more by doing more. And you have to have leverage, in the sense that the decisions you make have a big effect.

Measurement alone is not enough. An example of a job with measurement but not leverage is doing piecework in a sweatshop. Your performance is measured and you get paid accordingly, but you have no scope for decisions. The only decision you get to make is how fast you work, and that can probably only increase your earnings by a factor of two or three.

An example of a job with both measurement and leverage would be lead actor in a movie. Your performance can be measured in the gross of the movie. And you have leverage in the sense that your performance can make or break it.

[1] http://mitpress.mit.edu/sicp/front/node1.html

[2] http://www.paulgraham.com/wealth.html

But then why isn't, say, academia incredibly lucrative? It's got both measurement and leverage in the form of research grants, and leverage as well in publications.

Academia can be incredibly lucrative for rockstar grant writers, leaders in scientific fields, and authors.

These folks typically don't command astronomical salaries (though they do get paid much, much more handsomely than you'd imagine). Rather, some of them publish bestsellers (and demand the university's help in getting access to leading magazines, publishers, press, etc.). Some of them start their own labs, companies, or institutions that are funded by the schools. Some of them clean up on the lecture circuit. Some researchers -- especially in biomedical fields -- make fortunes (for both their universities and themselves) by selling or licensing patents to private enterprise.

For instance, I'm a writer by hobby. As such, I tend to hang around a lot of other writers (note: I don't recommend this). A common complaint among writers is that writing doesn't pay well. And, on average, that's true. But the world of writing isn't a neat, Gaussian distribution of income. It's a power-law distribution. Most writers are lucky to clear $50k a year in academia, or working for a large and respectable publication. But Malcolm Gladwell earns well north of $10 million a year. Michael Lewis probably makes even more. And don't even get me started on fiction. The woman who wrote "50 Shades of Grey" makes, on average, $3 million a week. J.K. Rowling is a billionaire. Stephen King is probably a near-billionaire. That guy who crapped out "The Da Vinci Code" could just about afford to purchase the Mona Lisa from the Louvre, were he so inclined.

Writing, like academia, is an all-or-nothing sport. There tends to be a very small slice of outsized winners, and a very large slice of abject losers. I don't mean "winners" and "losers" in a pejorative sense. I mean it in the sense that the pie is divided very, very, very unevenly. The average is very low, but the high end is quite high.

This is in stark contrast to, say, i-banking -- wherein a small proportion make an outlandish amount of money, but everyone else still makes a really large chunk of money.

- Research universities aren't generally for-profit organizations and so the simple calculus of profit generated versus salary paid isn't directly applicable.

- Professorship in many disciplines (i.e. liberal arts) is a glamour job, that is one with an effectively bottomless supply of qualified aspirants that do not have easy access to higher-paying alternative jobs.

- "Rainmaker" grant writers pretty much can write their own ticket within academia, but they won't necessarily be able to capture a lot of the grant money as salary. They should still be able to set their own hours while running projects with top-shelf employees and equipment. There's no guarantee that the most interesting, respectable research will be appealing to the types of people who fund research though. (n.b.: I am only guessing on this last point.)