It can make sense to invest in hedge funds even if you don't expect your hedge fund portfolio to beat the market on average, if the hedge fund is uncorrelated with market returns. This is a basic result of Modern Portfolio Theory, but the intuition is that sometimes with the stock market down you will have better performance in your hedge funds (and vice versa), so your account will be less risky in the sense of having shallower troughs. If you want to take on the same risk (i.e. have troughs that are the same depth as if you just held stocks) you can invest more of your capital in stocks/hedge funds vs. bonds/cash, which means you are making better returns. It should be noted that the hedge fund universe is extremely broad, and different funds have different risk/return characteristics.

Venture capital is different because it is both correlated with market returns and the average fund significantly underperforms the market,[1] so getting good returns in venture capital is pretty much a question of getting allocations in the top 20 funds, which are persistently the best. They have the best returns in part because they have the best reputations and thus access to the deals that provide the highest returns (top entrepreneurs would take Andreesen Horowitz's money over money from Unknown Partners on the same terms).

1: http://www.kauffman.org/~/media/kauffman_org/research%20repo...