What you are missing is that family offices usually farm out most of their vanilla public market securities portfolios to people like vanguard, but that 1/2 to 1/3 of the portfolio is private or alternative investments and that is where the investment team spends a great deal of time. Additionally, a lot of the "team" is often focused on things like reporting, tax, estate planning etc.