This is assuming the voters know enough to have a coherent volition. The clear majority of the people in the US think the government should spend less on foreign aid. If asked what percentage of government revenue should be spent on foreign aid they would name a figure much higher than we currently spend. But these aren't inconsistent, because the typical voter thinks we spend far, far more on foreign aid than we do. All this means that in terms of referendums there is no such thing as the will of the voters, there is only how you ask the question.

Now, if we could get the entire population of the US into a room to argue over the budget for a week with a vote at the end I'm sure that the result would be better than our legislators would do but of course that's logistically impossible. So barring that I'll take representative democracy over referendums any day. Legislators know that voters have a distinct tendency to vote the bums out of office if they feel things are getting worse.