That's a common, often unstated problem, in many places. A new family often doesn't pay enough in taxes to cover the services they need, schools in particular. Which puts pressure on everyone's tax burden.
I live in a rural/exurban town in New England and the tax issue definitely comes up in any discussion of approving any sizable development. (The town tends to be against development in general for traffic etc. reasons but taxes are part of it.)
Prop 13 can take almost exclusive blame for cities in California giving preference to jobs and retail over housing. It's like a direct incentive if they want to keep their budgets balanced.
I think it's fair to say that the real calculus in Brisbane is simply opportunity cost: they get a better ROI with commercial than residential. And this calculus simply makes it easier to defend anti-development sentiment, which is really what it all comes down to.
Geographic mobility really is a huge hurdle. A lot of business is becoming centralized around major long-distance transportation hubs, San Francisco being chief among them. Live in Concord, but want to work in Los Gatos? Good luck getting there by train. I lived in the Mission and to get to China Basin was a bus to a train to a bus. I opted for the ten minute drive instead of forty-five minutes of transferring. It's embarrassing how much better we could do so much better moving people around. Right now I opt to drive the 101 south instead of taking Caltrain, even though my employer covers the cost, because it would cost me an additional $10 a day to take other trains/buses to get to the Milbrae transfer, and by the time I've driven myself either to 4th/King or San Bruno I could have already been half way to work just driving the whole way. We need to do better.
Not to mention, no one is asking to knock down the beautiful hillside neighborhoods - they want to build housing in the flat baylands across the 5-lane highway Bayshore Blvd. The SF Muni T-line basically already services this proposed baylands neighborhood.
As for the mobility issue - I agree 100% that transportation is a crucial part of this whole equation. I also understand that you can't build effective transportation if you keep surrendering to people who want car-oriented sprawling development patterns, including prioritizing cars even in our dense neighborhoods.
Brisbane's city report from 2013 states there were ~6,500 jobs in the city. Add in children, unemployed, retired, etc... and that accounts for many times more residents than can actually live in Brisbane. Which means Brisbane is contributing heavily to the housing problem in the Bay Area.
I live in a mixed SFH/midrise apartment neighborhood in Oakland, and our population density is 24,000 per square mile.
So putting 24k people in that part of Brisbane wouldn't even look particularly built up.
The Nob Hill, Russian Hill and Telegraph Hill areas (beautiful, treasured, highly desired places) in San Francisco all vary from 30k to 60k per square mile.
Putting 45k people on Brisbane baylands would be awesome.
What's really happening is that Brisbane loves the sweet tax revenues from all the businesses they host, but don't want the (expensive) responsibility to house all those people that work in town.