Short of serious corruption, most cities are about as nice as their current tax base allows. Giving a company a tax break to incentivize that company to move into their city allows them to grow their tax base and improve their city for everybody. But it's not like cities are just sitting on potential attractors for employers to move in.
Policy is a highly subjective thing, but it also takes decades for it to make an impact.
Consumers and producers don't always get to decide. If you live in a small town that has a Wal-Mart then it's extremely difficult to compete directly with that store if you're just some independent. People might like your business, but you can't buy at the same volumes, you can't force the hand of city council for tax breaks, you have almost no power in comparison.
Absent good governance then corporations decide everything and we get to "decide" between the options they deign to give us.
That's not necessarily true. A city may not want everyone, or they may particularly want Amazon. Competition is competing, one part of which is incentives to localize in an area. Since so many places want an HQ2, it is Amazon's market not that of the cities.
Cities can rail against having to compete in a competition, but those cities likely will not win the competition.
Remember, if someone wants to move to a larger city they can. However, being smaller is actually appealing because you get less traffic etc. Further, city's are only important due to it's residents, if everyone moved somewhere else it's not like the land is going to get upset.
That's the problem, how do you spark the network effect? People don't setup in SF for ironic atari t-shirts, they're there for the talent pool and infrastructure (i.e. the stuff amazon will bring)
In short, you bring them in for other reasons. The most common being Universities, but it can also be Government or Private Sector research labs. One oft overlooked factor in Silicon Valley's success has been the great Universities nearby attracting some of the brightest people from all over the world.
Pittsburgh (CMU), Raleigh-Durham (UNC) and Austin (UT, TAMU) are succeeding in tech for similar reasons.
The Raleigh-Durham area is always talked about as kind of an unappreciated, affordable place for tech people to work. Both downtowns have improved over the past decade or two (like in many small cities), the three big universities have plenty of students, RTP is around, the only thing really against it is the suburban sprawl. I hear they are getting light rail but I've heard people talking about rail in NC since I was a child (30 some years ago).
as much as HN (me included) likes to complain about sprawl, don't forget that there's plenty of people who love the suburbs. Sprawl is not a downside for everybody - the people who like it just call it "affordable single-family homes" and "quiet neighbourhoods" instead of sprawl
I think on this I was thinking more the requirements of the HQ2 which specifically mentioned mass transit routes.
Also, more generally, the parent was talking about the 'network effect' of SF and the culture there. When you're isolated in your affordable single-family home and commute to your office park that vastly differs from the serendipity and collision of people, thoughts, opportunities that you have living in a denser place where you come across more people daily, one would think. I assume that's why cities are more liberal - that we have to put up with, tolerate, and can empathize with a wider variety of people than the isolation of house->car->work->car->house.
Just speculating there.
I get the race-to-the-bottom argument, but then what mechanisms can an underdog city use to ever compete?