Well, assuming you're American, you can look at the decline of Detroit to see what happens when you don't think of these things. Or Wales and the North of England, if you're from the UK.
> Are you suggesting that if we all collectively decide that building such things hurts people, no one will build them?
No, I'm suggesting that we consider the implications of what we're creating and try to ensure that, even as we build them, we ensure that we don't damage whole swathes of society in doing so. But that's just me, and it might just be that I'm a bleeding heart. All progress comes with its benefits and its downsides, and if you think rendering people unemployed without giving them an alternative means of supporting themselves is OK and/or shouldn't be part of the calculus of how progress is made, then that's your decision.
I don't think it's Silicon Valley that lacks empathy, I think it's the system that rewards profits and growth over anything else...
To be clear, I agree with what you're saying as an observation of how things work much of the time. I also don't think Silicon Valley is any better or worse than the rest of the 'system' (honestly I don't know).
But generally speaking I'm inclined to believe that 'this is just how things are' is one of the main reason why things don't change. Things don't have to be the way they are and it takes individuals working within (as well as outside) the systems that are in place to change this.