Plus the purpose of a house generally does not change much. If you buy one, you're not going to turn it into a bar one year, then turn it into a candle shop the next. The requirements usually stay the same.
Some business apps do need to change every year. There's not any common sense for this situation (like house building) because you'd need to predict the future.
>UX first can work if it's iterated in small cycles continuously
Absolutely agreed, what I've generally seen is that the first iterations are there to get the UX grammar down with non-technical users. Then they can see what they like and don't like and their requirements become more coherent.
It depends on if you're developing software that is more of a tool addressing a well understood and pervasive problem, as opposed to a service or solution where you're trying to adapt to the varying needs of a broader market.
If you build like a house you might have plans that meet the needs of a few customers but you're going to have to re-design the house for the next set of customers. Software is generally more of a continuously developed service that needs to adapt as problems, jobs and markets change -- houses don't have to do this.
But I agree, "UX" first can work if it's iterated in small cycles continuously and the technical knowledge is involved in understanding the jobs and problems rather than the requirements and specifications.