People tend to overestimate how much brown field land there actually is. And what does exist is often more suitable for commercial or industrial development anyway. A disused power station site (for example) could be better for a data centre, distribution center or another power station than residential housing.

Also, just because a parcel of land is green does not necessarily mean that it is a valuable ecosystem. Intensively farmed agricultural land can be degraded already. A well designed development will leave the hedgerows, introduce gardens and have wild areas. It can and should be better for wildlife.