That said, it's easier to make square sand round than round sand square. It's probably easier to crush rock into concrete-appropriate sand than to turn desert sand into concrete sand.

That's never stopped us before!

This study addresses the issue but is behind a paywall: http://www.scientific.net/AMM.174-177.604

For the Sahara example, I fear that part of 'strength' here is brittleness. The concrete may be strong enough, as long as you don't drop heavy objects onto it or hit it with a hammer.

Ignoring that, thicker walls in the Sahara probably are a good idea to keep rooms cooler, but can be problematic in high-rise buildings, as they weigh more, and thus require even thicker walls at lower levels. Floors are a problem, even in single-floor buildings.

[1] Actually, it is not just creation. Age plays a role, too. For example, you have to replace sandbox sand once the smaller grains have been rained out of it because it stops 'sticking'.