The conditions that are left when those above median have chosen theirs. If you want better you work for it. If you don't want better then you share the work those who worked for theirs produced. So you work just as much as you want, and others working more means those who don't have more.

It is true that if you want a luxury resource, like a very attractive location to live in, then others working more for that position means you also would have to work harder if you want to have it. But I don't consider people competing for luxury resources a problem, no that is a feature and is what makes the economy tick. If you don't want that rat race you have to live in median conditions.