VR versus pancake typically has nothing to do with disabilities. Display versus screen reader does. It's fine to exclude people who can choose to be included, but it's good to avoid exclusion when choice is not part of the equation.

If you're talking about building something that couldn't possibly be of any use to a blind person, that's one thing, but most websites do not fall into that category.