You can make that choice. On the other hand, if you decide on not making the situation right on that, you'll alienate not only accessibility-focused users, but also a very loud minority - some of whom are influential across tech (or tech-adjacent) communities.