Some people have significant health problems if they eat to much fiber. They are literally on a low fiber diet as a medical matter.

The rest of us have lower GI problems of some sort if we go too much over the daily recommended amount. That can cause significant practical and social problems. But you are correct that we probably won't die. It could be painful in some cases, though, and that would be enough.