Better, where possible, to implement general-purpose reusable functionality in libraries written in ordinary code in the language. That's a win-win approach: reusable library code can simplify codebases written in the language, but since it's plain old code that follows the normal rules of the language, a library doesn't add language complexity that the developer is forced to keep in mind.
Fundamentally, the key to making codebases in your language understandable is to be compositional: make sure that the developer can easily understand the combination of a and b if they understand a and b separately. Library functions do that, because you can understand how code that calls a library function behaves without needing to understand what the library function does. But language features like throw/catch don't do that: you need to understand what throw does if you are to understand code that calls a function that throws, even if the code you're trying to use doesn't throw itself.
Easily refuted observing that you could strip languages of many of their keywords (for example replacing for and while with if and goto), and you'd end up with less readable code. Keywords are often added to make languages simpler, at least because they declare the intention of the developer.
Otherwise, Brainfuck would be the least complex language to write code in.