This seems to be a general problem for what I normally refer to orchestration software. I.e. calling a lot of external dependencies, io and in general very slim on business logic. You've simply got to deal with too many dependencies, which may or may not have different versions, flavors and whatnot. It becomes impossible to test every permutation, so instead you rely on brute force to weed out bugs, simply fix the things once a bug report comes in.
It helps having good integration/e2e tests, but they only go so far, customers/developers customize and mess up their systems all the time.
Which is why I've choose binaries instead, they are mostly self-contained (except for various things about gnu and musl), and in general just rely on a higher level of built in functionality (because of the statically linked nature).
Turns out on one non-Linux operating systems, one of the utilities used for string manipulation, while POSIX compliant, behaved differently than the same utility in nearly every other operating system.
Wrong, wrong, wrong. Nearly everything you do in a shell script is an external dependency. And that's the biggest problem with shell scripts. The actual language is so narrowly scoped that you're dependent upon external utilities that are not so easy to upgrade.Count how many programs are in /bin. Those are your dependencies. Unlike e.g. python or ruby dependencies, you'll have a much harder time upgrading /bin. Having to fork a new process for trivial tasks is also one of the reasons why shell scripts are so slow.
Sure, bash has builtin versions of some of those utilities like [, but not every system has bash.