As with most things Emacs, there is no real ceiling to what you can do, because elisp. But as you observed, there's a reason that using syntax-table rules to write a highlighter has been called "black magic".

The sooner there is anything even remotely grammar-based for creating highlighters in Emacs, the better.