The concern with finetuning, even for specialized use-cases, is that you are binding yourself to the underlying model. Given rapid advancements in the field, this does not seem a prudent use of engineering time.

Having a hierarchy of prompts with context stuffing allows for rapid switching across models with a few (non-trivial) surface-level prompt updates while the deeper prompts stay static.