Prompt engineering is kinda like a basic form of data science. You have a dataset and some manually labelled results, and you hypothesize on prompts and try to improve on some metric. You'd be surprised how the tiniest of changes will alter a result. Thinks like a hyphen, semicolon, or capitalization can sway the metric. It's very finicky and very annoying.
e.g. why does the JSON output have silly whitespace/quotation sometimes? Obviously it's because the first token the model output was `{` and not `{"` like it should be. Obviously.
GPT-4-turbo's output context window is limited to 4096 so fixing this is relevant. You can use logit_bias for it.
You also need to know the tools to use and there's a lot of settings you play with that are hidden from the user in mainstream apps.