great question, i think rather than smaller inputs, you should be able to have as much context as needed in. Its more aboue smaller outputs, that are well typed and allow you to control workflow via standard coding mechanics. Therefore getting the best of both worlds, llms power of understanding unstructured info (which python sucks at) and being able to control things which python is great at.
i.e One could just use their own prompt for the binary classifier right?