Example: The stench of smoke and blood filled their nostrils as they crouched behind a fallen tree, its splintered trunk providing the only cover from the relentless hail of bullets, their heart hammering in their chest, threatening to burst free, as sweat mingled with dirt on their face, catching the frightened gaze of their comrades, their eyes wide with terror yet burning with determination, knowing they were all in this together with no turning back, seizing the opportunity as the gunfire briefly subsided, yelling "Move up! We've got to take that hill!" their voice breaking through the deafening cacophony of war, breaking cover without hesitation, boots pounding the earth as they ran through the hellish landscape, explosions erupting all around, the concussive force rattling their bones and sending tremors through the ground beneath their feet, seeing a fellow soldier fall, his anguished cry piercing their soul, diving to his side, dragging him into the meager cover of a shallow trench, his blood staining their hands, no time to mourn, continuing their charge, every muscle screaming in protest, lungs pleading for air, cresting the hill, the enemy bunker coming into view, sprinting towards it with a final surge of adrenaline, their hand shaking as they lobbed a grenade inside, the explosion tearing through the bunker, debris flying like deadly shrapnel, winning this small battle, but the weight of the war still pressing down on their shoulders, gazing out at the smoke-choked battlefield, wondering how many more battles they'd have to endure before the war was truly over.
Constrained generation i've found to produce the most creative-like results.
https://paperswithcode.com/paper/most-language-models-can-be...
I want to update that just because GPT-4 appears to be far better at following constraints, doesn't mean that it's anywhere near perfect at following them. It's better now at my easy example of "ban the letter e" but if you ask for several constraints, or mixing lexical and phonetic constraints, it gets pretty awful pretty quickly. Filter assisted decoding can make any LLM (no matter how awful they are) follow constraints perfectly.
I can't wait to get someone whose better at coding than me to implement these techniques in the major LLM frontends (oogabooga, llamma.ccp, etc) since my attempt at it was quite poopy research code: https://github.com/hellisotherpeople/constrained-text-genera... - especially with the idea of this but with all LLM types (e.g. bert), using a declarative syntax like langchain, and also supporting constrained beam search for sequence level constraints.
BTW - just to motivate the nay-sayers on why filtering LLM vocabularies may be a cool (and dangerous) technique even in a world where LLMs became "perfect" at following lexical, semantic, or phonetic constraints in the prompt. Imagine asking ChatGPT to "generate some social security numbers". Now imagine doing that again when it's only allowed to generate number or hyphens. This is one of the many reasons why full vocabulary probability distributions are not exposed by folks like OpenAI