Was a case of don't have the time or the patience to fight the model: just work damnit...
I was running something at work that involved an LLM, and kept getting safety refusals. The weird thing was that when I went to our in-house chatbot, set the model to the same one the program was using, entered the same text, it happily answered.
I spent quite a while trying to figure out why my program was being refused. Was I doing something wrong?
No - the chatbot was doing something "right". You see, the chatbot had some tools it injected into the prompt (current time, etc). These tools had no relevance to my query.
So I went back to my program and injected the same tools. Suddenly my request was very safe.
The phrase "security theater" was invented for stuff like this.
Total frustration while the work pressure was already high. Sol and astra thankfully complied and were very useful. Saved me hours compared to doing it by hand during log4j