Re:Better LLM response format – A simple trick reduces costs...
by livshitz on Tuesday July 25, 2023 @07:01AM
(#36860540)
Yes, minified JSON would be even less tokens than YAML.
But:
1- LLMs tend to have very hard time to produce minified (compacted) JSON in the output, consistently.
2- As for compacted JSON input- Empirically it seems that LLMs can process it quite well for basic cognitive tasks (Information Retrieval, basic Q&A, etc), but when it comes to bit more sophisticated tasks it fails compared to exactly the same input, uncompressed. I've mentioned and provided examples in the comments of this article.
[ Parent
| on HN ]
There are no comments on this comment yet.