Re:Llama 3.2: Revolutionizing edge AI and vision with open, ...
by noahbp on Thursday September 26, 2024 @02:44AM
(#41655228)
What kind of damage is done by constraining token generation to valid JSON?
[ Parent
| on HN ]
Re:Llama 3.2: Revolutionizing edge AI and vision with open, ...
by snovv_crash on Thursday September 26, 2024 @06:36AM
(#41656757)
Yeah, from my experience if you prompt something like:
respond in JSON in the following format: {"spam_score": X, "summary": "..."}
and _then_ you constrain the output to json, the quality of the output isn't affected.