News for hackers. Stuff that matters.
Re:Llama 3.2: Revolutionizing edge AI and vision with open, ... by noahbp on Thursday September 26, 2024 @02:44AM (#41655228)
What kind of damage is done by constraining token generation to valid JSON?
Re:Llama 3.2: Revolutionizing edge AI and vision with open, ... by snovv_crash on Thursday September 26, 2024 @06:36AM (#41656757)
Yeah, from my experience if you prompt something like:

respond in JSON in the following format: {"spam_score": X, "summary": "..."}

and _then_ you constrain the output to json, the quality of the output isn't affected.