News for hackers. Stuff that matters.

A comment on Ask HN: Is anyone doing anything cool with tiny language models?. Parent comment

Re:Ask HN: Is anyone doing anything cool with tiny language ... by sitkack on Tuesday January 21, 2025 @05:09PM (#42785696)
So you are using a local small model to remove identifying information and make the question generic, which is then sent to a larger model? Is that understanding correct?

I think this would have some additional benefits of not confusing the larger model with facts it doesn't need to know about. My erasing information, you can allow its attention heads to focus on the pieces that matter.

Requires further study.

[ Parent | on HN ]
Re:Ask HN: Is anyone doing anything cool with tiny language ... by mettamage on Wednesday January 22, 2025 @02:50AM (#42790194)
> So you are using a local small model to remove identifying information and make the question generic, which is then sent to a larger model? Is that understanding correct?

Yep that's it

[ Parent | on HN ]