I'm making LLMs work on small and old devices

Re:I'm making LLMs work on small and old devices by vector_spaces on Thursday November 12, 2026 @01:49PM (#50472218)

If you work in a highly regulated space subject to e.g. HIPAA, and you work with ancient on-prem systems within a slow moving and risk intolerant organization -- for instance, virtually every state run healthcare system in the US -- then these are all you can hope for.

In these settings, data is insanely messy beyond your wildest nightmares and hidden away in free-text fields that small models like this are great at cleaning up, normalizing, parsing, categorizing, etc

[ Parent | on HN ]

Re:I'm making LLMs work on small and old devices by mrashad74 on Friday November 13, 2026 @01:01AM (#50478952)

Agreed, and it's a new use case to me because actually in the cases I had the clients were okay with getting GPUs and handling the data mess for us, but none of them requested fine tuning, I believe trying to fine tune will be insanely complex from a regulatory prepsective.

[ Parent | on HN ]

Re:I'm making LLMs work on small and old devices by reasonabl_human on Friday November 13, 2026 @11:49AM (#50486164)

Do you work at an AI consulting company or how are you getting exposure to these regulated industries?

(In another comment you mention being involved in finance and healthcare)

[ Parent | on HN ]