News for hackers. Stuff that matters.
Re:Orbit. Mozilla's AI Assistant for Firefox by jiggawatts on Monday December 30, 2024 @09:09PM (#42555679)
Why are you confident to use such a tiny model on something so critical?

I would never use anything smaller than a 70B model for anything even vaguely medical related!

Re:Orbit. Mozilla's AI Assistant for Firefox by pilotneko on Monday December 30, 2024 @10:14PM (#42556071)
I don't use Mistral 7B alone, this is just a component in a RAG-based system. A system that is 1) not clinical facing, 2) not used in clinical decision making, 3) provides in-line references sources for end users to validate information, and 4) is inherently human-in-the-loop.
Re:Orbit. Mozilla's AI Assistant for Firefox by bigstrat2003 on Monday December 30, 2024 @09:41PM (#42555868)
> Why are you confident to use such a tiny model on something so critical?

Don't use any LLM for something critical. They can't be trusted innately due to their design, why on earth would you use them for something where you need reliability?

Re:Orbit. Mozilla's AI Assistant for Firefox by mdaniel on Monday December 30, 2024 @10:37PM (#42556216)
> why on earth would you use them for something where you need reliability?

As very best I can tell the answer is "because bags of cash"

Re:Orbit. Mozilla's AI Assistant for Firefox by add-sub-mul-div on Monday December 30, 2024 @09:26PM (#42555778)
A key thing to remember in this specific moment in history is that the vast majority of people will be as lazy as they can get away with being. People want to leave work and go home and if an LLM lets them do that faster, who cares if it's accurate? It can absorb the blame.
Re:Orbit. Mozilla's AI Assistant for Firefox by N2yhWNXQN3k9 on Monday December 30, 2024 @09:18PM (#42555730)
Its summarisation, who cares if its right as long as you feel confident after reading it? /s.

In my experience, even GPT-4o is terrible at revealing information from things longer than a few pages.

It might be an issue with dimensionality reduction in general though. If you think about it, you can't really take away much of what is contained within any given amount of text with text, unless the source was produced extremely inefficiently.

Producing outlines or maybe a form of abstract, it seems to be okay at, but you would never really know where it fails unless you read the entirety of the source text first to begin with. IMO, its not worth risking unless you plan to read the source anyway or its not really important.

Re:Orbit. Mozilla's AI Assistant for Firefox by someothherguyy on Monday December 30, 2024 @10:14PM (#42556074)
Try to walk through a Wikipedia article having an LLM summarize every few paragraphs, its often wildly inaccurate.