Building LLM Applications for Production

Re:Building LLM Applications for Production by iudqnolq on Friday April 14, 2023 @10:53AM (#35570212)

99.99% seems off by orders of magnitude to me. I don't have an exact number but I routinely see GPT 3.5 hallucinate, which is inconsistent with that level of confidence.

I've noticed this discussion tends to get too theoretical too quickly. I'm uninterested in perfection, 99.99% would be good enough. 70% wouldn't. The actual number is something specific, knowable, and hopefully improving.

[ Parent | on HN ]

Re:Building LLM Applications for Production by mrbombastic on Friday April 14, 2023 @12:19PM (#35571273)

I don’t have hard numbers but anecdotally hallucinating has gone down significantly with gpt4, it certainly still happens though.

[ Parent | on HN ]

Re:Building LLM Applications for Production by roflyear on Friday April 14, 2023 @07:59PM (#35576324)

I think it's way better than 70%, probably 95%+ even with bad data and poor prompts. I'd have to run more numbers but it's definitely better than 70%.

You can get to 99.9%+ with good data and well designed prompts. I'm sure it would be above 90% even with almost intentionally bad prompts, tbh.

[ Parent | on HN ]

Re:Building LLM Applications for Production by iudqnolq on Friday April 14, 2023 @08:29PM (#35576491)

It's definitely not that good if we share a definition of poor data/prompts.

This afternoon I tried to use Codium to autocomplete some capnproto Rust code. Everything it generated was totally wrong. For example, it used member functions on non-existent structs rather than the correct free functions.

But I'll give it some credit: that's an obscure library in a less popular language.

[ Parent | on HN ]

Re:Building LLM Applications for Production by roflyear on Saturday April 15, 2023 @09:14AM (#35580349)

> This afternoon I tried to use Codium to autocomplete some capnproto Rust code.

This isn't what I said at all. I said with summarizing data.

[ Parent | on HN ]