Emerging Architectures for LLM Applications

Re:Emerging Architectures for LLM Applications by akisej on Tuesday June 20, 2023 @04:02PM (#36409683)

Great starting point! These diagrams notably miss a LLM firewall layer, which is critical in practice to safe LLM adoption. Source: We work with thousands of users for logicloop.com/ai

[ Parent | on HN ]

Re:Emerging Architectures for LLM Applications by applgo443 on Tuesday June 20, 2023 @04:17PM (#36409870)

What do you mean by firewall layer? What tools do you use here?

[ Parent | on HN ]

Re:Emerging Architectures for LLM Applications by __loam on Tuesday June 20, 2023 @04:45PM (#36410250)

I imagine he's talking about preventing prompt injection (or making shit up)

[ Parent | on HN ]

Re:Emerging Architectures for LLM Applications by akisej on Tuesday June 20, 2023 @05:22PM (#36410840)

Yup, that's part of it but I mean it bidirectionally - users can accidentally leak data to models too, which is concerning to SecOps teams without a way to monitor / auto-redact.

[ Parent | on HN ]

Re:Emerging Architectures for LLM Applications by lukasb on Tuesday June 20, 2023 @04:56PM (#36410431)

That doesn't seem like the type of problem that can be solved with a drop-in solution.

[ Parent | on HN ]

Re:Emerging Architectures for LLM Applications by __loam on Tuesday June 20, 2023 @07:52PM (#36412422)

No but that won't stop them from making a startup to sell you some snake oil that doesn't work!

[ Parent | on HN ]

Re:Emerging Architectures for LLM Applications by applgo443 on Tuesday June 20, 2023 @08:07PM (#36412537)

I think we can detect atleast a few things like PII leaks etc. Don't you think those things alone are valuable?

[ Parent | on HN ]

Re:Emerging Architectures for LLM Applications by akisej on Tuesday June 20, 2023 @05:21PM (#36410826)

These common issues tend to prevent LLMs from being used in the wild: * Data Leakage * Hallucination * Prompt Injection * Toxicity

So yes it does include prompt injection, but is a bit broader. Data Leakage is one that several customers have called out, aka accidentally leakage PII to underlying models when asking them questions about your data.

I'm evaluating tools like Private AI, Arthur AI etc. but they're all fairly nascent.

[ Parent | on HN ]

Re:Emerging Architectures for LLM Applications by applgo443 on Wednesday June 21, 2023 @04:32PM (#36423985)

I’m a researcher in the space exploring few ideas with the intention of starting up. Would love to reach out to you and talk to you. Is there a way I can contact you?

My email is [email protected]

[ Parent | on HN ]