Ask HN: How to send context when you use LLMs via API for a B2B use-case?

Posted by msnkarthik on Monday July 29, 2024 @03:26AM
from the ask-hn dept.

In B2B use-case, we pass on the data to LLMs via API and get the answers and display to the users. It's like a wrapper on top of LLM API. But since API calls are one-off, there dont have much context about a whole chat. How do you send so much of context when communicating with GPT via an API and not directly through chat. Wondering this for a B2B saas use-case.

( 2 comments | on HN )

Re:Ask HN: How to send context when you use LLMs via API for... by fdarkaou on Monday July 29, 2024 @03:16PM (#41102915)

most LLMs today have a large context window where you can send a history of your chat

i've built multiple chat demo apps (see anotherwrapper.com) and there what i basically did was store a full copy of the history in the DB & then in a config file i specified how many previous messages i want to include in my chat history when interacting to the API

[ Parent | on HN ]

Re:Ask HN: How to send context when you use LLMs via API for... by msnkarthik on Tuesday July 30, 2024 @03:26AM (#41106837)

got it. this helps thanks. Which models are you using for this?

[ Parent | on HN ]