Show HN: ChatGPT Alternative with LLaMA Models

Re:Show HN: ChatGPT Alternative with LLaMA Models by he11ow on Thursday July 20, 2023 @07:01AM (#36798902)

This looks great!

Can I ask, are you using a HuggingFace endpoint to hit the LLaMA models, or deploying it yourself? Still new to this and trying to understand how putting the large models in production works...

[ Parent | on HN ]

Re:Show HN: ChatGPT Alternative with LLaMA Models by namanski on Thursday July 20, 2023 @10:10AM (#36800696)

Hey hey! We have deployed this on our cloud. It’s running on 2 A10Gs on AWS in the background.

We had the tech from our MLOps platform NimbleBox.ai that let us setup a managed service on all major cloud providers so we just frankenstein-ed it to work for LLMs as well :)

The prompt engineering, specially for web search, is powered by our open-source tool ChainFury (https://chainfury.nbox.ai/)

[ Parent | on HN ]

Re:Show HN: ChatGPT Alternative with LLaMA Models by he11ow on Friday July 21, 2023 @05:41AM (#36811873)

Thanks! Will check out these services, MLOps is definitely where the biggest pain points are right now.

[ Parent | on HN ]