Show HN: Exafunction NLP – Run models like GPT-J for 6x cheaper than OpenAI

Re:Show HN: Exafunction NLP – Run models like GPT-J for 6x c... by varunkmohan on Tuesday September 28, 2021 @02:34PM (#28685947)

Hi HN, I’m Varun, cofounder of Exafunction. We’re excited about all the use cases of large NLP models like text generation and language comprehension, but found the existing offerings to be quite expensive. Our goal is to serve these models in a cost effective way at scale.

The first model we’re starting with is GPT-J, the recently released open source large language model by EleutherAI. It’s comparable to OpenAI’s GPT-3 Curie in performance. With lots of optimizations, we are able to serve the model for 6x cheaper per token compared to OpenAI.

We have a simple HTTPS API that you can try out for free. Here’s a link with the details - https://www.exafunction.com/nlp-api

We’re also looking into supporting custom and fine-tuned models since it’s required to get good performance for a lot of applications. Especially for smaller models like BERT, we’re excited about the cost savings we can deliver for users who use many different models. In this case, we’re 100 - 1000x cheaper than others offering similar inference APIs like Hugging Face.

We hope this is interesting to you all and please email me at [email protected] if there’s a model you’d like to see supported.

[ Parent | on HN ]

Re:Show HN: Exafunction NLP – Run models like GPT-J for 6x c... by d13 on Wednesday September 29, 2021 @03:12AM (#28691069)

But how fast? I see other companies advertising 1.5 second response times for GPT-J, but a now assume that’s average per token, as for, say, a 200 word prompt response times can be well over a minute during heavy use periods like weekends when everyone is hitting their side projects.

[ Parent | on HN ]

Re:Show HN: Exafunction NLP – Run models like GPT-J for 6x c... by varunkmohan on Wednesday September 29, 2021 @06:57PM (#28701077)

For the public API, we should be under 100ms per token but don't have a strict guarantee. If you have a strict SLA, you can talk to us and we can get it to as low as 20ms per token at high load.

[ Parent | on HN ]

Re:Show HN: Exafunction NLP – Run models like GPT-J for 6x c... by GDC7 on Tuesday September 28, 2021 @06:40PM (#28688312)

Hi!

Please could you explain some use cases?

OpenAI is very strict with GPT-3. Say a company wanted to use Exafunction for email interactions or Discord interactions or even SMS interactions.

What would be the setup? How would that work with Exafunction?

Thank YOU!

[ Parent | on HN ]

Re:Show HN: Exafunction NLP – Run models like GPT-J for 6x c... by robbedpeter on Tuesday September 28, 2021 @08:36PM (#28689089)

https://nlpcloud.io/effectively-using-gpt-j-gpt-neo-gpt-3-al...

Here's a wonderfully written breakdown of some functionality that can be achieved with gpt-j and gpt-neo - it's not ExaFunction specific, but the language models are the same.

The exafunction api is fairly simple and powerful. Kudos!

[ Parent | on HN ]

Re:Show HN: Exafunction NLP – Run models like GPT-J for 6x c... by varunkmohan on Tuesday September 28, 2021 @06:59PM (#28688453)

Unlike OpenAI, which is serving a closed-source model, anyone can validate for themselves (even without using our service) whether open-source models like GPT-J are suitable for a particular application. We'd like to work with the open source community to make this kind of technology more accessible, essentially as an infrastructure provider.

If you have some specific use case in mind feel free to shoot us an email at [email protected]

[ Parent | on HN ]