News for hackers. Stuff that matters.

How to feed content to ChatGPT and make it answer questions about it?

I recently came across several startups that appear to feed content to ChatGPT and subsequently ask questions about that content.

Examples: 1. https://flowgpt.ai 2. https://www.chatbase.co

I'm curious about how this process works. Do these startups utilize the OpenAI API to train a model?

I attempted to train a model using the OpenAI API myself, but it seems like these startups have a different approach. I was unable to achieve the desired results simply by uploading content, even after spending hours creating JSONL files.

Chatbase, for instance, has a video demonstrating a user uploading a file and then asking questions: https://youtu.be/W4lzGger7_0

( 12 comments | 3 points | on HN )
Re:How to feed content to ChatGPT and make it answer questio... by fbrncci on Wednesday May 03, 2023 @01:15AM (#35798302)
Pretty easy, take a look at Langchain tutorials on YouTube. Basically you give it a set of documents, split these into smaller documents and then store these in a vector database and create embeddings (OpenAI, Jina, etc). Then when you interface with OpenAI on GPT-3 or GPT-4, you interface with those documents and embeddings and produce an answer based on the document set (or very near to it).
Re:How to feed content to ChatGPT and make it answer questio... by alexander-g on Wednesday May 03, 2023 @01:51PM (#35805459)
Thank you! It was an eye opener for me. We've been using slightly different approach (at https://jopilot.net) but vector database + langchain allows to process much bigger amount of data.
Re:How to feed content to ChatGPT and make it answer questio... by fbrncci on Thursday May 04, 2023 @01:45AM (#35811648)
No problem! You could probably improve it by fine-tuning GPT models on different categories of documents, prior to doing the vector retrieval from embedding. Fine tuning isn't available for GPT-3-turbo or GPT-4 yet, so I am waiting to try out this hybrid approach for when it does come available.
Re:How to feed content to ChatGPT and make it answer questio... by ksaj on Wednesday May 03, 2023 @12:58AM (#35798189)
I've only tried one json file I saw posted by someone, and it didn't work either. Well, it sorta worked, but it very quickly forgot that I had given it anything already.

I strongly suspect that the better results come through the APIs and not the web interface.

Re:How to feed content to ChatGPT and make it answer questio... by alexander-g on Wednesday May 03, 2023 @01:06AM (#35798253)
It's an API. Yes. But how it works? How to feed content and make it answer questions?
Re:How to feed content to ChatGPT and make it answer questio... by jn2clark on Wednesday May 03, 2023 @03:15AM (#35799124)
See here https://github.com/marqo-ai/marqo/blob/mainline/examples/GPT... and https://github.com/marqo-ai/marqo/blob/mainline/examples/Spe.... Multiple examples for answering questions from documents/manuals, text, or transcripts.
Re:How to feed content to ChatGPT and make it answer questio... by james-revisoai on Wednesday May 03, 2023 @07:34AM (#35801072)
Langchain --> query_with_sources function (semantic search context)

If you finetune, you'll get mostly the style and sometimes worse factual performance. Source through semantic search context/RAG is a better approach.

Source: Creator of AnyQuestions.ai which works like that plus some extra neat tricks. (released the first demo in August 2022 before the onslaught)

Re:How to feed content to ChatGPT and make it answer questio... by alexander-g on Wednesday May 03, 2023 @01:53PM (#35805478)
When are you actually billed? First, when you create embeddings at the beginning. Second, every time you call the OpenAI chat complete API?
Re:How to feed content to ChatGPT and make it answer questio... by james-revisoai on Saturday May 13, 2023 @07:54PM (#35933993)
Yes, that's correct. Then OpenAI bills you around the end of the month so you have some leeway of credit.

You can use local embeddings, such as sentence transformer models, too, to save on embeddings (and maintain embeddings you can move to another provider with new queries etc)

Re:How to feed content to ChatGPT and make it answer questio... by ian0 on Wednesday May 03, 2023 @08:07AM (#35801341)
This youtube series shows a basic intro to some techniques with code, he also runs through a great flow chart to show you how it all works

https://www.youtube.com/watch?v=ih9PBGVVOO4

Re:How to feed content to ChatGPT and make it answer questio... by sergiotapia on Wednesday May 03, 2023 @12:03PM (#35804033)
1. Take text of source material. Example: extract_pdf_text(pdf_path_string)

2. Segment text every 3000 characters.

3. Generate embeddings for every text segment and save the embeddings to PineconeDB. Make sure you also save the raw text segment as additional meta information so you can use it later. https://platform.openai.com/docs/guides/embeddings/what-are-...

4. Capture user's question and generate an embedding for it using the same OpenAI API.

5. Query your PineconeDB with the question's embedding you will get matches back.

6. Use these matches as context to hit the OpenAI chatgpt API endpoint. Example:

    Using this context answer this question: **QUESTION**

    **CONTEXT**
Re:How to feed content to ChatGPT and make it answer questio... by alexander-g on Wednesday May 03, 2023 @01:54PM (#35805491)
Thank you for the detailed explanation!