News for hackers. Stuff that matters.
Re:Exploring GPTs: ChatGPT in a trench coat? by siva7 on Wednesday November 15, 2023 @12:31PM (#38279555)
It's not so easy. You seem to be under the assumption that there will be just one static system prompt doing all the work that you can customise to your needs. This may be true for some apps but many useful apps will usually do a bit heavier lifting.
Re:Exploring GPTs: ChatGPT in a trench coat? by freedomben on Wednesday November 15, 2023 @12:39PM (#38279663)
I don't think even having multiple dynamic prompts removes the benefit, although for sure it gets a lot more complex to parse and understand as a human. Since the prompt(s) has/have to be rendered at some point though, even if at runtime immediately before use, it could still be made displayable to the user. Assuming this data is already in the database, it doesn't seem like an overly difficult feature to expose. And if it isn't, adding a column to capture it doesn't seem overly difficult either.

Regardless, if there's a "view source" option available on GPTs that opt for it, I'm likely to check those out whereas an opaque one I'm likely going to pass on. Even if it won't work for 100% of cases, it's still an improvement to the status quo.

Re:Exploring GPTs: ChatGPT in a trench coat? by siva7 on Wednesday November 15, 2023 @12:47PM (#38279779)
I feel similiar in this regard but let's not kid ourselves. "View source" isn't something that the general population does as long as it works.
Re:Exploring GPTs: ChatGPT in a trench coat? by simonw on Wednesday November 15, 2023 @12:55PM (#38279893)
View source is a power-user feature. Power users are important, because they're the people who figure out what something is capable of and help coach the general population in how to use it.
Re:Exploring GPTs: ChatGPT in a trench coat? by IanCal on Wednesday November 15, 2023 @01:53PM (#38280737)
GPTs are a prompt, knowledge base (files) and external function calls. There's no dynamic system prompt.

Edit - this is a simple fact, try making one yourself.

Re:Exploring GPTs: ChatGPT in a trench coat? by DebtDeflation on Wednesday November 15, 2023 @01:59PM (#38280830)
>knowledge base (files)

What is it actually doing with the files you upload? Is it just pasting the full text into the prompt? Or is it doing something RAG-like and dynamically retrieving some subset based on the query?

Re:Exploring GPTs: ChatGPT in a trench coat? by simonw on Wednesday November 15, 2023 @02:01PM (#38280859)
This is undocumented (frustrating) but it looks like it's chunking them, running embeddings on the chunks and storing the results in a https://qdrant.tech/ vector database.

We know it's Qdrant because an error message leaked that detail: https://twitter.com/altryne/status/1721989500291989585

It only applies that mechanism to some file types though - PDFs and .md files for example.

Other file formats that you upload are stored and made available to Code Interpreter but are not embedded for vector search.

Re:Exploring GPTs: ChatGPT in a trench coat? by IanCal on Wednesday November 15, 2023 @02:29PM (#38281243)
I've had good success putting code examples in a single txt file for our custom framework, and it seems to use that neatly for generating code. I'm surprised you've not had much success with them, I gave an assistant my wife's PhD thesis and while the API was working initially it seemed alright.
Re:Exploring GPTs: ChatGPT in a trench coat? by wddkcs on Wednesday November 15, 2023 @03:34PM (#38282052)
Does that error message really confirm qdrant for Chat? It's just failing to index a file called 'qdrant', and I don't see any further proof offered in that thread.
Re:Exploring GPTs: ChatGPT in a trench coat? by simonw on Wednesday November 15, 2023 @03:48PM (#38282237)
That's good enough evidence for me.

They're clearly running a vector store (you can get further hints at that by spying on the JSON using browser DevTools).

Qdrant is a very good vector store - it's powering all of the new Twitter features as of a few weeks ago.

Seems much more likely to me that they're using Qdrant than this is a weird error message coincidence.

Re:Exploring GPTs: ChatGPT in a trench coat? by jjwiseman on Wednesday November 15, 2023 @04:04PM (#38282462)
Interestingly, when I used an .md extension, GPT would write python code to try to pull parts out to answer queries (which worked miserably), but when I used .txt (for the same files), it seemed to put it in the vector store.
Re:Exploring GPTs: ChatGPT in a trench coat? by mmq on Thursday November 16, 2023 @04:21AM (#38287336)
Actually msft CEO mentioned in his presentation that OAI moved to Azure's vector search and AI search services for ChatGPT.
Re:Exploring GPTs: ChatGPT in a trench coat? by saliagato on Thursday November 16, 2023 @04:23AM (#38287349)
I think they are running their RAG on Microsoft's Azure Cognitive Search. Satya mentioned something about it in the yesterday Microsoft Ignite keynote
Re:Exploring GPTs: ChatGPT in a trench coat? by avereveard on Wednesday November 15, 2023 @02:18PM (#38281066)
I've seen gpts with a SQLite file that is loaded by the code interpreter to provide structured data as needed

I've seen gpts playing games with code that was attached to the prompt generating the turn results

I've seen gpt using API for converting YouTube to text to provide summaries

There's a lot of things that can be done, even if it's not the most dev friendly experience

Re:Exploring GPTs: ChatGPT in a trench coat? by NathanielLovin on Wednesday November 15, 2023 @02:04PM (#38280903)
If the file is short, they put the full text into the prompt. If it's longer, they use some sort of RAG with qdrant, it appears to be top-1 with context expansion, but nobody's knows for sure how they're doing the chunking.