News for hackers. Stuff that matters.
Re:Exploring GPTs: ChatGPT in a trench coat? by maCDzP on Wednesday November 15, 2023 @01:07PM (#38280072)
Regarding getting better results with RAG.

I have had some luck with that.

I use the Assistant API, which I believe is not the same thing GPTs. I have played with it through the web interface.

I had 100+ PDF:s files that were OCR:ed with Tesseract. I then had ChatGPT write a script that combines all files in to a single txt-file keeping the layout.

I uploaded the file and started asking questions. The files contains highly technical data regarding building codes in non English so I am guessing the model isn’t so used to that type of language?

Anyway, it worked surprisingly good. It was able to answer questions and the answers were good. Plus that it is supposed to annotate from where it took the answer, although I didn’t get that to work properly.

I tried to upload PDF:s, JSON-files, CSV:s. Raw text has worked best so far.

Re:Exploring GPTs: ChatGPT in a trench coat? by simonw on Wednesday November 15, 2023 @01:14PM (#38280176)
The thing I really want to get working is citations. When it answers a question using RAG I want control over the citation that is displayed - ideally I'd like to be able t o get that to link to an external website (the site that I built the context document from.)

Here's a screenshot illustrating what I mean: https://twitter.com/simonw/status/1721912151147979152

Re:Exploring GPTs: ChatGPT in a trench coat? by spdustin on Wednesday November 15, 2023 @01:23PM (#38280311)
The citations are built to reference the ID of the quote object in the metadata returned by the `quote_lines` function. I have been able to get them to point elsewhere, but not in the GPT itself; only with a userscript that intercepts the fetch for the completion and re-writes that metadata. Even then, encoding a URL for the real source would require a lookup somewhere to get the original source.

I had a little luck instructing the GPT to perform “an additional step after calling the `quote_lines` function of the `myfiles_browser` tool” so maybe that’s worth poking around further.

Re:Exploring GPTs: ChatGPT in a trench coat? by maCDzP on Wednesday November 15, 2023 @01:20PM (#38280270)
I had the exact same problem and just like you I believe it’s an important feature.
Re:Exploring GPTs: ChatGPT in a trench coat? by tiahura on Wednesday November 15, 2023 @01:10PM (#38280117)
I then had ChatGPT write a script that combines all files in to a single txt-file keeping the layout.

Mind sharing?

Re:Exploring GPTs: ChatGPT in a trench coat? by maCDzP on Wednesday November 15, 2023 @01:16PM (#38280216)
You mean the code? Sure, here is the ChatGTP conversation.

https://chat.openai.com/share/954f6b3e-7edc-4421-bfb1-89045e...

Re:Exploring GPTs: ChatGPT in a trench coat? by Ephil012 on Wednesday November 15, 2023 @02:17PM (#38281042)
Here's the catch. I did an analysis earlier myself of the assistants API and discovered this good performance is ONLY for if you combine into a single text file. If you try multiple files it fails.

Here's my post with the analysis. https://news.ycombinator.com/item?id=38280718