News for hackers. Stuff that matters.
Re:GPT-4 identifies SVB’s biggest risk & gives good advice u... by davesque on Saturday March 25, 2023 @02:52PM (#35305818)
This is exactly what I came here to say. Yes, GPT-4 may happen to give a correct answer. But what happens when it's wrong? And how would you know it's wrong without a specific background in some topic?
Re:GPT-4 identifies SVB’s biggest risk & gives good advice u... by bdg on Saturday March 25, 2023 @02:56PM (#35305855)
This is the major "magic wow factor" behind GPT. It has a basic level of knowledge in every topic which is enough to convince an average human that the AI produced a good result If-And-Only-If the human is unskilled in that topic.
Re:GPT-4 identifies SVB’s biggest risk & gives good advice u... by warent on Saturday March 25, 2023 @03:00PM (#35305894)
Except that GPT can and does produce good results even to experts, just not all the time. So your claim of "IFF" is unnecessary hyperbolic contrarianism
Re:GPT-4 identifies SVB’s biggest risk & gives good advice u... by shagie on Saturday March 25, 2023 @03:07PM (#35305961)
The interesting part here is "does it take longer for an expert to come up with the answer independently or verify that GPT is producing an accurate result?"
Re:GPT-4 identifies SVB’s biggest risk & gives good advice u... by jacobwilliamroy on Saturday March 25, 2023 @02:59PM (#35305885)
The other day I was having ChatGPT write some code for me and it started referencing objects and properties that didnt exist. I wish I tried correcting it. That would have been interesting.
Re:GPT-4 identifies SVB’s biggest risk & gives good advice u... by mckeed on Saturday March 25, 2023 @03:03PM (#35305929)
When you correct it, it always seems to say "You are absolutely right! I apologize for the mistake," and then it gives another wrong answer with the same fully confident tone.
Re:GPT-4 identifies SVB’s biggest risk & gives good advice u... by jacobwilliamroy on Saturday March 25, 2023 @03:15PM (#35306049)
Lol. One probably has to work at OpenAI to try and fix their LLM thing. The one they let the public use probably doesnt learn from its users. Hopefully. I wouldn't let just any data in my LLM. When those things get garbage in they just put garbage out.
Re:GPT-4 identifies SVB’s biggest risk & gives good advice u... by shagie on Saturday March 25, 2023 @04:27PM (#35306938)
There's a "thumbs up" and "thumbs down" next to each generated response.

While this model may not be getting updated in real time, I would be surprised if that feedback isn't used when looking at updating the model with good feedback being used for retraining the model.

Re:GPT-4 identifies SVB’s biggest risk & gives good advice u... by mclightning on Saturday March 25, 2023 @06:46PM (#35308328)
I would love to hear opinions from LLM experts. But I have a feeling they may not be able to update the model, at a fundamental level, so easily by live feedback.

Based on what I have been reading, training the core model is a work of statistical analysis on how often words follow each other. This produces a graph tree of words.

Now how do you re-evaluate the entire graph based on a single additional feedback without actually retraining the entire model with the said feedback (because it is costly, and result wouldn't be immediate, unless you're thinking of injecting the input as an initial condition to the readily trained model)

Re:GPT-4 identifies SVB’s biggest risk & gives good advice u... by shagie on Saturday March 25, 2023 @06:55PM (#35308413)
It is possible to tune the model, and I suspect that remains something that is done on a regular basis.

https://platform.openai.com/docs/guides/fine-tuning

In particular from https://platform.openai.com/docs/models/gpt-3

> With the release of gpt-3.5-turbo, some of our models are now being continually updated. In order to mitigate the chance of model changes affecting our users in an unexpected way, we also offer model versions that will stay static for 3 month periods. With the new cadence of model updates, we are also giving people the ability to contribute evals to help us improve the model for different use cases. If you are interested, check out the OpenAI Evals repository.

The feedback wouldn't be immediately injected back into the model (human curation of the responses is needed to see if the feedback is appropriate).

Some of the feedback would be used to train the moderation / supervisor model. https://platform.openai.com/docs/models/moderation

Re:GPT-4 identifies SVB’s biggest risk & gives good advice u... by pixl97 on Saturday March 25, 2023 @05:10PM (#35307392)
How long was your token list by then?

Once context gets long with the limited tokenization memory we currently have it seems to go insane rather fast. Would like to test on the 32k model to see how the same prompting differs.

Re:GPT-4 identifies SVB’s biggest risk & gives good advice u... by mckeed on Sunday March 26, 2023 @02:14PM (#35317573)
I thought I was using GPT-4, but it appears that's only available to paying customers so far? The question I was getting repeated wrong answers to was "Can you make up a palindrome that starts with 'Dude'?" If you want to try that with GPT-4 I'd be interested to see if it can do it now or at least knows to say "I can't".