Show HN: StonksGPT – A Natural Language search tool for Stocks and Finance data

Re:Show HN: StonksGPT – A Natural Language search tool for S... by yreg on Thursday June 01, 2023 @06:02PM (#36158017)

The catch is that the models don't have confidence. They cannot distinguish between knowing something for sure, guessing something and hallucinating false knowledge.

Perhaps researchers will find some clever solution to mitigate this, but for now hallucinating is a pretty good word precisely because the model doesn't give confidence.

[ Parent | on HN ]

Re:Show HN: StonksGPT – A Natural Language search tool for S... by T-A on Thursday June 01, 2023 @06:35PM (#36158369)

The Internal State of an LLM Knows When its Lying

https://arxiv.org/abs/2304.13734

[ Parent | on HN ]

Re:Show HN: StonksGPT – A Natural Language search tool for S... by yreg on Thursday June 01, 2023 @07:49PM (#36159028)

It's not surprising that this information is somehow saved in the internal state, but it is surprising that they were able to read it to some level.

I don't think the model can access this information, but an external "lie detector" would be interesting. Thanks for the paper.

[ Parent | on HN ]

Re:Show HN: StonksGPT – A Natural Language search tool for S... by shreyshnaccount on Thursday June 01, 2023 @10:57PM (#36160315)

that seems wrong.. don't llms runs on probability distributions? Won't they have confidence built in?

[ Parent | on HN ]

Re:Show HN: StonksGPT – A Natural Language search tool for S... by yreg on Friday June 02, 2023 @03:34AM (#36162212)

I don't think probability of tokens can be mistaken for confidence of knowledge being correct. I'd take it more as a fitness whether the tokens fit naturally in the sentence.

[ Parent | on HN ]

Re:Show HN: StonksGPT – A Natural Language search tool for S... by shreyshnaccount on Friday June 02, 2023 @08:37AM (#36164267)

Makes sense, but i wonder if there's some way to retrofit that functionality onto it

[ Parent | on HN ]