Re:OpenAI Vulnerability: 48 Days, No Response
by JyB on Tuesday July 15, 2025 @08:24PM
(#44577393)
I don’t see anything here that would prevent a LLM from generating these. Right?
[ Parent
| on HN ]
Re:OpenAI Vulnerability: 48 Days, No Response
by requilence on Tuesday July 15, 2025 @08:50PM
(#44577572)
In one of the responses, it provided the financial analysis of a not well-known company with a non-Latin name located in a small country. I found this company; it is real and numbers in the response are real. When I asked my ChatGPT to provide a financial report for this company without using web tools, it responded: `Unfortunately, I don’t have specific financial statements for “xxx” for 2021 and 2022 in my training data, and since you’ve asked not to use web search, I can’t pull them live.`.
Re:OpenAI Vulnerability: 48 Days, No Response
by Sebguer on Tuesday July 15, 2025 @09:08PM
(#44577696)
Do you understand what a hallucination is?
Re:OpenAI Vulnerability: 48 Days, No Response
by jojobas on Tuesday July 15, 2025 @10:21PM
(#44578066)
Coming up with accurate financial data that you can't get it to report outright doesn't seem like one.
Re:OpenAI Vulnerability: 48 Days, No Response
by Sebguer on Tuesday July 15, 2025 @10:50PM
(#44578213)
Models do not possess awareness of their training data. Also you are taking at face value that it is "accurate".
Re:OpenAI Vulnerability: 48 Days, No Response
by refulgentis on Tuesday July 15, 2025 @10:25PM
(#44578086)
I don't understand the wording
Accurate financial data?
How do we know?
What does using not-web-search not having the data have to do with the claim that private chats with the data are being leaked?
Re:OpenAI Vulnerability: 48 Days, No Response
by 01HNNWZ0MV43FF on Tuesday July 15, 2025 @11:47PM
(#44578502)
> I found this company; it is real and numbers in the response are real.
???
Re:OpenAI Vulnerability: 48 Days, No Response
by refulgentis on Wednesday July 16, 2025 @12:37AM
(#44578713)
Which of my questions does that answer?
Re:OpenAI Vulnerability: 48 Days, No Response
by queenkjuul on Wednesday July 16, 2025 @04:32AM
(#44579987)
That the financial data is accurate?
Re:OpenAI Vulnerability: 48 Days, No Response
by refulgentis on Wednesday July 16, 2025 @09:42AM
(#44582290)
It's an ourobos - he can't verify it's real! If he can, its online and available by search.
Re:OpenAI Vulnerability: 48 Days, No Response
by JyB on Wednesday July 16, 2025 @07:36PM
(#44588018)
Therefore what are the odds that this is just the LLM doing its thing versus "a vulnerability".
Seem like a pretty obvious bet.
Re:OpenAI Vulnerability: 48 Days, No Response
by BoiledCabbage on Wednesday July 16, 2025 @10:18AM
(#44582665)
> numbers in the response are real.
OpenAI very well may have a bug, but I'm not clear on this part. How do you know the numbers are real?
I understand you know the name is the company is real, but how do you know the numbers are real?
It's way may than anyone should need to do, but the only way I can see someone knowing this is contacting the owners is the company.