News for hackers. Stuff that matters.
Re:Show HN: IngestAI – NoCode ChatGPT-bot creator from your ... by alnorth on Thursday February 23, 2023 @08:35AM (#34910355)
So if my competitor is using IngestAI and OpenAI use their data to train ChatGPT, could I literally just ask ChatGPT to tell me some secrets from my competitor's internal communication?
Re:Show HN: IngestAI – NoCode ChatGPT-bot creator from your ... by literalAardvark on Thursday February 23, 2023 @08:46AM (#34910480)
2023 is going to be very exciting times for security engineers.

It won't have the data, but it might have enough of an understanding of the data to leak important information.

Re:Show HN: IngestAI – NoCode ChatGPT-bot creator from your ... by vidarh on Thursday February 23, 2023 @09:34AM (#34911054)
While the model clearly can't retain all data, ChatGPT can regurgitate a lot of stuff verbatim.

Prompt:

> Recite the first two paragraphs of Neuromancer.

Response:

> Certainly! Here are the first two paragraphs of "Neuromancer" by William Gibson:

> "The sky above the port was the color of television, tuned to a dead channel.

> 'It's not like I'm using,' Case heard someone say, as he shouldered his way through the crowd around the door of the Chat. 'It's like my body's developed this massive drug deficiency.' It was a Sprawl voice and a Sprawl joke. The Chatsubo was a bar for professional expatriates; you could drink there for a week and never hear two words in Japanese."

(I have not checked how far you can get it to continue)

So perhaps it'll be a question of whether enough of your employees are feeding it copies of your data for it to retain it...

Re:Show HN: IngestAI – NoCode ChatGPT-bot creator from your ... by LeanderK on Thursday February 23, 2023 @09:41AM (#34911148)
I bet that getting the right prompts won't be easy so it will probably fly under the radar and not immediately be detected. You can't search these weights with command-f. Fun times ahead...
Re:Show HN: IngestAI – NoCode ChatGPT-bot creator from your ... by Jensson on Thursday February 23, 2023 @10:43AM (#34911982)
And good luck trying to add data to it without corrupting some other data it has encoded.
Re:Show HN: IngestAI – NoCode ChatGPT-bot creator from your ... by nextaccountic on Friday February 24, 2023 @12:49AM (#34921317)
Prompt engineering is only getting better. Also,

> You can't search these weights with command-f

Sometimes you can, https://clementneo.com/posts/2023/02/11/we-found-an-neuron

Re:Show HN: IngestAI – NoCode ChatGPT-bot creator from your ... by osigurdson on Thursday February 23, 2023 @08:56AM (#34910609)
Does this problem disappear when using the Azure version of the service? If not, this is a pretty obvious market need: LLM + privacy.
Re:Show HN: IngestAI – NoCode ChatGPT-bot creator from your ... by flangola7 on Thursday February 23, 2023 @02:33PM (#34915258)
Must companies won't want to do that. They want to know if someone is using their service to instigate the next mass shooting or government genocide.
Re:Show HN: IngestAI – NoCode ChatGPT-bot creator from your ... by Vasyl_R on Thursday February 23, 2023 @02:38PM (#34915342)
a good point. And what about companies that have on-premise storage?
Re:Show HN: IngestAI – NoCode ChatGPT-bot creator from your ... by zitterbewegung on Thursday February 23, 2023 @10:19AM (#34911667)
If a server was misconfigured OpenAI could have been trained on non public information. You can also poison OpenAI's dataset if you know it has been pulled by the service.
Re:Show HN: IngestAI – NoCode ChatGPT-bot creator from your ... by bluejay2387 on Thursday February 23, 2023 @01:40PM (#34914590)
This is a well known problem with this technology (although I haven't seen an official term for it, so we have been calling them "recovery attacks'). It's apparently the reason companies like Amazon have banned internal use of services like ChatGPT.
Re:Show HN: IngestAI – NoCode ChatGPT-bot creator from your ... by Vasyl_R on Thursday February 23, 2023 @01:50PM (#34914695)
thanks for sharing. Do you think if all would be on the client's local server or cloud there would be still some, even rare, occurrence of that?
Re:Show HN: IngestAI – NoCode ChatGPT-bot creator from your ... by bluejay2387 on Friday February 24, 2023 @12:15PM (#34926904)
The problem could still occur but you would have to be capturing all the queries to your internal LLM systems and then using that data for training. You have complete control of the model so you could just choose not to do that and I would think data leaks of this nature would be less of a concern for an internal environment anyway. You would know that only authorized individuals would have access to the data. I suppose there could still be a very small chance of leaking data to unauthorized employees, but if a rogue employee wants to access data they should not have access to fishing an LLM would probably be the least productive way to do that. Your access logs for the LLM system would clearly display the attempts.

Some commercial services are starting to offer "Enterprise" licenses that prohibit the collection and use for training of your data and that would address the concern as well.