So if my competitor is using IngestAI and OpenAI use their data to train ChatGPT, could I literally just ask ChatGPT to tell me some secrets from my competitor's internal communication?

2023 is going to be very exciting times for security engineers.

It won't have the data, but it might have enough of an understanding of the data to leak important information.

While the model clearly can't retain all data, ChatGPT can regurgitate a lot of stuff verbatim.

Prompt:

> Recite the first two paragraphs of Neuromancer.

Response:

> Certainly! Here are the first two paragraphs of "Neuromancer" by William Gibson:

> "The sky above the port was the color of television, tuned to a dead channel.

> 'It's not like I'm using,' Case heard someone say, as he shouldered his way through the crowd around the door of the Chat. 'It's like my body's developed this massive drug deficiency.' It was a Sprawl voice and a Sprawl joke. The Chatsubo was a bar for professional expatriates; you could drink there for a week and never hear two words in Japanese."

(I have not checked how far you can get it to continue)

So perhaps it'll be a question of whether enough of your employees are feeding it copies of your data for it to retain it...

I bet that getting the right prompts won't be easy so it will probably fly under the radar and not immediately be detected. You can't search these weights with command-f. Fun times ahead...

And good luck trying to add data to it without corrupting some other data it has encoded.

Prompt engineering is only getting better. Also,

> You can't search these weights with command-f

Sometimes you can, https://clementneo.com/posts/2023/02/11/we-found-an-neuron

Does this problem disappear when using the Azure version of the service? If not, this is a pretty obvious market need: LLM + privacy.

Must companies won't want to do that. They want to know if someone is using their service to instigate the next mass shooting or government genocide.

a good point. And what about companies that have on-premise storage?

If a server was misconfigured OpenAI could have been trained on non public information. You can also poison OpenAI's dataset if you know it has been pulled by the service.

This is a well known problem with this technology (although I haven't seen an official term for it, so we have been calling them "recovery attacks'). It's apparently the reason companies like Amazon have banned internal use of services like ChatGPT.

thanks for sharing. Do you think if all would be on the client's local server or cloud there would be still some, even rare, occurrence of that?

The problem could still occur but you would have to be capturing all the queries to your internal LLM systems and then using that data for training. You have complete control of the model so you could just choose not to do that and I would think data leaks of this nature would be less of a concern for an internal environment anyway. You would know that only authorized individuals would have access to the data. I suppose there could still be a very small chance of leaking data to unauthorized employees, but if a rogue employee wants to access data they should not have access to fishing an LLM would probably be the least productive way to do that. Your access logs for the LLM system would clearly display the attempts.

Some commercial services are starting to offer "Enterprise" licenses that prohibit the collection and use for training of your data and that would address the concern as well.

Don't fully disagree but the reason I would care less in this case is I assume most/all of what you would be feeding it is non-sensitive documentation.

I don't think that matters? Sensitive could be just discussing designs, outages, hiring, interview feedback?

There's a lot of stuff in the average Slack account people don't want on the internet, let alone in a LLM which will potential expose it to the entire world?

Who is feeding interview feedback to a third party regardless if its IngestAI or not?

If you send all of your slack communications to IngestAI, it would include possibly channels where you discuss interview feedback. That's what the parent poster is saying.

And I am saying that was never the intended purpose of this product from what I read. That was my whole point in my OP. I agree that there are unique issues with products like these but its not alone, at the end of the day you should not be feeding sensitive data to third party applications like this.

please see my comment in this threads. We 're ONLY learning from your knowledge base and not from your slack chat history.

Yup, thats what I was trying to say from the beginning and why I was so confused with all the concerns.

hey, we are not learning anything from your slack history or channels.

The way IngestAI works is it takes your knowledge base as the input (markdown, docs etc) and answers the queries asked by the user on these knowledge base. ur primary usecase has been to simply learn from public documentation of companies and help answer the queries within their Slack/Discord community.

Probably gonna get downvoted for this but people on HN take the Privacy Policy of brand new websites way too seriously.

This is probably some template they downloaded from the web, not some sly document they had their nefarious lawyers put together.

On the other hand, our company has recently mandated that all the software we use needs to have it's license checked over by a lawyer because we've found some fairly nefarious things in some of the software licenses we're using. Don't want to reveal where I work but let's say there's plenty of people working there who understand software licensing already, and some of the things were still missed until under a microscope.

So maybe it pays to be skeptical / cautious?

As a SaaS vendor I’ve had an uptick in one off requests from legal teams to make changes to our terms.

So far, nothing deal breaking (largely because there isn’t anything nefarious in our terms), but I wonder if this will be a trend moving forward.

thanks for your comment! Are the data privacy concerns the main SaaS challenge from your POV ?

No, so far it has been about limiting their liability in the event of something bad happening on the platform.

Every app using GPT will go through OpenAIs API and all that data is stored and used for training future models. No amount of good will or non nefarious intentions from IngestAI side makes any difference.

thanks for your comment and support. That's what I'm trying basically to say in every third comment here. Generally, data privacy is a buzz topic since some time and with OpenAI it became even more viral.. It's not about IngestAI or any other SaaS, it's much wider topic and totally agree (thanks!).

To be clear I'm saying that you claiming that data won't leak into 3rd parties by storing it in aws is deceptive. There's no way you can protect the user of your product from having their data being soaked up by openai as long as you use their service instead of an open source LLM.

thank you so much for your support! Hope there're more people that think like you and are a bit 'empathic' about new services.. We improve with every single client, with every single request, and issue we face. So yes- Data Privacy is something very important but it's not only about IngestAI, I believe. Most of the SaaS solutions use API and are on AWS.. So, again, it's a wide topic, a buzz topic now.. Thanks again)

> This is probably some template they downloaded from the web

But that's worse. Don't you see how that is worse?

They want to have access to all the knowledge of the company and they can't even articulate what will and won't they do with it?

> people on HN take the Privacy Policy of brand new websites way too seriously.

People should take Privacy Policies of every website they work with more seriously.

We will work to fix this

This is probably the output of asking an LLM to generate a privacy policy.

It seems like the companies who throw caution to the wind and use this will end up being more productive in the short term and win out over more cautious entities. Thus we will only end up with more companies who use these tools or holdouts caving to pressure from competition. This will cause big problems by the time post-nut clarity arrives. We are being dazzled by the tech and forgetting our principles again. We are being pulled down this path whether we want to or not.

It also doesn't help that OpenAI is partnered with Microsoft. I would start with the mindset that all data given to OpenAI through any of these tools goes to Microsoft. Why would you give anything to a competitior?

This is hard to take seriously with such a policy. I think this is just somebody playing around with the OpenAI API. I expect serious competitors to have a better policy that is actually usable for corporations, it might take a year but I see no way this gets adopted in a broader way. So far OpenAI has the monopoly, but there will be competitors if you will give them some time.

sure, BERT is to come very soon and I think it will bring even wider adoption of AI. And yes, Data Privacy will become even more buzzed topic. Next few years will be of security engineers and of data privacy lawyers, don't you think?

thanks for your comment. If believe if Data Privacy is almost the only topic discussed here, that's good. Means, people like the idea but at the same time people are concerned about personal / sensitive data. And to be honest- I hope that would be our biggest challenge, as I believe it's a bit easier to get a good, solid Data Privacy policy compliant with all legal and moral requirements than to build a product. At least that's what I hope))

Most startups are clueless and ignorant about data governance.

would you be able to advise a good startup or service that provides good governance of Data Privacy management for startups? We would like to learn more and get this point as good as possible.. But you're right in some degree- we're builders, and we need help of professionals with data governance.

Ok so, you replied everyone on this sub-thread except the top-level comment. Why is that?

Your first task to improve your privacy policy is to review whether you really, absolutely, for reals, can require OpenAI to follow this: "they are obligated not to disclose or use the information for any other purpose."

Because, it looks like you can't, and OpenAI will absolutely use your customers' data for their own purposes, so you probably should remove this line from your privacy policy at minimum.

By using your data as training data, OpenAI might disclose your stuff to the world at large