Yeah, I think it's safe to assume this. One time as a contractor, I was working on a GIS-enabled game for some company. Their privacy policy was very clear that they will not use or sell your data for any reason. It was even on their home page, because it was a key selling point. By the time I was involved with this app, the money was running out, and I vividly remember being in a meeting with the leadership where they were enumerating their options, one of which was to find someone to buy all this user data they'd collected. Their commitment to privacy disappeared the instant it was tested. Ever since then, I just assume this is how every company works. No matter what they say, they're selling your data.

That's a fair assumption in my experience from running companies. I've seen management with ethics, but even then, eventually new people are in charge, shares (or the whole thing) get sold... That's what contracts are for, commitments from a company are commitments from people that might not be around very long.

Most EULA contracts have a clause that expressly permits the company to abandon their obligations upon sale of the business or other such events. However, that seems somewhat redundant, since these same contracts also assert that they can modify the terms whenever they want. For the average user, there's little to no recourse when the original contract gets violated.

Courts seriously need to crack down on these EULAs

This suggests that we require an explicit law that forbids changes in ToS from ever apply to data collected retroactively. With a real monetary punitive incentive that allows users to sue.

I absolutely believe that companies should have the freedom to change their ToS moving forwards, and that "promises" to "never" change a ToS are worthless (your example is a perfect reason why).

BUT, I simply don't see how it could ever be fair to use data retroactively. If you change your ToS, you should only get to monetize new user data going forwards. It seems like a basic legal principle.

Is there any existing law/precedent that suggests this is already the case, i.e. that such a company can be successfully sued but that people don't usually try? Or do we need new laws around this, and are there any government reps pushing for this?

Sounds good in principle because you're worked up and mad about your data (which is the correct reaction).

I just don't love the idea that my dinky little app has to ask every customer every time I add a new feature significant enough (debatable) or different enough (debatable) that uses their data in a way either I or they didn't anticipate (debatable). God forbid I try to monetize it (debatable). 'Control over your data' is meaningless in our current paradigm and I'll rue the day something like GDPR comes to the US in a meaningful way. No wonder the EU can't build.

As for this specific article, Zoom's (rightfully) getting heat for this but I don't blame them or any company for exploring how they can monetize every last morsel of data. In zoom's case (and many enterprise software companies), customers are paying a shit load of money and they didn't sign a contract and consent to give data for training an LLM.

> I don't blame them or any company for exploring how they can monetize every last morsel of data.

I absolutely do, if it's customer data that the company previously promised not to monetize. It's not their data to do with as they please, after all.

But the tech sector has fallen very far in terms of ethics so no company can be trusted. It's just a shame. The public views our industry in a very, very poor light and that view is 100% earned.

IANAL but it seems to me that this "law" already exists.

A TOS is a contract. It literally stands for "Terms of Service." Meaning, you give me money and here are the terms under which I will offer you the service you are paying for. How enforceable that "contract" is depends on a ton of things, differing in various jurisdictions (law is complicated), but it is - at the end of the day - a contract.

So I don't know how actionable it is, but the OP said that the company considered changing their TOS for currently active users. That could, in theory, be breach of contract and the customers might have a claim (again IANAL).

[There could have also been a clause in the TOS saying that they could change the terms at any time for any reason - though I suspect in many if not most jurisdictions, that would make the entire contract unenforceable].

In your case, don't make [potentially] contractually binding promises that you can't or don't want to keep.

> No wonder the EU can't build.

That was unnecessary.

> I don't blame them or any company for exploring how they can monetize every last morsel of data.

That's how a company works: try to do everything they legally can to make as much money as they can. Society has to decide of the framework into which companies optimize, and that is materialized with laws that the companies must follow. In the EU, there is a tendency to believe that users have a right to some kind of privacy.

Of course, this constrains what companies can do, and you could say "no wonder the EU can't build". I just call that cultural differences. In most countries in the EU, people don't have to start a crowdfunding campaign when they go to the hospital, because they actually have some kind of social security. I am all for GDPR.

> That's how a company works: try to do everything they legally can to make as much money as they can.

No, companies don’t need to be like that. This is a meme that needs to die. Companies can have a set of values (principles) and act according to those principles. Any investors can be told ahead of time the principles by which the company operates, and if they don’t want to buy stock on that basis, they’re welcome to stay out.

Bryan Cantrill has had some excellent rants about this over the years. Eg: https://youtu.be/bNfAAQUQ_54 . His take is that money for a company is like fuel in a car. You don’t go for a road trip (start a company) because you want to get more fuel. You go because there’s some place you want to get to. And fuel (money) is something you need along the way to make your journey possible.

Don’t let sociopathic assholes off the hook. They aren’t forced to be like that. They’re choosing to abandon their ethics and common decency. Everyone would be better off if this sort of behaviour wasn’t tolerated.

> No, companies don’t need to be like that.

Well, they don't need to. But the people at the top make more money if they are. And they are not at the top because they have principles: they are at the top because they want power or money.

> Companies can have a set of values (principles) and act according to those principles.

I would love it, but I just can't buy it. Like at all. How many big companies do you know where the executives don't get a much higher salary than the employees? Humans can't help it: if they are in a position of power, they will think they are worth more.

> Any investors can be told ahead of time the principles

IMO, if you have principles, you are not an investor. And investors want to get ROI, which is more likely from companies that don't have principles.

> His take is that money for a company is like fuel in a car.

Sounds exceedingly naive to me :-). The driver does not get fuel at the end of every month.

> Everyone would be better off if this sort of behaviour wasn’t tolerated.

Yes. We need laws, set by the society. We need the people to understand that they will never be one of those rich executives, and to vote for laws that prevent them to become indecently rich.

But your app doesn't have to do that.

The TOS are generally broad enough from the start that you can do anything you want with user data as is necessary to provide product features. Nobody's updating TOS every time they add a new feature.

Realistically, this is specifically about situations around selling data to third parties, and/or training for AI that is not related to product features. (There's a big difference between Zoom using chats for building LLM's, versus Google training on Gmail messages to build Gmail autocomplete.)

You don't need to ask about adding features, just put in the ToS that the data will be used for app features and metrics for improving user experience.

Monetization by adding paid features falls well within those boundaries. Monetization by selling user data to whomever will buy it does not.

I'd really love to have a GDPR specifically for people like you who feel entitled to do whatever they want with collected data. I'd love to have had it when reddit decided to charge outrageous prices for the API.

Yep. Their backtracking is just the part of this that we see. The initial decision was likely driven by many stakeholders who deeply believe this is a key advantage to acquire. Users are fighting an uphill battle, and no amount of pushback short of lost revenue will stop the forces at play.

Then it is time to take control by seeing such databases with false information. We could probably design data specifically to attract an AI system looking for well-documented data. I wonder how images of Steven Colbert uploaded to imgur it would take to convince the AI that he actually was.

I don’t think this is a possible approach. It would basically mean needing to feed in so much noise that a human wouldn’t be able discern reality from fiction given no priors. Modern ML is too smart

Given how easy it is to keep an average human from discerning reality (see the last decade of boomers/gen xers on Facebook as an example) and the massive potential to create slight variations with LLMs, I don't think a Stephenson-style misinformation propagation campaign is that far outside the bounds of possibility.

That's a lot of compute power to waste on it, but I would guess that that's what bot networks are going to be used for in the future (or already are, right now, if they're done mining bitcoin).

AKA encryption.

That's why I said president of canada. A non-intelligent AI would not find any priors indicating who was the president. There isn't one. So a few thousand images in the database, against zero information to the contrary, might be enough.

Our enterprise issued an ultimatum recently requiring no software outside of our MS contract. Which is going to be severely limiting, but painfully doable. This move really seems to be in reaction to the Zoom debacle.

Now we'll be forced to use Teams for online trainings after pretty much universally using Zoom since the pandemic. Our customers are gonna love that.

It's above my pay grade but I wonder if we've already signed our data over to MS for a certain price, with that stipulation.

Or they'll do it without updating the ToS. Nothing's stopping them.

Adults realize other adults do what benefits them.

The GDPR is stopping them.

But is it even enforced? Take software licenses. Most software out there uses open-source licensed dependencies. Even the permissive licenses require attribution.

But most software does not honor those licenses, and nobody cares. Enforcing such a law takes money.

I guess at least the GDPR can be enforced, to some extent, with Big Data. It seems like the fines are usually ridiculously low (they don't seem like an intensive for the company to change anything), but that's better than nothing.

It's not very well enforced but the fines are large, and big companies like Zoom will have suitably paranoid legal departments ensuring that it is followed.

I mean, if they have a bunch of enterprise customers there’s a fairly strong disincentive; any sensible corporate customer, if they did this, would presumably say, right, cool, MS Teams time.

I'm having trouble parsing your comment. Are you saying MS Teams is more or less likely to use your data to train AI?

I'm assuming it's safe because they make a big deal about compliance. But on the other hand MS have a huge incentive to obtain data for AI since they are going all in on it.

Microsoft is unlikely to _secretly_ do it, certainly; they would be sued into oblivion, and that’s before the EC gets to them (for GDPR enforcement grinds slow but exceeding fine, and also with exceedingly big fines). US regulators might also take an interest.

Also, modern Microsoft’s _whole thing_, more or less, is “we are your trusted enterprise partner who definitely won’t do bad stuff with the data you put in our cloud services”. They are unlikely to throw that away for a bit of flavour-of-the-month AI boosterism. Note that they’ve recently released a private ChatGPT thing; they can’t credibly acknowledge the problem with one hand and exploit it with the other.

I don't think that holds up.

Paying customers absolutely HATE the idea of their data being used to train AI models without their permission - this Zoom story is just the latest example of that.

Companies that try to do this will get burned. Zoom just got burned really badly, and I personally don't think they actually intended to even do this - they just didn't make it clear enough that they were NOT going to do it, which sparked a PR nightmare firestorm for them.

I think the incentives for companies are very much the other way round: if paying customers hate this, then the incentives are NOT to do it.

I promise not to use any of the data I collect about you!

Except to improve services (ML training!), Advertising (We'll sell your data to advertisers!), and by government order (pick your favorite three letter agency.)

Alternatively: They'll just use your data anyway and not tell you about it. How is anyone going to prove their data was used as the source? It's like all the NDAs people sign when they join a company and they pinky promise not to use it at the next job where they land a big fat raise and promotion... suuure they aren't going to take what they've learned and improve upon it to try and get more promotions and raises in the future.

It can't be stopped.

> It can't be stopped.

Maybe. Maybe not. Hard to tell without trying.

I applaud the EU and California for giving it a go with their data protection laws. I really hope their crackdown on this stuff is effective.

> It's like all the NDAs people sign when they join a company and they pinky promise not to use it at the next job where they land a big fat raise and promotion

Uh no. An NDA would cover proprietary intellectual property, not tools everyone else also uses. Unless you're now working for the previous employer's competitor, it's unlikely that proprietary tech would ever be used. Working for competitors and partners is usually also forbidden for some period of time after leaving.

> How is anyone going to prove their data was used as the source?

1) whistleblowing 2) compliance audits (think soc2)

How would a soc2 audit uncover that?

They're asking for evidence that your admin accounts are reviewed for permissions needed each quarter, that you are doing your DR tests, that you are following documented change management processes, etc, not asking to look at what your running containers are doing.

There is absolutely no way that any compliance audit I can think of would or would even attempt to uncover that kind of info.

It will become a process of boiling the frog. Companies that are in the business of renting access to your data will get increasingly clever at moving in this direction in small incremental steps, providing some user benefit along the way.

We can agree that Zoom did a terrible job of rolling out their new terms, regardless of what their intention was. What other companies will learn from this is to improve the roll out.

Once local/private inference becomes more viable, there will be even more of an incentive for the companies who store unencrypted data to use it as a competitive advantage.

This. I wouldn't be surprised if people give up after a while of searching and migrating to alternatives to each service that started using or selling data for AI training.

ehhhh I think some customers would be ok with it if it meant discounts. If your software is good enough, you can tell customers to go elsewhere. The theory is that AI done well can be incredibly powerful ($$$) so you have to build it or risk being left behind.

Sure, consumers are likely to say yes to that - but I'm talking about companies here who are much more sensitive to what happens to their data.

> Paying customers absolutely HATE the idea of their data being used to train AI models without their permission

Paying customers will be the ones to resist for the longest maybe but that frog too will be boiled eventually.

no need to even update the ToS

just call it "fair use", like OpenAI and GitHub

Yeah. And then regulars will argue for you that changing identifiers but otherwise copy and pasting code is perfectly cool fair use regardless of your codes licensing (I do not license any of my code for free corporate use, for example, but apparently, as long as chat GPT outputs the exact copy and paste of my code with a few things slightly changed, it’s cool).

I mean, around now would be a fairly convenient time for giant companies to change their mind about copyright. After all, that creative commons drawing and GPL code are only protected because of copyright, correct?

>a fairly convenient time for giant companies to change their mind about copyright

but then they would not be able to do this https://news.ycombinator.com/item?id=37100140

I am speaking tongue in cheek of course, if they weren't themselves protected at all it would not work out for them. They couldn't own the distribution system well enough to keep the cat in the bag.

But I can see them being okay with bringing copyright back to 10 years or something, they make most of the money from publishing a new television series early on. If they made it ten years, paying for streaming old television series doesn't require paying the copyright owners anything so it's pure profit for the streaming service.

Of course people can go elsewhere, but if they can only get things made in the last 10 years on some particular service, they'd just use the same to view old things... and pay the website without the creators getting anything.

So every distributor wins because they can all offer the older material as a pure profit for themselves and an enticement to join the service (on top of the actual unique material from the last 10 years).

Legislation should step in to forbid applying new or changed ToS to data collected before that ToS was explicitly accepted by a user. Would be a PITA for the business (as they now have to track under what ToS version each piece of data is stored under), but hopefully enough of a PITA to make them treat that data as the liability it ought to be.

Sounds almost reminiscent of Jamie Zawinski's Law of Software Envelopment: "Every program attempts to expand until it can read mail. Those programs which cannot so expand are replaced by ones which can."

Huh? Barely any programs on my computer can read mail. That’s a silly law.

It's probably outdated. A modern version might be "every program expands until it becomes a platform with instant messaging capabilities and an app store..."

I think most user data will be completely useless for AI training. No context, no quality control. Even assuming wisdom of crowds and on average you get the correct answer, you are training an AI to give a mediocre response. Back in the days of Big Data hype, many companies decided to store everything because data was valuable. But in most cases it wasn't worth the cost of the hard drives, because it only has value if there is a buyer willing to pay the cost of making use of it. There is a lot of data out there, but it is information and knowledge that has value.

Unlike consumers, many enterprises really care about data policies. Procurement, compliance, legal and audit teams inside enterprises often have the power to block purchases and contract renewals. Many of those departments actually read ToS too.

This makes it difficult for B2B companies like Zoom to use customer data for AI training.