News for hackers. Stuff that matters.
Re:Some argue that synthetic data can make AI systems better by WalterBright on Friday February 18, 2022 @05:17PM (#30391859)
If a computer program is generating the training data, aren't you just training the AI to do the same thing as the already existing computer program does?
Re:Some argue that synthetic data can make AI systems better by csee on Friday February 18, 2022 @06:09PM (#30392423)
Not at all. The existence of the Gran Turismo game (a simulator) is not the same thing as an AI that can play Gran Turismo.
Re:Some argue that synthetic data can make AI systems better by stingraycharles on Friday February 18, 2022 @06:35PM (#30392677)
But a better analogy would be if an AI generated a computer game that another computer can learn to play.

In the end, it’s more like an anonymization layer than anything. If a computer is trained to generate input data for other computers to train with, there’s not a lot special going on.

Re:Some argue that synthetic data can make AI systems better by manmal on Friday February 18, 2022 @07:00PM (#30392901)
Generating an environment and acting within it are wildly different things. Eg Tesla generates virtual camera footage of traffic situations they want their vehicles to handle correctly. The footage generator is basically a scripted video game director, while the trained AI is one of the most complex software projects ever.
Re:Some argue that synthetic data can make AI systems better by csee on Friday February 18, 2022 @07:06PM (#30392944)

  "But a better analogy would be if an AI generated a computer game that another computer can learn to play."
Doesn't the latter AI have a policy which contains novel information that does not exist in the former AI?

Even if what you say is true in some abstract information theory sense (and I would question that), there is a world of practical difference in the usefulness of a trained self-driving AI and the game engine within which that AI functions.

Re:Some argue that synthetic data can make AI systems better by taeric on Saturday February 19, 2022 @12:23AM (#30394637)
But.... if you were to train a model on the simulator of the game, you would expect it to pick up on the rules programmed into the simulator.

This is really no different than it picking on the rules embedded in the data gathering of real world data. Any implicit and hidden decisions in that space would be expected to find their ways into the ML.

Re:Some argue that synthetic data can make AI systems better by redytedy on Saturday February 19, 2022 @01:15AM (#30394818)
The tricky part is that the simulator itself may not have easy to understand rules. Waymo has a Neurips talk about training world agent models that are used for car behavior in the simulation itself. Trying to make world agents that are indistinguishable from real-world vehicle behavior (e.g. minimizing jenson-shannon curvature entropy) is a completely different task than training a model to safely transport you somewhere.
Re:Some argue that synthetic data can make AI systems better by taeric on Saturday February 19, 2022 @11:10AM (#30398127)
Right. That was my angle. Sometimes, they are ready to see after the fact. But, just like bias can enter into data collection, so too can it enter simulation.
Re:Some argue that synthetic data can make AI systems better by csee on Saturday February 19, 2022 @03:05AM (#30395270)
Yes but isn't the optimal policy novel information?
Re:Some argue that synthetic data can make AI systems better by taeric on Saturday February 19, 2022 @11:10AM (#30398132)
But it's it free of bias and a good execution policy in the real world?
Re:Some argue that synthetic data can make AI systems better by csee on Saturday February 19, 2022 @01:37PM (#30399327)
That probably depends on how realistic the simulator is. Your trade off is more bias (due to simulation inaccuracy) in exchange for unlimited data (and hence less variance).
Re:Some argue that synthetic data can make AI systems better by taeric on Saturday February 19, 2022 @02:05PM (#30399570)
Agreed. I could see a good case of training heavily on simulated data, verify/validate on collected data.

My question is ultimately how this really helps with making the system ethical. Just moved the bias from collection to simulation. And... I can't see simulation being less impacted with bias.

Re:Some argue that synthetic data can make AI systems better by csee on Sunday February 20, 2022 @07:01AM (#30405060)
Ethical is not the right question here, we simply want it to crash less and kill less people
Re:Some argue that synthetic data can make AI systems better by taeric on Sunday February 20, 2022 @11:02AM (#30406676)
The headline of the article brings in ethics.
Re:Some argue that synthetic data can make AI systems better by wolverine876 on Friday February 18, 2022 @06:28PM (#30392603)
I thought the same thing ...

Simulations are daily parts of life, in research and development from real-world simulations in research (e.g., climate) and industry (often in spreadsheets); to the theory of gravity (gravity simulated with mathematics) - and every other theory of science, social science, and humanities; to develping your iPhone app on your laptop or just reading the train schedule or using a mapping program.

So what is the difference? Those simulations were built from reality. The Theory of Gravity was built from and confirmed with empirical observations, not from someone else's simulation of gravity! That essential foundation of science, reality (it's not science otherwise), is what is missing.

Also, we already have the problem of our biases and preconceived notions infecting training data, and AI becoming a simulation of that rather than reality. By then training on 'simulated data' (yikes!), we seem to create more of a loop.

Re:Some argue that synthetic data can make AI systems better by rapjr9 on Sunday February 20, 2022 @12:50PM (#30407535)
If machine learning (ML) is trained on human behavior it can never be better (for some measure of better) than people are. So if racism is widespread, it will influence decisions trained into a ML algorithm. That raises the question, can we train an ML algorithm to make the decisions we _want_ rather than base its decision making on what people currently do. Training on generated data might be one way to do that, to build in implicit biases because we want more fairness in decision making than people currently exhibit. But then who gets to choose what those "goal" biases are? Someone could add a bias that improves life for people of male gender and worsens it for everyone else for example. This seems like a really important problem that has no clear answer. It's very much related to electing politicians where the choice of politician is also a choice of future goals. We don't know of any objective way to always decide what goals are best for everyone (voting certainly is not objective since many voters are not aware of all possible policy/goal implications and can be tricked into voting against their own interests). Yet it seems certain that researchers are working to introduce selective biases into algorithms, if just to adjust for known biases that are problematic. As an opaque input into an opaque algorithm, biases become invisible in deployments and could become very difficult to reverse or fix later. Even with continuous learning, if people's behavior is the input, the result will never be better than people are currently, which can create problems for the future. Some kind of intentional bias aimed towards goals seems necessary, yet also seems very dangerous since it can introduce biases decided by a very small set of people.
Re:Some argue that synthetic data can make AI systems better by wolverine876 on Monday February 21, 2022 @10:53PM (#30423823)
Excellent points. It's human beings all the way down.
Re:Some argue that synthetic data can make AI systems better by RicoElectrico on Friday February 18, 2022 @06:33PM (#30392649)
The fact that you can compute y = f(x) doesn't imply you already know x = f⁻¹(y).
Re:Some argue that synthetic data can make AI systems better by WalterBright on Friday February 18, 2022 @07:38PM (#30393232)
If you want an AI that can recognize cats, and train it with computer generated pictures of cats, you may wind up with an AI that only recognizes cat pictures from that particular generator.
Re:Some argue that synthetic data can make AI systems better by dmabuf on Friday February 18, 2022 @08:47PM (#30393656)
This tradeoff is known as synthetic domain shift and is still an active area of research. https://paperswithcode.com/task/synthetic-to-real-translatio...
Re:Some argue that synthetic data can make AI systems better by tomp on Friday February 18, 2022 @05:38PM (#30392089)
No. We had realistic 3D graphics 20 years ago.

I'm not aware of any true 3D computer vision system that could reliably play those games (from just vision).

Re:Some argue that synthetic data can make AI systems better by ynfnehf on Friday February 18, 2022 @05:37PM (#30392083)
You can train an AI to do the inverse of the existing program (as is the case for the self-driving described in the article.) Take some input, generate output using the existing program, and then train the AI with the input/output reversed.
Re:Some argue that synthetic data can make AI systems better by btdmaster on Friday February 18, 2022 @05:36PM (#30392063)
Re:Some argue that synthetic data can make AI systems better by ausbah on Friday February 18, 2022 @05:31PM (#30392008)
at least in the case of reinforcement learning, no. just because you can simulate the problem doesn't mean you know how to optimally solve it - ex driving a car
Re:Some argue that synthetic data can make AI systems better by renewiltord on Friday February 18, 2022 @06:50PM (#30392811)
An obvious counterexample. I was young in 2005 when Tesseract was open sourced. I wanted to use it to do something. But I decided to trial it out first by writing something in Notepad and then screenshotting it and trying. Synthetic data! But no, I didn’t know how to make the existing computer program turn image into text.
Re:Some argue that synthetic data can make AI systems better by 8note on Friday February 18, 2022 @06:18PM (#30392512)
as a contrast to the other comments, you will still get biases from where the simulation differs from reality.

eg. if your simulated traffic lights dont blink at 60hz, the model trained on it wont know to handle it

Re:Some argue that synthetic data can make AI systems better by throwawaynay on Friday February 18, 2022 @05:22PM (#30391907)
Generating a realistic city and a self driving AI are two wildly different tasks
Re:Some argue that synthetic data can make AI systems better by SomewhatLikely on Friday February 18, 2022 @05:30PM (#30391998)
Even a simpler task like image classification such as: does the picture contain a lion. Imagine you have 3d model of a lion. You can render it from lots of different angles, lighting conditions, backgrounds, stretched out, curled up, etc. You know the ground truth classification on all renderings is that the picture contains a lion, but being able to generate images of lions is a very different task from recognizing lions in images.
Re:Some argue that synthetic data can make AI systems better by hahajk on Friday February 18, 2022 @06:48PM (#30392797)
The potential issue with using synthetic data to simulate the problem (like image classification) is that recognizing a lion in synthetic imagery and recognizing a lion in real imagery may also be very different tasks to a computer.
Re:Some argue that synthetic data can make AI systems better by throwawaynay on Saturday February 19, 2022 @03:41AM (#30395460)
not that much actually because the image are handled at a lower resolution and with a smaller set of colors, to make the models train faster(a lot faster)

so they look pretty much the same to a computer vision algorithm