News for hackers. Stuff that matters.
Re:It's Still Funny That 1k People Gave Tesla $250k for a Ro... by ActorNightly on Wednesday November 29, 2023 @04:12AM (#38457208)
>All that is needed is to train them on enough examples of different situations that occur on the road.

The only real way this would work is if you build a pipeline where you have a simulated world with realistic physics, rendered through some pipeline to represent camera images (in relatively high fidelity) and/or lidar outputs, and teach the network to predict the evolution of these scenes, and do this with a shitload of random data

This is probably orders of magnitude more expensive to do than training GPT models, since for vision, you would have to render it with ratyracing.

Re:It's Still Funny That 1k People Gave Tesla $250k for a Ro... by ArtTimeInvestor on Wednesday November 29, 2023 @04:18AM (#38457248)
Human drivers are proof that this is not needed.
Re:It's Still Funny That 1k People Gave Tesla $250k for a Ro... by ActorNightly on Wednesday November 29, 2023 @05:12AM (#38457615)
I hate that companies in the vision self driving space like Tesla and Comma AI have made the act of driving seem like its trivial.

Human drivers drive with EXCEPTIONALLY more "software" under the hood. We simulate the world around us using learned rules across multiple domains. For example, if someone has never driven at night, but understand that the cars have tail lights that are red, we can deduce that pairs of red lights are cars on the road without ever being subjected to the visual of driving at night. Its this kind of processing that lets us be safe.

In the current form, ML models are way subpar to this. A superhuman driving agent should be able to drive through a construction site with cones and debris, or a busy parking lot, or a grass field full of logs. No model out there can do this. This is why you absolutely need simulations if you want to do it traditional way. Comma AI has gained quite a bit of success in training in a simulated environment that simulates deviations from straight line driving on real world video.

Re:It's Still Funny That 1k People Gave Tesla $250k for a Ro... by ArtTimeInvestor on Wednesday November 29, 2023 @05:32AM (#38457746)

    we can deduce that pairs of red lights are cars on the road
Generalization, abstraction, understanding. That is what neural networks are about. Current NN's are already very good at that. FSD constantly has to understand lighting situations it has not seen before.
Re:It's Still Funny That 1k People Gave Tesla $250k for a Ro... by ActorNightly on Wednesday November 29, 2023 @03:01PM (#38464465)
You still need training data.

Even if you had a simulation set up for training, and enough compute on the vehicle to run something like MuZero real time inference for self driving, you would still run into the problem of humans setting up all the possible driving scenarios, which will likely leave some out and the model will never learn the right actions for those - meanwhile, to any human, the action would be very obvious. Over time you could probably get close enough to a very very small error rate, but you would still be hesitant to trust the system.

A true superhuman driving agent would most likely be able to take a picture of a scene, and then give a prediction of evolution of object position in 3 dimensions for a given time window, and give a confidence score on that. For example, if the single image is from few on a highway, it should be able to predict that cars are in fact moving, because the chance of cars standing still on a highway is very low.

And to train a model like that, you most likely need a base model that can "understand" physics.

Additional images from the past or the future would generate prediction that is more accurate, and also improve the model self assessed confidence. And then you would have some heuristic algorithm like MCTS based on confidence levels on the best course of action.

Re:It's Still Funny That 1k People Gave Tesla $250k for a Ro... by bagels on Wednesday November 29, 2023 @05:29AM (#38457730)
It's not just objects, but it has to predict what people are doing, going to do, or are trying to communicate. For example, police redirecting traffic, a passenger having a medical emergency, other drivers driving erratically, etc.
Re:It's Still Funny That 1k People Gave Tesla $250k for a Ro... by chx on Wednesday November 29, 2023 @04:27AM (#38457304)
And current generative AI has jack all to do with humans.
Re:It's Still Funny That 1k People Gave Tesla $250k for a Ro... by amenhotep on Wednesday November 29, 2023 @05:31AM (#38457737)
Have you not seen the results that have been shown recently with Gaussian splatting? Photorealistic rendering looks like it'll become a hell of a lot cheaper than raytracing soon enough, and this might actually be a realistic idea.
Re:It's Still Funny That 1k People Gave Tesla $250k for a Ro... by ActorNightly on Wednesday November 29, 2023 @03:05PM (#38464530)
I dunno if statistical methods can be applied to generating photorealistic images from simulation, where you don't have enough information in a scene, since you are starting with 3d models.