News for hackers. Stuff that matters.
Re:Tesla FSD Timeline by mg on Wednesday December 13, 2023 @06:39AM (#38625869)
What I would find really interesting: If someone drove the same route with every version of FSD and counted the interventions.

That would give a nice chart over time from which one could see the progress.

Is there anything online similar to this?

Another way to look at it would be to look at the revenue that is generated via FSD. Buyers probably do some due dilligence what FSD offers. If the number of buyers goes up, that could be an indicator that FSD is getting better. But a quick Google search did not bring up much data about this published by Tesla. In their earnings report, there is a chart showing the cumulative miles driven with FSD. Not sure if one can draw conclusions from that. It seems very smoothed out and only goes back to 2021.

Re:Tesla FSD Timeline by xendipity on Wednesday December 13, 2023 @06:46AM (#38625920)
A YouTuber named DirtyTesla has done a series of drives over the years throughout his hometown where he's tracked the intervention and disengagement rates per mile. He often shows those numbers at the end of his videos.
Re:Tesla FSD Timeline by mg on Wednesday December 13, 2023 @06:59AM (#38626075)
Is it the same route every time?

If not, they might take harder and harder routes as FSD gets better. Making it look like there is less progress than there really is.

Re:Tesla FSD Timeline by buro9 on Wednesday December 13, 2023 @07:08AM (#38626187)
You just know that Tesla would tune the software for this specific route, if the value of the comparison number became meaningful.

Just like you cannot trust benchmarks, Goodhart's law will kick in https://en.wikipedia.org/wiki/Goodhart%27s_law

> Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes

Re:Tesla FSD Timeline by mensetmanusman on Wednesday December 13, 2023 @08:48AM (#38627279)
Their model architecture don't allow for individual route tuning.
Re:Tesla FSD Timeline by krisoft on Wednesday December 13, 2023 @10:23AM (#38628690)
I think that is not true. They can just take data from the route, annotate it and put it into their training set. That is how you would do route tuning.
Re:Tesla FSD Timeline by mlrtime on Wednesday December 13, 2023 @09:13AM (#38627623)
Sort of off-topic but I believe Openpilot allows this. You can record every route you take, copy it off to your PC and run it in a simulator. You can then tweak the model for this particular issue.
Re:Tesla FSD Timeline by tsimionescu on Wednesday December 13, 2023 @09:00AM (#38627442)
They claim.
Re:Tesla FSD Timeline by numpad0 on Wednesday December 13, 2023 @07:37AM (#38626463)
There is no publicly known archive of Tesla software images, at all. Which, with the benefit of hindsight, do sound wrong.
Re:Tesla FSD Timeline by jjallen on Wednesday December 13, 2023 @06:52AM (#38625990)
Would have to drive many times over different times of day, days of week, seasons and weather conditions for them to be comparable.
Re:Tesla FSD Timeline by simion314 on Wednesday December 13, 2023 @07:00AM (#38626089)
That would depend on traffic, like it could perfectly work for you 100 times and the 101 time when a truck with some weird painting on it is present on the road and it will just hit it, or because of day of the year the shadows are different.

Tesla keeps the true data secret and cherry pick the results they want. Would be inserting to data mine that data, like

- are there roads where humans always (most of the time ) take over

- how many near accidents are caused by FSD/autopilot

- are there time of day variations

- are there weather variations

- are there issues after software updates

- what are the good roads and bad roads see if we can identify what is special about them.

Probably there could be more the public would like to know if the data would be made public, and IMO makes no sense to keep this public safety data secret.