I think also a critical weakness is that LLMs are trained on the code people write ... and our code doesn't annotate what was written by a human and what was suggested by a tool. In your analogy, this would be like if your sat nav system suggests that you turn right where other people have turned right ... because they were directed to turn by their sat nav.

In fact, I'm pretty sure this already happens and the results are exactly what you'd expect. Some of the "alternate routes" Google Maps has suggested for me in the past are almost certainly due to other people making unscheduled detours for gas or whatever, and the algorithm thinks "oh this random loop on a side street is popular, let's suggest it". And then anyone silly enough to follow the suggestion just adds more signal to the noise.

Long before any use of LLMs, OsmAnd would direct you, if you were driving past Palo Alto, to take a congested offramp to the onramp that faced it across the street. There is no earthly reason to do that; just staying on the freeway is faster and safer.

So it's not obvious to me that patently crazy directions must come from watching people's behavior. Something else is going on.

In Australia the routes seem to be overly influenced by truck drivers, at least out of the cities. Maps will recommend you take some odd town bypass when just going down Main Street is easier.

I imagine what you saw is some other frequent road users making choices that get ranked higher.

OsmAnd doesn't use traffic data. You can enable traffic map layers by feeding a reverse-engineered URL, though.

I'm not talking about use of traffic data. In the abstract, assuming you are the only person in the world who owns a car, that route would be a very bad recommendation. Safety concerns would be lower, but still, there's no reason you'd ever do that.

safety concerns would probably actually be higher since the most dangerous places on roads are areas where traffic crosses and conflicts (the road you cross to get from the offramp to onramp)

> if you were driving past Palo Alto, to take a congested offramp to the onramp that faced it across the street

If you're talking about that left turn into Alma with the long wait instead of going into the Stanford roundabout and then the overpass, it still does that.

I've seen this type of thing with OsmAnd too. My hypothesis is that someone messed up when drawing the map, and made the offramp an extension of the highway. But I haven't actually verified this.

Google Maps has some strange feedback loops. I frequently drive across the Bay Bridge to Delaware beaches. There are 2 or 3 roughly equal routes with everyone going to the same destination. Google will find a "shorter" route every 5 minutes. Naturally, Maps is smart enough to detect traffic, but not smart enough to equally distribute users to prevent it. It creates a traffic jam on route A, then tells all the users to use route B which causes a jam there, and so on.

It hadn't even occurred to me that there are places where enough people are using Google Maps while driving to cause significant impact on traffic patterns. Being car-free (and smartphone-free) really gives a different perspective.

Are you Theodore Kaczynski ghost? :)

Seriously what job you do that allows you to not have a smartphone ?

Not OP: I have a smartphone for my own personal use but don't use it for work at all. If my employer wants me to use specific phone apps they can provide one for me like they do a laptop.

yeah so you got a smartphone, the dude was saying he doesn't have a smartphone.

no smartphone make it a real pain in the ass to do most things nowadays, job is one of the biggest but it's not the least. to even connect to my bank account on my computer i need a phone.

Cell phones exist which are not smartphones, and everyone who uses phones for 2FA is happy to send 2FA codes to a "dumb" phone. They only have your phone number, after all.

Well my bank doesn't give me the possibility to use SMS code. It's obligatory to use the app on a verified phone and put a pin.

This is also problematic in cases where navigation apps are not updated and drivers start taking routes they are no longer authorized to take.

An example I notice frequently on interstate highways that go through large cities is GM suggesting you get off on an exit, take the access road and skip one of no exits, then get back on at the next ramp. It does it especially often during rush hour traffic. Denver is where I've noticed it the most, but it's not limited to that area by any means.

That already happens. Maps directs you to odd nonsense detours somewhat frequently now, that you get better results by overriding the machine. It's going down the way of web search.

> It's going down the way of web search.

This is an interesting idea. There's an obvious force directing search to get worse, which is the adversarial desire of certain people to be found.

But no such force exists for directions. Why would they be getting worse?

Probably my cynicism but its the more stores you drive past the more likely you are to stop off and buy something.

Exactly! This is an amazing observation and analogy.

For now LLMs in coding are more like an LLM version of a GPS, not the GPS itself.

Like imagine you’d ask turn-by-turn directions from an LLM and then follow these directions ;). That’s how it feels when LLMs are used for coding.

Sometimes amazing, sometimes generated code is a swamp of technical debt. Still, a decade ago it was completely impossible. And the sky is the limit.

> LLMs are a GPS, not a highway.

I love these analogies and I think this one is apt.

To adapt another which I saw here to this RoR thread, if you're building furniture then LLMs are powertools while frameworks are ikea flatpacks.

It’s best the analogy of the month. I don’t think cab drivers today are the same cab drivers that knew the city by heart of the past.

So, it’s been a privilege gentlemen, writing apps from scratch with you.

walks off the programming Titanic with a giant violin

The problem is now that the LLM GPS will lead you to the wrong place once a day on average, and then you still need either open the map and study where you are and figure out the route, or refine the destination address and pray it will bring you to the correct place. Such a great analogy!

Strangely this reminds me of exactly how you would navigate in parts of India before the Internet became ubiquitous.

The steps were roughly: Ask a passerby how to get where you want to go. They will usually confidently describe the steps, even if they didn't speak your language. Cheerfully thank them and proceed to follow the directions. After a block or two, ask a new passerby. Follow their directions for a while and repeat. Never follow the instructions fully. This triangulation served to naturally fill out faulty guidance and hucksters.

Never thought that would one day remind me of programming.

Indeed. My experience in India is that people are friendly and helpful and try to help you in a very convincing way, even so when they don't know the answer. Not so far off LLM user experience.

Try asking your GPS for the Western blue line stop on the Chicago L. (There are two of them and it will randomly pick one)

What is “your GPS” meant here. With Google Maps and Apple Maps it consistently picks the closest one (this being within minutes to both but much closer to one), which seems reasonable. Maybe not ideal as when either of these apps will bring up a disambiguation for a super market chain or similar, but I’m not witnessing randomness.

Perhaps they have improved their heuristic for this one, though perhaps it was actually Uber/Lyft that randomly picks one when given as a destination...

To be clear, above i was talking about LLMs. Randomness in real GPS usage is something I have never encountered in using Google maps already since 15 years or so. 99 percent of the time it brings/brought me exactly where i want to be, even around road works or traffic jams. It seems some people have totally different experiences, so odd.

I'm the kind of guy who decently likes maps, and I pay attention to where I'm going and also to the map before, during, and after using a GPS (Google maps). I do benefit from Google maps in learning my way around a place. It depends on how you use it. So if people use LLMs to code without trying to learn from it and just copy and paste, then yeah, they're not going to learn the skills themselves. But if they are paying attention to the answers they are getting from the LLMs, adjusting things themselves, etc. then they should be able to learn from that as well as they can from online code snippets, modulus the (however occasional) bad examples from the LLM.

> I do benefit from Google maps in learning my way around a place.

Tangent: I once got into a discussion with a friend who was surprised I had the map (on a car dashboard display) locked to North-is-up instead of relative to the car's direction of travel.

I agreed that it's less-convenient for relative turn decisions, but rationalized that setting as making it easier to learn the route's correspondence to the map, and where it passed relative to other landmarks beyond visual sight. (The issue of knowing whether the upcoming turn was left-or-right was addressed by the audio guidance portion.)

It's neat to hear that I'm not the only one who does this. It makes a night-and-day difference for me.

When the map is locked north, I'm always aware of my location within the larger area, even when driving somewhere completely new.

Without it, I could never develop any associations between what I'm seeing outside the windshield and a geospatial location unless I was already familiar with the area.

> LLMs are a GPS, not a highway. They tell you what to do and where to go

It still gives you code you can inspect. There is no black box. Curious people will continue being curious.

The code you can inspect is analogous to directions on a map. Some have noted in this thread that for them directions on a map actually do help them learn the territory. I have found that they absolutely do not help me.

That's not for lack of curiosity, it seems to be something about the way that I'm wired that making decisions about where to navigate helps me to learn in a way that following someone else's decisions does not.

You have to study the map to learn from it. Zoom in and out on surroundings, look up unfamiliar landmarks, et cetera. If you just follow the GPS or copy paste the code no, you won’t learn.

The problem is that coders taking this approach are dominantly ones who lack the relevant skill - ones who are taking that approach because they lack that skill.

The ones that until now copied and pasted everything from Stack Overflow.

One small change to using a GPS radically impacts how much you know about the area -- do you use the default, always-forward view, or do you use the slightly-less-usable always-north setting? If you use the latter, you will find that you learn far more about the city layout while still benefitting from the GPS functionality itself.

I think LLMs are similar. Sure, you can vibe code and blindly accept what the LLM gives you. Or, you can engage with it as if pair programming.