At least in my corner of the industry (commercial) , we've been asking for better test cycles because we don't want to wind up in a situation where we are chasing fuel economy on the test cycle at the expense of fuel economy in the real world. The regulators want to keep the same test cycles so they can measure how emissions and fuel economy are changing over time.

Internally, we have much better test cycles correlated to real world routes that we use to validate fuel economy simulations.

I don't understand why regulators don't just do additional, more representative tests so they can track long-term trends and also have a better test? How much time/resources do the tests actually take?

A more representative test (like running a car on a treadmill) requires a dedicated testing facility that nobody wants to pay for or wait in line to use.

The article suggests on average real-world emissions are 2-5x more than in the lab except for a 20x outlier. VW was 40 times more than in the lab.

I don't think most of these cars are hiding anything, it's just an example of how easy it is to optimise for a fixed test.

exactly, beside, a disconnect between test regime and real world results is expected.

what VW did was not just optimize for the test, but cheat the test, which is: those cars in the article are able to pass the tested restriction in the test regime, while disabling the cheating code wv engines were found not to be able to match test restriction even under test regime.

no one here was busted for real world emission. that's kind of understood already by regulators which built the test assuming the result would trickle down in real world usage even if not 1:1

It's not as though there haven't been issues with gaming these tests to a degree for a long time. Almost certainly this has been an issue of walking backwards into a big mistake little by little, which is the typical process of making big mistakes. You start by figuring out how to game the system right at "the line" as much as you can, then the day comes where you decide to be more explicit about it and build it into the software. You probably decide that you just want to play around at the edges, with incremental differences between "cheat mode" and real driving in the same realm as the bit by bit gaming they've been doing already. But once the system exists the natural process of optimization would push it to the limit of what was capable, and it turned out that was a lot. But by then you're committed, and it would take a lot of fortitude to walk back from such a mistake.

I'm sure the same sorts of dynamics have played out in many other auto makers, though it's notable that some, with a more developed sense of ethics seemingly, have not gone down that road.

Maybe we have a different idea about what a 'defeat device' is. Its a physical piece of hardware. Not some code. So to find out if these other cars have it, you stick your head under that car and look.

The term "defeat device" refers to any mechanism, physical or otherwise, designed to defeat emmissions controls. It's not literally a standalone gadget. I assume the term is a holdover from the 1970s.

    A defeat device is an AECD “that reduces the effectiveness of the emission
    control system under conditions which may reasonably be expected to be
    encountered in normal vehicle operation and use. Motor vehicles equipped
    with defeat devices cannot be certified.”

And the AECD is defined as

    any element of design which senses temperature, vehicle speed,
    engine RPM, transmission gear, manifold vacuum or any other parameter for
    the purpose of activating, modulating, delaying or deactivating the
    operation of any part of the emission control system.

http://www.autocar.co.uk/car-news/industry/vw-emissions-scan...

That's an extremely narrow and frankly useless definition...

What physical defeat device is VW using? (There isn't one)

Two temperature-sensing devices, according to the report.

Link to said report? Temperature sensors (EGT) are a normal part of cars.