News for hackers. Stuff that matters.
Re:Volkswagen Chief Martin Winterkorn Resigns Amid Emissions... by harryh on Wednesday September 23, 2015 @01:06PM (#10266295)
Two things that are surprising to me about this story:

1) The secret was kept for so long. How many programmers were involved with the relevant code? How many project managers signed off on it? Surely VW does its own emissions testing internally so some of them must have known. How high up in management did this go? It seems like it must have been quite a few. Amazing that none of them got mad and told someone outside the company.

2) I would never have guessed that emissions from a car engine could vary so widely. 20%? Sure. 50%? Sure. But news outlets are reporting that these cars are emitting at least 10x and possibly as much as 40x NOx as they should. This is clearly because of my ignorance of the details of the engineering here, but I was shocked that such a difference could happen.

Re:Volkswagen Chief Martin Winterkorn Resigns Amid Emissions... by hackuser on Wednesday September 23, 2015 @02:03PM (#10266763)
> The secret was kept for so long. How many programmers were involved with the relevant code? How many project managers signed off on it? Surely VW does its own emissions testing internally so some of them must have known. How high up in management did this go? It seems like it must have been quite a few. Amazing that none of them got mad and told someone outside the company.

We should not be surprised; we need to learn to expect it and manage that risk. Institutions of all types have long histories of such conspiracies to cover illicit activity. Think of the many recent financial industry scandals, the Catholic Church (and Penn State University) covering widespread rape of children, various military scandals, US government spying on citizens, IT companies spying on their users, performance enhancing drugs in baseball ... in fact we should be surprised when someone steps forward.

The #1 rule of any institution is loyalty. Rape will be covered for; blowing the whistle is a mortal sin. It shouldn't be that way but we can't deal with these problems until we accept that it's human nature, at least in certain situations.

EDIT: I don't mean to preach. A certain part of me is always surprised too.

Re:Volkswagen Chief Martin Winterkorn Resigns Amid Emissions... by baakss on Wednesday September 23, 2015 @05:16PM (#10268295)
I would call it a fear issue more so than loyalty. I would be willing to bet not a lot of people knew to begin with, and the few that were told or stumbled upon it didn't want to make it into a big deal for fear of getting caught up in a problem.

In the past, when I've seen a situation like this, one person in middle to middle-upper management made the call, and seriously only 1, maybe 2 devs were involved. If those devs said anything, it was usually a halfhearted attempt to escalate the risk, and was ignored as "well, that manager must have some reason" or "That's too hot of an issue."

I know, it's a crappy mentality, but most people don't want to get caught up in a big ethics issue at work. It's extremely risky for the person escalating.

As far as how only a couple people might know this even happened, corporate codebases are sometimes a total mess. There usually aren't things like code reviews. Project managers sometimes have absolutely no idea what the project actually is or what the requirements are. Devs often work in a bubble with shoddy requirements worked out by managers in the business and/or IT management. Not every company works this way, but I've seen three like this where the SDLC was miserably flawed.

Re:Volkswagen Chief Martin Winterkorn Resigns Amid Emissions... by notvladputin on Wednesday September 23, 2015 @03:31PM (#10267518)
The used of PEDs in baseball was a well known issue dating back to the 1980s. Players, front office types, writers, and anyone in the public paying attention knew about it. I can't comment on the other items, but it's pretty clear that everyone knew about PEDs but just didn't care enough to ban them.

(That said, my personal opinion is that steroid testing done in baseball is much more about using the issue as a wedge to weaken the MLB Players Association, rather than "fixing" the game.)

Re:Volkswagen Chief Martin Winterkorn Resigns Amid Emissions... by hackuser on Wednesday September 23, 2015 @03:48PM (#10267655)
I don't agree: Back then I was a very knowledgable baseball fan (I probably could have named almost every player on every team) and I didn't know. I heard occasional rumors but nothing more persuasive than that, and nothing at all which indicated, in any convincing way, how widespread it was. When Jose Conseco talked about it, he was ridiculed as a nutcase and shunned (as expected; see my comment above about loyalty).

I remember when home runs and offense increased dramatically, to record levels, and everyone was saying that the ball was juiced - nobody suspected it was the players who were juiced.

Re:Volkswagen Chief Martin Winterkorn Resigns Amid Emissions... by notvladputin on Thursday September 24, 2015 @07:26PM (#10275503)
In 1991, the commissioner tried to institute a policy banning steroids. He didn't have the authority, so it was no different than if I tried to write that. But it's a clear indication that MLB was aware of the steroid problem and tried taking a small step to fix it.

Regarding the "players being juiced", it's never been shown that juiced players performed better. There have been a number of studies comparing players who were caught using PEDs, before and after, comparing to clean players, etc. None of them show any indication that taking steroids helped players perform better on the field.

If you look at the power-on-contact or ISO of players across periods (the 90s, 00s, current game), you find that there is just as much power on contact in the game now as before. The different is strikeouts, which is clearly tied to strike zone enforcement and some other tradeoffs hitters are making now that teams understand strikeouts are fine if you have power.

I apologize if my tone has some frustration in it, but my pet peeve is people saying that steroids = strength = McGwire hit ball far. The reality is far more nuanced, and if you are able to come into the analyses without preconceived conclusions, it's pretty clear that the 'steroid slugger' era happened to overlap with the power era, but it does not appear to have driven the higher rate of home runs and offense during the time.

Re:Volkswagen Chief Martin Winterkorn Resigns Amid Emissions... by hackuser on Friday September 25, 2015 @12:12AM (#10276330)
I don't know about studies, but the evidence is strong. Let's look at home runs:

* Since 1920 [1], excluding the PED era, only twice have players hit 60 or more home runs in a season and both barely passed the threshold: Babe Ruth hit 60 in 1927 and Roger Maris hit 61 in 1961.

* In the 4 years from 1998 to 2001, the feat was accomplished 6 times by 3 different players (2 very strongly associated with PEDs, the other widely supsected of it), many blowing away the former threshold, hitting 73, 70, 66, 65, 64, and 63 home runs.

* Since PEDs were banned in baseball, only one player has hit more than 55 home runs in a season.

[1] 1920 is when hitters embraced the modern strategy of trying to hit home runs, led by innovator Babe Ruth. Before that, a period called the 'Dead Ball Era', they generally just tried to get any hit they could and home runs were much more rare. Also, there were technological and rule changes which may have facillitated home run hitting. Before 1919 players regularly led the league with 10-15 home runs; in 1919 Ruth hit 29, a record at the time and more than most entire teams hit; in 1920 he hit 54.

Re:Volkswagen Chief Martin Winterkorn Resigns Amid Emissions... by notvladputin on Friday September 25, 2015 @06:08PM (#10281015)
The problem with this type of evidence is that it assumes the single change in the game was the presence or absence of steroids. To learn more from the data, check out the references I included at the bottom. The TL;DR is that there is no compelling statistical evidence that suggests steroids impacted homers, or offense in general.

The facile argument fails to address the fact that the game underwent expansion (weakening the average pitcher). The strike zone was changed a number of times (if you normalize for strikeout rate, the power in today's MLB is the same as in the 90s). Umpires got a lot better once PitchFX was able provide them feedback. (Lots were let go as well). Colorado started using a humidor, reducing home runs at the greatest hitters park of all time.

There's not really any way of knowing who did or did not use. MLB and mainstream sports media "strongly suspected" PED usage is closer to HUAC findings than real evidence. Also, it doesn't acknowledge the fact that a lot of pitchers were found using PEDs.

In short, it's incredibly facile to look at some superficial stats and hints of suspicion in order to reach the "steroids = homers" conclusion. But when you look at the underlying component numbers and adjust for the changing environment in the game, there is no evidence to support that conclusion. (That does not mean steroids had no effect, just that we cannot detect them with our best efforts.)

Here's an incredibly strong counter argument from Joe Sheehan's paywalled newsletter [I added it to pastebin for reference, I doubt Joe will mind]. http://pastebin.com/DXW0HSSt

Dan Szymborski is one of the top baseball data analysts in the country. From [0] (paywall) "Despite the rhetoric surrounding PEDs, players caught for steroid/testosterone use do not show a pattern of overperforming their projections in the years leading up to the drug suspension or a pattern of underperforming their projections in the years after a drug suspension."

In layman's terms, if all you knew about players the past 10-15 years was their past OPS+ and whether they were busted for steroids -- now or at any time in the past -- it appears the PEDs had no noticeable effect on the projection of their future OPS+.

What this means is that, even with the knowledge of what outliers such as Barry Bonds accomplished while allegedly using performance-enhancing drugs, as a whole, there's extremely limited evidence of a significant effect on statistics of the drug users as whole. And without double-blind research studies of PED use among major leaguers and/or detailed information of what players are using, all we have to go by so far is the bottom-line results.

Now, none of this should be taken as endorsing the idea that MLB should simply open the floodgates and allow players to do whatever they want. Instituting drug testing is a very good thing for the sport -- but that improvement is for reasons other than the record books, such as the long-term health of players and the public trust. As far as the record books being tainted by PED use, well, it appears there isn't much evidence of that."

[0] http://insider.espn.go.com/mlb/insider/story/_/id/10922627/m...

Re:Volkswagen Chief Martin Winterkorn Resigns Amid Emissions... by harryh on Wednesday September 23, 2015 @03:10PM (#10267359)
You are clearly correct and you're probably right that I underestimate the power of loyalty. Just something in my lizard brain remains surprised. Defect in my circuitry I suppose.
Re:Volkswagen Chief Martin Winterkorn Resigns Amid Emissions... by Uhhrrr on Wednesday September 23, 2015 @03:20PM (#10267434)
For 1), it could be that one set of engineers was tasked with gathering emissions data and finding which parameters got the legal result, another set wrote the software which gathered the data to determine the run profile, and another was tasked with setting the parameters in software for a given run profile. And, the internal emissions testers are probably told to replicate the various government trials. It could be probably done with just one or two directors knowing the whole story.

Reminds me of the denouement of "Real Genius": https://www.youtube.com/watch?v=HoT-h0S1gkE

Re:Volkswagen Chief Martin Winterkorn Resigns Amid Emissions... by greggman on Wednesday September 23, 2015 @08:04PM (#10269099)
What was actually built

http://fas.org/spp/starwars/program/news99/991001-news-madde...

Sorry there are other references. Search for TRW + laser + San Clemente

Re:Volkswagen Chief Martin Winterkorn Resigns Amid Emissions... by Uhhrrr on Thursday September 24, 2015 @01:47PM (#10273228)
Interesting that I can't easily find articles about it after 2001.
Re:Volkswagen Chief Martin Winterkorn Resigns Amid Emissions... by harryh on Wednesday September 23, 2015 @03:22PM (#10267449)
Excellent movie reference!

And ya, that's a reasonable hypothesis. I wonder if we'll ever find out what really happened. Probably not. :(

Re:Volkswagen Chief Martin Winterkorn Resigns Amid Emissions... by Uhhrrr on Wednesday September 23, 2015 @04:12PM (#10267817)
I guarantee* that multiple books will be written about this. We'll probably find out some pieces of what happened.

*insofar as a pseudonymous Internet commenter can guarantee anything

Re:Volkswagen Chief Martin Winterkorn Resigns Amid Emissions... by x0x0 on Wednesday September 23, 2015 @01:27PM (#10266464)
My understanding is that diesel cars had to be manufactured with either a NOx storage catalyst or injectable urea, both designed to convert the NOx emissions into something less harmful. What VM did seems to be the equivalent of just removing the catalytic converter.

It beggars belief that the ceo and many executives were unaware. Considering that, in order to make emissions standards, competitors have to add hardware to the emissions system or inject urea, which increases costs and hassle to the driver (you have to fill the AdBlue, every piece of hardware is a maintenance cost waiting to happen) and to VW (plumbing, harder to convince drivers to buy diesel.) VW figured out a way to avoid both, a distinct advantage to their diesel products. And nobody in the executive team inquired at all how this engineering miracle they pulled off worked? I don't believe it. If nothing else, someone in the executive team asked how long it would take their competitors to copy VW's innovation. Or suggested patenting it.

Re:Volkswagen Chief Martin Winterkorn Resigns Amid Emissions... by mzs on Wednesday September 23, 2015 @02:05PM (#10266784)
It is possible that CEO was unaware. The line was that VW engineers had solved it for small engines with better CDI and engine/transmission management. Theoretically it could be enough and they did patent that stuff.
Re:Volkswagen Chief Martin Winterkorn Resigns Amid Emissions... by teddyh on Wednesday September 23, 2015 @01:19PM (#10266412)
Programmers are susceptible to venality just like anybody else.

http://www.commitstrip.com/en/2015/09/22/software-worth-bill...

Re:Volkswagen Chief Martin Winterkorn Resigns Amid Emissions... by harryh on Wednesday September 23, 2015 @02:03PM (#10266770)
Of course. My surprise has nothing to do with my opinion of programmers vs other professions. Instead it has to do with the ability of large groups to keep big secrets. That's a pretty hard thing to do.
Re:Volkswagen Chief Martin Winterkorn Resigns Amid Emissions... by nitrogen on Wednesday September 23, 2015 @05:52PM (#10268512)
Instead it has to do with the ability of large groups to keep big secrets. That's a pretty hard thing to do.

Honestly, I wonder if this is a myth started by a large group that was keeping a lot of secrets. It seems like it would be pretty easy to keep secrets under wraps by compartmentalizing knowledge, emphasizing and rewarding loyalty, and intimidating or marginalizing potential whistleblowers.

Re:Volkswagen Chief Martin Winterkorn Resigns Amid Emissions... by PhasmaFelis on Wednesday September 23, 2015 @10:58PM (#10269578)
Part of it is also that announcing this sort of thing online doesn't do much unless it goes viral, and going viral seems to have a lot to do with random luck. There's so many horrible things happening in the world that it's difficult to pay attention to more than a few of them.
Re:Volkswagen Chief Martin Winterkorn Resigns Amid Emissions... by goatforce5 on Wednesday September 23, 2015 @01:20PM (#10266419)
Agreed.

I haven't been following the story at all closely, but if VW was able to get (wildly?) better figures than competing companies, I would have thought those other companies would be buying VWs and testing them and/or pulling them apart to figure out how they were doing it... And at some point much earlier than today those other companies would have realized the numbers didn't add up.

Re:Volkswagen Chief Martin Winterkorn Resigns Amid Emissions... by mzs on Wednesday September 23, 2015 @01:25PM (#10266453)
Mazda engineers for example thought that there was something in the atomization they could still improve cause that is what they guessed VW did better which now proves to have been a snipe hunt sadly.
Re:Volkswagen Chief Martin Winterkorn Resigns Amid Emissions... by CydeWeys on Wednesday September 23, 2015 @01:43PM (#10266591)
To be clear, it's not that VW was able to get wildly better figures than competing companies, it's that, in one engine map, they get NOx emissions levels that are around the same as other companies (i.e. it meets emissions criteria). Then, in another engine map, when it only uses when it suspects it isn't being tested for emissions, it runs a different fuel/air mixture that allows increased economy (which might only be a 10% improvement or whatever), but at the expense of severely increased emissions.

Other companies running the same sort of emission map that optimizes for fuel efficiency at the expense of all emissions would get similar figures of 40X for emitting way too much emission. VW is the main one we're aware of that's cheating in this manner.

Re:Volkswagen Chief Martin Winterkorn Resigns Amid Emissions... by usrusr on Thursday September 24, 2015 @02:16PM (#10273470)
Organizations keep those secrets by rationalizing that their behavior is not illegal at all. "This might be against the spirit of the law, but hey, here is this clever loophole that we found." This way, what could have been a perfectly whistle-blowable secret is just a dirty little implementation detail that won't be shared because the people involved are not particularly proud of it. What makes this pattern so powerful is that its persuasiveness does not correlate much with the legal quality of the rationalization: as long as the rationalization is not routinely tested in court, people will happily believe just about anything if it makes their job easier. Doubly so in a corporate environment where optimism is rewarded and pessimism is punished.

Nonetheless it has been very much knowable, for years if not decades, that some level of benchmark-gaming is happening in the car industry. But that's an inconvenient truth to anyone who is living a car centric life, so even people outside of the car industry proper were likely to not make much of it when a car engineer had a spell of loose lips. I am convinced that it is neither a coincidence nor a conspiracy that only a study initiated by a quite car-critical organization could break that wall of motivated disinterest.

Re 40x: a clean combustion combines all the inputs into mostly harmless outputs. Inert gases that have shed all chemically bound energy and won't react much with the molecules that make us alive (unfortunately, we don't know how to perfection that chemical efficiency without sacrificing thermal efficiency). Those mostly harmless gases might still be bad (think CO2), but are not terribly bad as in directly poisonous. In an imperfect combustion, all that stuff can combine into much more reactive and thus dangerous gases. If you set your regulatory threshold sufficiently close to perfection, even a very slight imbalance in the combustion can cause "orders of magnitude beyond threshold". It's the same mathematical principle that makes "many nines" uptime guarantees such a beast: if you fail, you will most probably fail quite hard.

Re:Volkswagen Chief Martin Winterkorn Resigns Amid Emissions... by HNaTTY on Wednesday September 23, 2015 @01:12PM (#10266346)
From what I've read so far, this has to do with heat necessary for the catalytic converter to absorb NOx -- without enough heat, NOx isn't absorbed, so in emissions test mode, they're just running the engine in such a way that more heat is generated to make sure the cat works properly.
Re:Volkswagen Chief Martin Winterkorn Resigns Amid Emissions... by mzs on Wednesday September 23, 2015 @01:22PM (#10266436)
Fuel economy improves at higher temperatures, but NOx production increases dramatically then. Conventional catalytic systems cannot keep-up so either a selective catalyst (expensive) or improved egr system (performance sapping) must be used.
Re:Volkswagen Chief Martin Winterkorn Resigns Amid Emissions... by harryh on Wednesday September 23, 2015 @01:15PM (#10266376)
Interesting! How much heat are we talking about here? When an engine is just running in normal operation it generates a fair amount of heat naturally. I guess that's not enough?
Re:Volkswagen Chief Martin Winterkorn Resigns Amid Emissions... by ams6110 on Wednesday September 23, 2015 @02:25PM (#10266963)
Diesel engines run with a surplus of air so they only burn the fuel they need to. They don't really generate a lot of heat unless they are under load (relative to gas engines). Just cruising down the highway a diesel will not need much fuel and will run fairly cool.
Re:Volkswagen Chief Martin Winterkorn Resigns Amid Emissions... by durin42 on Wednesday September 23, 2015 @03:46PM (#10267635)
The owners manual for the VW TDIs actually warns you to not try and preheat the engine by letting it idle in the winter, because it won't work.
Re:Volkswagen Chief Martin Winterkorn Resigns Amid Emissions... by tkinom on Wednesday September 23, 2015 @01:22PM (#10266437)
I too am curious on how it really happen.

If one burn 1 gallon of diesel get x amount of NOx, wouldn't 40x NOx need to burn 40x amount of diesel?

What does the SW do that? Dial down the power of the engine to 1/40th of "normal operations" during the testing?

Re:Volkswagen Chief Martin Winterkorn Resigns Amid Emissions... by tkinom on Wednesday September 23, 2015 @01:27PM (#10266465)
Re:Volkswagen Chief Martin Winterkorn Resigns Amid Emissions... by x0x0 on Wednesday September 23, 2015 @01:41PM (#10266569)
from the article

   [Dan] Carder [director of West Virginia University’s Center for Alternative 
   Fuels, Engines and Emissions (CAFEE)] did, however, shed light on diesel 
   exhaust. The problem, he says, is one of thermal management.
   
   Catalytic converters break down NOX only when they’re hot, and keeping them 
   hot may sometimes require that you burn more fuel than you need for the 
   performance you want. You might want to run the engine on a lean fuel-air 
   mixture, but you have to run it rich.
   
   “There’s also a trap for NOX that acts like a sponge,” Carder points out. 
   “But if you saturate it with NOX, then you have to go to a rich-burning 
   condition to use the catalyst to reduce the NOX” and thus wring out the 
   sponge.
Re:Volkswagen Chief Martin Winterkorn Resigns Amid Emissions... by ubercore on Wednesday September 23, 2015 @01:30PM (#10266482)
I don't know if it directly answers your question, but NOx emissions are dependent on combustion temperature I believe, so the ECU could potentially adjust fuel and timings to create conditions that don't spew NOx, but don't perform as well either. Then when on the road, it opens back up to increase performance but liberate more of the NOx that otherwise might be bound up in less harmful substances or caught by the particulate filter/catalytic converter.
Re:Volkswagen Chief Martin Winterkorn Resigns Amid Emissions... by mzs on Wednesday September 23, 2015 @01:28PM (#10266471)
No, but good question. NOx production varies wildly on factors such as atomization of fuel, ratio vs stochiaometric in mix, temperature of combustion...