NO2 output is higher for a modern diesel engine, and NO2 is poisonous. Not like "Oh my god I think I breathed some NO2 now I need to go to hospital" poisonous, but it's bad for you and wasn't there anyway. If Europe had decided it hated diesel and petrol remained more popular (as in the US) there'd be lower NO2 emissions in London.
PM2.5 (tiny particles) is higher for diesel. Again, breathing in a tiny particle that will then permanently get stuck in your lungs, damaging them in a minute but essentially irreversible way, is not going to send you to ER immediately, but it keeps happening and cumulatively, especially over a population, that means deaths.
As the article also points out PM2.5 doesn't even go away if you got rid of internal combustion engines. Just rolling at speed causes tiny pieces of the surfaces that come into contact to break off, probably a racist cyclist even emits PM2.5 though nothing like a diesel truck. So this isn't a problem we can entirely solve, but things like ULEZ aim to make it better, the argument is largely that they aren't enough (and apparently for this journalist, won't ever be enough).
Oh man i hate those racist cyclists :P
So yes, whether racist or not, and indeed even if using some less conventional wheeled transport like a skateboard or shopping trolley - all fast wheeled travel creates PM2.5 pollution and we just have to suck it up. I presume maglev doesn't do this, but maglev isn't exactly at the top of the list of cheap inner city transit options.
Catalytic converter a pull most of the NOx out of a gasoline engine's exhaust, but don't work on diesels. Urea traps are an alternative, but an expensive one.
Moden SCR systems (they inject DEF, aka, urea, into the exhaust) do a really good job getting rid of NOx.
Diesel engines are more efficient at using the energy in the fuel and as a result produce less CO2.
However, diesel engines generally produce more NO2 and particulate emissions which causes local pollution. There are technologies which mitigate these pollutants for diesel engines (such as urea injection and exhaust gas recirculation) which are required in the US, but I believe there is less stringent requirements for them in Europe.
One cheating diesel car in 1000 cars increases ozone, PM2.5 pollution by 1.33%, 1.99% respectively.
Each additional cheating diesel car in 1000 decreases child birth weight by 0.19%, causes low child birth weight 1.9% more often and causes 8% more visits per quarter per 1,000 children of age 0 to 4 to doctors because of asthma.
So yes, comparable American cities would be healthier. Having less taxes on diesel was a huge mistake in Europe. My guess is it was an attempt to burden transportation industry less with taxes. Plus lorries can do tax arbitration by filling up outside of countries with high diesel tax (so you wouldn't want too high of a diesel tax because of the "laffer curve"). This caused this whole innovation of having consumer diesel cars, even though they are more expensive, complicated and heavy. If you take into account weight and production cost it becomes difficult to justify a diesel engine via CO2/fuel savings outside of tax advantages (this doesn't apply to lorries).
So the intent at least originally was to discourage the plebs from buying diesel cars, whereas those that used a lot like trucks, taxis etc. got cheaper fuel (for them the fixed yearly tax was a trivial cost compared to the advantage of cheaper fuel and a more efficient engine).
I agree this is a mistake that should be rectified, but it is of course politically very difficult.
Its not the ignition (if I ran a diesel engine with CH4 I'd get no particulate emissions), or even the type of fuel (if I burn diesel in an appropriate furnace I can eliminate most soot).
It's burning diesel in a diesel engine that's the problem.
Diesel is thick and has a low vapor pressure. When it is injected, therefore, it is more akin to tiny droplets of fuel whose surface burns. Eventually there is -locally- no air around the droplet and the droplet pyrolizes and becomes a particle of soot.
Ironically, diesel engines run very lean (excess O2). But in the neighborhood around the droplet, without good mixing, you get low concentrations of O2.
Gasoline has a low vapor pressure, so it is essentially a gas, not a droplet in the combustion chamber. Therefore there is a flame front across a fairly homogeneous mixtures of gases and there isn't a local lack of O2 (assuming the air ratio is right).
For a gasoline engine this breaks down in two scenarios: Cold engines and direct fuel injection:
- In a cold engine the T might not be high enough to vaporize the fuel and the engine won't start. Therefore more fuel is used and it is in the form of droplets. This is what the choke is for in carburated engines, and now the computer does the same.
- In a direct injection engine there might not be enough time to vaporize the fuel. High P are used (and to shove the fuel into a high P chamber) to encourage vaporization. But its not as effective as port injection.
Direct fuel injection, however, allows for leaner fuel mixtures and more control of the burn (higher compression ratio) so it's basically necessary to achieve efficiency standards.
Another feature/problem of diesel is their high compression ratio. This is how the diesel cycle is more efficient than the Otto cycle (otherwise, the Otto cycle - all else being equal - is more efficient). However, high compression ratios imply high NOx emissions (NOx is an endothermic reaction that lowers P -> high T and high P favor its formation).
So you can see that environmental goals in ICE work against each other. Lowering CO2, or rising fuel efficiency, means rising NOx or soot emissions (or finding a way to shift the curve). You need a transformative technology to improve NOx, soot and CO2 (like fuel injection in the 80s).
In F1, there are no emission rules engines don't have to last 300 000 km but there is a very strict efficiency rule. Here Mercedes claims to have achieved 50% thermal efficiency, which is insane for a small ICE running at 10k rpm and producing 500hp/L. A typical gasoline car gets at best 25~30% in its most efficient state.
Look at, say, WWII airplane engines. They had 4 sodium cooled valves per cylinders, overhead camshafts, power density around 50 kW/L, pretty darn good bsfc, all this decades before anything like that appeared in civilian cars.