News for hackers. Stuff that matters.
Re:Ford F-150 Lightning: Fast Truck, Slow Computer by jfim on Tuesday August 16, 2022 @11:06PM (#32491843)
Do they have the margins that allows them to pay equivalent salaries?

From what I recall, Meta and Google have gross revenues on the order of one million in revenue per employee, while SAP AG has revenue per employee around 200k.

Re:Ford F-150 Lightning: Fast Truck, Slow Computer by assttoasstmgr on Wednesday August 17, 2022 @12:22AM (#32492320)
This. The OP wrote:

> They tried to lowball him badly. Like 1/2 the rate he asked for (contract) and probably 1/3rd or 1/4 what we made at Google

FAANG arrogance never ceases to amaze me. It's like these guys live in a different universe blissfully unaware of the world around them. Did it ever occur to him maybe Ford is paying normal average human salaries and not grossly inflated Google wages?

Remember Ford (and this analogy extends to most companies in traditional industries) still has to be able to sell cars to the average blue collar Joe at a price point he can afford, and still turn a profit. Outside SV most people don't drive $80k Teslas. Somebody's still gotta write software for the $16k econoboxes. I'm sorry but nobody deserves to get paid more than e.g. a specialist doctor to hack JavaScript code I don't care how smart you think you are, when there's a guy in India probably just or nearly as good at it who'll do the job for $40k/year. FAANG is swimming in money and can afford to piss away $$ in talent wars, artificially driving up salaries and the prices of everything else, which is why no average person can afford a house in SF anymore. Those that won the FAANG lottery and managed to cash in? God bless them. But don't let the reality distortion field make you think that's how the rest of the world works.

Re:Ford F-150 Lightning: Fast Truck, Slow Computer by AlotOfReading on Wednesday August 17, 2022 @01:56AM (#32492817)
So for context, I'm in automotive on the FAANG side and work closely with people at traditional companies.

There's some misunderstandings here. For one thing, there are obviously skilled people working at every salary level. What you're missing is that this is largely irrelevant. Relative compensation (in software) is a shockingly good proxy for all the associated support and people investments to support them. Employees that cost less are less supported and thus less enabled to produce high quality work than their better-paid counterparts, as a very rough rule of thumb.

It's my experience that OEMs are significantly less invested in their developers and the quality of the software they produce than any FAANG. They don't consider software to be a core competence and they rarely invest in the technical expertise to understand quality or push it internally.

The scale of dev costs is also pretty minor for vehicles as a whole. Amortized labor costs are typically on the order of 10-20% per vehicle. Devs are a small portion (~100s-1000s) of the tens of thousands of engineers that work at major OEMs and their suppliers. An extreme 2-3x wage increase would end up as a few percent increase, but the truth is there are cheaper ways to do that than salary. "All" they have to do is care about the software components the way many do about the mechanical components. It's a cultural problem rather than a strictly monetary one.

Re:Ford F-150 Lightning: Fast Truck, Slow Computer by assttoasstmgr on Wednesday August 17, 2022 @02:38AM (#32493031)
> OEMs are significantly less invested in their developers and the quality of the software they produce than any FAANG.

We must be using different FAANG software because the ones I use on a daily basis are slow and buggy as all hell. The last few versions of macOS/iOS have been atrocious. Every version of CarPlay I've used has bugs. Amazon apps are full of glitches. Compare that to developing engine ECU software which I think is a far more difficult engineering task that the average overpaid UI JavaScript slinger could not handle. You can't just update ECU software every week because you sloppily overlooked something. Unless you're Tesla of course.

I don't deny that there are many outstanding, technically excellent engineers at FAANG that deserve every penny they make. But the sad reality is many of them are just TC-chasing posers who got lucky and have subsequently deluded themselves into believing that whatever narrow expertise they now possess is worth far more than it is on the open market.

Re:Ford F-150 Lightning: Fast Truck, Slow Computer by AlotOfReading on Wednesday August 17, 2022 @11:09AM (#32497422)
Want to know something terrifying? ECU code is often nearly as sloppily written as that JavaScript. They follow MISRA and usually have a couple "QA" teams to catch the most egregious errors of course, but the overall quality is eerily similar to what you'd get if half the team came straight out of a time machine from 1995. I've had to give informational sessions on why boot systems should have redundancy and what undefined behavior is, for example.
Re:Ford F-150 Lightning: Fast Truck, Slow Computer by mrguyorama on Wednesday August 17, 2022 @04:56PM (#32501619)
And yet the ECU shipped with nearly every car for the past few decades almost never fails through millions of cycles.

I think all this "ECU code is bad" stuff is just people who don't like the style of the produced code. Are you sure those edge cases aren't accounted for in other ways? Maybe they decided that everything should be "best case" code, and any error just resets the ECU which boots up before the next spark event needs to be triggered.

I just cannot square how much people complain about the code in ECUs with how utterly reliable they are. Consider the group that investigated VW cars cheating on emissions. The system was complex and obfuscated, but also powerful, reliable, and configurable by the manufacturer.

Re:Ford F-150 Lightning: Fast Truck, Slow Computer by AlotOfReading on Wednesday August 17, 2022 @05:47PM (#32502141)
A similar argument was made by Toyota about their systems during the lead-up to the unintended acceleration debacle. I recommend Koopman's talk [1] on the subject for details, but long story short the Toyota ETC was apparently utterly reliable and installed in some of the most popular vehicles on the market at the time. However, detailed investigation of the systems and code behind the few publicized incidents revealed basic system issues like the watchdog failed to detect task crashes, rampant memory safety issues, and even a failure to follow their own internal code guidelines. All of that was manifesting in people's vehicles, but it was only rarely turning lethal and even those cases went mostly unrecognized.

Things have improved significantly since the Toyota acceleration issue, in large part because all those details came out. Model-based design is now basically standard for most ECUs, which eliminates large amounts of buggy human-written code. Formal methods tools that don't entirely suck exist. However, most of the issues Koopman points out (especially memory safety) still exist in many places on modern vehicles and all of his points about the issues with relying on testing to surface quality problems remain relevant.

[1] https://users.ece.cmu.edu/~koopman/pubs/koopman14_toyota_ua_...

https://youtu.be/DKHa7rxkvK8

Re:Ford F-150 Lightning: Fast Truck, Slow Computer by iso1337 on Wednesday August 17, 2022 @10:57AM (#32497281)
I think you’re conflating the UI of Amazon with the more difficult problem of getting a large distributed system to work at scale.

Conversely, if your ECU has minor glitches, will you notice? Unless it results in catastrophic failure, you will not even know if happens to result in the occasional 0.1 mpg increase in consumption.

And I’m sure VW and the other companies put a lot of resources into software engineering the exhaust emissions defeat workaround.

Re:Ford F-150 Lightning: Fast Truck, Slow Computer by hef19898 on Wednesday August 17, 2022 @02:14AM (#32492899)
>> They don't consider software to be a core competence

Because it isn't for OEMs, theirbequipment is their core competency and aoftware, especially the entertainment side of things for automotive OEMs, is just part of that.

Re:Ford F-150 Lightning: Fast Truck, Slow Computer by AlotOfReading on Wednesday August 17, 2022 @10:37AM (#32497013)
Would you argue that software isn't a core competency for Apple or Samsung because they sell phones? Same situation.

Modern vehicles are just distributed computers with tires, and self driving vehicles are worse. Virtually every feature is either massively supported or entirely enabled by software.

Re:Ford F-150 Lightning: Fast Truck, Slow Computer by kelnos on Wednesday August 17, 2022 @01:28AM (#32492681)
I think that's besides the point. If a qualified developer can get paid more elsewhere, they'll go elsewhere, and Ford will end up with marginally- or un-qualified developers. That's just how the job market works.
Re:Ford F-150 Lightning: Fast Truck, Slow Computer by Shacklz on Wednesday August 17, 2022 @05:15AM (#32494007)
Only if money is the only thing you're optimizing for, which isn't necessarily the case for everyone.

I once went for a job because of the money alone - it was the worst experience of my life and made me realize that money is not worth optimizing for beyond a certain point of comfort.

Re:Ford F-150 Lightning: Fast Truck, Slow Computer by saiya-jin on Wednesday August 17, 2022 @07:51AM (#32495013)
Sure, but you have to look at total costs, span over lifetime. How long will it take you to pay for that house you probably live in? How much time do you waste commuting? How draining is your job? How much do you spent when going out on some show, restaurant etc? How close to true nature are you?

I've followed money in my job decisions in the past and it has worked out OK for me so far, but of course I have no clue how my life would look like with other choices. But living in cramped place like SF for double the salary (thats max) and double the costs... no thank you.

Re:Ford F-150 Lightning: Fast Truck, Slow Computer by WatchDog on Wednesday August 17, 2022 @01:20AM (#32492639)
I agree mostly with your points, other than when you get into how much the cars cost.

The unit price of the cars isn't really relevant to the cost of developing it's software. A low cost model that moves sells a lot of units, can likely afford to spend more on software compared to a high cost model that moves only a few units.

Re:Ford F-150 Lightning: Fast Truck, Slow Computer by hef19898 on Wednesday August 17, 2022 @02:15AM (#32492905)
That's not how unit costs and margins are calculated.
Re:Ford F-150 Lightning: Fast Truck, Slow Computer by WatchDog on Wednesday August 17, 2022 @04:53AM (#32493907)
I don't think I described how unit costs and margins are calculated.
Re:Ford F-150 Lightning: Fast Truck, Slow Computer by hef19898 on Wednesday August 17, 2022 @05:18AM (#32494025)
>> The unit price of the cars isn't really relevant to the cost of developing it's software. A low cost model that moves sells a lot of units, can likely afford to spend more on software compared to a high cost model that moves only a few units.

You didn't? Of course software costs are part of the unit costs for each car produced, just how those development costs are actually allocated is up to Finance (accounting and controlling) to decide. And of course those costs are relevant, as all the other costs of a car maker.

Re:Ford F-150 Lightning: Fast Truck, Slow Computer by cmrdporcupine on Wednesday August 17, 2022 @08:11AM (#32495215)
Putting your accusations of arrogance aside, the point is ultimately this: unlike the work done by a worker on an assembly line, software work scales out "infinitely." You may pay that team of developers a huge amount but the work they have done is reproduced at "zero" cost hundreds of thousand of times.

It makes sense to invest in quality there.

I have made the point that ad revenue distorts FAANG practices elsewhere. I agree on this. You'll note that myself and my friend had no intention of seeking a FAANG salary elsewhere, it's unrealistic. However. There are limits. And sane practices.

The low compensation band reflects a lack of recognition of a) the importance of software and b) the economics of it. It isn't just that they pay bad, it's that they produce bad software because they don't get it.

Re:Ford F-150 Lightning: Fast Truck, Slow Computer by cultofmetatron on Wednesday August 17, 2022 @02:27PM (#32499938)
on the other side, I'm not beholden to work at less than what I could potentially make just so some blue collar worker can have a car.
Re:Ford F-150 Lightning: Fast Truck, Slow Computer by grog454 on Tuesday August 16, 2022 @11:12PM (#32491878)
What is their profit per employee ignoring employee compensation? That tells you more about how much they can pay employees.