> This included making direct code changes to the Tesla Manufacturing Operating System under false usernames and exporting large amounts of highly sensitive Tesla data to unknown third parties.

This sounds like something out of the 1990s, that dark and romantic era of version control when we thought CVS was pretty cool actually and we didn't know what key-based authentication and 2FA were.

There are volunteer-ran projects that don't have this problem.

Edit: to be clear, I presume no one is debating the fact that someone with high enough credentials can push code to production. The questions that the email raises are:

1. Why can anyone, regardless of credentials, push mission-critical code without review (or, alternatively, if the changes did go through review, why did the review process not catch multiple malicious changes?)

2. Why can someone compromise several high-level credentials without anyone figuring it out (the changes were made, apparently, under "false usernames")?

Or a contractor that was fired had their credentials appropriated by this manager, perhaps by that manager removing them from a "delete these accounts" list.

Those are a couple of mundane ways of getting a false username to a production machine. This is even easier when there is a lot of flux at the company with many people coming and going, a lot of account management happening etc.

It could have been that the accounts were local to specific machines and not managed by the company as a whole.

And -- keeping in mind that production type machines operate machinery that can kill -- this sounds okay to you?

Not to mention this:

> Or a contractor that was fired had their credentials appropriated by this manager, perhaps by that manager removing them from a "delete these accounts" list.

...keeping in mind that production type machines operate machinery that can kill, does it sound OK to you that anyone can get access to an account that they don't own and control it?

This particular case would be enough to have PCI certification come into question (if not for it to be revoked), and that's just about money, not life-and-death stuff.

Then again it could be just as simple to create an alternate fictitious identity without going through IT but just by accessing the systems you have permission to access anyway.

The former restriction is maybe difficult enough to efficiently implement in an organization that it's excusable (we have a scheme for it at $work, but it unfortunately means that sometimes people show up at work and the paperwork isn't ready yet and some of the accounts they need aren't yet ready).

The latter, on the other hand, is security 101 and not implementing it on the production floor is just irresponsible. I really hope it's not what happened.

And I'm not joking - recently I was asked whether something (that was designed for a clustered Linux environment) could run on Windows XP because that's what was on the machine they wanted it to run.

Why do you suppose the unauthorized party was following the company's development practices? Maybe it was from the sysadmin side, somebody who worked on the toolchain used for reviewing and pushing things to production. So he was able to sidestep the normal review process. This can happen, what is important is that such things are discovered.

He should not have been able to sidestep the normal review process. That's the problem in the first place. Even if you're from the sysadmin side. It should not be possible to do it.

You may think that looks exaggerated but I've worked in two places where we implemented such a process, both of them far more boring than Tesla and, I suspect, far less money to burn on infrastructure.

> This can happen, what is important is that such things are discovered.

No, what is important when working with mission-critical code is that such things are mitigated. Discovering such a problem in production code is already a problem, not a solution.

No, seriously.

1. Sign every review 2. Use the review signatures + the manufacturer's keys to sign reproducible builds of the production image (i.e. you cryptographically certify that "this image is authorized, and it includes this list of commits, that has gone through these reviews"). 3. Use a secure boot scheme of your choice to ensure that only signed images can be installed on production servers 4. Keep anyone with 'wheel' access away from the image signing keys, and anyone who can generate images away from 'wheel' access.

This way, you make sure that no one who has 'wheel' access can install a sabotaged image, that any image that can be installed has gone through an auditable trail of reviews, and reduce the attack surface that a malicious developer has control over to stuff that requires root access (which is still a lot of surface, but is harder to sneak past a review).

Root access to production servers does not need to mean that you can install arbitrary code on them, and with the right systems engineering, you can ensure that it does not trivially result in arbitrary code being run on production equipment.

Edit: this is all in the context of "questions that Tesla's answer raises". For all I know, the answer might be that they hired some brilliant genius who figured out how to sneak by whatever secure boot scheme they're using. The point is -- the post that sparked all this is not naive. This is real stuff. Companies that are concerned about it can ensure that unauthorized commits are so difficult to get into production that a disgruntled employee would rather just quit than go through with it.

It's possible a sysadmin with low-level access can exploit that and a variety of zero-day exploits and escalations of privilege in the layers above to systematically compromise the boot images, steal or falsify credentials and signing keys, and circumvent the safeguards and alarm systems which should be in place to prevent malicious modifications of the source code and the compiled binaries, while hiding his actions from his co-workers, sure whatever.

And if that's what happened to Tesla, wow, sucks to be them, that's amazing.

But if there are no safeguards, no review process, no alarm bells to go off and any damn person can submit malicious code effortlessly and they were basically working off the honor system... I'm going to blame the victim a little bit.

Depending on what those "other user accounts" had access to, it could go in many different directions. :)

2. If you already have privileged accounts you can escalate in pretty much any environment. And they obviously did figure it out.

There is really no reason for this to be the case. Certainly all code that actually runs on the car can be required to go through review and be verifiably built, even if server code standards are more lax.

If they found a way to use more than one username they may very well have run it through the review process and approved it themselves

Programmers: read-write to repository

Staging team: read-only on repository, read-write to test servers and staging zone

Deployment team: read-only on repository and staging zone, read-write to production

It wouldn't prevent malicious code going out but at least would require a chain of cooperation between employees, which would be harder to achieve.

No, it just requires a chain of cooperation between authorized accounts. That's a very important distinction, especially here, where the email in question alleged the following:

This included making direct code changes to the Tesla Manufacturing Operating System under false usernames

* Storage engineers: Generally have access to most storage (all of dev/test/prod) in their group. Sometimes their access is silo'd, sometimes not.

* Backup engineers: Generally have read/write access to _everything_, and all historical versions of it, as backup systems need to be able to do both read/write. Fairly often there are ways for this access to be "unlogged" too, so the actions aren't captured into any system auditing logs (otherwise it can screw things up). I've not (yet) seen access for backup engineers ever be silo-d, but some places might be doing it and I've just never seen it. :)

It very much sounds like thats the case here - production code was edited, and subsequent auditing has found what should've been deployed and what is deployed differs.

And what stops a single member of the staging or deployment team patching the build scripts, binaries or just installing their own software to a server?

Given this, I think it’s much more likely there were few or bad controls, than a person in an incredibly privileged position working in the margins.

That's completely unnecessary and should not be the case. If you need something pushed quickly, you can get a colleague with review bit and get them to ack for "urgency" reasons after a quick lookover.

I believe this policy, used only a few times per year, has saved us 8 figures in outage costs over a decade. (More than half of the benefit is from a clear statement and instilled sense of ownership, and only secondarily the defusing/unraveling of people would otherwise wait or insert tangles of “best practice” red tape while the website or a factory was hard down.)

I based it on 14 CFR 91.3 (in intent) which says, in part:

91.3 Responsibility and authority of the pilot in command.

(a) The pilot in command of an aircraft is directly responsible for, and is the final authority as to, the operation of that aircraft.

(b) In an in-flight emergency requiring immediate action, the pilot in command may deviate from any rule of this part to the extent required to meet that emergency.

(I made part c of the law, the reporting requirement, mandatory where it’s only on-demand in the aviation law.)

When we explain the policy to new employees, we often cite the aviation law directly, to help clearly communicate our intent.

[0] https://www.law.cornell.edu/cfr/text/14/91.3

Often it boils down to not taking sufficient measures against social engineering. In this case the claim is they used fake usernames - most places I've worked, successfully getting a fake account if you already work there would tend to "only" require a willingness to lie on a form or two ("fake" a contractor) and then request elevated privileges. Very few places I've done work requires sufficient checks or counter-signatures to require additional accomplices or make it harder than that. They do exist, but they're rare.

The state of security most places is quite depressing at times. Then again, most of the time it's enough.

Not that that's necessarily better... Manufacturing equipment's at about the same danger tier as cars.

While you're absolutely right, it is far more likely that someone was able to do this because there is a lack of security in their software engineering practices. This isn't aimed at you, but I think a lot of people are blinded by their support of Elon Musk and Tesla to acknowledge that ultimately he's one man, and Tesla are just a company that makes cars. People and companies are fallable.

There are areas that I couldn't push code willy nilly, namely in the security space. But I'd be willing to bet that a majority of teams at any bigN could have a single bad actor cause some damage... That's just the maturity of the industry.

What? Why?

Nobody on my development team has access to the production code signing keys. And nobody - at all - has the ability to remotely make a production system take an unsigned update.

And in my specific case, the group with the singing keys is also the group paid to tell us "no" whenever a release is blocked by process reasons.

I got dragged into defending my technical solution, but my point was that if the ability isn’t needed, don’t have it. You can break the glass when when you need to. Good process make deviation from it more visible. Our code signing keys belong to a team we already need wet ink signatures from to release software.

I can go into the biohazard labs with shorts on easier than I can leverage our technical disaster recovery. I’ve only ever had to do the latter.

You know that backup system your production servers have?

That also has perfect _write access_ to everything, and it's needed so _restoring_ after something goes wrong works.

I have very different experiences from a SEC regulated company. With SOX there are controls to prevent such a thing to happen. If this is a SOX breakage, Tesla is in deep trouble with the SEC.

So Sarbanes-Oxley came about because Enron were stating investor-facing metrics that didn't match reality. SOX compliance comes about if you:

  * are publicly-listed

  * announce numbers to investors

So, if you were some web app that mentioned Monthly Active Users in your quarterly results: bam! Your release process now has to be SOX-compliant, since the MAU calculations come from analytics (from the frontend, or from access logs) which could be altered by some nefarious code.

Note that the intent of the act isn't to prevent such a thing from happening, but to make sure there is enough information for external auditors to detect it if it occurs.

One person writes a requirement, this needs to be OKed by another person, then a third person writes this code and it's pushed to production. Controls are setup - e.g. checking JIRA tickets in git logs - that no code without proper authorization (corrrect JIRA status) is pushed and deployed.

People need to be able to trace every code change to the requirement and the OK.

In the core this only applies to systems that are in one way or the other relevant to financial data (like ordering), but auditors usually want to be better safe than sorry. But Tesla might have a SOX-IT and non-SOX-IT.

I would expect that there are limits to this. If a rogue employee engages in fraudulent behaviour against you, using "false usernames" to subvert your security as this employee reportedly did, then I don't see how the organisation could be considered responsible.

Many orgs don't track github usernames to real human mappings well or at all, mostly because the single sign on version of github is 3x the price.

As a developer I'm not allowed to have write access to any production system, except in an emergency via a break-glass mechanism, which is audited to the hilt and back.

It also means we're not allowed to deploy software to production systems. This has to happen via a specific chain development > regression / user acceptance testing > production.

All those environments need to be physically seperated with very specific access requirements.

The deployment process needed to be signed off by outr auditors.

I can't speak for other banks, but they probably need to implement the same -, or a similar system.

Neither can I speak for SOX requirements regarding software fo car manufacturing.

What if the employee were a sysadmin-level person that sidestepped the normal process?

E.g. run quarterly reports for changes to prod that didn't go through the regular release pipeline and note down which P0 they corresponded to.

Most companies do not have good security. Even when they do, it's hard to get it right, especially for internal attacks. Don't underestimate what a single individual can do when they're already inside and well-informed.

No, not for a company doing what Tesla does. In the environment I used to work in, that would have been flat out impossible. No change would have been allowed that did not follow process unless an explicit and documented exception was made.

Even if you did manage to commit code to the release trunk without review, every single change to the codebase was checked before the release process started in order to verify that the right process was followed. If we ever found a change that no one could find paperwork for, it would be reverted.

So, yeah, it's possible if you want to do it badly enough.

Code reviews are a thing, but physical/mathematical assurance that zero people in your organization can bypass them are not. However amazing your tower of automation and policy may be, there's at least one sysadmin underneath it all, and at least one guy with keys to the cabinet underneath him. That only starts to diverge when you're running on FIPS 140-2 Level 3+ HSMs and you need to assemble a quorum of operators to do anything. And those are quite easy to sabotage - just trip the tamper protection mechanisms.

If it was a totally new user, shouldn't the system have a list of approved users and a strict protocol to validate a new user?

What does being rude add to the discussion?

Edit: see closeparens comment above. Complicated systems can always be subverted when trust is broken.

Most businesses who's primary money maker is not defense or security related... are not running their operation with a focus on defending themselves from themselves.

Too often the focus is entirely on outside attacks, with little consideration given to insider attacks. Previous job was at a cyber security firm. We'd routinely come under attack from criminal and, we believed, occasional nation state attacks as our researchers attributed a few "Axis-of-Evil" nation-state operations.

Then you'd come back from lunch and find the mantrap doors propped open, or someone left a workstation unlocked with root access to something important, or random guests just wandering around. It's a miracle we never were compromised by a disgruntled employee (of which there were many).

New job, new industry, same behavior. Folks leaving "secure" doors open, workstations unlocked with root access, and all that jazz. They're happy to cite a vague SEC regulation that may have applied at their last job, but doesn't apply to ours and even then, it's not like an attacker would ever give a flying fuck about what the SEC thinks.

I'm starting to wonder if this isn't all a comically bad dream.

Maybe not the most technical solution to this, but one of my previous employers had a workplace culture of setting the desktop background to a rainbow with unicorns if someone left their computer unlocked. Sometimes even teams of two would work together to pwn some of the more alert members, by temporarily keeping them distracted while the other "attacker" did the business.

3 or more fails and the background bumped up to something pretty gross that nobody wanted, simply as a practical matter.

I actually thought gamifying it made it pretty fun, and at least for workstation locking, we were elite. Even today I'm sharp about it. Maybe it could be extended to physical access to areas as well.

On top of that, anything that relies on human discipline is going to fail. Everyone forgets sometims and an attacker only needs in once

Beginning with the simple workflow:

1. Get many burner phones for cash.

2. Get many prepaid credit card for cash.

3. Sign up for many self-driving car services.

4. Targeted denial of service.

(I'm avoiding the very-slow-cruise-missile scenario because nobody wants to think about it.)

It used to be really easy, but law enforcement is cracking down on the ability to buy and activate these anonymously for obvious reasons.

Are there ways around these? Sure, but it's not nearly as easy as you're proposing.

Production line software is almost certainly handled separately from the software that runs their vehicles, and isn't "production" in the usual web sense of being customer facing. This sounds more like someone messed with their factory automation setup.

Manufacturing process (including mfg software tools) are a huge potential source of product failure. I don't know about automotive QMS specifically, but I work in embedded software for safety critical systems. If an unreleased procedure or unreleased software is used to build the hardware, or unreleased software is run on the hardware, it's not even suitable for QA (let alone going to the field).

> Are you trying to us that less stringent controls on manufacturing software is in any way acceptable?

Yes.

It is simply uneconomical to develop all software in a no-bugs-allowed NASA style process. One of the classes of software that can easily be placed in a lower scrutiny bucket is industrial software where the the result of the industrial process will be separately checked by a quality control process.

This is in dramatic contrast to safety critical software in cars, for example the battery management software or the fly-by-wire systems. Defects in these systems, in contract to industrial software, are much more difficult to discover by inspection, and failures are much more costly.

>> I find it concerning that one person was able to push malicious code to 'production'.

>Production line software is almost certainly handled separately [...] This sounds more like someone messed with their factory automation setup.

For a production line, I would still expect a level of control over exactly what software was running and how it was configured, even if it was SOUP.

edit: SOUP not SOAP. I need to stop posting in the morning.

It's not really comparable to your work where you're writing the OS or system code for a safety PLC or something, and you quite rightly have to maintain a very strict process where everything is fully documented and triple-checked so you can prove that it meets your target performance level. All of the stuff that does need that kind of precision is locked away into modules (like the aforementioned safety PLC) which just gets plugged together like Lego by most of the controls team.

(All this is kind of moot, though, because it says that the culprit used a few different logins to make their mischief. Depending on the permissions set for those logins they could have bypassed pretty much any formal workflow, at least temporarily.)

Furthermore, what if Tesla has the same kind of "security" in the software division, too? Can just about anyone in the Tesla team develop and push malicious code to the Autopilot?

I've always thought that Tesla has "better than most" software security for its cars, but I've never thought even that was "good enough". I don't think any carmaker has the kind of extensive security measures put in place that should be required for something that drives you to places on its own at high-speed, can be accessed remotely, and receives OTA updates.

Are they building highly sensitive software?

I would suggest that the factory automation systems would be sensitive to downtime more than anything else, and the ability to respond quickly with changes would be very valuable. Merging policies would only slow this down.

I would also suggest that there are many other internal tools, from machine learning infrastructure to small internal webapps, or supply chain management software that really aren't sensitive, and where the whole dev team having the _ability_ (whether that's used often or not) to push to master is advantageous.

Where is it not appropriate? Customer databases, HR tools, the autonomous driving software, perhaps?

Different development teams can work in areas with wildly different sensitivities and priorities, and in many cases it creates a better culture, faster delivery and more ownership to have developers able to push to master. Not always appropriate, but worth striving for when it is.

Yes!

> I would suggest that the factory automation systems would be sensitive to downtime more than anything else

If somebody pulls some shenanigans on the sofware, which is responsible for actually building the cars I'd wager that such software is even more critical than the software in the cars.

Just imagine some twiddling with the break systems being manufactured.

So I don't believe downtime is the major concern. Security of the final product, which is directly derived from such software, is.

A car is a potential murder weapon. So yes. Especially considering it may be remote-controlled, and deployed by the millions.

And to those were wondering about robots sabotaging security sensitive car equipment: This should not be possible. In such situations you'll always two independent chains of control, one in HW and one in SW. In the car. HW breaks down eventually and SW is always partially broken.

There was someone running torrents in the past.

But yeah, I suppose there are other areas of the company that might have more relaxed requirements

While I was on the Enterprise services team, looking upon IRadaS for something that broke after a push we always debated was it malicious or incompetence.

Then again, what do I know.

So why did it never happen? Or if it did, why did Bill Gates not leak an email saying they found the culprit, and that they believe he was hired by those unscrupulous companies, Lotus or Oracle?

Regardless, I think configuration should be treated as code (use version control and reviews, write tests), since the damage you can do through misconfiguration is often as great as anything you can do with code.

I have no idea what has happened at Tesla, it's perfectly possible they lack processes they really should have, but you cannot prove that from the information available.

It's almost childs play to hide a security vulnerability into a large 1000 line PR, down some error path, through a use after free, hidden deep in a library, a buffer overflow, a debug handler, or anything else really.

Most companies allow you to add on code and then push it after it has already been approved (mainly to account for small nitpick changes)

> get deployed?

I would assume through the normal process? You don't have to immediately try and blow up everything causing an immediate rollback. If you are smart, you'll pull a stuxnet [0] and carefully fuck up the machinery to try and make it go unnoticed.

[0]: https://news.ycombinator.com/item?id=17099969

Codereview is a methodology to get high quality code you can stand behind and put your brand to.

Maverick engineers making hodgepodge changes and knocking things together quickly with no oversight is the way you get a proof of concept prototype working fast.

For the factory, I take it they're still in (or have returned to) the latter methodology.

Could it be that folks started looking at code commits when vehicles started behaving badly?

Says horrible things about Tesla's security policies, especially for a company claiming that it will unleash true self-driving sometime in the next 6 months.

On a vehicle that claims to be able to adjust the strength of your brakes over-the-air, no less.