Ok, now help me understand how that can be true given this quote from the advisory:

The findings of our security partner’s investigation concluded there was no evidence of abuse or misuse of Linode’s infrastructure that would have resulted in the disclosure of customer credentials. Furthermore, the security partner’s assessment of our infrastructure and applications did not yield a vector that would have provided this level of access.

Did they get compromised or not?

"no evidence" or insufficient logging. The wording is spun to their favor throughout the entire advisory. Like "Security Investigation Retrospective" making it sound like they have the whole thing wrapped up. Also that everyone will be fine once their passwords are reset, when it sounds like they still don't really have any solid idea if they fixed the problem or not.

My reading was that:

1. Their security partner (whoever that is) didn't see the Lish vulnerability, either because Linode's security team had already fixed it or because they just missed it.

2. Their records weren't sufficient to show the breach itself.

I'm not just trying to be argumentative. Here, let me quote more specifically:

no evidence of abuse or misuse of Linode’s infrastructure that would have resulted in the disclosure of customer credentials.

I feel like I must be misreading something. Didn't they say earlier that they found secrets from their account credentials database on a customer instance that was used to attack (apparently) PagerDuty? That's not "no evidence of abuse or misuse of Linode’s infrastructure"?

you forgot "that would have resulted in the disclosure of customer credentials".

I'm speculating, but having been a fly on the wall at this kind of meeting before, here's my theory of how this went down:

Manager: "So somebody got the key to generate one-time password tokens for PagerDuty. How did that happen?"

Engineer: "I have no idea."

Manager: "What about that Lish vulnerability? Could it have been that?"

Engineer: "Could have been."

Manager: "We don't know?"

Engineer: "I mean, they could have gotten it that way, but we don't have logging for security vulnerabilities, because, you know, we would just fix them instead of logging that they occurred..."

Manager: "Okay, I want a second opinion. Time to bring in $SECURITY_PARTNER."

$SECURITY_PARTNER: "What Engineer said. And here's a bill for our time."

Manager: "What about if the customer screwed up and lost a device? Could it be that?"

Engineer: "It could be that."

Support: "PagerDuty said something about wiping their phone."

Engineer: "If they did lose a device, it would be indistinguishable from the evidence we have."

Manager: "So, maybe we weren't hacked after all?"

Engineer: "Maybe not."

Manager: "Any other theories?"

Engineer: "I just want to go on record... AGAIN... that I don't trust our ColdFusion infrastructure at all. The people responsible have all left, and none of us understand it. We need to rewrite in a cooler language, like Python. And hire some actual security people."

Manager: "Sigh. We've gotten enough flak over this that I guess I'll put it in the budget. Okay, good meeting everyone. I'll write this up in a blogpost, because we need to show transparency. That way customers are at least as confused as we are."

In our case someone logged in to the Linode Manager first. Whomever logged in was confident they knew the password of the user and the MFA token.

That would indicate that Linode's password hashes may have been cracked offline.

Hah, pretty close except the only person responsible is still there.

Honestly, it is likely because they are spinning/lying that it is confusing to you.

If you just assume they are attempting misdirection it makes perfect sense for them to insist things are fine.

from the disclosure:

> After examining the image from our July investigation, we discovered software capable of generating TOTP codes if provided a TOTP key. We found software implementing the decryption method we use to secure TOTP keys, along with the secret key we use to encrypt them. We also found commands in the bash history that successfully generated a one-time code. Though the credentials found were unrelated to any of the unauthorized Linode Manager logins made in December, the discovery of this information significantly changed the seriousness of our investigation.

so, it was an inside job then?