So in short, the audit is an advertising stunt and will say nothing at all about the security of the actual product.

That's an overly cynical way of looking at this. Security audits of your code are a matter of course best practice and are unrelated to "does the VPN live up to its marketing claims?"

It's like publishing that you passed health and safety inspection for a factory that makes safes. I don't think anyone would reasonably confuse a company publishing that its factory passed inspection for a claim that its safes are hard to break into.

They released the full report here and it's pretty clear on what they did and didn't audit: https://blog.mozilla.org/security/files/2021/08/FVP-02-repor...

You don't expect it. But if they were to publicize the result of the inspection, promote it and after the said inspection, start manufacturing plutonium safes, you probably would not be happy as a buyer.

(I know plutonium is not a good choice for the comparison as in the VPN case, we don't know if what will replace the script will be secure or not whereas plutonium is known to be unsafe, but the idea remains that changing something critical to safety after the audit is not nice to hear at all.)

I think what you're saying is you don't know whether future versions of the code are safe?

That's fair, but also I don't know any companies in the industry that pay for a fine-toothed-comb audit like this one for every major/minor release because it's simply not practical. I don't think this report pretends or is intended to pretend that a one-time audit is representative of future code any more than a negative COVID test is representative of whether you have COVID two weeks later. But that's not an argument against disclosing that you came up negative on your last test a few days ago, because disclosure is still better than the opposite.

The way Mozilla is handling this is a textbook implementation of typical pretty-good transparency/disclosure practices. A post discussing the big issues, and the full report available publicly. I think it's a cynical take specifically because it's the least charitable take on someone following best practices.

If the code in question was standard code, this would not be such an issue. However, it is code that runs with all the privileges, and is therefore the one where security issues would hit the biggest. That is even more true as this code manipulates what is running on the computer, which is easy to get slightly wrong (For instance, for a long time, the LDD binary could be used to execute arbitrary code and it was therefore unsafe to run dracut with dependency resolution on unsafe binaries)

I am not saying that I am against Mozilla's transparency, especially as they were clear on this issue and said by themselves they intended to change this code before release. I'm simply explaining why some may find it either a bad faith or strong security issue.

Yes, because a VPN is a service and you can only meaningfully audit software itself. Any service audit is meaningless one second later as the operator of the service can change it to serve their purposes.

People all too frequently confuse software and services. They aren't the same. This is a software audit, and the thing that needs trust is the service.

Mozilla VPN uses the Mullvad VPN service for most of its backend, and Mullvad had Cure53 complete an audit of its infrastructure last December:

https://mullvad.net/en/blog/2021/1/20/no-pii-or-privacy-leak...

As I mentioned, this is irrelevant one second after it's published.

If they tie a hash of the software to the report, then you can verify the software is the same.

Not sure how to handle updates later though. What level of udpates would require an entirely new audit?

But how do you verify the backend code and infrastructure?

Thats the most important question for a VPN.

The audit commoner is referring to is of their backend infrastructure, not their published software.