You can't say that without saying how large the sample is. Certainly, if the random sample size is "100% of all ballots" it would be enough.
Risk Limiting Audits take that into account. There's a rigorously backed formula for determining how big the sample size needs to be based on the margin of victory. The closer the election, the more ballots are included in the sample.
In fact, you're right that a _very_ narrow margin requires a random sample that ends up being very nearly or actually all votes.
Georgia is a perfect example of this. Georgia applied a Risk Limiting Audit to the Presidential Election. Because the margin was so close, the RLA required them to sample every single ballot cast. The first "complete hand recount" of Georgia was actually a Risk Limiting Audit.
Here is the GA SOS reporting out the results of that first statewide hand recount: https://sos.ga.gov/index.php/elections/historic_first_statew...
As a made up example: maybe some precincts had scanners that worked 100% perfectly, and some had units that occasionally mis-scanned with a rhythm.
Georgia was the closest percentagewise in 2020. 4935487 votes were cast between Trump and Biden, and the difference was 11779 votes. However, even if Georgia went to Trump, Biden still wins, so you're forced to check states with even larger gaps.
However, even in Georgia, to claim the election should have gone the other way with 99% certainty, it is not hard to check wherever you want. This was done by many groups, for example here [1]. Of course, given the sheer idiocy of Trump followers and news support, they don't educate their supporters on any of this, because the facts destroy their claims.
Since voter records are public in pretty much every state (every one I've looked at over the past few elections let you download them), it's trivial for researchers to do this sampling. They do this every election, and that is how they make the statements they do about the extremely low rates of fraud - they did do the testing.
Trump and co did it too, and found nothing. They had the lists immediately, because they started publishing names they claimed were fraudulent, and in each case the media showed them wrong. So they were looking at the names.
And even with those names, and the ability to sample, they still didn't put forth any solid evidence. Sampling irregularities would have been enough to get more eyes looking. But they knew (just like Trump knew in 2016 with his Clinton 3M illegal votes charade that he eventually dropped) that this is a lie.
[1] https://sos.ga.gov/index.php/elections/3rd_strike_against_vo...
The identity of the voter is on the outer envelope which is separated from the ballot and destroyed after their signature is checked. An attacker could insert manufactured ballots to be tabulated after this process and there would be no way to know if they are legitimate or not - they’re anonymous now too so you can’t check with the voter and the voter can’t verify their vote.
You can recount the ballots all you want if this attack occurred, because the fake and real ballots would be indistinguishable and unauditable.
The prevailing opinion of election security professionals is that State election systems should be designed assuming the machines are insecure and the process should be tolerant to that fault.
That's why many states that use electronic voting or tabulation produce a voter verifiable paper trail. That means, when you're done voting the machine gives the voter a piece of paper that has their votes on it. The voter can check the paper to ensure it matches their votes, then deposits the paper record like a ballot.
State's can then use Risk Limiting Audits to verify a necessary amount of randomly sampled ballots to ensure that the electronic count matches the voter verifiable paper ballots.
So, if your concern is that the GA election was overturned by electronic tampering you would need to explain why the voter verifiable paper trail extremely closely matched the electronic tabulation.
This is a long-winded way of saying I agree with you. There are some states that use electronic voting or tabulation without a voter-verifiable paper trail and risk limiting audits. Fortunately, none of those states were "disputed" in this election. But those states should absolutely switch to these kinds of systems.