For example, you might try to wait for all stylesheets to have loaded, and then run requestAnimationFrame until the latency is below some threshold (say, 100ms) to represent a somewhat-loaded-and-not-too-laggy page.
The point is, you want some measure that plausibly excludes ad-related resources since clearly you don't care how long it takes to load those. It'd also be polite to be transparent about the limitations of your measure.
The event does not have to come from within the vanilla browser, without human intervention. It could be from a plugin you create to collect human feedback on load times.
Of course, the human element will significantly decrease the number of trials you can do within a fixed time period.
Even that is not enough. A number of sites (especially ad-heavy sites on mobile devices) look ready to read, but as soon as you try to scroll, are jerky and semi-unresponsive. It's particularly a problem on ad-laden sites of small news publications. Blocking ads sometimes mitigates this, though not always, as some sites insist on using Javascript to implement "fancy" scrolling which inevitably breaks.
(Semi-related sidenote: please, for the love of God, don't use Javascript to hijack browser scrolling. It's almost certain you will not get it perfectly right and some users will suffer. It's almost never worth it. This is the #1 reason I end up having to switch to Reader Mode in Firefox).
And then the page completely re-flows because some big advertising asset popped into existence further up that you weren't expecting.
The only reason I can think of where I'd be interested in how quickly people can navigate a site without having loaded all the content yet, is when optimizing a site for people with really slow/bad internet connections. But that seems like a whole other topic that has barely any influence on the goal of the comparison.
I don't see why not. It's exactly what they want to show, how much quicker pages can load if you don't have to load the ads. And I personally am not satisfied with a page that has loaded to the extend of usability. Either a site loads fast or it doesn't.
> opening the developer tools causes various side-effects
As long as it's done for all the tests it should not matter. Whatever "side-effects" are introduced will be applied for all tests. Of course this means that the values themselves can't be used to compare the result against other tests, but the goal here was anyways to compare between the different ad blocks and not to generate values for general statistics.