Re:Apple M2 Ultra Isn't Faster Than Last Gen AMD&Intel Deskt...
by sudosysgen on Sunday June 11, 2023 @12:53AM
(#36278209)
The fact that the 5x outlier may be qualitatively worse is something that you pin down using percentiles, which is what everyone does, either 1% or 0.1% low. The argument about time-weighted frame staleness is not mathematically valid.
Re:Apple M2 Ultra Isn't Faster Than Last Gen AMD&Intel Deskt...
by Retric on Sunday June 11, 2023 @12:58AM
(#36278239)
There’s nothing inherently mathematically special about 1% or 0.1%, you can also use 0.0042% or 0.00069% which highlights this kind of extreme outliers more.
Benchmarking sites use 1% or 0.1% because they better highlight the commodity PC hardware rather than the games bugs and architecture.
Re:Apple M2 Ultra Isn't Faster Than Last Gen AMD&Intel Deskt...
by sudosysgen on Sunday June 11, 2023 @09:54AM
(#36281378)
That's not true. You can't use 0.0042% because then you don't have enough frames to be statistically significant without running for a very long time. It's also true that a frame pacing issue that is so rare that it isn't even impacting the 0.1% frame time isn't going to be noticeable, really.
These benchmarks are controlled for game bugs and architecture by simply averaging over ~40-50 games, on repeatable loops.
It's not clear to me: are you saying those two scenarios should be quantified as equally bad? Because it seems pretty obvious to me that the 5x outlier is qualitatively much worse.