It's hard to imagine the attack you describe being viable.
The idea is that you don't measure only once. You measure thousands, tens of thousands of times and more. Then any difference however small it will be also multiplied thousand times and it will be possible to measure it. Even if you will have other random processes such as OS scheduler, network latency, etc - it will be pretty obvious after thousands of measurements which ones to ignore. For example, check the histogram graphs in this paper: https://mlq.me/download/netspectre.pdf
With a timing attack, you can iterate the bytes of the MAC, itself probably based on a cryptographic hash like HMAC(sha256), until you get something that gets to the second stage.
You reduce MAC-forging complexity from 2 to the power of the number of MAC bits, to 256 times the number of bytes in the MAC.
This has been the conventional thinking of software engineers for the past few decades. It turns out to be completely wrong.
I mean who would have thought measuring memory access latency could turn speculation into an arbitrary kernel read gadget yet here we are.
If the function to test a password that has been entered stops as soon as there is a wrong character, it's trivial to write a program to send it a modified password each run and see how long it takes to come back. If it takes slightly longer than last time, then you have one more character correct. Iterate until you have the complete password.
It's one cheesy thing that Hollywood does that at least worked at some point in the past, ie. the hacker that uses a program to crack a password one character at a time (preferably as the clock ticks down to some impending disaster)