In abstract, it's impossible to design a test to take that into account because if your subjects have great pitch memory they can recall a reference pitch heard a long time ago and then use relative pitch to fill in the rest. This is indistinguishable from absolute pitch in almost every way except maybe speed.

In practise, spacing the tests out more than a day and asking subjects not to listen for other things between tests might help, without being foolproof.