If you open the audio context you should see consistent latency as long as you keep it open, but each time you open a new audio context you might end up with different latency. You should not see jitter within a single context.