The published hyperparameters are usually ridiculously conservative, for the simple games like breakout and pong you can usually converge in far fewer frames than in the papers.