Perhaps you could request a 1 or 2 week deferral of your start date at A, citing last-minute personal reasons. Such a thing would not be unheard of, and that way you neither do anything dishonourable* (as long as you don't lie) nor close any options prematurely.

* Well to be fair, assuming your pending employment is at-will, there's nothing actually wrong with quitting immediately after your start (after all, maybe you simply won't like it there); it's a just a bit inconsiderate toward A to start if you don't really believe you're committed to it yet.