That historical detail is actually somewhat irrelevant to the quote. This is from "Either/Or" and Kierkegaard was making a point about the nature of decision-making and regret, not about marriage. The fuller quote:

"If you marry, you will regret it; if you do not marry, you will also regret it; if you marry or if you do not marry, you will regret both. Laugh at the world’s foolishness, you will regret it; weep over it, you will also regret it; whether you laugh at the world’s foolishness or weep over it, you will regret both. Believe a girl, you will regret it; if you do not believe her, you will also regret it; if you believe a girl or you do not believe her, you will regret both. [...] This, gentlemen, is the sum of all practical wisdom."

And later:

"Were I to wish for anything I would not wish for wealth and power, but for the passion of the possible, that eye which everywhere, ever young, ever burning, sees possibility. Pleasure disappoints, not possibility."