And the taxes aren't always a bad thing. In my case, my partner makes much less income than me, so starting this year we can shift medical expenses to her tax return to save with the medical expenses tax credit. We'll also qualify to share health insurance plans despite not being married.
Let me circle back and invite you to make sure you know all the limitations of being common law partners (which you seem to be). For my wife and I for example, the fear that her parents might have the final say in her health decisions if she's incapacitated meant we needed more than simply common law.
In our case, there are strong financial reasons not to get married yet: she's a student receiving Quebec financial aid, and since we're unmarried and childless at the moment, Quebec doesn't consider my income or assets in student aid decisions (unlike how e.g. Ontario treats common-law partners). Once that changes, we may reconsider.