If you are lucky enough to live in one of the few remaining places in the world where common-law marriage is still allowed, you can do precisely this. You essentially cohabitate for some amount of time (in my jurisdiction there is actually no set minimum time though I imagine the longer you live together the more legal water it holds) and then just tell people in public that you are married. No paperwork necessary.

I managed to make it four years under such an agreement with my partner doing things such as buying a house, relocating internationally, sharing taxes, finances and medical insurance, without actually needing paperwork. Relocating internationally was the trickiest, but after explanation to the new country of the law they were willing to accept the document that showed we bought a house together as evidence of our marriage.

Eventually this year we finally had to get some paperwork, forced to by, believe it or not, government bureaucrats who would absolutely not allow me to be placed under my partner’s medical insurance (government employee) without marriage documents. Fortunately our jurisdiction allows you to get a piece of paper that just says something like “We decided we were married on this date and the county affirms it”. The paper is stamped by the county and filed like typical marriage paperwork, takes about 5 minutes and does not require a judge.

Overall if someone wants the benefits of marriage without all of the pomp and circumstance and happens to be lucky enough to live in one of the jurisdictions where common law marriage is allowed, I can recommend it.

https://en.wikipedia.org/wiki/Common-law_marriage#Ontario

The registration bit is the most important part, it’s not bureaucratic. For the same reason that you need to have documents drawn up to form a business partnership, it’s for when things go sideways. The registration is the proof that both parties entered into a contract, enforceable by the government.