You can build and distribute them, but it's very ugly. When a person first tries to open your unsigned and unnotarized application, they get a big warning that the app cannot be opened. They need to actively whitelist it by going to the system settings or by right-clicking the application bundle and selecting "Open" from the menu. And the warning that it gives to users is such that your average user won't even have any idea that whitelisting is an option.

https://support.apple.com/en-us/HT202491

While the process is fairly simple, it basically excludes all normal people from using your application if you don't sign them. So any application that wants to distribute outside macOS experts needs to get their application signed and notarized.