actually I often see stating that you should rely on a 3rd party authentication library since its mostly well known and well tested. While I would agree here I know that mostly the library will be your weak point since you have no idea how its implemented or whenever a new version comes around if it maybe has a security issue or if you keep your version if your vulnerable to something.

Actually most people should take their authentication mechanism as something really really important and not just rely on something. It's really important that your authentication/authorization stuff works and is hardened, not just for you, also for your users. I know that auth could be really hard, especially when the language of your choice contains some bugs, but its a nogo to just use something when it is that important. I don't mean that you can't use a library, but if you do please double check the code twice on each version.