1. Forcing people to create an account on a service they don't use or trust to access your website. Remember, quite a lot of people despise Facebook and Google, and by forcing them to use those services, you're making them choose between their privacy and your service.
2. Giving said services too much leeway on how the setup works. Do Facebook or Google or Twitter or whoever now charge money to use their account login system? If so, tough luck; you're gonna have to either rework your whole system or pay up.
3. Giving said services too much control in general, and letting them centralise the internet. This is a bad thing for numerous related reasons, ranging from easy access to personal data for tyrants to dependence on a walled garden run by a large corporation).
4. Making them the world's number 1 target for hackers. Admittedly they're already likely this, and homegrown systems have their own security issues in this department, but I certainly don't like the idea that the whole of the internet is basically now compromised the minute a data breach hits Google or what not. Any centralised system is basically a giant bullseye for every bad actor on the planet.
So no, it's not necessarily accurate that no one should try and implement password management or account systems or farm it out to third parties. That's a good way to give whatever controls remains over to large corporations.
Authentication is such a core part of the UX of your product. You should know how it works. If you're factoring out to a black box because you couldn't do it yourself, then I'm not convinced you're any more secure.
If your team is too incompetent to do it correctly (there are so many resources online), then how are they going to do anything else correctly, like ensure I can't access your private messages when I change /<my_id>/messages to /<your_id>/messages in the URL bar, or query the database without sqli?
This "don't do it, it's too hard" meme does nothing but scare people away from core competencies of our profession.
For example if you have multiple social logins, what do you do if the user forgets which account(e.g., google or facebook) they used?
You could just let them login with any as long as the email matches. But then what if you have a social login type that doesn't require verified email? For example let's say you add an imgur login but the user doesn't have an already have an account there. Then an attacker can just create the imgur account with that unverified email and then use that to log into the same email on your site.
There's also a bunch of edge cases with handling users who use different emails for different social accounts so you can't assume one-to-one correspondence between email address and users anymore.
And there's a bunch of other issues with how to initialize a good default display name from these different social profiles. And also the issue with how to handle if they unlink all their social login methods. And how/who to send reset password. And so on and so forth.
In the end this was all too hard and I gave up and only allowed logging in with google and google only. Which kind of sucks.
I don't remember if I used Google, Facebook, or SO's own auth when I created my SO account. Some sites have upwards of 5 to 10 different ways to login and I find it very confusing.