Whether your site gets owned or not is dominated by its popularity rather than its source code license. There's a number of proprietary forums out there too, they tend to suck as much as the open variants. (And if it's for sale, your attacker has a copy.) Being custom-developed protects you from the script kiddies (but you'll have to fight spam yourself--fortunately custom non-OCR-requiring captchas can have higher blocking success rates than even recaptcha), being closed gives you little more than a time advantage if you do become popular. Security through obscurity is a real security layer, it's just incredibly thin.

The better argument for "roll your own", if you have the time, is that it can be a great learning experience. But regardless of what's used, if you ever do get popular enough to become a conscious target it's just negligent to not do security audits from people whose job is security. It's also helpful to design from the perspective of "everything is compromised, what will I regret."