Some more color into our situation:
- we've successfully put in place the core metrics for PPC and set up conversion tracking
- we're utilizing the Analytics API to track custom goals, in our case, things like email signups or purchases
- we've tested different landing pages, choosing one that had 300x better conversion rate but I feel there's still more room to improve
- we've run Facebook Ads, AdWords and Display on Google's Content Network and search has been the most effective at getting what we need (emails)
From my POV: I've done it myself and can continue to teach myself, but there are too many moving parts (from what I can see) for me to successfully lower our CPA and manage on a day to day basis while running the company effectively and efficiently
Also don't go for an agency that gets payed a percent of spend or per thousand clicks. You will usually get more conversions off an agency that charges a flat management fee per day.
The entire agency model is geared towards brand advertisers who spend obscene amounts of money on "branding" without tracking performance or looking at results.
The problem here is a profound misalignment of incentives. An agency gets paid a percent of spend, so it's in their best interest to spend as much of your money as possible testing without thinking about conversions or results.