All of our acquisition methods specifically target sports fans and are zero cost. For instance, we have partnerships with some sports content sites that run our ads for free. We also drive quite a bit of traffic by posting links to Sidelines in team fan forms.
The on-boarding process for Sidelines takes a few steps. One of the things we do, is as soon as you sign-up via the landing page we ask you to select your teams. This gives the user something to do and we find that 95% who sign up complete this step. In the sign up flow, we automatically subscribe new users to other users on the site who are active and follow the same teams, and this is explained clearly to the user in the sign up flow. Once the user completes the sign up flow, he lands on the feed, which shows interesting stuff the people he is following have shared about his teams. Since the feed is such a universally well-understood UI construct, users get it, and we see a good percentage of our first time users reshare an item or comment on an item on their first visit.
We also send out a welcome email (which we’ve been experimenting with) which explains more details about the site and how to get the most out of it. We’ve had fairly good success in bringing users back to the site with the welcome email.
We’re currently in the process of implementing a tour which will walk the new users through the features of the site, that should further drive engagement on the first visit.
For our landing page tests we use Google Analytics which now includes Google Website Optimizer. We track engagement by logging activity time stamps per user in Redis. We also track how many other friends a user invites as a means to measure our k-factor and we are definitely biased towards targeting users that are more likely to be engaged and bring other users to the site.
I think the results here speak more than any guesses we might make, but I'm just astounded with the results.
Side note: I was able to click "Sign In", and then from that page access the "About" page from the footer. That told me at least a little bit more about what the service is actually for.
I know it's a chicken and egg problem, you want a large enough sample base to get users using the product but with vague messaging you are attracting people that wont use your product (people that you shouldn't target) and people that WILL use your product but don't understand the vague messaging. I would suggest taking all versions of the page and signup methods to test usability of the product. Optimizing on the initial signup is great, but if your users don't use the product after they signup bc it's so vague.. then you've just wasted all that optimization effort. Unless the goal is to create a huge list of signups.
Also, where is traffic coming from? Paid, organic search, social media, press release, tc, etc? That has a huge difference on your results. And what tools are you using to run the tests and what is the sample size? I think readers would love to hear those tidbits too
Thanks for sharing your findings.. good luck =)