I once believed optimization was the secret weapon that could make almost any startup successful. It was certainly a critical part of reaching millions of users in each of my first five startup marketing roles. At a couple of startups we saw a tripling of conversion rates from a single experiment. When we tripled conversion rates, we tripled the effectiveness of every future marketing dollar.
I first became a fan of funnel optimization at one of my early startups where we had hit a wall trying to develop scalable customer acquisition channels. We decided to temporarily stop trying to find new customer acquisition channels and focus instead on improving conversion rates. A few months later we resumed channel building and were able to scale the same previously tested channels to support 100X the marketing spend with the same target ROI per dollar spent. Beyond the clear benefit of enabling scalable marketing campaigns, the improved user experience also resulted in a multifold increase in free organic growth. User growth immediately hockey-sticked and years later still hasn’t diminished. All the while, the company maintained cashflow positive results.
These benefits probably have you chomping at the bit to start your own optimization program. But be careful, optimization can easily kill a startup when not done right (or at the right time).
Here are the three most common optimization mistakes startups make:
1) Premature optimization – Optimization is about improving the path that users take to reach a certain destination within your website. For most sites it’s ultimately about getting people to experience and buy your product. While this seems like an important goal from the beginning, it’s not. If the value of your core product is weak, doubling the percentage of users that get there won’t help much. And it will actually hurt you because every unit of effort put into optimization is one less unit that you can put into improving your core product. Products that don’t become a “must have” almost always fail.
My recommendation for startups is not to begin optimizing until at least 40% of your randomly surveyed users say they would be “very disappointed” without your product. That doesn’t mean you shouldn’t try to have a great first user experience, rather it means you shouldn’t start iterating flows until the core product meets this threshold. The only exception to this is if your value proposition will increase because of a network effect (like eBay). I’ll try to write a post on this scenario soon.
2) Not being deliberate –To execute full funnel optimization you test multiple changes at every step in the acquisition process. Since every change is also an opportunity to screw things up it’s extremely important to measure the actual results of a change. Unfortunately traditional analytics programs aren’t helpful here since they don’t track specific user cohorts moving through the funnel (AKA groups of users). In the early startups I worked with we spent months building systems internally to track conversions at the user level. Fortunately “off the shelf” systems are now cropping up that make user level funnel tracking much easier (I’ve been advising KISSmetrics on such a system for over a year and I’m now using it in a couple projects). With the right system you can track your “measures of success” and roll back any changes that havea negative effect on these metrics.
This presents a new problem. Anyone with a basic understanding of statistics will realize that optimization is a numbers game. If you test enough things you will definitely find something that improves your key measures. That’s the theory, but the reality is that you’ll never get past the first few tests if the early ones don’t yield improvements. People quickly lose faith in the process. Therefore it is essential to vet every test idea before asking the development team implement it. Prioritize test ideas so that the easiest and/or most likely to improve results are implemented first.
3) Killing the love – One thing that is rarely measured in an optimization project is a reduction in the core value perceived by your most passionate users. Your ability to deliver an experience that creates passionate users is your most important asset as a business and must be protected. It can be improved, but it must be done very carefully. The first step in protecting it is to understand it. I never attempt an optimization project without first doing a project that helps me understand the use cases of the most passionate users. After this initial project, which I combine with messaging optimization, I am in a much better position to safely optimize the full conversion funnel.
Effective optimization requires the right tools, qualitative research/understanding and a systematic approach to testing. When executed properly it can easily result in 2X – 10X improvements in conversion rates. No business will come close to its potential without a concerted optimization effort, but be careful to avoid the mistakes listed above.
For more context on where optimization fits into the overall startup marketing priorities, see this post on The Startup Pyramid.
Andrew Chen
Brant Cooper
Dave McClure
Eric Ries
Jamie Siminoff
Steve Blank
Fantastic post, Sean. I think all your points sum up to saying that optimization should only be done after making sure that the constraint in conversions is the message communicated and not the actual product. This becomes little complicated because perceived value of the product can also be optimized but I guess customers are smart enough to realize that they have been duped if you are not actually offering what you promised in your optimized messaging.
Premature optimizing is often the cause of losing faith in the process. Typically startups will start optimizing their landing page before they even know if the problem they are solving is worth solving. If product is not good, no amount of optimization can help.
Hey Sean…Good post. I’d add that you need to look at which customer SEGMENT’s funnel you should optimize and when. As you know with Jooners we can optimize on the converting Organizers and/or optimize on converting volunteers. Each is achieved via somewhat of a different path.So looking at your different customer segments and deciding what to optimize and when (priorities) is another aspect to think about.
Hey Nazila, Great point. In your case I’d say the customer acquisition funnel is for organizers. The funnel for volunteers is part of your core product.
When most other businesses think of segmenting, it will be for different types of users moving through variations of the main customer acquisition funnel.
Pingback: Radio Advertising Advice – Keep it Simple — Radio Advertising Tips | Advertising Marketing Wisdom
Excellent post. Sean. I’m especially happy to see “killing the love” listed.
Too many people treat an optimization problem as purely mechanical, assuming that anything that results in a short-term revenue increase is good. Of course, that’s how you end up with the customer experience of a major bank. I’ve never heard anybody say, “Gosh, I love Bank of America”; instead, the most common comment about any large bank involves irritation mixed with grudging tolerance. They might be able to get away with that, but how many startups can?
Hi Sean:
Great post, just a quick question. Can’t early optimization also help a company achieve PMF more quickly? Optimization of funnels that drive users to the core experience should increase the speed at which you get enough users to start surveying for PMF and it should increase the frequency at which you can survey, thus, shortening the feedback loop and iteration cycle. Doesn’t that jive with your assertion that it’s important to remove bottlenecks and roadblocks from the core experience to expose the love? Isn’t that essentially what optimization is about?
Great post. Two questions though – I’m not really sure what you mean by “user level funnel tracking”. Could you please elaborate on that a bit?
And when you say off the shelf analytics software won’t track the user level funnel, do you mean Google Analytics – or just other simpler analytics package?
Hey Chris,
I agree that more active users would help most startups achieve product market fit more quickly. However, I don’t believe it’s worth borrowing resources pre p/m fit from core product just to get a higher percentage of users through the funnel. This is debatable and I’d love to hear thoughts on why you think it may be worth the tradeoff.
Your business is one of the exceptions I mentioned – existing users benefit when you add more active users. Because of this network effect, you guys should be optimizing from the beginning.
Pingback: AOL Sees Display Decline; Another Googler Leaves To Run A Company; For Q2, Sorrell Seeing Strength And Caveats; MediaMath Offers MathClarity
Pingback: Lesenswert: Passwörter, Stefan Glänzer, Soundcloud, ClickandBuy, Facebook, Optimization Mistakes :: deutsche-startups.de
Thanks Seph,
The details of tracking are pretty confusing but I’ll do my best to explain them (with some help from my friend Hiten Shah at KISSmetrics). User level funnel tracking refers to using people as the base unit of measurement, NOT pageviews, visitors or events. Think of this as being able to drill down to see individual people who dropped off at a specific point in the funnel versus just seeing a count.
Most packages, including Google Analytics, won’t track user level funnels. In the past this type of tracking was only available by building it yourself. Packages on the high-end such as Omniture do have user-level tracking, as part of their comprehensive solutions, but the costs for high-end packages make them not an option for startups. Hope this helps clear it up.
Hey Sean:
Thanks for the response. A few quick thoughts. First, when I say, I think optimization can help accelerate the path to PMF, I don’t mean that the aim of the optimization should be to maximize conversion through the funnel to the point of marginal returns. I just mean that you should optimize just enough to get enough users so you can consistently test them. So, the focus is on maximizing the number of times through the iteration loop by shortening the feedback loop not on maximizing your ROI on conversion. I’m not convinced that I am necessarily correct on this, just putting it out there for feedback. Second, and I would love to hear your thoughts on this, as it could shed light on my first point and clear it up for me, there seems to be something of a chicken and egg problem here. To achieve PMF, one needs to survey users and then realign the product around what resonates with the must have users, but you need users to survey. Assuming you go live with too few users to survey, you need some way of getting users, otherwise, you can’t collect the data you need to achieve PMF. Going from zero users to enough to survey probably requires some level of optimization as you need to get people from visitor>sign up>experiencing the core experience. The tradeoff would be worth it when you are going from zero users to enough to test.
Pingback: ArticleSave :: Uncategorized :: How to Prepare for Conversion and Optimization
Sorry for the slow response this time 🙂 I missed your second comment. I think we’re on the same page here. It’s great to have more users to be able to iterate faster – but you don’t necessarily have to acquire these early users in the most efficient way. It’s worth thinking about a good first user experience, just no need to obsess over fine tuning it. Before scaling growth you definitely should obsess over an efficient acquisition/conversion process.
Optimization and testing are concepts that are still very much alien to more traditional marketeers, and to say that I’m learning loads from your blog would be the understatement of my career. Your insight is proving invaluable in my ongoing marketing education, Sean, and I salute you for that.
Regards
Donna