The Cost of Landing Page Optimization

Many people have opined that landing page testing is a silver bullet. You pick some page elements to test, collect your data, and all of a sudden you have a better performing landing page.

In fact, not all of your test plan ideas will make a positive impact on your conversion rate. Unfortunately, you don’t know ahead of time which of your variable values will succeed. If you did, you wouldn’t even need testing.

A certain kind of mental reframing is required as part of the proper perspective for ongoing landing page optimization.

Each Landing Page Test Has a Cost

You will have to expend time and effort to set up the test, monitor data collection, and analyze the results. Even if your test is successful, you still have to consider alternative marketing activities that you could have been engaged in rather than testing, and the lost opportunity cost of not doing them.

In other words, if you have bigger opportunities to impact your program’s profitability, attend to those first.

A Test may not Yield any Positive Results

It’s possible that the tuning elements you select won’t create any noticeable conversion improvement. A few may actually make things worse and lower your conversion rate.

At the end of the test, your original baseline recipe for the landing page may still remain the champion and will have bested all challengers. This isn’t a problem.

You can’t base your testing program on the outcome of a single test. You will guess wrong a significant percentage of the time when selecting alternative variable values to test.

But this shouldn’t deter you from trying. Testing is an ongoing activity, and until you have completely exhausted all of your meaningful ideas, you should keep trying. However, it’s likely that you will see a law of diminishing returns if you repeatedly tune the same page.

Chances are you’ll see the biggest gains during your early tests because at that point your landing page is in its worst shape and your ideas are most numerous and original. During subsequent tests, you will probably tinker with smaller refinements that aren’t as likely to produce dramatic conversion improvements. Soberly evaluate whether another test on the same page is warranted.

Performance may Drop During a Test but Still Lead to Positive Results

The mix of alternative recipes you’re testing could perform worse than the baseline recipe. Often, some of your variable values are worse than their baseline counterparts while others are better, creating a sampled blend of recipes that has lower overall conversion. As a result, you will often see a significant drop in revenues early in your test.

Don’t panic or abort your test. Have the self-discipline to collect data with high statistical confidence. If some variable values or recipes continue to underperform, you can eliminate them from your testing mix.

Eventually, after several experiments (or follow-up test runs), you should be able to cut out all of the poor performers and focus on what works the best. This may get you into positive territory (where your final challenger recipes perform better than the original baseline).