Success Through Testing: Don’t Forget the Scientific Method

Test TubesYou remember middle school right? That time of awkward adolescence when we tried on multiple personalities to see which one really fit?

Middle school kids are kind of like a website, always trying to test out something new to see if it makes them better (or cooler).

Another product of middle school was what we learned in science class, an analytical practice called The Scientific Method.

Unlike the middle school student, digital managers have an opportunity to use the Scientific Method to make smart decisions about what makes the site or marketing programs better. The middle school kid? I don’t even want to know how they came to decisions.

We often hear “lets test it” or “we can test it out to see if it has legs” when discussing opportunities to improve digital performance.

However, we’ve probably all been part of tests that have gone awry because the schematics were not planned out effectively. We’ve also been privy to clients (or bosses) who believe in “perpetual testing” – the idea that we should always be testing, for testing’s sake. However, if we apply the principles of the Scientific Method to our marketing tests, we can alleviate both of these outcomes.

By using the Scientific Method, we’ll ensure the test works at delivering an answer, as well as provide a foundation for eventual testing completion. So, what exactly are the principles of the Scientific Method and how can they help? Read on.

Principle 1: Characterizations

Characterizations are “observations, definitions and measurements” of the subject. Think of these as your baseline performance statistics.

  • What’s the CTR and conversion rate? 
  • What’s the current hero image?

By truly understanding your starting point, we form the basis of measuring the effects of change on the program. Many times we “test” without having a performance standard by which to measure success, so when we look back, we still can’t determine if the test was successful, so we test again.

Principle 2: Hypotheses

Hypotheses are suggested explanations of a phenomenon or a reason assumption suggesting possible correlation. Not to be confused with predictions (an equally important part of the Scientific Method), hypotheses seek to provide answers as to why things happen, which can then be experimented against in order to determine its accuracy.

Potential digital marketing hypotheses could be:

  • Active shoppers are more inclined to convert and buy than passive shoppers (search marketing anyone?)
  • Consumers respond more to brands they are familiar with
  • Consumers won’t spend more than five minutes checking out from a shopping site

Hypotheses are the key to unlocking new thinking. Developing and testing new hypotheses improves the marketing for our clients and pushes results further.

Principle 3: Predictions

Predictions are often mistaken for hypotheses (I know, I do it myself), but whereas hypotheses seek to explain why behavior happens, predictions predict what will happen based on change.

Some examples of digital marketing predictions:

  • By changing the call to action, our CTR will improve 50 percent.
  • If we remove steps 3 and 4 from the order process, conversion rates will improve 10 percent.
  • Search re-targeting will have a 20 percent lower CPA than behavioral targeting.

Clients are seeking this type of thinking from agencies or digital departments. If we set forth on a test, we should have some expectation of what will happen (or we would not propose the test, right?) By establishing the predictions prior to the test, we create goals for defining success of the test; thereby lessen the requirement for perpetual testing.

Principle 4: Experiments

Experiments obviously are the tests themselves. When conducting the experiment, there are two important factors:

  • Control Group: This segment doesn’t change and is used as the barometer by which to measure the effect of change.
  • Variable Isolation: Variable isolation is the practice of isolating a single variable and changing it to determine its impact on the prediction. It is this isolation that allows for tests to be conducted successfully. Without isolation the ability to prove causation is severely limited and often leads to re-testing.


The Scientific Method is alive and well within the digital (and direct) marketing landscape. It is a powerful tool in driving performance forward.

Hypotheses and predictions really begin to show what should be tested, whereas characterizations and experimentation show how to test.

And in the words of my middle school science teacher, “Keep on trucking testing!”

Related reading

SEO strategy website redesign migration
siddharth taparia SAP on embarking on search transformation projects
How to improve SEO using data science
Four tools your business needs for better metric reporting