Explanation Devils

10 Nov

hhhOnce results come in from an advertising campaign, explanation devils rear their ugly heads. What are they? Those traps of illogical thinking that we can all fall into when we try to explain why a campaign succeeded or failed. There are many explanation devils, but here are three of the most common to watch for.

Post-Hoc Fallacy: The assumption that event A caused event B just because A came first. That telemarketing push increased our sales! Why? Well, we did the telemarketing, then our sales went up. Hello, explanation devil. Remember that correlation does not imply cause. Consider that event C (say, the weather) caused event B (increased sales). Or A and B are unrelated coincidences. Or B caused A.

Regression Fallacy: When a data point deviates far from the average, it is highly probable that the next point is going to be much closer to average. A deviate result is also likely to prompt human intervention. After 75 years of consistent business, a store has a terrible summer. They renovate the store, and voila, the next summer, business comes back. Must have been the renovations! Explanation devil. Unless there was a seismic shift in their company or industry, it is likely that business would have come back towards average without them doing a thing differently.

Confirmation Bias: We all like to be right. And we all have our pet theories. Confirmation bias affects our perception of current events and our memories of earlier events to support our theories. Direct mail is the best advertising medium! Every time it’s used, the store is filled with customers and sales go through the roof! Really? Or are you playing into your biases, only noticing your busy store after your DM hits the market to confirm your theory? Explanation devil!

Watch out for these devils when doing post-mortems on campaigns. Unless you’re doing controlled experiments, there’s no way to know for sure why one thing worked and other thing didn’t. That uncertainty is the “art” of advertising – a frustrating joy.


5 Responses to “Explanation Devils”

  1. dougbrowncreative November 10, 2009 at 11:20 am #

    As advertising moves increasingly away from “gut” into “science” (performance metrics, ROI), my guess is these little hellions will look for other industries to bedevil. And you’ll be free to romp blissfully through bias-free results Shaner!

  2. Shane November 10, 2009 at 11:34 am #

    Performance metrics establish what the measurement for success is. ROI determines if the benefit outweighs the cost. Neither one attempts to answer “why” something was successful – which is where the devils thrive.

    Advertising is thrust into a social environment full of a million influencing factors on any measured variable, making it impossible to conclusively determine ‘why’, unless you test it in an controlled lab situation. I doubt any client would see the value in doing that. I don’t think the ‘science’ of advertising will be coming along any time soon. (to me, accountability does not equal science)

  3. dougbrowncreative November 10, 2009 at 3:51 pm #

    However online advertising and the ease of testing gives us a pretty good controlled lab situation. And if changing only one element of the ad – for example the position of the call to action – and keeping everything else identical leads to greater response, can we not then conclude that it was not an unrelated coincidence, but a cause and effect? That’s science to me.

  4. Shane November 10, 2009 at 4:04 pm #

    That’s a great point Doug. Creative testing of one version vs another is a smaller-scope question that can be more easily answered than the age-old doozy – “did my ad campaign work?”, which ROI quantifies but never really tries to confirm that advertising was the hero, not weather, economy, etc.

  5. dougbrowncreative November 10, 2009 at 4:36 pm #

    Great post by the way….

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s