Tag Archives: Measurement

I don’t know (and neither do you.)

Jan 04 2010
3

Q. What copy will perform best on my website?
A. I don’t know.

Q. What page should I direct search traffic to in order to get the highest conversion rate?
A. No clue.

Q. What time of day will get the best open rate on my email marketing campaign?
A. Dunno.

A big part of any consulting-style job, be it advertising, PR, business process or any other role in which the central function is to incite action across a broad group of people, is answering questions. To be successful, you need to be part futurist, part expert and part hand holder and teller that everything’s going to be okay-er.  But the reality of this type of work – especially that which deals with newer or unproven media or channels – is that sometimes the answer to the question the client is asking is “I have no idea.”

But that’s not much comfort to clients.

So, we answer the question based on our experiences, our intuition, our understanding of media and consumers, and we guess… just a little bit.  But more and more, the guesswork is coming out of the profession.  We have access to analytics, measurement systems and tracking that communicators have never before had access to, and for the most part, they’re inexpensive and easy to integrate. We can tell where our best-converting sales leads are coming from, what types of posts have the best engagement, and when the best time to send out our email communications is – all from easily collected real-time data.

So why are we still guessing?

Part of the reason is that advertising people have always been regarded in part as soothsayers. It was this confidence in our knowledge of the medium, the message and the huddled masses yearning to be sold to that garnered multi-million dollar ad budgets. Conversely, it’s the expertise and ego that got CMOs through the ranks to where they are today.  So where is the incentive for anyone in this equation to ever utter the phrase “I don’t know?”

As a society, we’ve always had an odd fascination with mediums and psychics who pretend to be able to see the future, telling us, to our amazement, what would happen to us, would we find love, how we would die. Of course, these were all parlour tricks based on intuition and a controlled situation. And that’s exactly what we’re doing when we predict what colour “buy” button will perform best, or where to put the call to action on the landing page. Our experience, knowledge and understanding of the media combines with our intuition to make an educated guess, but that’s usually all it is.

If, on the other hand, both the client and the agency embraces the fact that they don’t know the answer – that’s when real answers can be found.  We can A/B test copy to see which performs best.  We can analyze data to deduce why people are coming to our site.  We can built multiple landing pages for multiple keywords and choose the one to go with based only on which one makes the client more money.

Of course, it’s impossible to test everything, which is why we rely on experts in the first place.  If you’re Google, you’ll test 42 shades of blue to determine what colour your background should be, but that’s impractical at best, and an impediment to creative thinking at worst.  Instead, look to solve problems by focusing on the most probable solutions (based on intuition), and determining which one works the best (based on numbers).  It’s not as sexy as a crystal ball, but it’s certainly better for business.

Creative Commons License photo credit: Frogman!

The meaning behind the measurement

Mar 26 2008
6

Sitting at a conference a couple of weeks ago, I sat in stunned silence as a member of a panel on gaming and engagement suggested that if interactive, gaming and social media wanted to grow its slice of the advertising pie, we would have to learn how to measure in terms of CPM.  On the surface, it sounds like a reasonable proposal – after all, a unified system of measurement for advertising can only help us all compare and evaluate results.

The problem, of course, is that there can be no truly unified measurement between radically different systems any more than you can measure the distance between New York and Vancouver in kilograms.  This is not a weakness of either the tactic, its measurability or the understanding of the medium, but rather a weakness in the desire to do things the way we’ve always done them.

Traditional media is measured very simply.  An advertiser buys reach and frequency based on the distribution of a particular medium, for a price ususally expressed in cost per thousand.  Whether the medium is newspaper, television or radio, there is a relative assurance that the advertiser will receive the number of impressions that they have paid for, and the value of those impressions is based on the value of the eyeballs it will be supposedly reaching.   The key thing here is that regardless of the medium, there is always a comparative metric and a comparative value.

When the web was still in its infancy, still being formed by the collective fingers of its users, advertisers co-opted this new medium in very much the same way.  If you wanted the eyeballs of Yahoo! users, you bought impressions based on CPM.  But as the way we used the web became more complex, and the technology and infrastructure that supported it became much more sophisticated,  it became impractical to reach people through a traditional broadcast approach.

So, the good agencies and strategists looked at how people were using the web, and eventually changed the way they thought about marketing to people online.  The problem is that the way we think about evaluation and measurement has largely remained rooted to the past, ignoring the realities of human interaction and attempting to retrofit social interaction with reach and frequency, or worse – attempting to express a complex interaction on par with a casual glance at a display ad in a newspaper.  The problem with this, of course, was that the measurement was left in the hand of those who were selling the advertising space.  What resulted was the baffling notion that 2.5 people read a newspaper cover to cover, or that 20,000 Neilsen families that nobody has ever met decide who’s watching what based on what they watch for a minimum of 2 minutes.  Understandably, when new marketers, who were measured not by guaranteed placement, but by human behaviour had to compare results, they couldn’t stack up side-by-side.

The challenge for new marketers is not measurement – any interactive medium offers no shortage of things to measure.  The challenge will be to move an entire industry away from its desire for comparing apples to oranges and establishing new benchmarks for success that do not rely on comparison to an irrelevant and inherently flawed system that is clung to simply because it’s the way things have always been done.  In order to do that, we need more case studies like Direct2Dell or Fiskateers, which may not have measured up in terms of CPM, but did more for a brand and for the bottom line than any print ad could have.