I'm a web strategist and aspiring know-it-all with a passion for all things digital. I've worked in PR, advertising and not-for profit industries, and now I run a creative agency. These are the things I think about, and am sometimes compelled to write. More...

Category Archives: Measurement

I don’t know (and neither do you.)

Jan 04 2010

Q. What copy will perform best on my website?
A. I don’t know.

Q. What page should I direct search traffic to in order to get the highest conversion rate?
A. No clue.

Q. What time of day will get the best open rate on my email marketing campaign?
A. Dunno.

A big part of any consulting-style job, be it advertising, PR, business process or any other role in which the central function is to incite action across a broad group of people, is answering questions. To be successful, you need to be part futurist, part expert and part hand holder and teller that everything’s going to be okay-er. But the reality of this type of work – especially that which deals with newer or unproven media or channels – is that sometimes the answer to the question the client is asking is “I have no idea.”

But that’s not much comfort to clients.

So, we answer the question based on our experiences, our intuition, our understanding of media and consumers, and we guess… just a little bit. But more and more, the guesswork is coming out of the profession. We have access to analytics, measurement systems and tracking that communicators have never before had access to, and for the most part, they’re inexpensive and easy to integrate. We can tell where our best-converting sales leads are coming from, what types of posts have the best engagement, and when the best time to send out our email communications is – all from easily collected real-time data.

So why are we still guessing?

Part of the reason is that advertising people have always been regarded in part as soothsayers. It was this confidence in our knowledge of the medium, the message and the huddled masses yearning to be sold to that garnered multi-million dollar ad budgets. Conversely, it’s the expertise and ego that got CMOs through the ranks to where they are today. So where is the incentive for anyone in this equation to ever utter the phrase “I don’t know?”

As a society, we’ve always had an odd fascination with mediums and psychics who pretend to be able to see the future, telling us, to our amazement, what would happen to us, would we find love, how we would die. Of course, these were all parlour tricks based on intuition and a controlled situation. And that’s exactly what we’re doing when we predict what colour “buy” button will perform best, or where to put the call to action on the landing page. Our experience, knowledge and understanding of the media combines with our intuition to make an educated guess, but that’s usually all it is.

If, on the other hand, both the client and the agency embraces the fact that they don’t know the answer – that’s when real answers can be found. We can A/B test copy to see which performs best. We can analyze data to deduce why people are coming to our site. We can built multiple landing pages for multiple keywords and choose the one to go with based only on which one makes the client more money.

Of course, it’s impossible to test everything, which is why we rely on experts in the first place. If you’re Google, you’ll test 42 shades of blue to determine what colour your background should be, but that’s impractical at best, and an impediment to creative thinking at worst. Instead, look to solve problems by focusing on the most probable solutions (based on intuition), and determining which one works the best (based on numbers). It’s not as sexy as a crystal ball, but it’s certainly better for business.

photo credit: Frogman!

Video and Measurement

Sep 08 2008

For a lot of reasons, I’ve been thinking a lot about video lately – not only in terms of how it can help build brands and community through social media and PR, but specifically how we can understand what the results of this brand and community building mean.

In most cases, unless you’re spending a ton of money on analytics packages and hosting videos internally, you get one metric about your online video efforts – views. In many cases, what a “view” means is not even particularly clear – is it based on people watching a whole video or just the first few seconds? How are offsite views counted? In general, measuring views of a video is a blunt metric that gives a general idea of how many impressions a video gets. Once again, we’re measuring eyeballs.

Katie Paine, of KDPaine and Partners, often calls out measurement companies and consultant for focusing on impressions in online media, when what really matters is action, and I think it’s just as important for understanding how online video fits into the architecture of persuasion. Consider which is the more valuable insight – knowing the number of people who watched some part of a video, or knowing how many took action at a specific point in a video, and the percentage of people who took action as a result of one message versus another.

One of the reasons that this has been on my mind in the past few months is due in large part to the fact that one of my clients – Overlay.TV – is making this type of measurement possible by allowing content producers to make any video interactive by adding links, text, animation or even video-in-video and in-video chatting.

So now, in addition to measuring impressions, by simply adding a few elements to a video, content producers can measure, through their existing analytics software, how many of those viewers actually took action and visited a site, purchased a product through an affiliate, or entered the sales funnel as a direct result of the video. By using the chat widget, they can also gauge real-time reactions to a video and begin to understand more about the content they’re producing and how it relates to the audience.

Overlay.TV came out of beta late last week, and is now available for public registration. It currently offers a number of widgets that make it easy to add measurable interactivity to any video in just a few seconds. If online video is part of your marketing mix, I highly recommend checking it out and experimenting with the tools. Overlays can be as complex or simple as you need, so it’s easy to add a few quick links or create a fully interactive video, and even easier to measure the actions that come out of it.

As online video evolves as a marketing medium, measurement that goes beyond impressions and allows marketers to better understand how a video drives action will be more important than ever. To see it in action, check the newly-launched Overlay.TV or read about the company on TechCrunch.

The meaning behind the measurement

Mar 26 2008

Sitting at a conference a couple of weeks ago, I sat in stunned silence as a member of a panel on gaming and engagement suggested that if interactive, gaming and social media wanted to grow its slice of the advertising pie, we would have to learn how to measure in terms of CPM. On the surface, it sounds like a reasonable proposal – after all, a unified system of measurement for advertising can only help us all compare and evaluate results.

The problem, of course, is that there can be no truly unified measurement between radically different systems any more than you can measure the distance between New York and Vancouver in kilograms. This is not a weakness of either the tactic, its measurability or the understanding of the medium, but rather a weakness in the desire to do things the way we’ve always done them.

Traditional media is measured very simply. An advertiser buys reach and frequency based on the distribution of a particular medium, for a price ususally expressed in cost per thousand. Whether the medium is newspaper, television or radio, there is a relative assurance that the advertiser will receive the number of impressions that they have paid for, and the value of those impressions is based on the value of the eyeballs it will be supposedly reaching. The key thing here is that regardless of the medium, there is always a comparative metric and a comparative value.

When the web was still in its infancy, still being formed by the collective fingers of its users, advertisers co-opted this new medium in very much the same way. If you wanted the eyeballs of Yahoo! users, you bought impressions based on CPM. But as the way we used the web became more complex, and the technology and infrastructure that supported it became much more sophisticated, it became impractical to reach people through a traditional broadcast approach.

So, the good agencies and strategists looked at how people were using the web, and eventually changed the way they thought about marketing to people online. The problem is that the way we think about evaluation and measurement has largely remained rooted to the past, ignoring the realities of human interaction and attempting to retrofit social interaction with reach and frequency, or worse – attempting to express a complex interaction on par with a casual glance at a display ad in a newspaper. The problem with this, of course, was that the measurement was left in the hand of those who were selling the advertising space. What resulted was the baffling notion that 2.5 people read a newspaper cover to cover, or that 20,000 Neilsen families that nobody has ever met decide who’s watching what based on what they watch for a minimum of 2 minutes. Understandably, when new marketers, who were measured not by guaranteed placement, but by human behaviour had to compare results, they couldn’t stack up side-by-side.

The challenge for new marketers is not measurement – any interactive medium offers no shortage of things to measure. The challenge will be to move an entire industry away from its desire for comparing apples to oranges and establishing new benchmarks for success that do not rely on comparison to an irrelevant and inherently flawed system that is clung to simply because it’s the way things have always been done. In order to do that, we need more case studies like Direct2Dell or Fiskateers, which may not have measured up in terms of CPM, but did more for a brand and for the bottom line than any print ad could have.

Are Marketers Addicted to Transaction?

Nov 14 2007

I talk to a lot of brand managers and marketers about their online campaigns and the goals that they’re trying to achieve. At the end of the day, clients are looking for one thing when they hire an agency – increased revenue. True – community and connection have external rewards for both brands and the community it serves, but the end goal is always the same… this is business, after all.

When I bring up social media with them, I get one of three responses. Either they fully understand the value of community, have not considered it but are open to the idea, or they believe that community, content creation and “joining the conversation” are notions that fall outside the realities of marketing.

For some reason, the third group is most often made up of the marketing set. Their reason, when I prod a bit, is that their job isn’t to create community or become a broadcaster – their job is to create leads, measurable impressions, drive sales. I can understand this feeling – as I said, it is their job as marketers to create more revenue, and transaction is measureable, therefore a marketer who needs to justify his or her existence to a CEO or CMO can point to a transaction and say “Look what I did.” In most cases, if they create a lead that doesn’t turn into a sale, the blame lies outside of the marketing department and has to be sucked up by bizdev.

These are the marketers who eschew blogs for mailing lists, social networking for advertising and focus their marketing plans around creating quantifiable transactions rather than creating a vibrant and evangelical community.

The problem is that the buying process for most purchases is much more complicated than lead, opportunity, close. When consumers have unlimited choice at their fingertips, there are hundreds of factors that will affect intent to purchase. Landing pages may convince people to enter their email for more information, but will they increase trust? Will they convince me to tell my friends about a great service? Probably not.

Social Media can sometimes be a hard sell because most of the time, it’s not easily measured on a transaction basis. On top of that, it’s not about selling so much as it’s about educating and showing, which is very hard to swallow for a hyper-competitive sales-focused Marketing Director. In the end, however, a strong social media campaign executed for the right type of client strengthens reputation, trust, evangelism and a company’s listening power – all of which are incredibly important to reaching the almighty sale.

Transaction is important – there is no question about that – but ignoring the importance and power of community because it can’t be measured as easily strikes me as completely crazy. As social media people, it’s our job to educate client on how that grey area between community and sales actually works, but in order to do so, we have to understand it ourselves and accept that we may have to explain it with numbers to some clients. This is difficult, as social media is uncontrolled and deals in the realm of reality rather than metrics, but it’s not impossible. The key is understanding the marketing landscape for each client and showing them how community can move the numbers that they need to move.

Quantum Marketing and the Paradox of Measurement

Jul 10 2007

I’ve recently had a change in my job description that has caused me to put more focus on how I think about measurability in marketing and PR. To the advertising industry, measurement is often regarded as a nuisance… after all, burying your head in data after all the sexy stuff is done is not why most of us got into the industry. The reality, however, is that if you’re in charge of a brand, either as a manager or as someone representing a client, measurement is a necessary evil.

For a while, “branding” was the order of the day. This was great for agencies, because it’s in many ways an unquantifiable, ethereal concept that has very little in the way of tangible qualities. In other words, as long as you were “growing the brand,” it was much harder to hold anyone accountable, since there wasn’t anything to account for. This is not to say that measurement wasn’t done – but that it was much less of a concern than it is today.

The era of brand-centric marketing has given way to results-driven marketing, and with it has come an avalanche of tools with which to measure, quantify and analyze. Google Analytics democratized online measurement when it bought one of the top analytics companies around, rebranded it, and offered its services free to anyone who wanted it. Online CRM solutions like Salesforce.com allows us to measure our sales funnel in real time, and tools like Vocus allow us to measure the efficacy of our media relations, right from the comfort of our browsers.

The current reality of marketing communications management is that it is simpler than ever to measure the results of our efforts, but with this new simplicity comes a new problem – a disease of affluence. With all the data available to us with relative ease, there is a real temptation to measure everything we can, or to structure programs in a way that would allow things to be more measurable. In many cases, though, we can affect the outcome of our program by the way we measure or track it.

Say for instance, you have a brilliantly researched, written and targeted ebook / white paper / manifesto that you are providing to your clients free of charge in order to establish yourself or your company as a knowledge leader in your space. There are now two competing schools of thought on how to handle this. The first – the marketing-centric approach, would associate the document with an integrated CRM in order to funnel visitors into a sales funnel, qualify them, and follow up. The second, influenced by social media and the share-alike mentality of the blogosphere would provide it to everyone who visited their site without registering, and encourage their visitors to send it to as many people as they can, to post it on their blogs for download and to print it out and share it with friends.

The first approach is very measurable, and can be tracked all the way from download to sale. Marketers like this, but the audience is limited by the locked-down manner of measurement. The second is virtually impossible to track, but reaches much farther than any marketing department can, using existing nodes of influence to share an idea.

There is no “right” way to track a campaign of this nature – both have their own place. The thing to keep in mind is that how we decide to measure a campaign can directly affect its outcome. Success is defined in many ways, but it’s important to remember that sometimes, the greatest successes cannot be measured directly.