David Raab Article Archive

New Techniques for Marketing Measurement
David M. Raab
Information Management
November / December 2010

It’s fitting that the most famous quote about marketing measurement – “Half the money I spend on advertising is wasted. The trouble is, order I don’t know which half.” — is itself unreliable: although it’s typically attributed to John Wanamaker, pilule there is no definitive citation.(1) Nor has much changed since Wanamaker’s time: to paraphrase what Mark Twain (2) didn’t say about the weather, everybody still talks about marketing measurement but few do anything about it.

The problem isn’t lack of data. Marketers today can track the messages received and actions taken by their customers in more detail than Wanamaker could imagine. It’s true that the information is still not complete or fully accurate, but even perfect information would not yield the answer. The problem is fundamental: even if we knew everything that happened to a customer and everything that the customer did, we couldn’t calculate the how individual events affected the final result. All we see is what a particular set of inputs will produce in aggregate. It’s like following a recipe in a cook book: you know what the combination of ingredients will produce, but not how each one contributes.

Measuring the impact of individual marketing programs requires breaking apart the combination of inputs and testing the effect of changing them individually. This is why even complete data cannot solve the measurement problem. Only tests allow marketers to measure the incremental impact of specific activities.

There’s nothing new about this insight. But, with the glaring exception of direct marketing, testing has traditionally been expensive and inexact. As a result, marketers limited testing to a handful of major issues and sought additional insights from statistical correlations such as marketing mix models. But mix models are effective only in industries with very large customer bases and marketing budgets, and even there they can only measure broad effects.

In today’s data-rich environment, testing is much easier. Marketers can more often build databases of their customers and track many of the messages those customers see and respond to. Even formerly anonymous media such as broadcast and print advertising often drive consumers to trackable behaviors on the Web.

But tests face another obstacle: the time lag between many marketing contacts and a customer’s eventual purchase. Although revenue is the ultimate measure of marketing success, many marketing efforts occur early in the purchase process. Waiting until a purchase is completed before measuring their results introduces a lengthy delay. It also allows time for intervening events that could affect the final outcome. For both these reasons, marketers need a different measure they can read more quickly.

Much of the current innovation in marketing measurement is directed as finding such measures. The general approach is to divide the buying process into stages and to measure the impact of marketing efforts on moving prospects from one stage to the next. This is similar to the sales funnel or pipeline traditionally monitored by sales automation systems, except that that it begins earlier in the purchase process. It’s also similar to psychological models such as AIDA (awareness, interest, desire, action), which are often used to guide brand advertising.

The difference from past psychological models is that marketers are now tracking the status of individual consumers. This is what today’s new data sources make possible. In the past, companies rarely knew more about non-customers than basic demographics (age, income, gender, location). But these can’t track movement through purchase stages. Movement must be inferred from fresh information, whether provided directly through survey responses or inferred from behaviors such as Web site visits. Behavior is a much richer source because there’s more of it and it’s not limited by what prospects choose to provide.

In sum, then, marketers are finding they can greatly improve marketing measurement by using behavior to track consumers through stages in the purchase process, and using tests to correlate changes in this movement with changes in marketing programs. This creates several technical requirements, including:

– identifying individuals consistently over time and across channels, so you can build a database of their behaviors and marketing contacts. This database provides the input for tracking their movement through the purchase stages.

– data mining and analytical tools that uncover patterns, such as x Web visits within the past Y days, which indicate a prospect’s location in the purchase process.

– scoring systems that can apply complex patterns and other information to identify the current stage of thousands or millions of prospects on a continuous basis and to identify movement from one stage to the next. (In practice, movement does not always follow a fixed sequence: some prospects will stagnate, move backwards or skip ahead several stages at once.)

– tagging of content by stage, to help the scoring system. While the volume of behavior is important, the specific information consumed gives much more insight into a prospect’s current state of mind. Three visits to the white paper library means they’re just doing some research, while three visits to the contract terms means they’re seriously considering a purchase. Although content tagging can be major task, the tags are also needed to help select the right message for each individual. So the measurement system should be able to use tags that are already in place.

– maintaining a history of previous statuses, since they cannot be reconstructed using only current information. This is a classic “slowly changing dimension” in data warehouse terms.

Converting this data into actual measurements adds still more requirements:

– testing features including random selection, tagging of test group members, and test/control reports.

– stage movement measures such as average time per stage and continuation rates from one stage to the next.

– correlation of marketing contacts with prospect stages. This is used to calculate the marketing cost per stage and to show the impact of different contacts on stage progression

– integration of purchase history with prospect profiles, needed for revenue-related calculations such as return on investment.

– projections of future revenues from the current prospect pool. This uses the number of prospects in each stage, the expected continuation rates from one stage to the next, and the expected value of ultimate purchases. It is used to forecast business results to estimate the financial impact of differences revealed by tests.

Few of these capabilities are wholly new to marketing systems. Most are required for other purposes such as offer selection and segmentation. But making them available for measurement requires exposing them in different ways, so marketers and IT staff still need to ensure they are available for measurement projects.

The advantages of stage-based measurement make it worth the trouble. Breaking the buying process into stages lets marketers understand what’s driving results, identify problem areas and find opportunities for improvement. It lets them measure the true incremental value of individual marketing contacts, replacing arbitrary “revenue attribution” on the first or last touch or simplistic fractional weighting. Perhaps most important, stage-based measurement provides near-immediate feedback, allowing marketers to quickly reallocate resources to the most productive programs.

Stage-based measurement isn’t a complete solution to all marketing measurement problems. It has a short-term, incremental focus that may not capture the deeper value of branding programs. It measures each marketing program in isolation, making it difficult to assess interactions among several programs. And it often relies on the assumption that changes at one stage in the buying process ripple through to the end more or less undiminished. This last assumption is particularly dangerous, because the opposite is often true: positive changes at one stage often have negative consequence later on. (For example, a free introductory offer may attract more orders but fewer conversions to paid customers.) Sophisticated marketers will be aware of these issues and compensate for them. They are a reasonable price to pay for finally knowing which half of your marketing budget is wasted.

1. The Quote Verifier: Who said what, where, and when, Ralph Keyes, New York, NY: St. Martin’s Press, 2006, pg 2
2. It was more likely Charles Dudley Warner. See http://quoteinvestigator.com/2010/04/23/everybody-talks-about-the-weather/

* * *
David M. Raab is a Principal at Raab Associates Inc., a consultancy specializing in marketing technology and analytics. He can be reached at draab@raabassociates.com.