.

In case you haven’t noticed, the Internet has fundamentally changed how people gather information. This has affected business marketers in particular. Because Web sites now provide so much information that previously came from salespeople, marketers stay engaged with prospects for much longer. This means they must do a better job of understanding and responding to prospect interests, and of deciding when it’s finally time to turn them over to sales.

The change from simply generating leads to actively nurturing them is probably the main engine propelling growth of “demand generation” vendors like Eloqua, Vtrenz, Manticore, Market2Lead and Marketo. Their products, and at least a dozen competitors, manage traditional lead generation campaigns. But the goal is no longer just getting a name and handing it to sales. Instead, it’s to draw people to the company Web site, where they will join equally anonymous visitors from print ads, Web ads, trade shows, search engines, and other sources.

The real work of the demand generation system starts with that first Web visit. It begins tracking visitors’ behavior, trying to deliver the information they need at the moment they need it, and convincing them to surrender information about themselves in return. If this sounds like a seduction, that’s because it is one.

The moment of truth comes when marketing sends the lead over to sales. If the lead isn’t ready, then sales will complain about low quality. If marketing waits too long, opportunities may be missed. Like Goldilock’s porridge, the leads must be not too cold or too hot, but just right.

The instrument used to measure their temperature is lead scoring. Demand generation vendors recognize how important this is and are rapidly improving their scoring systems in response. Typical enhancements include increasing the scope of data that can be scored, adding precision to the score calculations—for example, by reducing the value assigned to each event based on recency—and making it easier to set up the scoring rules.

But these efforts face a fundamental problem. Traditional lead scores were built by marketing and sales experts deciding how what weight to assign to each attribute. This worked well when not much information was available: typically little more than source, company, job title, and BANT (budget, authority, needs, timeline), gathered at the start of the process. In fact, jointly defining the scoring rules was one of the best ways for marketing and sales to align their understanding of lead quality.

Today, the volume of data has exploded. Demand generation systems track each page view, document download, and email open. They combine information about different visitors from the same company, based on a shared Web domain. And they look at the timing of these events to understand when prospects’ interest is reaching a peak.

Rules of thumb collapse under so much detail. Marketers need formal data mining projects to identify the most important events and behavior patterns. These projects correlate prospect attributes and behaviors from the demand generation system with results captured in the company’s sales automation applications.

Assembling this data is relatively easy, since the demand generation systems are design for tight integration with sales automation systems, and Salesforce.com in particular. But these systems do not provide data mining and predictive modeling tools.

This is no problem for data mining, where most work is done by statisticians who prefer their favorite systems anyway. But for predictive modeling, most scoring formulas are too complex to replicate manually in other systems. The demand generation systems will eventually need to import scoring formulas from external modeling systems, or to call those systems to generate the scores and return them.

Other lead scoring enhancements will follow. Current systems require marketers to manually assign a weight to each event or class of events. The work involved limits how precisely the weights can be tuned to each item. But content analysis systems already exist that could automatically classify the actual message within each item, allowing more precise weighting with no manual effort. Similarly, existing systems that search the Web and assemble information about a company or individual could easily enrich the prospect profile with new scoring inputs.

Content classification and Web searches will initially be provided by third party systems. The demand generation vendors may eventually build these directly into their products, but a better solution in most cases will be to simplify integration with external specialists through APIs or Web services. This will let the demand generation vendors focus on their core products and let their clients benefit from continued progress in other fields.

These enhancements will be valuable even if they are not immediately integrated with lead scoring. Salespeople already use demand generation systems to generate automated alerts based on customer and lead behaviors, and then to list those behaviors for manual review. Better content classification and automated external search could make the alert rules more powerful and better organize the data presented for review.

The fundamental challenge for demand generation vendors will be to add these and other capabilities without making their systems too hard for marketers to use. This is a painfully common dynamic in the software industry: competitive pressures force vendors to add features, and complexity grows as a result. Demand generation vendors face an unusual counter-pressure from systems targeted at small businesses, which are purposely kept simple. We’ll see if this keeps them from following the usual path.

* * *

David M. Raab is a Principal at Raab Associates Inc., a consultancy specializing in marketing technology and analytics. He can be reached at draab@raabassociates.com.

.

.

Last month’s column presented several return on investment calculations for data quality projects. These were the measures that business people look at: profit per customer, promotion effectiveness, value per response, return on promotion expense. Let’s look at technical measures of data quality for those same cases.

- profit per customer. An automobile dealer made service history available to salespeople while a new car purchase was being negotiated. The business value came from targeted offers to increase use of the highly profitable service department.

Technical data quality measures included:

- speed of access: this is the time it takes the salesperson to retrieve data for a customer. Multiple queries may be needed before the system returns a satisfactory result, and salespeople will not bother if it takes too much effort. Elapsed time would be gathered from system logs.

- match rate: this is the proportion of successful matches returned by the system. There are separate statistics for correct matches, false matches, and missed matches. Match accuracy is often difficult to measure because the correct answer is not known. But in this case, the customer will know whether she has previously used the service department. The salesperson should therefore know when to look and keep trying until the system returns a match. This means the most important measure is “correct results returned on the first try,” as shown by the number of successful single-search sessions. Successful searches are followed by a request to view the underlying data. Abandoned searches are not.

- service data quality: this includes all quality components—accuracy, completeness, consistency, currency, and suitability to task. Since the service history is derived from the service department’s billing system, it should be reasonably accurate, current and complete. This would be confirmed by the company’s normal auditing functions. Consistency is measured by profiling the data over time to identify unexpected values or value distributions. Profiling can also detect improper or fraudulent billing—something the service manager may or may not be particularly eager to explore.

Suitability to task is a particular challenge, since the data is being used for something other than its original purpose. The system must summarize the raw service data to show aggregate purchases, changes in usage patterns, types of work (e.g. all routine maintenance or only major repairs), and inferences about customer needs (high mileage, off-road travel, heavy loads, etc.). Summarization depends on the core data quality measures of accuracy, completeness and consistency.

Even summarized data can be difficult for a salesperson to interpret, so the system should also recommend a best offer. Recommendation quality is measured by tracking how many recommendations are presented by the salespeople, how many of these are accepted by the customers, and their long-term impact on customer profitability. Presentations and acceptances can be measured directly so long as salespeople record their results. Long-term impact requires tracking customers over time.

Similar technical data quality measures apply to the other three cases. Space is limited, so, briefly:

- promotion effectiveness. This was a project to improve accuracy of a packaged goods manufacturer’s lists of distributor contacts. The business value was better execution of retail promotions. Technical data quality measures include:

- list accuracy: determined by random telephone calls to the distributors to verify the names on the existing lists. Returned mail and rejected email addresses may also provide information.

- update speed: determined by tracking how often the sales force provides list updates. This will identify salespeople who are not participating.

- value per response. This described an online marketer’s project to reduce bad debt and improve product recommendations through better real-time access to customer history. Technical data quality measures include:

- match rates with internal systems: measures include the percentage of successful matches, the percentage of confident matches (using a system-generated confidence score), and the percentage of multiple matches (more than one customer record matches a single input). Here, independent validation of match accuracy may not be available.

- match rates from external sources: confidence scores may not available, so the only measure is the match rate itself. Some verification is needed to measure false matches—a particular issue with external vendors who are paid on the number of hits.

- quality of results from internal systems. Completeness is measured by the scope of data provided: purchases, payments, returns, refunds, and service interactions. These may originate in several different systems. Currency is measured by how long it takes a new transaction to become available. It can range from milliseconds to a month.

Important non-data quality measures include response time and prediction accuracy.

- return on promotion. This described direct response marketer who used lifetime value to optimize promotion spending. Technical data quality measures included:

- cost data: accuracy, completeness, and consistency. The most important measure is percentage of missing values, since many marketers fail to record the necessary information in the marketing system. Another key measure is variation between the marketing system and the accounting system, since entries in the market system may not be revised to reflect actuals.

- customer integration: accuracy and completeness. Records for the same customer are set up independently in several systems and then merged. Measures of incomplete merges include refunds without a corresponding purchase, and repurchases without an initial order.

* * *

David M. Raab is a Principal at Raab Associates Inc., a consultancy specializing in marketing technology and analytics. He can be reached at draab@raabassociates.com.

.

Everyone agrees that data quality is important, but that doesn’t make them willing to pay for it. Any manager asked to approve a significant data quality project will rightly want to understand its return on investment.

Sometimes the justification is a simple cost reduction: fewer duplicate mailings, fewer misdirected shipments, fewer customer service transactions to correct mistakes. But often the benefit comes from higher revenue: better data will allow more effective promotions or more precise (that is, higher) pricing. Data management professionals are frequently unfamiliar with the details of such analyses. Even the business counterparts who are supposed to provide the necessary input may not know how to structure them to satisfy financial gatekeepers.

The challenge usually lies with the value calculation: the “R” in ROI. After all, investment is not much different from any other systems project. But how do you estimate the value of incremental revenue—or, indeed, what that revenue might be?

Here are several real-world projects that involved benefits from improved data quality. Although they cover just a handful of the possible situations, they may inspire insights that apply to your own business.

- higher profit per customer. An automobile dealer wanted to increase revenues from its service operation, the primary source of business profits. A proposed integration project would make the service history of existing customers available to salespeople while a new car purchase was being negotiated. The value came from helping salespeople to make the most appropriate service-related offer for each purchaser. Existing service department users would be offered a long-term contract to lock in their business, while sporadic users would be given discount coupons to encourage them to come back. In this situation, returns were measured in terms of increased profit per customer. This incorporated a company-wide view that included profit on the sale itself, profit on future service revenues, and profit from financing activities.

- improved promotion effectiveness. A consumer goods manufacturer relied heavily on retail promotions executed through its distributors. The manufacturer was aware that it often paid for promotions that never reached the store aisles. A little detective work found that materials for these promotions were sometimes not delivered to the distributor or, more often, they were not placed in the store by the distributor’s field staff. Digging further, it turned out that the company’s contact lists for distributor field staff were often outdated. As a result, distributors often did not receive notice of planned promotions or know who to contact when materials were received unexpectedly or expected materials were missing. The value of the project to fix these lists was based on a reduction in wasted promotion materials—which accounted for about half the total marketing budget—and the revenue gain from increasing the number of promotions that were actually executed.

- increased value per response. An online marketing organization used email and Web advertising to generate orders. For each response, the company had to decide whether to require payment in advance of shipment. This was a delicate balancing act, since pre-payment reduced the number of orders, but credit often resulted in bad debt. In theory, the company could identify likely no-pay customers based on previous behavior, but poor matching and disconnected fulfillment systems meant only a portion of this history was available while the order was being processed. The project to improve matching and data access was justified by the value of better credit decisions: orders would increase because more good customers were given credit, while bad debt would drop because more bad customers had to pay in advance. Access to previous purchase history also would allow the company to better recommend additional products to buy once the initial order had been accepted. The profits from higher add-on sales per responder might actually exceed benefits of the improved credit decisions.

- optimal return on promotion expenses. A direct response marketer acquired customers at a loss in order to make profits on future sales. It had a wide of choice of products to offer for the initial promotion and a wide range of channels to reach them. Each product group traditionally evaluated acquisition promotions based on cost per order and expected revenues for future sales within its own group. Obstacles to a company-wide measurement of each customer included multiple account numbers for the same customer, and lack of accurate cost data for lifetime value calculations. The value from removing these obstacles would be measured by the increase in company-wide profit from reallocating promotion expenses to the most profitable acquisition product and channel combinations. This required calculating the lifetime value of customers acquired by increasing or decreasing promotion spending for each combination—a demanding but doable task. Moving investment from the least to the most profitable options would result in a substantial long-term profit increase with no change in promotion expense.

In one sense, each of these projects is justified in the same way: by higher company profits. But identifying the specific mechanisms that will generate these profits yields credible, understandable return on investment calculations. These are much more likely to result in project funding than a generic appeal to the value of data quality. Although details for your projects will be different, a similar approach should also serve them well.

* * *

David M. Raab is a Principal at Raab Associates Inc., a consultancy specializing in marketing technology and analytics. He can be reached at draab@raabassociates.com.

.

Bank marketers are increasingly recognizing the opportunities of customer-initiated contacts such as telephone calls and Web site visits. Outbound messages risk being ignored, ill-targeted or intrusive, while inbound messages start with the customer’s attention, can be tailored directly to the situation, and are clearly triggered by the customer’s own actions. As a result, they are much more likely than an outbound message to yield a productive result.

Given these advantages, why isn’t inbound marketing more common? The problem is simple: like tongue-tied adolescents as a school dance, banks don’t know what to say when an opportunity presents itself. So they stare down at their metaphorical feet, listen silently to the hold music, and eventually wander off without having tried to make a connection.

But help is on the way. New technologies can teach banks to understand what customers want and how to offer it to them. They can even deliver the right message at the exact moment it is needed. Think of them as can’t-miss pickup lines for financial institutions.

The first challenge in successful inbound marketing is listening. Having a human involved helps—but call center and branch agents focus on solving the customer’s immediate problem, not assessing the situation for marketing opportunities. Nor do most agents have the skills, training or personality to do a good job of marketing. So whether an agent is involved or it’s a fully automated interaction on a Web site or ATM machine, technology should handle most of the marketing-related listening.

This listening has at least three components. One is literally understanding the customer’s words. The process might begin with spoken words that are converted to text through speech analysis software, or it may originate as text in an email message, Web query or agent call notes. Either way, text analysis software will then parse the message to identify key attributes such as products mentioned, terms demanding attention (e.g., “attorney”), and emotional content. Current technology can extract these sorts of items, but it hasn’t reached the stage where it can reliably understand the exact meaning of how they are being used. That is, the software might recognize that a conversation involves free checking accounts, but not assess whether the customer already has an account or is considering opening a new one, let alone precisely which features would be most important.

This brings up the second component of listening: tracking specific activities in company systems. Technology can monitor the events during an interaction—accounts opened or closed; deposit, transfer, and withdrawal transactions; balance inquiries; data from forms; search terms entered; Web pages viewed; and so on. These are much less ambiguous than streams of text.

Current transactions can be further enriched by the third component of listening, which is placing the current interaction in context. This brings in customer data such as existing accounts and balances, past transactions, service history, and background information in company systems or from external sources like a credit bureau. It can also include non-customer data such as the current workload in the call center, current promotions, and profitability of specific products.

Taken together, these three forms of “listening” provide a rich view of a current interaction: a much richer view, in fact, than a human agent could assemble on her own. It can be hard work to assemble all this data, and the initial implementation of an inbound marketing system is unlikely to be complete. But even partial data can be adequate input to the next task: making sense of what’s happening and deciding what to do about it.

This step usually involves a combination of business rules and statistical models. The models predict specific behaviors, such as probability of accepting a particular product offer or of closing an account. The business rules make decisions, or recommendations if a human is involved: offer this product, waive that fee, present these selling points. The rules themselves often incorporate model scores, which helps keep the rules simple: the rule might indicate it’s time to make a product offer, but let the model select the specific product based on likelihood of acceptance, profitability, expected impact on retention, and other factors. Rules and models can also provide non-marketing guidance, such as flagging a transaction for fraud review or identifying a credit risk.

Once the system has decided what to recommend, this must be fed back to the system conducting the interaction itself—the call center, branch workstation, Web site, ATM, or another. Modern “customer-facing” systems are designed to make this possible without major modifications. Older systems can be harder to work with, but technologies exist to allow superficial integration even if the inner workings of the customer-facing system remain hidden.

The final step in the inbound marketing process is learning from the results. The system records the decisions it has made, what was actually presented to the customer (which may not be the same thing if a human discretion is involved), and how the customer responded. Some systems automatically analyze this information and adjust future recommendations to be more effective. In other systems, the information is analyzed separately and then reviewed by business people who decide whether changes are needed. Automated and non-automated approaches each have their advantages, but in practice, even an automated system must be watched closely by human beings to ensure it doesn’t spin out of control.

What benefits can marketers expect from these sorts of systems? Published results can be hard to find, but here are a few. Key Bank increased revenue per call by 23% after installing a call center recommendation system from eglue www.e-glue.com. Barclay’s Bank doubled the number of inquiries on portions of its Web site using Omniture’s Touch Clarity www.omniture.com to tailor content based on observed customer activity. Holland’s Spaarbeleg retail bank added $30 million in sales on one million calls to its service center with SPSS www.spss.com PredictiveCallCenter.

In other words, this isn’t science fiction. Inbound marketing is a proven approach with very substantial benefits. It’s one you should give a good close look at your own institution.

* * *

David M. Raab is a Principal at Raab Associates Inc., a consultancy specializing in marketing technology and analytics. He can be reached at draab@raabassociates.com.

.

Last month I wrote about the importance of scenarios in helping to select the right software. But even scenarios only help buyers understand how a product works. To make a truly sound decision, they must know how it will impact their company.

This means the software must be assessed in the context of other company systems. A list of requirements can be misleading because some of the requirements may already be met by other resources. But you can’t just ignore those requirements: if a new product is better at them than the existing tools, that value of that improvement should matter.

To place the new product in context, the evaluation team must first describe how the company functions today, and then see what would change if the new product were added. If several products are being considered, they must assess the combination of each product with the existing systems.

The mechanics of this approach are not so different from conventional assessments. It still starts with a requirements list, preferably built by analyzing company processes through scenarios. What’s different is the next step: preparing a set of scores for how well each requirement is met today. This is the “base case”. Then the team creates additional sets of scores for the base case plus each new system (less existing systems that would be removed, if any).

The practical impact is that a new system gets credit if meets a requirement better than the existing systems, but is not penalized if it doesn’t. Instead of comparing the new systems in isolation, you’re measuring the value they add to your business. This is what you really want to know.

Converting “requirements met” to “business value added” is not easy. With analytical systems, the business value often comes from making better decisions, not from processing transactions more quickly or at lower cost. How do you measure the contribution that different analytical systems will make to better decisions?

One approach is to measure who does the actual work. The logic is this: in most analytical systems, the primary goal is to answer managers’ questions. So the best system is the one that gets managers their answers the most quickly. This depends on how far from the manager each question must travel before reaching someone who can answer it.

In other words, analytical systems involve tiers of users, starting with the manager herself. If she can answer a question without help, she will. If not, she will ask a someone on her staff who has greater technical skills and probably more powerful tools. Let’s call that person a business analyst. If the analyst can’t answer the question, he will hand it off to specialist such as a statistician working in a corporate decision management group. If that group can’t get an answer—usually because it requires data that hasn’t already been exposed in a data warehouse or service environment—they turn to the IT department.

Each step in this chain adds time. Therefore, the value added by an analytical system can be measured by how many answers it moves closer to the manager. For example, automated predictive modeling systems shift the ability to build models from statisticians to business analysts, speeding up the process and lowering the cost dramatically.

The table below provides a concrete illustration. It defines five levels of effort to answer a question: read an existing report, drill down in a business intelligence system, analyze data extracted from an existing BI or reporting system, analyze data in a data warehouse (but not exposed to end-users in a BI system), and add data not already in the warehouse. Numbers represent the percentage of questions of each type that can be answered by each user role (they add to 100% reading across). Column totals show the capabilities and workload for each group.

In the example, “Base” shows the current situation and “New” shows capabilities after adding a new business intelligence tool. As the “Change” row illustrates, the new tool slightly empowers Managers and greatly increases the capabilities of Business Analysts. The workloads of Statisticians and IT are reduced accordingly.

% Answers Provided by Each User Type
Case	Question Type	Manager	Business Analyst	Statistician	IT
Base	Read Report	100	-	-	-
	Drilldown in BI	40	40	20	-
	Analyze from BI	30	40	30	-
	Analyze from Warehouse	-	30	70	-
	Add New Data	-	-	30	70
	Score Total	170	110	150	70
New	Read Report	100	-	-	-
	Drilldown in BI	50	50	-	-
	Analyze from BI	30	60	10	-
	Analyze from Warehouse	10	80	10	-
	Add New Data	-	40	30	30
	Score Total	190	230	50	30
Change	Score Total	+20	+120	-100	-40

So far so good. But how do you find the best choice among several systems? I’ll answer that question next month.

* * *

(Second of a two-part series)

The first part of this series showed why the traditional approach of comparing software products against each other less useful than evaluating how they complement existing company systems. It also proposed a new measure for the value of analytical systems: the number of user groups a question must pass through before reaching someone who is able to provide an answer.

The table below repeats the example from last month. It shows five types of analytical questions, requiring different levels of technical skills and resources to answer. The “Base” case shows the percentage of questions that can be answered by four types of users with existing systems, and the “New” case shows the percentages with a new system added. The change in score totals shows that the new system allows Managers and Business Analysts to answer more questions, while shifting work away from Statisticians and IT. The bar chart illustrates the same changes even more clearly.

% Answers Provided by Each User Type
Case	Question Type	Manager	Business Analyst	Statistician	IT
Base	Read Report	100	-	-	-
	Drilldown in BI	40	40	20	-
	Analyze from BI	30	40	30	-
	Analyze from Warehouse	-	30	70	-
	Add New Data	-	-	30	70
	Score Total	170	110	150	70
New	Read Report	100	-	-	-
	Drilldown in BI	50	50	-	-
	Analyze from BI	30	60	10	-
	Analyze from Warehouse	10	80	10	-
	Add New Data	-	40	30	30
	Score Total	190	230	50	30
Change	Score Total	+20	+120	-100	-40

…

This approach gives a good picture of how a new system will affect the organization. If several new systems are being considered, comparing the charts for each would be enlightening. But it wouldn’t tell you which one to buy.

Picking a single system requires combining the workload figures into a single number that can be used to rank the alternatives. To do this, the detailed figures must be weighted on two dimensions: question type and user type.

Questions types are ultimately the same as user requirements. Since weighting on user requirements is part of any evaluation methodology, this poses no new challenge. (The example simply added the five requirement scores, which implicitly weights them equally.)

But weights for user types are something completely different. These reflect the relative value of having different users answer the same question. Our original premise is that answers from users “closer” to the manager are worth more because they are received sooner. They are probably cheaper too. So we know the value weight is highest for answers from managers and lowest for answers for IT.

It’s possible to calculate precise ratios between these weights, based on factors like turnaround time, labor cost, accuracy, and opportunity cost. But in most cases an intuitive estimate will suffice. In the example below, weights of 4, 3, 2 and 1 have been applied to the original New and an alternative business intelligence product, New 2.

Summary Value Calculation – New vs New 2
Case		Manager	Business Analyst	Statistician	IT	Combined Value
New	Score Total	190	230	50	30
	x Value Weight	4	3	2	1
	= Value	760	690	100	30	1,580
New 2	Score Total	200	140	130	30
	x Value Weight	4	3	2	1
	= Value	800	420	260	30	1,510

Examining the user-level scores, we see that New 2 gives Managers slightly more capability