David Raab Article Archive

In the gold rush mentality of today’s software industry, being first in a new territory is the accepted key to riches. So once developers decided that marketing departments were the next, and maybe last, major unautomated business area, hordes rushed to stake their claims. Marketers, who until recently had little to work with beyond a spreadsheet and word processor, are now being offered a confusing jumble of shiny software baubles. So the new challenge for marketers is telling the different systems apart, and figuring which comes closest to meeting their particular needs.

Industry terminology provides little help. The most popular umbrella term, Customer Relationship Management, has been applied to just about everything from call center systems to data mining algorithms. The next-most-popular label, Marketing Automation, is barely more specific. While software vendors have many good reasons to apply these labels to their products, helping buyers to make informed purchasing decisions is not among them.

Fortunately, once you look past the labels, it is possible to discern meaningful differences among today’s various marketing-related software products. After analyzing dozens of these systems, seven distinct groups emerge. Each group has its own unique specialty, although individual products may straddle more than one category. It’s even possible to arrange the groups on one of the two-dimensional matrices so popular with industry analysts.

But let’s save the matrix for dessert, and hold off on the categories themselves until we establish a broader framework. Marketing systems don’t exist in a vacuum. They are generally acquired to support some larger corporate strategy related to the treatment of customers and prospects. Of course, since just about everything an organization does is somehow related to customers and prospects, a description of this larger group of systems looks a lot like any other description of the over-all corporate information system architecture–a topic that others have discussed in great detail. Still, and with apologies to the cognoscenti for necessary oversimplifications, it’s worth taking a look at Figure 1, which presents a marketer’s-eye-view of the corporate information structure.

Most pieces should look familiar. On the left are the “front office” or “customer facing” systems, like sales force automation, point of sale, call centers, customer service, help desk, Web sites and e-mail. These are systems that come into direct contact with customers and prospects. For some, like an e-commerce Web site or automated teller machine, the customer does the actual data entry; for others, like a telephone call center, the customer talks to a company employee who punches the keyboard. But in either case, the system is accepting and reacting to inputs as a transaction occurs. In other words, these are real-time, online systems that are at least theoretically capable of affecting the interaction between the customer and the company as it happens. Because these are real-time systems, their technology is similar to other transaction processing systems, typically using a highly normalized relational database structure designed to process individual transactions quickly.

In the past, each of the front office systems would have functioned independently, maintaining its own database of customers and transactions. But today, these systems are increasingly integrated in the sense of using a common database–represented in Figure 1 as the single vertical box they are all plugged into. This configuration–various front-office systems sharing a common database–is the domain of vendors like Siebel, Vantive, and Pivotal, who are pretty much at the heart of the Customer Relationship Management market. So even though the term CRM has today been stretched beyond almost any specific meaning, it is probably useful to call to mind this configuration when the term appears.

In the center of the diagram are the “back office” systems including manufacturing, distribution, finance, and human resources. These are also online transaction processing systems and today are often tightly integrated in ERP packages like SAP and Baan. As the double headed arrow indicates, they may exchange information in real time with the front-office systems for tasks such as checking prices or inventory availability during order entry.

Moving up the diagram, both the front office and back office systems feed the data warehouse. The warehouse in turn feeds analysis systems, and analysis systems feed into database marketing. This is oversimplified and others may feel strongly about some of the details. But the only point at hand is that these systems are mostly updated via batch, rather than real-time, processes, and they are structured for analytical rather than transaction processing. This distinguishes them quite sharply from the front-office and back-office systems.

Within this group, the “database marketing” label itself may need a little clarification. This is yet another term that has lost much of its specific meaning. Here, it refers to what is sometimes called first or second generation database marketing, which by definition involves creation of lists for non-interactive marketing efforts such as direct mail promotions. Such systems are updated through batch processes and therefore do not allow for real-time interactions. At best, they approach near-real-time through updates that occur nightly or several times a day. More about this later.

The very top of the diagram also shows external data sources, such as third-party demographic data. This can feed into the warehouse or analytical system, or sometimes directly into a front-office database. Such data is increasingly important, although it is not particularly relevant to the current discussion. It is included only for the sake of thoroughness.

The database marketing box has two outbound arrows–and here is where things again get interesting. The lower arrow, heading off into the market, refers both to traditional outbound promotions, and to slightly interactive marketing delivered through front office systems. “Slightly interactive” seems to make as much sense as “slightly pregnant”. It refers to messages that are generated in advance and then delivered via a front office system–but without regard to the nature of the current interaction. Think of a bank database marketing system that decides, “the next time this person calls, offer her a home equity loan.” While that may be the right thing to do when the analysis is run, it could be utterly inappropriate if the next call is, say, to close all accounts because the customer is selling her home. Even if the list of recommended messages is updated several times a day, such an approach will never be truly interactive. But it is still–rightly–considered to be sophisticated database marketing.

The other arrow out of database marketing leads to something labeled “interaction management”. This describes systems that are specifically designed to guide transactions as they unfold. This truly is a real-time system, gathering input from both front- and back-office systems as they execute transactions and providing them with instant feedback that reflects customer-specific marketing considerations. A typical front-office application would be a product recommendation provided during order entry; a typical back-office application would be determining whether to waive a penalty fee for a late credit card payment as statements are prepared.

At the core of interaction management systems are business rules that determine how to act in particular situations. The rules may draw on customer strategies or segments defined in the database marketing system and on data taken from the warehouse. This is why the diagram shows both systems providing inputs. But the application of the rules happens within the interaction management system itself. For clarity, it’s worth stressing that the interactions actually execute outside of the interaction management system, in the front- or back-office systems. The interaction management system only provides those systems with some marketing-oriented advice.

The reason for going through this diagram is to provide a context for understanding how the seven types of marketing systems fit into the larger information architecture. Next month’s column will look at the definitions of the systems themselves.

Making Sense of Marketing Software – Part 2

October, 1999

Today’s marketers have many needs–and software vendors are eager to meet them all, plus a few more the marketers may not even have known existed. While all this attention is better than being ignored, it does force marketers to try to make sense of their options. After looking at dozens of marketing-related systems, we have found they fall into seven reasonably distinct classes, each built around a particular capability. Here they are, in roughly increasing order of complexity:

– email campaign management. These are systems whose primary function is to broadcast email messages and track response. Core capabilities include maintaining a permanent database of customer profiles, selecting records using characteristics in the profiles, maintaining an inventory of personalized text and HTML email forms, embedding a unique promotion ID or Web page (URL) in the outbound email, broadcasting the emails to selected records, and providing response reports based on replies containing the promotion ID or hits against the specified Web page. Most systems can also capture additional information, such as replies to surveys embedded in a response form.

– interactive customer support. These systems manage interactions with customers, primarily in response to email inquiries. They are used mostly for customer and technical support, but sometimes for lead processing as well. Their key feature is a structured database of replies to common questions. They use this to classify inbound messages based on key words or text analysis, identify an appropriate reply, and send the reply automatically or forward it to a human agent for review. Like conventional call center systems, they also track performance statistics such as response time, successful resolutions, and agent productivity. Most products in this group can maintain a permanent database of customer/prospect profiles and contact history, generate Web forms and capture data from them, and run outbound e-mail campaigns.

– conventional campaign management. These are classic database marketing systems, designed to identify file segments, generate direct mail or telemarketing lists, and report on response. Their key distinction is the ability to split a file into multiple, mutually exclusive segments in a single step–as opposed to executing many independent queries that select one segment each. Many products in this group also let users create multi-step campaigns, which define a sequence of message that are executed automatically over time. Most campaign management systems support sophisticated marketing programs with random sampling, test/control splits, and methods for inferring promotion results when direct mechanisms such as response codes are not available. Selections are typically executed in batch processes.

– marketing automation. This term has been applied to a very wide array of systems, but the heart of the category is business-to-business lead management. This involves three clusters of functions: support for budgeting, project management, approvals work flow, reporting and other administrative tasks; creation of Web-based marketing materials including e-mail and Web pages; and design and execution of multi-step campaigns to nurture immature leads and deliver ripe ones to the sales force. Of these functions, administration is the glue that holds the others together and is also the least likely to be found in any other type of system. So while administration is not the only function of a marketing automation system, it can be considered the main identifying feature.

– Web site personalization. These systems create personalized Web sites, which can present different contents based on a visitor’s stated and inferred preferences. They provide tools to develop personalized Web pages and other content, link to external systems for data look up and order entry, maintain databases of available content, store and update visitor profiles, and hold rules to determine which content is sent to which visitors. Pages are generated in real-time and may change in response to user actions. But these systems do not execute the multi-step contact sequences of conventional campaign managers or marketing automation software.

– collaborative filtering. These systems return real-time estimates of the affinity between an individual and a set of options, such as a list of books, banner ads, or solutions to technical problems. Answers are based on how similar individuals have behaved in the past. The particular feature of these systems is the high degree of automation: they build and adjust their predictive models automatically as they observe customer behavior over time. This lets collaborative filtering systems work where there are too many options to manually define a rule to handle each one.

– interaction management. These systems provide marketing input to real-time interactions, such as an Web site visits or call center conversations. Their key feature is the ability to assess a specific situation and recommend a response that is optimal from a marketing perspective. This is usually done through sets of manually-crafted business rules, which can take into account business issues–such as the impact on long-term customer loyalty–that are not captured by looking at what similar customers have chosen in the past. Although the interaction management systems themselves cannot determine the long-term impact of a decision, manual rules allow marketers to execute strategies that reflect their own judgements.

Next month’s column will look at how these categories relate to each other and the larger corporate information architecture. Future columns will examine each category in more depth.

Making Sense of Marketing Software – Part 3

November, 1999

Last month’s column described seven classes of marketing software: email campaign management, interactive customer support, conventional campaign management, marketing automation, Web site personalization, collaborative filtering, and interaction management. These systems were selected because they are all used by marketers to manage direct customer interactions–which is why the list excludes other familiar categories such as predictive modeling, Web site analysis, and non-Web customer service, telemarketing, sales automation and product configuration systems. Modeling and Web site analysis systems are used by marketers, but don’t execute interactions–rather, they provide information to improve results from other systems. Customer service, telemarketing and other front-office systems do execute interactions, but are usually run by operational groups, not marketers. This is an admittedly arbitrary rule, and individual cases may be debatable within it. But it’s probably not worth much discussion.

A more interesting question is how these categories relate to each other. Last month’s article described the key capabilities of each group. These are summarized in Figure 1.

(Figure 1 – Marketing Software Categories)

As Figure 1 indicates, some features are shared by multiple categories. For example, both marketing automation systems and conventional campaign management systems let users define “multi-step campaigns”–a sequence of system-initiated contacts, with different branches on how the customer reacts initially. This is something that today’s email campaign managers do not generally provide.

While it’s possible to define the category features with some clarity, the boundaries get fuzzier when you look at individual products. Vendors are continually jockeying for advantage, and one key dimension of competition is breadth of product functionality. As vendors develop or acquire new capabilities, they produce systems that straddle the categories listed above.

This sort of consolidation is typical of a maturing industry, as point solutions are replaced by integrated packages. Although some buyers prefer to purchase “best of breed” solutions and do their own integration, most find the cost and speed advantages of buying an integrated system to be overwhelming. This is particularly true in the marketing arena, where time and technical resources are especially scarce.

Still, it will be some time before all seven categories are combined into a single Great Marketing System in the Sky. As Figure 2 shows, there are significant differences among the classes of marketing systems along two key dimensions: the processing interval, which ranges from real-time to batch, and campaign complexity, which ranges from none (responding only to the current situation) to complex (considering long-term goals and strategies). In broad terms, the processing interval is related to technology while campaign complexity relates to functionality.

(Figure 2 – Marketing System Grid)

As it happens, each of the seven categories falls into a different place on the grid. On the right, conventional campaign management, marketing automation and interaction management systems can all execute very complicated campaigns that respond to customer actions in a context of tactical and strategic considerations. But conventional campaign managers do this via batch processing–rarely more frequently than daily, and often no more than monthly. Marketing automation systems can manage near-real-time response to email and Web forms, but still don’t handle the real-time interactions of an online interaction manager. In the center column, email campaign managers use standard queries to select names for broadcast, and personalized Web sites link standard contents to preferences indicated by a Web site visitor: both approaches can be considered simple business rules. Of course, email messages are near-real-time at best, while Web pages are generated immediately. On the left, interactive support and collaborative filtering systems react to a specific situation–finding the best response to a question or the product a customer is most likely to purchase–without any particular concern for long term campaigns. Again, the interactive support via Web and email is near-real-time or worse, while collaborative filtering provides responses as the interaction unfolds in real time.

It is just a coincidence that each category falls into a different box on the grid. If other classes of software were added, some boxes would get more crowded: for example, front office systems like telemarketing or sales automation would fall into the same group as personalized Web sites: they are also real-time systems governed by simple campaign rules. Adding more groups could also fill the two blank boxes: simple list generation systems belong in the batch/no campaign category, and first-generation campaign management tools like bank MCIF (Marketing Customer Information File) software fall under batch/simple campaign. Both groups are technically qualified for the list; they are just too old-fashioned to be of much interest.

Next month’s column will explore what the grid says about product integration.

Making Sense of Marketing Software – Part 4

January, 2000

Last month’s column described a grid that classified marketing software based on processing interval (technology) and campaign complexity (functionality). This is reproduced in Figure 1.

(Figure 1 – Marketing System Grid)

What’s important about the grid is that it gives some idea of which product groups are the best prospects for integration. From a technical standpoint, the horizontal axis dominates: real-time systems have to be built in certain ways to generate quick responses, regardless of the complexity of their campaigns. So it will be technically easier to combine collaborative filtering with Web site personalization than to add interaction management to a marketing automation product.

It also seems logical that making a system run slower is easier than speeding it up: that is, adding batch or near-real-time capabilities to a real-time system is easier than adding real-time functions to a batch process. If true, this would mean it’s easier for a product to expand up the grid than down. But the fundamental requirements for high-performance batch processing and high-performance real-time processing are so different that this isn’t necessarily true.

The vertical axis dominates where marketing functions rather than technical issues are involved. For example, the campaign functions in a conventional campaign management system are much more similar to the campaign functions in a marketing automation system than an email campaign manager. This means that system designers and marketers will find it intuitively easier to understand how systems in the same column should work together, whatever the technical challenges of achieving this integration. Unlike the horizonal axis, the vertical axis has a definite hierarchy: it’s much easier to make a system run simpler campaigns than to add campaign complexity. So adding email campaigns to a marketing automation system is much easier than transforming an email campaign manager into a marketing automation product. But the hierarchy applies only to campaign management functions. When all aspects of a system are considered, a product on the left may be considerably more complex than one to its right. A good customer support system has many more moving parts than an email campaign manager.

Taken together, these observations provide some specific guidelines for vendors seeking to expand their products and for buyers considering a multi-purpose system.

– Products on different horizontal levels may appear similar functionally, but have different underlying technologies. So vendors who want to expand in this direction should anticipate keeping at least some separate data structures and processes to support the new level. This means their current technology (and technical staff) may not be appropriate for the new level–so they should consider purchasing an existing product, and pay close attention to the technology they are buying. Then they can expect a substantial integration challenge at both the technical and personnel levels.

– Marketers purchasing a system that promises to work at multiple horizontal levels should pay close attention to technical issues. If the system was initially developed to work at one level, how did the vendor expand its capabilities to the other levels? If it developed the expansion in-house, did it use different technologies for each level or try to use one technology for all? If it grew through acquisition, how were the different products and technologies combined? If the system is strongest at one level, is it the level you care about most?

– Products on the same vertical level are functionally different but share similar technologies. Vendors who expand in this direction will find it easier to develop an integrated system with their current technical resources but may find it harder to understand the marketing and business requirements of a new application. This means they are likely to need design help from users or specialists in the new field. They may also find themselves selling to a new group of users. If they grow by acquisition, vendors should plan to add the new functions to one or the other existing product rather than keeping separate data structures and processes. Which product they keep will depend on the relative complexity of the different systems, not their position on the grid.

– Marketers considering a system that crosses several vertical columns should look carefully at its functional sophistication in each area. As always, it’s useful to know the genealogy of the product–most systems were originally designed to provide a single function, and are still stronger there than in areas they added later.

– Marketers should also look closely at how a vertically-integrated system will communicate with external products. Point solutions are intentionally designed for easy communication, because they expect to be part of a larger group of systems. But integrated solutions hope to be self-contained, so links to external products are often less of a design priority. The problem is that most integrated solutions don’t extend to all contact methods–Web, phone, direct mail, field, and so on. So if the integrated functions are not accessible to external systems that support these methods, the user will need to duplicate the integrated functions or do without. For example, imagine personalized Web site software with embedded collaborative filtering. Sounds great. But what happens when somebody decides to do collaborative filtering at the call center as well? If the Web site collaborative filtering system isn’t accessible, the user has to buy another one. Now the embedded collaborative filtering doesn’t sound so great after all.

Making Sense of Marketing Software – Part 5: Conventional Campaign Management

February, 2000

The initial articles in this series defined seven major categories of marketing systems. The next group of articles will each describe one category in detail.

Conventional campaign management is the oldest of these categories, so it’s a good place to start. These systems trace their roots to the 1970’s, when banks built Customer Information File (CIF) systems to identify all accounts owned by the same customer. By the mid-1980’s these had evolved into Marketing Customer Information File (MCIF) systems that let marketers select names based on a complete view of each customer relationship. MCIFs were initially used to identify the best customers for specific products, thereby maximizing the efficiency of each marketing campaign. This closely mirrored traditional catalog and magazine direct marketing, which often split an audience into hundreds of cells and analyzed the performance of each one. But MCIF systems also allowed a more customer-oriented approach, where marketers defined a matrix of offers and chose the best one for each customer segment. Although relatively few bank marketers actually implemented matrix marketing, it was widely recognized as the best practice. In a more sophisticated form, it still is.

Both the product and matrix approaches relied on the campaign manager’s ability to hierarchically assign records into segments. That is, the system would first find all records that qualified for an initial segment, then check which of the remaining records qualified for the second segment, then check which of the still-unassigned records qualified for the third segment, and so on. In product-oriented campaigns, the segments let marketers measure the results of different customer groups to help find the best prospects. In the matrix approach, the segments controlled which customers got each offer.

In fact, the ability to execute hierarchical, multi-segment selections is precisely what distinguishes conventional campaign managers from other marketing systems. The most efficient campaign managers are engineered to do these selects in a single pass through the file–determining where each record falls in the hierarchy and then moving on the next. Unfortunately, single pass processing is sequential, while relational databases and SQL work with sets. As a result, most SQL-based campaign managers execute a separate query for each segment, at a considerable cost in processing time. The cleverer products gain back some efficiency by using special indexes or querying an interim result set rather than the original universe. Campaign managers vary greatly in how they handle this problem, so buyers who expect campaigns with dozens or hundreds of segments should examine the mechanics of each product closely before making a purchase.

A first-rate campaign manager needs several functions that can be found in other products as well. High on the list is tracking promotion history, typically by adding a new promotion history record each time a customer is selected. This allows reports to compare the people who received a promotion with those who responded. At a minimum, the promotion history record holds a customer ID and promotion ID. More powerful systems also let users include a snapshot of other data elements, to use in later analysis.

Other important functions include complex queries–things like ranking, ad hoc calculations, and comparisons across multiple records–and random sampling to set up tests and control groups. These are also difficult for standard SQL, although most systems handle them fairly effectively. Many products sacrifice the advanced query functions for ease of use–a compromise most users seem to find acceptable.

Finally, nearly all of today’s major products can automatically execute campaigns on a regular schedule. This is a relatively new function, added when systems changed from a monthly or quarterly updates to daily or weekly. The greater frequency allows timely reactions to events such as a major purchase. But despite the goal of near-real-time response, most systems run automated campaigns using the same batch processes as their original, manual campaigns. That is, the campaign “trigger” is a query that runs periodically against the entire file. Since such queries can consume significant resources, they impose a practical limit on how often the trigger campaigns can be run. A few systems avoid this problem by placing triggers on the database itself, so events are identified as they occur. But adding a trigger to a database field takes a fair amount of technical skill, which makes it unacceptable to many marketers.

Even if marketers loved setting up database triggers, the fact would remain that today’s campaign management products have been optimized for batch processing. Handling real-time interactions would require a major reengineering that few vendors can afford. The issue is now becoming critical as users demand support for email campaigns. While generating a list of outbound emails is pretty much the same as generating a list of mailing labels, the real user requirement is to receive the email replies, post them to the marketing database, and react–more or less immediately. So far, the leading campaign management vendors have slipped by with loose integration with external email and Web site software. But they will probably need a more sophisticated approach to remain central to their clients’ marketing efforts.

Making Sense of Marketing Software – Part 6: Marketing Automation

March, 2000

“Marketing automation” is one of those seductively vague labels that seem to promise greater benefits than any software can really deliver. Taken broadly, the term appears to imply automation of the entire marketing process, from product development through to sales and service. While this might indeed describe the ambitions of some marketing automation vendors, their products have been considerably more limited. Specifically, most marketing automation systems have focused on the primary operational responsibility of a traditional business-to-business marketing department: lead generation.

In the past, lead generation involved running various kinds of promotion campaigns, including direct mail, print advertising, trade shows, and seminars. Prospect names gathered through these efforts would be sent fulfillment materials, qualified using survey responses or external data, and then sent to sales for telephone or personal follow-up. Some marketing groups also placed the prospect names on a mailing list that received sporadic promotional mailings or, in a few cases, a well-thought-out sequence of follow-on messages. But most contact with prospects was the responsibility of sales, not marketing.

Today’s marketing departments still perform the traditional functions–and marketing automation software is designed to support them all. But business marketing has also been deeply affected by the Internet. Marketing often runs a large portion of the company Web site, replies to email inquiries, and engages in long-term dialogues with prospects. This gives marketers a new set of quasi-operational responsibilities that require more constant attention and faster turnaround than traditional lead generation efforts. Help in keeping up with the increased workload is a key reason for marketers to buy a marketing automation system.

The functional capabilities of marketing automation systems closely mirror their users’ business activities. These capabilities fall into three intertwined groups: marketing administration, campaign execution, and Internet interactions.

– marketing administration includes scheduling, budgeting, forecasting results, gaining approvals, and managing the execution of marketing projects. This focus on administration is unusual among marketing systems, but makes sense in the context of a marketing department: since it is often difficult to correlate marketing activities with specific revenues, managers must focus instead on execution costs and efficiency. After all, marketing is typically a cost center, not a profit center.

Administration often revolves around an integrated promotion calendar, which shows all marketing activities by time period, medium, product group and, sometimes, market segment. The calendar is usually presented in a Gantt chart format, allowing online users to “drill down” within the chart by clicking on the bar representing a particular activity and viewing its detail. Rubric (www.rubricsoft.com) and MarketFirst (www.marketfirst.com) are marketing automation vendors with especially strong calendar functions.

Approvals and project management are integrated with workflow functions that let the user assign tasks to individuals, track completion of tasks, and record approvals of key project components. Approvals may cover everything from the initial proposal and budget to specific pieces of copy. A good system will also provide a database to manage the promotion materials themselves, with information on both electronic components such as HTML snippets or telemarketing scripts, as well as physical items such as printed brochures and letters. Some products go further to include a database of vendors and internal resources. Ideally this would link to external databases such as email directories and accounts payable files, although such links are rare.

Specialized facilities for tasks such as trade show management, event registration and dealer certification programs are more common, although they vary from system to system. Most products let users create task lists and schedules for specific types of projects, and then store and reuse these as templates. Attune (www.attuneinc.com) has a particularly comprehensive set of marketing administration functions.

– campaign execution involves sending a sequence of messages to prospects and customers. Marketing automation systems generally offer a flow chart interface that lets users set up the campaign sequence with no programming. A typical campaign builder lets users specify the promotion materials sent at each step in a campaign sequence, define the interval between the steps, and create branching conditions that treat different prospects differently. These branches might be based on responses to a survey or on actions after the campaign is under way. Some marketing automation tools can also perform random splits within a sequence, usually to allow testing of different marketing treatments. But this sort of testing is more common in consumer than business marketing, so not many vendors have thought to include it. One exception is Annuncio (www.annuncio.com), which is more oriented to consumer marketing than most marketing automation vendors.

Most marketing automation products do provide separate tables to store business and individual data, a structure that is common in business marketing but many consumer systems do not offer. Marketing automation systems also offer automated scheduling capabilities so that multi-step campaigns can run unattended–another function missing in some consumer-oriented campaign management tools. A few marketing automation systems include sophisticated query and segmentation functions, but most are limited to simple and/or selection logic or just assume that records for each campaign will be imported from an external list. Similarly, only a few marketing automation systems can identify duplicates between an imported list and the existing database. Like conventional campaign managers, marketing automation systems automatically keep track of the promotions sent to each prospect and make this history available for queries, selections and analysis.

The same campaign functions are generally used both for initial lead processing and for subsequent contact programs. Despite the importance of lead generation to business marketing departments, most marketing automation products do not extend to lead distribution and management capabilities such as allocation based on territories or product specialization, charge-backs for leads sold to salespeople or dealers, lead acceptance and disposition tracking, and rules to handle capacity constraints. This sort of fine-grained lead processing is generally the responsibility of the sales organization, not marketing, and thus beyond the scope of a marketing tool. Sales organizations obtain these functions either within their sales automation system, or through specialized systems for lead management (such as Saligent or MarketSoft) or channel relationship management (such as ChannelWave, Partnerware, or Webridge). The standard functions of most marketing automation products could be applied to provide basic lead distribution capabilities if necessary.

– Internet management is the third major area supported by marketing automation products. As with campaign execution, the goal is not to compete with the most powerful dedicated tools, but rather to provide an adequate set of capabilities that the marketers can use without technical support and are tightly integrated with other marketing functions. All marketing automation can generate personalized emails containing unique URLs and/or customer codes to track response. Most can also let users set up survey forms, either as HTML forms that are sent via email or as Web pages that the user is directed to. These systems can automatically post the survey replies to the underlying marketing database, and then use the answers to help generate appropriate replies.

Because email response generation needs to be done in near-real-time, it usually requires a more active technology–such as an application server that constantly scans the email inbox for new messages–than the batch update used in conventional campaign management systems. This is one of the major distinctions between the two sets of products.

Although most marketing automation products can generate personalized HTML pages, they generally do not attempt to provide full-scale Web site creation. One exception is Imparto (www.imparto.com), which lets users link content to Web page templates, thereby allowing marketers to update their Web site with little technical assistance.

* * *

The initial marketing automation products appeared about two years ago, and the more mature ones are now on their second or third release. Technically they tend to look like campaign management systems, with batch feeds from external systems and detailed historical databases. But most also allow at least limited direct updates to support posting of survey results and near-real-time email response. Another distinction is that marketing automation systems nearly all use their own data models, while most modern campaign management systems are built to map to any external data structure.

Nearly all of these systems were built on Windows NT and SQL Server, although a few support Unix and other enterprise relational databases. The NT/SQL Server combination is might raise some scalability concerns, but should be adequate for the relatively small volumes of most business marketers. Although these products are offered for purchase with conventional software licenses, nearly all of the vendors also offer a hosted or application service provider option to reduce investment and speed initial deployment.

Making Sense of Marketing Software – Part 7: Interaction Management

April, 2000

The purpose of these articles is to discuss software purchased by marketers. This means they exclude the call center, sales automation and customer support products generally referred to as customer relationship management (CRM) systems. Those are operational systems purchased and run by operational departments, not by marketers. This is true even though these systems are often sold on their ability to coordinate all contacts with each customer–a process with value only if the coordination is guided by marketing input. Otherwise, CRM systems simply let companies do the wrong things more efficiently.

Interaction management systems are important precisely because they provide the connection between marketers and CRM. More formally, interaction management systems assess each customer contact and recommend the optimal response from a marketing (that is, corporate) standpoint. It is this marketing standpoint–which takes into account the long-term impact of each choice–that distinguishes interaction managers from recommendation engines such as collaborative filtering systems. This later group makes immediate predictions such as which product is the customer most likely to buy or which document is most likely to answer to his question. While these sorts of predictions are important, they do not address the strategic issue of whether making a recommendation is the right thing to do in the situation at hand. For example, in the classic book-buying scenario, the best choice in some cases may be not to recommend a related book, but to offer the customer a discount certificate or even just say thanks for past business. A recommendation engine could not make this sort of judgement; an interaction manager can.

To function effectively, today’s interaction management systems must meet three challenges. The first is to integrate in real-time with the touchpoint systems–call centers, automated teller machines, Web sites, etc.–that execute the customer interactions themselves. Real-time operation is unusual among marketing systems, which are generally more analytical and batch oriented. So the technology of interaction managers is significantly different from traditional marketing products.

The second requirement is to handle an astonishing volume of data–every click on a Web site or every transaction on a bank statement. This volume would be a challenge under any circumstances, but is especially daunting when the system must assimilate and react to the data in real time.

The third requirement is to simplify the overwhelming variety of real-world situations that can arise. Because so much detailed information is available, no two customers may have exactly the same history. So an ability to group individuals and find common patterns of behavior is essential to limiting the number of cases to consider to something manageable.

This last point deserves a bit more explanation. A fully automated system might in theory be able to handle the naked complexity of thousands or millions of unique customer histories by independently calculating an optimal approach to each. (Whether the results would justify such massive processing is a separate question.) But today’s reality is that interaction management systems use rules that are conceived, created and maintained the old fashioned way–by human beings. This means there is a natural, and relatively low, limit to the number of cases that can be treated separately. So the ability to define a reasonable number of significantly different situations is essential.

Several products to exist to perform these functions, including Black Pearl Knowledge Broker (www.blackpearl.com), Harte-Hanks Allink Agent (www.harte-hanks.com), RightPoint Real Time Marketing Suite (www.rightpoint.com, now owned by E.piphany) and Verbind LifeTime (www.verbind.com). Each takes a slightly different approach to the three challenges. Still, the general process is roughly the same:

– the system identifies situations that require intervention. Generally this is done by having the touchpoint system call the interaction manager in specified circumstances, such as a particular point in a Web page or telemarketing script. This usually involves modifying the touch point system to call the interaction manager via an Applications Program Interface. It may also involve running a part of the interaction management application on the touchpoint server.

Alternately, the interaction manager may scan a stream of replicated transactions and intervene when a specified transaction or set of transactions appears. This gives more control to the interaction manager and its users (the marketers), since they can use the interaction manager’s own interface to specify when it will intervene. This also lets the interaction manager employ specialized scanning and data storage techniques to handle the data volume. But even with the scanning approach, the interaction manager needs some integration with the touchpoint system to deliver whatever messages are finally selected.

– the system gathers data necessary to make a decision. This involves information provided by the touchpoint system about the current interaction as well as historical information stored elsewhere. Some systems access external databases through standard SQL calls–an approach that is flexible but may cause performance problems. Other products generate their own profiles for faster access; these may be updated in batch, real-time or both. Like scanning, internally-generated profiles permit systems to deploy special techniques to update, store or access large data volumes efficiently.

– the system chooses a response. This is where the interaction manager truly diverges from a recommendation engine. Most recommendation engines rely on some sort of automated predictive model or scoring algorithm to select the “best” answer, while interaction managers usually apply hand-built business rules expressed in some form of Boolean logic or SQL query. The rule-based approach allows much more flexibility, since the rules can embody whatever business strategies the marketers come up with. Some systems allow the rules themselves to include scoring algorithms or even to call a recommendation engine as part of the decision-making process.

The rules are often arranged in hierarchies. This lets the user establish priorities and manage complexity, while it permits the system to minimize processing by testing only rules that are relevant to the situation at hand. Rule definitions also often include references to standard customer segments that are defined outside of the rule itself, which further simplifies maintenance and enforces consistency.

A good interaction manager assigns explicit start and end dates to rules and lets users group related rules into strategies or campaigns. It also lets users set up random sampling to test alternative treatments. The system needs a graphical interface to help users visualize the rules they have created, and security to let different users maintain rules for different functional areas or customer segments.

For systems that rely on external data rather than prebuilt profiles, it is important to ensure the data needed for all rules is gathered in advance–otherwise separate queries will be generated for each rule, slowing the process unacceptably.

– the system generates the chosen response. At a minimum, the interaction manager must be aware of whatever content–say, a database of Web pages–is available to the touchpoint system, so it can tell that system which item to deploy. At the other extreme, the interaction manager could generate and transmit the chosen message itself. However the responsibilities are divided, the two systems must coordinate closely with each other to ensure that each is aware of changes made in the other. Since marketers control the interaction manager and operational managers control the touchpoint system, this is an organizational as much as technical challenge.

– the system reports on results and adjusts as appropriate. Because most of today’s interaction managers rely on manually-created rules rather than automated scoring mechanisms, there is little opportunity for the system to be self-adjusting. This is another difference from recommendation engines, where self-adjustment is extremely important. But interaction managers can report how often each rule is executed, the results it achieved, trends in response, and perhaps even the characteristics of people who reacted differently. These reports should be available in real time or close to it, so users can detect and adapt to changes in customer behavior as they occur. A good system would provide alerts or exception reports to bring urgent matters to the user’s attention quickly. Again, relating individual rules to larger categories such as customer segments, business objectives (like retention vs. acquisition) or long-term strategies is essential to making sense of the mass of detail. The system must also keep a log of changes to rules over time, so historical reports can accurately determine which set of rules were active when particular results were achieved.

* * *

The details of each of these functions provide major points of differentiation among interaction management products. Systems vary significantly in the throughput they can handle, the sophistication of the situations they can distinguish, and their degree of touchpoint integration. In the future, differences are also likely to arise in the degree of automation they apply to rule definition and evaluation. As ever, each user’s specific situation and requirements will determine which is the most appropriate product to deploy.

Making Sense of Marketing Software – Part 8: Collaborative Filtering

May, 2000

Collaborative filtering software uses the behavior of people with similar preferences to identify products an individual is likely to purchase. It is the best known type of “recommendation engine”, which includes systems using many different techniques to decide what to offer in a given situation. Although the classic recommendation application involves product selections such as books or films, recommendation engines can also target advertising such as Web banners or select information such as replies to technical support questions. Alternatives to the group-based approach of collaborative filtering include market-basket analysis, which looks at what products are typically purchased at the same time; pattern analysis, which looks at what products are typically purchased in sequence; knowledge base analysis, which looks at which replies are most likely to be successful in a given situation; as well as standard predictive modeling.

Recommendation engines are a relatively new application for most marketers. Although marketing has traditionally identified the most likely buyers for each product, it was usually up to sales to recommend specific products to specific individuals. The growth of interactive systems, both on the Internet and in telephone call centers, has changed this by creating a need for automated systems that make recommendations without expert human intervention. The potential for significant revenue increases with little or no added expense makes this an especially attractive investment, particularly since the incremental sales are so easily measured. This is a welcome contrast to the leap of faith required by many other “customer relationship management” applications that promise to make customers happier but have less directly measurable payback.

The most prominent feature of collaborative filtering is a high degree of automation. It is typically applied in situations where there are thousands of options available and preferences can shift quickly–conditions making it impractical to employ hand-crafted rule sets or conventional statistical models. By its nature, a collaborative filtering system automatically performs the critical tasks: recording the behavior of each individual, applying the behavior to predict future behavior, and adjusting its predictions in response to results. Some other recommendation techniques can also achieve similar degrees of automation, although this is not always built into the standard software packages. Instead, these packages often limit themselves to discovering relationships among different behaviors, and rely on a human to transform these into business rules.

The automation of recommendation engines is something of a mixed blessing: it lets the systems handle high volumes and behavior shifts, but also limits the output to the tactical question of what a customer is mostly likely to want. The response can be broadened somewhat to incorporate simple business needs–for example, adding a margin calculation would let the system recommend the offer with the highest expected profit rather than just the highest expected response rate. But deeper strategic considerations–such as whether to make a product offer at all in a given situation–are outside the recommendation engines’ scope. As noted last month, these require an interaction management system, which may in fact call on a recommendation engine as part of its larger decision making process.

There are about a half-dozen collaborative filtering software packages on the market today, including (alphabetically) Andromedia LikeMinds, Gustos, HNC/Aptex SelectResponse, Manna FrontMind, Net Perceptions, and Trivida. Although there are substantial technical differences among them, most users don’t–and needn’t–care about these details. Instead, they need to consider a number of general issues when making a selection:

– data types: some systems require users to provide explicit ratings of sample products, such as books they liked or disliked. Other systems can look at behaviors and infer preferences without an explicit rating. Many systems can combine both types of data. Systems also differ in whether they are limited to purchase data or can consider other activities such as viewing a Web page or putting an item in a shopping cart, in how easily they incorporate data from non-Web legacy systems, and whether they can consider non-behavioral data such as demographics or original source.

– outputs: the standard outputs of these systems are recommendations of products the customer is most likely to want. But some systems can also make “cross sell” recommendations of items most likely to be purchased with something the customer has already selected. Other recommendations may involve Web banner ads the viewer is mostly to click on or the documents most likely to answer a question. Each of these applications has somewhat different requirements, so it is important to ensure a particular recommendation engine is suited to your expected use.

– constraints: a simple product recommendation may involve no constraints other than eliminating items the customer has already purchased. Even this can be harder than it seems; for example, a system needs to exclude different editions of the same book or both the black-and-white and colorized versions of the same movie. Applications like cross-sell might involve more complicated constraints, such as ensuring products come from different categories (a tie matching a shirt, rather than a second shirt). Advertising and sales applications also often require limits on how many times the same offer can be made to an individual and how long to wait before showing the same offer again. Other constraints such as inventory level and profit margin might also come into play. Sophisticated marketers will want to throw in an occasional random selections to help validate system accuracy and to test customers for unknown interests.

– integration: most of these systems are designed to feed recommendations to external “touchpoint” systems that will execute the interaction itself. Systems built with integration in mind will provide documented API’s, standard objects, multiple database connections, and other tools. But some recommendation systems are part of a larger package and may be difficult to use with anything else. Even when a system is designed for integration, buyers must ensure the methods chosen by a particular vendor are compatible with whatever technology is deployed elsewhere in their organization. This is particularly true as companies seek to deploy recommendation engines at all touchpoints, not just the Web. Since the different touchpoint systems may themselves use different technologies, the ability to handle several different kinds of integration may be essential.

– scalability: this goes hand-in-hand with integration, since integrating with inadequate response time is pretty darn useless. Scalability for recommendation engines involves many dimensions, including response time, transactions per minute, customers on file, data per customer, products rated, attributes per product, and number of data sources. Systems vary greatly in their resource requirements: some store highly compressed profiles while others keep every piece of detail. Systems that store detail can be very powerful, but may need impractical amounts of disk space or processor cycles when dealing with large volumes. Some products scale well through parallel processing, in-memory databases and other high-end technologies; others have bottlenecks these won’t address. As always, careful testing is needed to be sure any system will scale beyond the size of its existing installations.

– accuracy: it seems odd to put the accuracy of a system’s recommendations so low on the list, but the practical requirements already described do come first. Of course, quality does matter, and there are significant differences among the various products. But different systems will have different strengths, so there is no real alternative to testing with your own data to find which system works best for you. It’s important to test how well each system deals with varying amounts of data–you will probably face a range of situations from new customers and new products with no history, to established customers and products with vast amounts of detail. Since this is one of those areas where averages can be highly deceptive, it’s important to measure separately how each system handles each situation. Then you can decide where your priorities lie.

– reporting: these systems provide a wide range of information, including operational statistics such as processing volumes, response times and resource consumption; confidence and probability values returned with individual recommendations; performance data on actual response rates; and analytical reports such as how often each product is recommended and purchased. A good system will display trends as well as snapshots of its information, and maybe even issue automated alerts when there is a significant change in recommendation accuracy or demand for particular products. For systems that build models or rule-bases that define relationships among behaviors or data elements, ways to visualize and summarize these are also important.

Making Sense of Marketing Software – Part 9: Personalized Web Sites

June, 2000

If the Deep Thinkers of marketing agree on anything today, it’s that the Internet is the key to the future, and personalization is the key to the Internet. So Web personalization software should receive more attention than any other type of marketing system, right?

Evidently not. Attend a marketing conference, read an industry magazine, or listen to marketers talk about their problems, and you’ll find most of the attention goes to CRM and campaign management systems. Even Internet specialists spend at least as much time considering Web-based customer service and email campaigns as personalization products. And if you distinguish personalization from collaborative filtering and interaction management, the mind-share occupied by personalization grows smaller still.

Why the disconnect? Is personalization really so much less important than the pundits would have us to believe? Or are most marketers still working on basic Web functions, and just not yet ready to add personalization?

Neither explanation seems very sound. Visit almost any major consumer Web site, and you will be offered some form of personalized interaction. So it seems that personalization is the Web’s clean little secret, something that everybody does but nobody talks about.

It would be fun to blame this mysterious silence on an extraterrestrial conspiracy or World Federalist plot or at least on Microsoft. But the real reason is much less exciting. People rarely talk about personalization systems because these no longer exist as stand-alone products. Instead, personalization has become embedded in basic tools used to build and run major Websites.

Since these tools are bought and operated by the site builders themselves–who once reported to marketing but now reside within IT or perhaps their own domain–marketers don’t concern themselves with these tools as much as they used to. And even the site builders consider personalization as just one of many capabilities to evaluate in a product.

But precisely because personalization can easily be taken for granted when evaluating a Web system, it’s important to examine what it takes to do it properly. There are some fairly specific issues to keep in mind. These relate to the three main components of a personalization system: content, profiles, and rules.

– Content refers to the specific messages that will be sent to individuals. Of course, every page on a Web system is content, and extremely sophisticated systems have been developed for content creation, maintenance, distribution and access. But personalization adds its own demands to standard content management. One requirement might be called granularity: while static Web sites can be built and managed at the page level, personalized Web pages are typically assembled from many smaller components. This means the content must be stored at the component rather than page level.

It also means the system must include ways to categorize these components–often assigning multiple attributes to the same item. In part, this helps manage the much larger number of items that result from working at the component rather than page level. Perhaps more important, it also lets the system identify the components appropriate in a given situation without having the user explicitly specify them in advance. This opens the door to automated content selection mechanisms such as predictive models, which will ultimately be needed to replace hand-built rules.

Personalization also expands the types of content a system must present. Many personalization schemes rely on selecting information from a database, whether it’s leisure airfares to a favorite city or products most commonly purchased with each other. Other schemes involve geographic considerations, such as maps to the nearest drugstore. Personalization can also involve manipulating a graphical image, such as showing how a dress look in a particular color or pattern on the viewer’s body type. At a more basic but still challenging level, personalization can involve storing content and navigation messages in different languages.

The final impact that personalization has on content relates to system performance. Personalization greatly increases the number of content items that must be quickly accessible. This means that content must be stored in ways that allow high-speed retrieval. Thus, personalization implies a greater concern with in-memory caching, indexing, multi-threading, distributed servers, and other advanced technologies.

– Profiles refer to information about individuals. The simplest profiles are limited to data that is gathered within the Web system itself. These can include information provided by customers through media such as registration forms and surveys, data inferred from click-stream and other behaviors, and transactions such as purchases and customer service requests. One key issue is how much and which kinds of this data a system can store. For example, marketers add new survey questions all the time–a system must be able to accommodate the new answers in its profile database, and ideally will be able to ask users for missing information without repeating questions they have already answered.

Most marketers want their profiles to include non-Web information. This might come from a data warehouse or legacy transaction processing system. The simpler method is to import selected values directly into the Web system’s profile database. But while this is technically straightforward, it can involve massive amounts of data–much of which will never be used if many customers never visit the Web site. The alternative is to query the external systems in real time as data on individual customers is needed. This of course assumes a common key, such as a standard customer ID, is available in both the Web and non-Web systems. Beyond that, it requires the system design and resources to provide adequate response times. If the warehouse or legacy system was not designed for this type of access, real-time queries may not be practical. A good Web personalization system should be able to handle either situation.

Most Web personalization systems store their profiles in a flat file or small number of relational tables, with standard indexes to provide high-speed access to individual records. Where very large volumes are involved–either in number of customers or data per customer–some form of compression or summarization may be needed to ensure performance and keep hardware costs within reason.

While personalization is the main reason profiles are gathered and made accessible on the Web, they can also be used to target Web ads and to target non-Web direct marketing campaigns. Personalization itself is rarely seen as a privacy issue, but the data in the underlying profiles is often considered quite sensitive. Systems need to protect this data through encryption and access limits. They may eventually need to comply with additional constraints on profile usage, such as restrictions on permissible applications, limits on data transfer, required expiration dates, and customer review and consent. Although today’s systems do not such functions, it would be a good idea to consider what it would take to add them to any system you are about to buy.

– Rules represent the final component of a personalization solution. They determine which content is presented to which profiles. Rules can vary from a straightforward lookup of the customer’s preferred language, to simple if/then logic, to rankings based on automatically generated statistical models. Most of today’s Web personalization relies on manually-defined rules, sometimes drawn with a slick graphical flow chart but more often written in Java or another programming language. A few systems include data analysis capabilities that let them discover relationships and propose new rules on their own, although even in those cases it is (wisely) left to users to decide which new rules to implement. Most systems can nest rules within rules, allowing considerable complexity.

Like content, rules pose a significant administrative challenge: they have to be reviewed and authorized by the right people; they may need to start and expire on specified dates; they must fail gracefully if something unanticipated happens. In fact, rule and content administration must be closely coordinated, since rules often call on a specific piece of content that had better be available. The challenge in a large organization is that rules and content will often be created by different people. This means test and validation processes are needed to ensure a rule can be executed properly. One approach to mitigating this problem is to have a rule call for a class of content–for example, “find the best cross sell offer”–rather than a specific item. That makes it easier for the system to use whatever content is available, and to provide a default response if nothing better can be found. But this approach does require considerably more sophistication, and more real-time processing, than calling a specified piece of content.

The real challenge with rules is integrating them into the rest of the system. In early products, and still today in systems aimed at smaller installations, rules and content were intermingled: that is, a little snippet of logic would be embedded directly in an HTML page. While better than nothing, this has obvious problems, since any rule change requires changing the Web pages themselves. Today’s enterprise-level systems generally separate rules from content, so marketers can make changes without affecting the integrity of the site. For similar reasons, content itself is often separated into different classes–for example, there might be standard page templates constructed by the Web staff, with placeholders for content (or rules) that can be changed by other people. A similar division of labor may also apply where content interacts with profiles–allowing users to create survey questions, but giving administrators control over where the results are stored.

Not all Web personalization issues relate to content, profiles and rules. Architecture and scalability are critical concerns; so are integration with other systems’ data and processes. These are more related to the underlying Web site system than personalization itself. Personalization reporting also generally relies on the reporting functions of the parent Web site system, or on whatever third-party Web analysis tool a company has chosen. But personalization does add some specialized reporting requirements, such as the need to track the use and results of individual rules. So a few specialized reports are often built into the personalization process.

Making Sense of Marketing Software – Part 10: Email Campaign Management

July, 2000

Of all the systems that marketers purchase today, email campaign managers are by far the simplest. After all, what can be complicated about generating a list of names, sending them an email, and tracking the replies?

Indeed, email campaign managers fall short of the most advanced rank in both campaign complexity and processing interval, the two dimensions used to classify marketing systems in these articles. (See figure 1.) This is unusual: five of the seven systems are on the leading edge of one dimension or the other. Systems in the other exceptional category, interactive customer support, are quite complicated but in ways the matrix doesn’t capture.

(Figure 1 – Marketing System Grid)

In terms of campaign complexity, email campaign managers generally send the same letter to everyone in a promotion, and usually include only one message and its response in a single campaign. By contrast, campaigns built with conventional campaign managers often assign different messages to different segments and include a sequence of messages, responses, and new messages. Some of the email systems’ limits can be overcome through implementation techniques, such as segment-based personalization in a letter or links across multiple campaigns to create a sequence of messages. But these approaches are hard to set up, require extra processing, and don’t automatically generate integrated reports. So the advantage of the conventional campaign management systems is real.

In terms of processing interval, email managers are near-real-time systems. That is, they must respond quickly to incoming messages, but need not react immediately. This requires less sophistication than true real-time products like interaction managers or personalized Web sites. It means that batch update cycles are possible, although in reality most email campaign systems use online application servers to scan for inbound messages and reply as they are received. The initial campaign selections may be done either in batch or online: although this has significant technical implications, it usually doesn’t matter much from the marketer’s perspective.

Despite their relative simplicity, email campaign managers do have their own set of issues. Probably the most important is the ability to elicit replies that are easily posted back to an underlying database. This involves two key capabilities: generating a unique key to identify the incoming record, and capturing data in a structured format.

– Unique keys must identify both the recipient and the promotion. These are often embodied in either a unique URL printed in the body of the outbound message, or a response key stored in the message that will be returned. Both methods work, but the unique URL is generally preferred since it requires a smaller outbound message–the actual promotion can be stored on the sender’s Web site rather than physically emailed to the recipient. This is especially useful for complicated messages such as surveys and other forms. Pointing to a URL also lets the system retain greater control over what happens after the response, since all the capabilities of a personalized Web site can be brought to bear. In some systems, the visitor is first sent to a site that tracks response and them automatically redirected to the company’s main Web site. This gives the email campaign system a way to track results without needing to create a separate site.

– Capturing data usually involves filling out a form created as an HTML page. The alternative is to have users submit text, but the parsing required to process such input is more difficult and less precise. Often the forms can be pre-filled with data that is alreay known, such as the customer’s name and address. This saves effort. Some systems can automatically apply these forms to an underlying database, while others post them to an intermediate file where they are reviewed and cleaned before being accepted. Systems also vary in whether they store replies to each form independently, or require that questions fit into a single profile record. Some products can trigger additional actions based on the contents of a reply, such as sending a message to an account manager or passing information to an order processing system.

Of course, email campaign managers provide functions beyond data capture. These start with the ability to maintain a customer database: some systems build and maintain an internal database while others can be mapped to external tables. If there is an internal database, the structure may be fixed, extensible around a standard core, or totally custom. Most systems provide some degree of flexibility within a standard structure that includes customer data and contact history.

Once the customer database is set up, marketers need to select from it. Most email campaign managers include a standard SQL-based query builder. This is generally adequate, although less capable than the advanced query functions provided strong conventional campaign managers. One problem is that not all systems offer random sampling, which is essential for proper test procedures. Most systems can combine separate SQL selections into one campaign list, with duplicates automatically eliminated.

Email campaign managers vary more substantially in the tools they provide to build the email messages themselves. Nearly any system can produce text messages that allow users to specify database variables–such as a name field–that will be merged into the message when it is delivered. Today, most products can produce personalized HTML messages as well as text. But only some can extend beyond simple value substitution to conditional logic that will send different messages in different situations. Applications can range from sending different offers to different segments, to asking different questions depending on what information is already known about a customer, to listing the products a customer is most likely to want based on past purchase history. To some extent, this sort of personalization can substitute for the conventional campaign management approach of sending different offers to customers in different segments. But the conventional approach generally gives control over segment treatment–such as specifying maximum quantities for any segment or allowing random splits to test alternate versions–that embedded personalization does not. Even a simple count of how many customers will receive each treatment is harder to get when the decision is made within the body of the letter.

Systems also vary in their administrative functions. Some generate test output, ensuring that all the email forms associated with a campaign produce valid output and point to live Web pages. Most let users schedule a campaign to execute once or repeatedly. Systems built for large-scale implementation also let users specify the time of day and the number of records released per hour to avoid overwhelming the email server. Some track which email addresses are successfully delivered and will retry those that fail, or mark them as invalid in the database. Some also track whether a customer’s email reader can read rich HTML and send future mail in the appropriate format. A few offer standard marketing administration functions such as project task lists and financial analysis. Many offer automated posting of unsubscribe messages from customers who wish to opt out of future communications.

Reporting nearly always provides basic campaign statistics including the number of records selected; number of messages sent, delivered and opened; and number of responses. Some systems report on the information included in the responses, such as counts of answers to survey questions. Most reporting is provided against the actual customer database, so figures are updated as responses are posted.

Although stand-alone email campaign management systems were once common, their functions are increasingly being offered within other marketing systems. Given the relative simplicity of the technology, this is fairly easy to accomplish. But as with other components of integrated product suites, buyers must know their actual requirements before they can judge whether the suite’s integrated capabilities are truly appropriate.

Making Sense of Marketing Software – Part 11: Interactive Support

August, 2000

Among the software categories currently purchased by marketers, interactive customer support systems most come closest traditional operations. Certainly conventional customer support–that is, providing customer service and technical assistance by telephone and mail–is a classic operational task. Interactive customer support is a marketing system only because it runs over the Internet, which has been a marketing responsibility.

Or at least it started out that way. In the year since these articles began, an increasing number of organizations have stopped treating the Internet as a marketing project and started integrating it with their regular operations. So today, interactive customer support systems are likely to be purchased by the customer support group, not marketers. Within the technology itself, this is reflected in a trend toward systems that combine email and Web with traditional telephone and field channels. This makes perfect sense in today’s environment, where the ultimate goal is to ensure consistent treatment of customers across all points of contact. How better to do that than by using the same system for all of them?

It also makes perfect sense because the functions of interactive and conventional customer support systems are quite similar. Both kinds of systems are organized around service requests or cases, which must be received, tracked, resolved and reported on. Broad requirements include classification schemes to organize new cases; customer and case history lookup to identify who is calling and their previous experiences; routing rules to assign cases to the appropriate resources; prioritization schemes to treat the most important cases first; knowledge bases to help identify correct solutions; workflow, scripts and form letters to provide consistent treatments; escalation processes to ensure cases are resolved in a timely fashion; service level reports to monitor performance and backlog; and tracking data to keep a history of how each case was handled. As with most operational systems, each of these areas has a stack of specific requirements needed to operate efficiently.

Both kinds of systems must also help to manage internal resources. This requires reports to track the speed and accuracy of customer support agents; tools to forecast workload and schedule staff; databases of agent skills to help with routing; queues to assign cases to specific agents and reassign them if necessary; and options to make outbound marketing calls when inbound workload is light.

Still, interactive customer support systems do have their own nuances. The most obvious is that they are more automated than conventional support products, which are primarily oriented toward telephone call centers. For email systems, this automation is generally limited to parsing of inbound messages to classify them and choose an appropriate response. The capabilities of such parsers are still quite limited, as anyone who has received an inappropriate robo-reply already knows. Most parsers just scan for a list of key words, either in the subject line or body of the message itself. Some apply more intelligence, looking for combinations of words or assigning replies based on automated learning algorithms that look at previous interactions. But most vendors today recommend their automated systems be used only to route messages and identify suggested replies, with the final message reviewed by a human agent before it is sent. Still, even this limited form of automation can more than double agent productivity.

It’s fairly easy to insert a human agent into an email support process, where acceptable response is measured in minutes or even hours. But Web site visitors will not wait for a human to receive and process their request. So interactive support systems also encompass self-service facilities that give Web site visitors direct access to knowledge bases and search engines. This is generally based on the same technology used by service agents to look up responses, although the interface must be simplified for untrained users and new data capture formats such as Web forms must be added. As with parsers, much work remains before this technology offers consistently useful answers.

This is a major issue, since visitors will require expensive human support if the self-service system fails to solve their problem in a reasonable period of time. The problem is somewhat mitigated by the hybrid options that let a Web site user chat with a live customer service agent by telephone or instant message. But it is still important to test the accuracy of a self-service solution before making a major investment.

Ideally, one knowledge base would serve telephone agents, email and Web self-service support systems. These systems usually rely on humans to define the broad structure of how information is organized. This is typically a hierarchy of key words or patterns connected by questions and rules. Some systems also include self-learning algorithms that can identify patterns in behavior–such as which answers are most successful in which situations–and adjust the rules accordingly. Even those that don’t should provide reports on how often different rules and pieces of information are being used and with what results.

Interactive customer support products are evolving rapidly, even by Internet standards. The most ambitious vendors are seeking to make them the foundation of a complete relationship management system, extending not only to conventional communications channels but also to functions including ecommerce, sales automation, campaign management, data analysis and even management of the consolidated customer database. Kana Communications (www.kana.com), with acquisitions or alliances in all these areas, has been particularly aggressive. Other major vendors include eGain, Quintus and ServiceSoft. In turn, these firms face new competition as vendors from other sectors extend their own functionality in the race to offer a truly complete customer management system. Eventually, the distinction between marketing and operational buyers will become irrelevant, as companies purchase large integrated customer management suites to serve the needs of both groups.

Making Sense of Marketing Software – Part 12: Summary

September, 2000

In the year since these articles began, the marketing software industry has continued to evolve. The most prominent trend has been an expansion of functionality as vendors attempt to offer a complete marketing solution in a single product. This expansion is sometimes accomplished through in-house development, but more often by alliance or outright acquisition. The stock market’s willingness to purchase equity in unprofitable new companies has given many software vendors a currency to make such acquisitions.

The expanded product offerings fall into two fairly distinct groups. The first combines data extraction, analysis and campaign management–that is, all the batch processes used in a conventional database marketing system. Not surprisingly, this approach is favored by vendors with roots in one of those areas. This includes campaign management vendors like Xchange, Inc. (formerly Exchange Applications), Prime Response, Recognition Systems and Unica and data mart vendors like E.piphany, Broadbase, and to some extent Sagent and Informatica. Pure analysis and reporting vendors like SAS, MarketSwitch, Hyperion and Microstrategy have tended to form alliances with campaign management vendors rather than trying to develop an offering of their own. (One exception is Unica Corporation, originally a modeling software vendor that has introduced its own campaign manager and is now moving to add real-time marketing.) Despite these vendors’ origins in batch processing, the overwhelming importance of the Internet has lead them to include real-time or at least near-real-time email and Web marketing as well.

The second group of integrated marketing products comes from vendors of operational contact systems, such as sales force automation, order processing, and customer support. This includes vendors of traditional systems such as Siebel, e-mail specialists such as Kana and Annuncio, and personalized Web site builders like Broadvision, Vignette and ATG. Unlike the first group, these vendors have always built real-time systems. So they have tended to add real-time marketing analysis and segmentation that run directly against the operational database, rather than creating batch extracts into a separate analytical data mart. Not so long ago, any experienced database marketer or data warehouse developer would have rejected this approach as laughably inadequate for two reasons: it doesn’t consolidate customer data across all touchpoints, and operational data structures are unsuited for serious analysis. Today the first objection has less power, since many operational products are now at least theoretically capable of handling all touchpoints in a single system. The second objection remains fundamentally valid, although the vendors might counter with an approach that uses a separate mart purely for planning and analysis, but not as the main store for customer data. This is roughly the approach taken by Siebel, whose latest marketing product uses a series of star schemas for complex analysis and segmentation, but loads selection tags back into the operational data. Campaigns then run against the operational data itself.

Integrated suites involve a Faustian trade of easy deployment for vendor lock-in, which makes them a distinctly mixed blessing. But that’s a topic for another day. In the context of these articles what’s noteworthy is that both groups of suites include systems not traditionally purchased by marketers (which was this series’ definition of a “marketing system”.) Specifically, the batch-oriented suites include data extraction tools traditionally run by IT and sophisticated analysis tools traditionally run by statisticians. The operational-oriented suites are of course built on systems run by operational staff.

It’s possible to interpret the broad scope of these systems as evidence that marketing is taking a central role in today’s organizations, presumably as a result of increased corporate focus on customer management. But does anyone believe marketing can or should be responsible for data extraction or order processing? Realistically, the expansion of these systems reflects the closer integration of marketing with other corporate functions, but not marketers’ control over those functions. In other words, the integrated marketing suites will not be purchased solely by marketers. Some marketers may mourn this loss of autonomy, and it could signal trouble for vendors whose products have traditionally been purchased by marketers acting autonomously. But tighter integration is probably a net benefit to the corporation and its customers.

In addition to integrated suites, the past year has seen a rapid maturing of technologies needed for real-time interaction management. These include the interaction managers themselves, as well as supporting systems for transaction scanning, content management, recommendations, Web site analysis and optimization. Today these are still delivered primarily by stand-alone products, although they are rapidly being incorporated into product suites. At present this happens mostly through alliances, although acquisitions will probably become more common. The past year also has seen forward-thinking vendors and industry observers develop a fairly clear understanding of the requirements and components of the interaction management process. In particular, the importance of automated optimization and reporting has become apparent as people realize that manually-defined rules and analyses cannot manage the extreme volumes and fast reaction times implied by real-time, enterprise-wide interactions. As this shared understanding continues to percolate through the industry, better products and more knowledgeable buyers will follow.

Whatever the details of these future developments, the framework presented in this series should provide a useful context for understanding it all. Basic distinctions of real-time vs. batch, operational vs. analytical, and reactive vs. strategic response will remain relevant; the interaction manager will still occupy a key position connecting front-office systems with marketing strategy. Like a snapshot of a growing child, these articles have presented a detailed but static look at a changing subject. We can now look ahead to watching it grow over time.

* * *

David M. Raab is a Principal at Raab Associates Inc., a consultancy specializing in marketing technology and analytics. He can be reached at draab@raabassociates.com.