David Raab Article Archive

Autonomous Marketing Messages
David M. Raab
Information Management
July / August 2010

Here’s a metaphysical puzzle: can you send a marketing message before it’s created?  The answer used to be no: messages remained unchanged after they were sent, either because they were broadcast and vanished immediately or because they were physically persistent but inert, like a printed catalog or recorded TV ad.  Even messages that were dynamically tailored to a specific individual and situation were rendered and then frozen before they were sent.  A Web site might adjust its offers over time, but each offer was itself fixed.

Because marketers knew the contents of each message when they sent it, the only subsequent information they needed was who received it and how they reacted.  Indeed, most marketing measurement boils down to answering those two exceedingly difficult questions.

But marketers today face an added challenge: capturing the message itself.  Paradox notwithstanding, an increasing number of messages can now change after they’re created.  Consider:

- Adobe’s latest design software, CS5, can create ads that send different messages to different individuals and record the results.  Specifically, Web designers can embed the testing, segmentation and automated optimization of Adobe’s recently-acquired Omniture Web analytics system.  The solution relies on the Omniture server to execute tests and store results.  But the next logical step is to embed test logic, tracking and automated self-optimization within ad itself so it can function when a server connection is unavailable.  This would result in a truly autonomous marketing message.

- Vendors including smartFocus, Genius.com and Genoo have extended social media sharing to tag each item with the ID of the individual who shared it, so they can be credited as the source of later visits by recipients.  In other words, if Jane posts a link to this article on Twitter, marketers will later know not just which visitors came from Twitter, but also which came from Jane’s Twitter post.  This lets them measure how much traffic Jane generates and identify the members of Jane’s social network.  In effect, the original message is being modified by adding the identity of each sharer, which must then be captured with responses.

- Barcodes on products and advertisements are being linked via mobile phone applications to Web sites that vary their content based on location.  Here, the original message is being enhanced with the viewer’s location and, perhaps, actual identity.  One obvious use is to deliver different offers based on local weather and competitive promotions.  Vendor StickyBits make a buzzword triple play by adding social media to mobile and geo-targeting, with separate social media sites for different locations of the same UPC code.  Since the social site also evolves, marketers can only know the message received by each consumer if they capture a snapshot of the site as it appeared to each visitor.  They must also capture whatever contextual variables (time of day, weather, competition, current promotions, etc.) play into offers and results.

These examples point to the emergence of autonomous marketing messages: communications that are launched into the world to operate more or less independently, occasionally phoning home like a dutiful college student to report results and perhaps get some advice.

The concept poses challenges for everyone involved.  For marketers already struggling with the transition from one-way broadcasts to peer-based communities, it’s a further loss of control.  For technologists serving those marketers, it’s another set of delivery systems and reporting systems to manage.  For marketing analysts, it’s a new type of data to incorporate.

But the concept also creates new opportunities for success.  Messages that can track their own movement from consumer to consumer can provide important insights into the always-mysterious connection between messages sent and resulting customer behavior.  Autonomous viewer logs, testing and optimization can enhance media where continuous real-time connections to central servers are unavailable.  Periodic contacts with central servers let the applications download their information and update their libraries of offers, models and business rules.  Combining mobile, location and social media provides rich information about consumer behavior, along with direct opportunities to deliver highly targeted messages.

Autonomous messages add to the flood of data already generated by digital marketing.  This increases the need for ways to load, store, access and analyze tremendous volumes at reasonable cost and speed.  Similarly, the complex and variable structure of the new data reinforces the existing demand for technologies that can easily incorporate new data types and models.  Autonomous messages also require improvements in techniques to automatically uncover significant patterns within the data and infer appropriate marketing treatments.

The major new challenge posed by autonomous messaging is portability.  Autonomous systems must somehow incorporate decision rules, self-adjusting analytics, alternative treatments and data capture mechanisms while making minimal demands on host resources.  This is a particular issue in mobile environments, where bandwidth, storage and processing power are scarce.   The messages must also find efficient ways to exchange data with central servers.

Customer identification is another issue.  Autonomous messaging could ease some privacy concerns by tracking and responding to behaviors without sharing them externally.  But it also extends to platforms where customer identification is more difficult than usual, making it still harder to gain the most value from data that marketers have permission to use.

Progress will be incremental.  The immediate future will see hybrids that combine different aspects of autonomy with centralized techniques.  Marketers and technologists will need to assess the strengths and weaknesses of each approach and look for opportunities to combine them to deliver solutions more powerful than any one method provides by itself.

*                            *                           *

David M. Raab is a Principal at Raab Associates Inc., a consultancy specializing in marketing technology and analytics.  He can be reached at draab@raabassociates.com.

Tags: marketing technology

Marketing Systems for OnLine Media
David M. Raab
Information Management
March / April 2010

Online media account for about one-third of consumers’ time (1) but receive less than one-sixth of all advertising expenditures (2).  So it’s a safe bet that marketing systems will increasingly need to support online advertising.

This is a challenge for current products, which were developed for outbound campaigns via mail, email and telephone.  Even real-time interactions have been treated as extensions of traditional campaign flows rather than versions of online advertising.

The distinction is significant.  Outbound marketing assumes that marketers know who will receive their messages, can control the delivery of those messages, and can directly capture responses.  It further assumes that the promotions are the primary influence on consumer behavior.  This lets marketers execute carefully structured champion-challenger tests to measure the impact of alternative customer treatments.

Advertising results have never been anywhere near as measurable.  Advertisers in traditional mass media had only the vaguest notions of who was seeing their messages.  Most measurements relied on consumer samples such as panels or surveys or on statistical correlations captured in marketing mix models.  Despite formidably complex statistical techniques, these are inherently imprecise measures of marketing impact.

The flood of data provided by online advertising seems to offer an escape from the uncertainly of traditional media measurements.  Unfortunately, it won’t work.  The key assumptions of outbound campaigns – that marketers control who receives their messages, what those messages contains and how the customer can reply – are not true in the online advertising world.

- In online advertising, consumers decide where to direct their attention.  Marketers can access an audience with presumed interests and attributes, but cannot target specific individuals in advance.

- Online advertisements are delivered in the context of some other consumer activity, such as visiting a Web site or conducting a social interaction.  This greatly reduces the amount and type of content that can be delivered.  More important, the online environment contains information from competitors, neutral experts, and other consumers.  Even if the consumer is lured to the marketer’s own Web site, she can exit at any time.

- Although many interactions are captured, few are directly linked to an actual purchase.  This invalidates the central simplifying assumption that a single promotion can be credited as “causing” a specific purchase.  With this cornerstone removed, champion-challenger testing requires elaborate analytical scaffolding for support.  The challenge is magnified because interactions may be recorded but not linked to the same identity, making it difficult to understand the full set of contacts that influenced each consumer.

If a common problem threatens the outbound campaign systems, it’s the need to handle unstructured data.  Online advertising can capture huge amounts of detailed information about each consumer and each interaction, including Web pages the consumer has visited, competitive ads the consumer has seen, comments she has posted, profiles gathered by third parties, geographic location, and even the interaction device (computer, smartphone, kiosk, etc.).  But the structure of this data and the significance of individual attributes is not always known in advance.  Thus, systems must analyze it in many different ways before they can identify features that are relevant for marketing.  From a technology perspective, this implies specific capabilities including storage and easy access to unstructured data; text and semantic analysis to extract and classify contents; audio and video analysis to deal with non-textual content; time-series and pattern analysis to identify significant behavior patterns; and network analysis to understand social media influences.  None of these are handled easily by traditional outbound campaign systems, which assume a highly structured environment.

The second new core requirement is better prediction of relations between content and subsequent behavior.  Traditional champion-challenger tests can’t isolate the impact of the large number of contextual variables that apply in online marketing interactions.  Even if they could, consumer behavior is too volatile to assume relationships remain valid weeks or months after they are collected.  Rather, marketers must deploy self-adjusting models that can observe consumer behavior and predict which content (both tested and untested) is likely to be effective under different circumstances.  Beyond predicting immediate response, marketers need new ways to track long-term results.  This requires linking fractured consumer identities from first- and third-party cookies, member profiles, social media personas, email addresses, IP addresses, location, customer accounts, and other sources with varying degrees of anonymity, all while respecting legal and ethical privacy constraints.  As with traditional advertising, marketers will often find themselves relying on consumer panels and surveys to gather data that cannot be assembled without individual cooperation.

A third key requirement is data-gathering, analysis and execution across complex networks in real-time and near-real-time.  These capabilities will evolve over time: it will be easier to execute dynamic models on your company’s own Web site than through a third-party ad network.  But as the advantages of superior content selection become apparent, marketers will increasingly press for the ability to deploy them more broadly.

All of these capabilities – unstructured data management, self-adjusting analytics and real-time execution – are outside the scope of traditional outbound marketing systems.  Whether existing vendors can adapt to support them or whether new systems emerge will determine who dominates the growing market for online marketing execution.

(1) Forrester Research, “Consumer Behavior Online: A 2009 Deep Dive”, July 27, 2009
(2) PricewaterhouseCoopers and Wilkofsky Gruen Associates, “Global Entertainment and Media Outlook: 2009-2013”, June 16, 2009

*                            *                           *

David M. Raab is a Principal at Raab Associates Inc., a consultancy specializing in marketing technology and analytics.  He can be reached at draab@raabassociates.com.

Tags: marketing technology, online media

Marketers May Shift from Analytical to Transactional Databases
David M. Raab
Information Management
May 2009

Marketing systems were early adopters of analytical database engines.  This was purely from necessity: list segmentation and analysis were the core application of most marketing systems, and standard relational databases require much tuning and considerable hardware to handle them effectively.  Since marketers rarely had the technology budget to pay for this, they were forced to seek the lower-cost alternatives provided by specialized databases, typically using columnar structures.  Corporate IT departments were rarely enthusiastic about these non-standard systems, but, since marketing was usually a low priority, IT found it easier to let the marketers buy what they wanted and take responsibility for the consequences than to step in and do the work themselves.  No one was quite happy, but an uneasy truce prevailed and everyone got their work done.

Today, specialized analytical databases are a hot fashion in corporate IT.  These include not just columnar structures but also in-memory and shared-nothing parallel processing systems.
So you’d think that marketing systems using of those databases would be the center of attention.

Not necessarily.  It’s true that big marketing systems with big analytical databases are more common than ever, and that IT is more comfortable with them.  But the classic list segmentation and selection functions are less important because today’s marketing is increasingly focused on managing real-time interactions with individuals.  Those interactions largely require conventional transaction processing technology.  As a result, the analytical databases are receding to the role of supporting systems that prepare and stage data for front-line interactions.  The really interesting business and technical challenges are making the front-line systems work better.

The implications of this run deep.  For marketers, it means the traditional divide between marketing and sales is now much less distinct.  Customers and prospects all interact with the same company systems (Web sites, call centers, email, etc.) and external social networks.  To ensure that each visitor is handled correctly, marketing (responsible for prospects) and sales (responsible for customers) must jointly define the business rules and treatments.  This is much more cooperation than those groups are used to.

The change also calls into question the fundamental architecture that has kept marketing databases separate from sales and customer service systems.  Even though Customer Relationship Management (CRM) vendors have long claimed to integrate all three, a look under the hood has usually found that sales and service ran on a transaction-oriented database while marketing ran on a separate, analytical database.  This was partly due to the technical requirements of marketing queries, which made the analytical data structure more effective.  But the separation between sales and marketing activities meant there was no particular penalty for running fundamentally independent systems.  Companies needed just a bit of data synchronization, which might be done in real time but could often run nightly without causing serious problems.

The equation changes drastically when marketing and sales do need to integrate.  Now the advantages of a separate, analytically-structured marketing database must be weighed against the significant impediments to sales and marketing coordination that database creates.  This gives companies more reason to consider alternatives.

The simplest approach is to just get rid of the separate analytical database altogether.  This is increasingly viable because today’s relational databases and hardware can more easily support transactional and analytical processing.  The very largest marketing databases will probably still run on separate analytical systems, but companies with smaller files will increasingly be able to get adequate performance for marketing applications from their transactional databases.  Even if this requires a bit of tweaking to add some extra indexes or summary tables, the overhead will usually cost less than maintaining a truly separate marketing database.

Another option is to keep the analytical database but shrink its role.  Under this scenario, the analytical database is truly used for analytics – that is, research and reporting – while the quasi-operational tasks such as list selection migrate to the transactional system.  This is arguably a more logical structure anyway: since some of the tasks were assigned to the analytical database because it was the only system the marketers had available, not because they required an analytical structure.  For example, many analytical systems work best with batch updates, but have been forced to take real-time updates so that marketers can work with current data.

The fundamental point to remember is that whatever happens to the analytical database, marketing will do an increasing portion of its work on the transactional systems.  That’s the inevitable result of managing individual interactions at touchpoints such as the Web site.  In theory, marketing systems could evolve to include their own transactional components.  But that makes little sense when most companies already have these in the sales and service components of their CRM systems.  It’s more likely that the long-standing promises of CRM systems to encompass marketing will finally be fulfilled, and separate marketing systems, like separate analytical databases, will play a smaller, more specialized role.

*                            *                           *

David M. Raab is a Principal at Raab Associates Inc., a consultancy specializing in marketing technology and analytics.  He can be reached at draab@raabassociates.com.