Marketers May Shift from Analytical to Transactional Databases
David M. Raab
Information Management
May 2009
Marketing systems were early adopters of analytical database engines. This was purely from necessity: list segmentation and analysis were the core application of most marketing systems, and standard relational databases require much tuning and considerable hardware to handle them effectively. Since marketers rarely had the technology budget to pay for this, they were forced to seek the lower-cost alternatives provided by specialized databases, typically using columnar structures. Corporate IT departments were rarely enthusiastic about these non-standard systems, but, since marketing was usually a low priority, IT found it easier to let the marketers buy what they wanted and take responsibility for the consequences than to step in and do the work themselves. No one was quite happy, but an uneasy truce prevailed and everyone got their work done.
Today, specialized analytical databases are a hot fashion in corporate IT. These include not just columnar structures but also in-memory and shared-nothing parallel processing systems.
So you’d think that marketing systems using of those databases would be the center of attention.
Not necessarily. It’s true that big marketing systems with big analytical databases are more common than ever, and that IT is more comfortable with them. But the classic list segmentation and selection functions are less important because today’s marketing is increasingly focused on managing real-time interactions with individuals. Those interactions largely require conventional transaction processing technology. As a result, the analytical databases are receding to the role of supporting systems that prepare and stage data for front-line interactions. The really interesting business and technical challenges are making the front-line systems work better.
The implications of this run deep. For marketers, it means the traditional divide between marketing and sales is now much less distinct. Customers and prospects all interact with the same company systems (Web sites, call centers, email, etc.) and external social networks. To ensure that each visitor is handled correctly, marketing (responsible for prospects) and sales (responsible for customers) must jointly define the business rules and treatments. This is much more cooperation than those groups are used to.
The change also calls into question the fundamental architecture that has kept marketing databases separate from sales and customer service systems. Even though Customer Relationship Management (CRM) vendors have long claimed to integrate all three, a look under the hood has usually found that sales and service ran on a transaction-oriented database while marketing ran on a separate, analytical database. This was partly due to the technical requirements of marketing queries, which made the analytical data structure more effective. But the separation between sales and marketing activities meant there was no particular penalty for running fundamentally independent systems. Companies needed just a bit of data synchronization, which might be done in real time but could often run nightly without causing serious problems.
The equation changes drastically when marketing and sales do need to integrate. Now the advantages of a separate, analytically-structured marketing database must be weighed against the significant impediments to sales and marketing coordination that database creates. This gives companies more reason to consider alternatives.
The simplest approach is to just get rid of the separate analytical database altogether. This is increasingly viable because today’s relational databases and hardware can more easily support transactional and analytical processing. The very largest marketing databases will probably still run on separate analytical systems, but companies with smaller files will increasingly be able to get adequate performance for marketing applications from their transactional databases. Even if this requires a bit of tweaking to add some extra indexes or summary tables, the overhead will usually cost less than maintaining a truly separate marketing database.
Another option is to keep the analytical database but shrink its role. Under this scenario, the analytical database is truly used for analytics – that is, research and reporting – while the quasi-operational tasks such as list selection migrate to the transactional system. This is arguably a more logical structure anyway: since some of the tasks were assigned to the analytical database because it was the only system the marketers had available, not because they required an analytical structure. For example, many analytical systems work best with batch updates, but have been forced to take real-time updates so that marketers can work with current data.
The fundamental point to remember is that whatever happens to the analytical database, marketing will do an increasing portion of its work on the transactional systems. That’s the inevitable result of managing individual interactions at touchpoints such as the Web site. In theory, marketing systems could evolve to include their own transactional components. But that makes little sense when most companies already have these in the sales and service components of their CRM systems. It’s more likely that the long-standing promises of CRM systems to encompass marketing will finally be fulfilled, and separate marketing systems, like separate analytical databases, will play a smaller, more specialized role.
* * *
David M. Raab is a Principal at Raab Associates Inc., a consultancy specializing in marketing technology and analytics. He can be reached at draab@raabassociates.com.