2009 Jun 01

When Does On-Demand Business Intelligence Make Sense?

When Does On-Demand Business Intelligence Make Sense?
David M. Raab
Information Management
June 2009

Back in February, this column asked “Does On-Demand Business Intelligence Make Sense?” and answered a tentative “Yes”.    The gist of the argument was that the major obstacle to BI success is the skilled labor needed to build the systems, and on-demand vendors can reduce this through strategies including automation, end-user-driven interfaces, advanced database engines, and pre-built solutions to specialized problems.

True as this may be, it is coming at the issue backwards.  On-demand BI should not be a solution in search of a problem.  The real goal is to identify situations where on-demand BI is a better choice than conventional solutions.  In other words, the right question is not whether on-demand BI makes sense, but when.

The general answer is that on-demand BI is the right choice when it meets two conditions.  First, it must solve problems that conventional solutions cannot.  Second, it must not face insurmountable new problems created by the on-demand model itself.

Regarding the first condition, we’ve already identified the required amount of skilled labor as the major problem faced by conventional BI solutions.  But labor isn’t the only potential roadblock.  A project might cost more than it’s worth, take longer than the time available, or require more hardware than the company can immediately deploy.  On-demand solutions generally offer advantages in these areas, so projects that face these issues are good candidates for an on-demand approach.

The second condition addresses on-demand weaknesses.  Common objections to on-demand systems include: the perceived security risk of moving data off-premise; difficulties if close integration is required with other company systems; the cost of outages if the on-demand system goes down or you can’t connect to it; the risk of becoming dependent on a system you don’t control; and the impossibility of enhancing or customizing the application if it doesn’t meet your present or future needs.

This is where picking the right application and the right on-demand vendor become more important.  Let’s start with security.  Most on-demand vendors meet high security standards because their customers demand it.  Still, particular applications or data sets may require even tighter security than a particular vendor can provide.  This might limit your choice of on-demand vendors to those who can prove they meet that standard, force you to load only less-sensitive data, or prevent you from using any on-demand system at all.

Similar considerations apply to the other issues.  Tight integration between an on-demand system and other systems is much easier than it used to be, because some on-demand vendors have provided APIs or otherwise engineered their systems to permit it.  But the details vary from vendor to vendor.  On-demand BI systems in particular are generally not built with real-time external integration in mind.  Nor, for that matter, do most BI applications require it.  You have to look carefully at what you need and what the systems you’re considering can do.

The cost of outages also relates primarily to applications that are closely coupled to other company systems.  For example, a BI system that made real-time credit decisions about new orders could cause major problems if it were unavailable even for a few minutes.  But most BI systems are used for statistical analysis rather than operational transactions.  Although you might be able to find an on-demand BI vendor who could guarantee to meet stringent availability standards, an application that required extreme reliability would probably be a poor choice for on-demand.

Vendor dependency is an issue with any externally-owned system, not just on-demand.  For that matter, reliance on internally-built systems also raises dependence issues.  But on-demand does compound the usual risks of vendor lock-in (unreasonable price increases, unresponsive service, deterioration in quality) with concerns that the system could become unavailable if the company went out of business or its system crashed.  Again, most BI applications are not mission-critical in the short term, so the worst-case scenario – permanent loss of access – is usually an acceptable risk.  Where continued access is truly essential, you may be able to negotiate appropriate back-up arrangements as part of the on-demand agreement.

The final issue is inability to enhance or customize the system.  Unlike the previous concerns, this applies to BI applications in particular.  While requirements for operational applications are usually well known and fairly stable, BI requirements are often poorly understood until users can load and analyze their data.  In addition, the requirements change frequently as new projects and questions arise.   This means that a BI system selected with a particular task in mind may not meet future needs.  The solution lies primarily in the vendor choice, and in particular in matching the vendor’s approach (automation, simple interface, flexible database, predefined solutions) to your situation.  For example, if you expect to work with many large files, a BI solution built around a flexible, scalable database is most likely to accommodate future needs even if you can’t define them precisely.  But if your major constraint is a lack of skilled staff, a highly automated system may be your best bet.

In the end, on-demand systems cannot solve every BI problem.  Some BI projects are best built in-house with conventional technology, and others should be outsourced completely.  You must assess the requirements for each project and match it against the alternatives to select the most suitable approach for every case.

*                            *                           *

David M. Raab is a Principal at Raab Associates Inc., a consultancy specializing in marketing technology and analytics.  He can be reached at draab@raabassociates.com.

2009 May 01

Marketers May Shift from Analytical to Transactional Databases

Marketers May Shift from Analytical to Transactional Databases
David M. Raab
Information Management
May 2009

Marketing systems were early adopters of analytical database engines.  This was purely from necessity: list segmentation and analysis were the core application of most marketing systems, and standard relational databases require much tuning and considerable hardware to handle them effectively.  Since marketers rarely had the technology budget to pay for this, they were forced to seek the lower-cost alternatives provided by specialized databases, typically using columnar structures.  Corporate IT departments were rarely enthusiastic about these non-standard systems, but, since marketing was usually a low priority, IT found it easier to let the marketers buy what they wanted and take responsibility for the consequences than to step in and do the work themselves.  No one was quite happy, but an uneasy truce prevailed and everyone got their work done.

Today, specialized analytical databases are a hot fashion in corporate IT.  These include not just columnar structures but also in-memory and shared-nothing parallel processing systems.
So you’d think that marketing systems using of those databases would be the center of attention.

Not necessarily.  It’s true that big marketing systems with big analytical databases are more common than ever, and that IT is more comfortable with them.  But the classic list segmentation and selection functions are less important because today’s marketing is increasingly focused on managing real-time interactions with individuals.  Those interactions largely require conventional transaction processing technology.  As a result, the analytical databases are receding to the role of supporting systems that prepare and stage data for front-line interactions.  The really interesting business and technical challenges are making the front-line systems work better.

The implications of this run deep.  For marketers, it means the traditional divide between marketing and sales is now much less distinct.  Customers and prospects all interact with the same company systems (Web sites, call centers, email, etc.) and external social networks.  To ensure that each visitor is handled correctly, marketing (responsible for prospects) and sales (responsible for customers) must jointly define the business rules and treatments.  This is much more cooperation than those groups are used to.

The change also calls into question the fundamental architecture that has kept marketing databases separate from sales and customer service systems.  Even though Customer Relationship Management (CRM) vendors have long claimed to integrate all three, a look under the hood has usually found that sales and service ran on a transaction-oriented database while marketing ran on a separate, analytical database.  This was partly due to the technical requirements of marketing queries, which made the analytical data structure more effective.  But the separation between sales and marketing activities meant there was no particular penalty for running fundamentally independent systems.  Companies needed just a bit of data synchronization, which might be done in real time but could often run nightly without causing serious problems.

The equation changes drastically when marketing and sales do need to integrate.  Now the advantages of a separate, analytically-structured marketing database must be weighed against the significant impediments to sales and marketing coordination that database creates.  This gives companies more reason to consider alternatives.

The simplest approach is to just get rid of the separate analytical database altogether.  This is increasingly viable because today’s relational databases and hardware can more easily support transactional and analytical processing.  The very largest marketing databases will probably still run on separate analytical systems, but companies with smaller files will increasingly be able to get adequate performance for marketing applications from their transactional databases.  Even if this requires a bit of tweaking to add some extra indexes or summary tables, the overhead will usually cost less than maintaining a truly separate marketing database.

Another option is to keep the analytical database but shrink its role.  Under this scenario, the analytical database is truly used for analytics – that is, research and reporting – while the quasi-operational tasks such as list selection migrate to the transactional system.  This is arguably a more logical structure anyway: since some of the tasks were assigned to the analytical database because it was the only system the marketers had available, not because they required an analytical structure.  For example, many analytical systems work best with batch updates, but have been forced to take real-time updates so that marketers can work with current data.

The fundamental point to remember is that whatever happens to the analytical database, marketing will do an increasing portion of its work on the transactional systems.  That’s the inevitable result of managing individual interactions at touchpoints such as the Web site.  In theory, marketing systems could evolve to include their own transactional components.  But that makes little sense when most companies already have these in the sales and service components of their CRM systems.  It’s more likely that the long-standing promises of CRM systems to encompass marketing will finally be fulfilled, and separate marketing systems, like separate analytical databases, will play a smaller, more specialized role.

*                            *                           *

David M. Raab is a Principal at Raab Associates Inc., a consultancy specializing in marketing technology and analytics.  He can be reached at draab@raabassociates.com.

2009 Apr 02

Tools to Support Social Media Marketing

Tools to Support Social Media Marketing
David M. Raab
Information Management
April 2009

Social media in all its forms—blogs, social networks, forums, file sharing, Wikis, and the rest—continues to evolve at a head spinning pace. But for marketers and IT departments, it has already moved past “my cat is cute” to become a can’t-ignore business tool.

Yet “can’t ignore” isn’t the same as “can manage”.  There are literally thousands of software applications to help you monitor and interact with social media.   Picking your tools is a major project in itself.

To bring some order to this process, first step back and define your objectives.  Common social media applications include:

- gathering market intelligence about attitudes towards your company and competitors
- executing campaigns for outbound marketing, publicity and community development
- identifying potential customers based on their published comments
- connecting with people through common acquaintances and recommendations
- tracking behaviors and attitudes of sales prospects and customers
- measuring campaign results (including, of course, the results of social media campaigns)
- providing support and responding to complaints in public or private

Many of these applications share some requirements, although the mix is different for each.  Here is a brief list of primary requirements by application:

- gather market intelligence: keyword alerts to identify references to the company or product; sentiment measurement to understand the nature of the references; external site traffic and link measurement to measure influence; trend reporting and analysis

- execute campaigns: content creation and posting; external traffic and link measurement; search engine optimization and paid advertising to build traffic to your own sites; platforms to build and manage your own communities

- identify potential customers: keyword alerts, social network research to profile  individuals

- connect with people: personal network and reputation development; social network research

- track behaviors: social network research, Web site visits and behaviors; behavior-based alerts

- measure results: traditional Web analytics (traffic sources and volumes); keyword alerts; sentiment measurement; search volume and destinations; link tracking; content rating on sites like Technorati and Digg; track use of company-generated content

- support and complaints: keyword alerts, sentiment measurement, social network research, external traffic and link measurement

There are dozens and sometimes hundreds of tools to address each requirement.  Social media themselves are a great place to identify the candidates, since expert bloggers often post information on products within a given specialty, and community members are usually eager to share their own experience.  Many of these people can also be hired as consultants—something that can be well worth the investment.

But before you jump into the details of individual products, you’ll want to define a strategy.  Almost every company today has several social media applications on its agenda, so you’ll want to inventory the projects to see which requirements are shared.  Ideally you’ll then select a company standard tool for each requirement.  The reason is less to save on software costs than to avoid supporting more than one platform.  Even if each department runs the tool independently, a shared standard will let them compare notes and help each other out.

Your strategy should also consider the need to integrate the different products, in reports if not actual operation.  Most Web-oriented systems are designed to be open, which in practical terms translates to Application Program Interfaces (APIs) that allow external access to their data and functions.  But the scope and efficiency of those APIs varies widely because vendors must balance openness against external demands on shared servers and the never-quite-dead goal of customer lock-in.  So there’s no avoiding the need to define the required points of integration and to assess carefully whether proposed systems can support them.

Another part of your strategy must estimate the value expected from each application and from meeting individual requirements.  This allows you to add the different supporting systems in the sequence that makes the most sense from both business and technical perspectives.  These choices are essential in the current economy, where every penny and every moment must be spent as productively as possible.

Your strategy must also set standards for vendor stability.  This is especially challenging for social media tools, which are often cobbled together by tiny start-ups.  The good news is that switching to a new system is often inexpensive, even counting the cost of staff time and business interruption.  So unless a particular function is truly mission critical, you can usually take a chance on a great product from a shaky vendor.  In reality, most of your initial social media projects are experimental, so learning quickly and cheaply is more important than finding a system you’ll be able to keep for years.  Much of what you learn is about the underlying technologies and business applications, so that knowledge will retain its value even if you change tools..

Indeed, the most important part of your social media technology strategy has to do with people, not tools.  It’s a truism, but you really do need to train your staff to understand the business objectives of the various projects, and the functions, data and processes needed to support those objectives.  These basic principles will change much less rapidly than the technology itself, and a well-trained staff working in a sound framework will easily adjust to whatever the social media geniuses come up with next.

*                            *                           *

David M. Raab is a Principal at Raab Associates Inc., a consultancy specializing in marketing technology and analytics.  He can be reached at draab@raabassociates.com.

Previous Entries Next Entries