David Raab Article Archive

Privacy concerns have grown from a distant cloud on the bright horizon of relationship marketing to a looming thunderhead that may burst any second. The immediate catalyst has been the Internet, where consumers are acutely aware that their every movement can be tracked, recorded and analyzed. But database marketers have collected reams of personal data for years, a practice that generated fewer complaints only because consumers were less familiar with the details. Now that the Internet has made privacy a public issue, all kinds of data gathering are being examined.

Many marketers are genuinely perplexed by consumers’ concern. While marketers are often willing to play on public distrust of government snooping when it serves their purposes, they see little potential for abuse of data held in private hands. After all, a private business can’t put you in jail or make you pay taxes. The commercial reason for gathering personal information is benign: to provide advertisements and services that are tailored to individual interests. So long as consumers have the option to reject any offers they receive, what’s the harm? As these marketers see it, the whole issue has been manufactured by a handful of privacy nuts who are either genuinely paranoid or cynically use privacy to further their own political agendas. Proponents of this view point to opinion surveys, low use of existing opt-out services, and consumers’ willingness to trade data for slight compensation as evidence that the real value people place on data privacy is quite low.

But such arguments miss the point. True, most people don’t care about trivial annoyances, such as advertising, and therefore won’t pay even a low price in effort or cash to avoid them. But personal data gathered by businesses can be used for decisions with considerably greater impact than ad messages. Remember, the fundamental goal of relationship management is to tailor all interactions to build the optimal relationship with each customer. This goes beyond advertising to include which products are offered, how they are priced, and what level of service is provided.

This sort of data-driven tailoring is usually described as treating customers differently, but a less polite way to put it is that some will be treated better than others. Think about the airlines: your status with the company determines everything from how quickly they answer the phone to take your reservation, to how long you wait to check in, to how soon you can board, to when your luggage comes off, in addition to mention your legroom, food and drinks during the flight itself. Interestingly, this particular set of privileges seems to attract little hostility from the have-nots, perhaps because they still perceive air travel as a luxury. Compare this to the anger that banks generate when they charge additional fees or limit the services available to low-balance customers. This type of discrimination is perceived as hugely unfair, presumably because banking is considered an economic necessity, or perhaps because it seems to target lower income customers.

Although the superior treatment that airlines, banks and others give their best customers raises some issues of fairness, it is not primarily a privacy issue because the distinctions are based on transactions between the customer and the business itself. That is, there is no hidden data collection and no sharing of data from other sources. There are also no inferences based on predictive models or scorecards: the criteria are objective measures such as miles flown or balances maintained. This means that even though a policy may be inherently unfair, it is at least accurately and consistently applied.

By contrast, the worst privacy nightmares involve data that may be collected surreptitiously, inappropriately shared with others, applied incorrectly or just plain wrong. This is where the real harm gets done: someone is denied a job because they visited a gay Web site; someone is charged a higher health insurance premium because the insurer sees their past medical bills; a loan application is rejected based on a defective statistical model; a service request is denied because an external database suggests the buyer will not be a good future customer.

A privacy skeptic might point out that most of these situations involve a problem unrelated to privacy itself: people shouldn’t be penalized for visiting unpopular Web sites, people shouldn’t lie about their past medical bills, statistical models shouldn’t be defective, data shouldn’t be incomplete or inaccurate. But such problems do exist and always will; it would be silly to ignore them, assume they will all be fixed, or pretend we can legislate them out of existence. They are part of the privacy issue because they only hurt people when large amounts of personal data are widely available: otherwise, no one would be able to check what Web sites someone had visited, look at their medical bills, use certain variables in statistical models, and so on. In effect, privacy wraps a blanket of ignorance around each individual that prevents companies from even trying to discriminate among them, for good reasons or bad. It seems likely that a visceral understanding of the protection provided by privacy underlies most people’s concern for its loss.

Of course, no one has absolute privacy, and giving up information confers benefits as well as costs. So the privacy debate is really about striking a balance between the value that data can provide and the problems that it can cause. It’s tempting to argue for a free market solution, of letting individuals negotiate with companies about which data to share, for what uses and with what compensation. But things aren’t so simple: if most people decide to share a particular piece of information, then those who do not may be falsely assumed to be hiding something, or simply lumped into a category that gets worse treatment than others. So even voluntary data sharing results in effective coercion of those who would prefer to opt out. This means, paradoxically, that only government regulations can make data sharing truly voluntary, by protecting those who choose not to share.

Obviously there are economic and social costs to such regulation, so it should be limited to types of data that are worth controlling. The government also needs to ensure that companies use data only in the agreed ways, just as it would enforce any other contract. Finally, society may decide that some types of data should never be collected or should only be used for particular purposes. So, free market fantasies notwithstanding, there is no alternative to government involvement in this area. In reality, extensive government regulation already exists in areas such as credit reports and medical information.

The real question, then, is not whether the government should be involved in privacy regulation, but how. This ultimately depends on the social aims the regulation supports. One aim is privacy itself–the sense that what a person does should be nobody else’s business unless there is a good reason otherwise. Today such a right is widely, and legally, acknowledged, although it is still questioned in some circles. Another fundamental social goal is fairness: the idea that everyone should be treated equally unless they are different in a significant, relevant way. In today’s United States, differences such as race and religion are almost never acceptable grounds for differential treatment; other differences, such as income and education, are suspect but not automatically forbidden. Fairness is justified as a fundamental moral imperative–it is simply the right thing to do. In addition, fairness has a practical justification: giving everyone equal opportunities helps to ensure that society gets the benefit of all of its members’ talents.

Since the bedrock principle of fairness is that people should be treated equally unless there is a reason not to, any differential treatments–including the treatments of relationship management–need some justification before they are accepted. In the airline and banking examples mentioned earlier, the justification is fairly easy: the differences are based on relevant past behavior by the specific individuals involved. But what about differential treatment that is not based on concrete information about specific customer transactions? If the information is likely to be inaccurate or lead to false conclusions, it is considerably less defensible because people may be discriminated against unfairly. And–here’s where privacy comes back in–much of the personal data that is surreptitiously shared among companies is prone to exactly this sort of problem. There may be errors in the data itself, errors in matching data to the right person, approximations used in place of real data, and any number of other flaws. Nor, unfortunately, are the errors likely to be unbiased: data based on averages will penalize people who belong to disadvantaged groups and reward those who belong to more privileged strata, regardless of their personal characteristics. This means that differential treatment based on inadequate data is not only immoral, but diminishes the social mobility that is a key practical benefit of fairness. The utilitarian argument could be extended to oppose differential treatments even when they are based on accurate data: the theory is that treating everyone the same effectively subsidizes less well-off people, making it easier for them to ultimately succeed. But such subsidies are harder to defend on purely ethical grounds.

This is pretty abstract stuff, and there may be some holes in the logic. But what it boils down to is this: data privacy raises serious personal and social issues. It is not merely the irrational concern of a handful of paranoid Luddites. Relationship marketers should not fight blindly for the widest possible freedom in how they use personal data but should carefully seek to balance valid business and social considerations. Those who do may well find themselves supporting considerably greater restrictions on data sharing than they originally expected.

* * *

David M. Raab is a Principal at Raab Associates Inc., a consultancy specializing in marketing technology and analytics. He can be reached at draab@raabassociates.com.

By all rights, service bureaus should be extinct. Burdened with high-cost, slow-moving mainframes, these dinosaurs of the marketing services world should long ago have been supplanted by more nimble, less costly in-house systems. The growing desire for real-time, company-wide integration between marketing and operational systems, which seems to require that the marketing database reside in-house, should have been the final nail in the coffin.

Yet service bureaus continue to prosper. The reason is simple enough: most firms lack the skills to build and maintain a serious marketing database themselves. Such skills have always been rare, and today’s shortage of all types of computer staff has made them still harder to find. Combine this with the risk of in-house development and the need to move quickly in an ever-more-competitive marketplace, and the bureaus’ promise to deliver a sophisticated system in a reasonable time at a controllable cost is nearly irresistible.

But even in the relatively stable world of marketing service bureaus, change does occur. One of the more interesting recent developments is a shift away from traditional volume-based pricing–where customers were charged on a per-thousand basis for every processing step–to a flat-fee model where customers buy a certain amount of hardware, software and staff capacity and are free to use it pretty much as they please. Although few firms have moved to a pure flat-fee model, many have moved in that direction. (In fact, a recent study by Raab Associates found only three of ten proposals relied exclusively traditional unit-based pricing; in half the proposals, over 50% of the fees were not volume-related.)

Part of the reason for the change is technical. Traditional mainframe technologies involved large computers whose capacity greatly exceeded the needs of most individual service bureau clients. The databases were typically maintained through large, periodic batch updates during which one client’s work took over most of the computer’s resources for a brief period of time. In this environment, it made sense to share one large computer among many clients, and to charge those clients based on the proportion of that computer’s capacity that they consumed. Pricing was set by finding some measure of utilization such as processor cycles, figuring how many units of that measure could be processed when the machine ran at its practical capacity, calculating the cost to operate the computer (including downtime and overhead), and arriving at a cost per unit to charge clients. The effect was to translate a largely fixed cost–maintaining a large mainframe computer–into variable unit costs. This approach may sometimes result in prices that do not reflect true underlying costs: if the computer is used at less than the expected load, costs are not fully covered; if demand so exceeds capacity that a new computer must be added, the incremental cost is much higher than the price charged the customer. But avoiding these dangers forces managers to pay close attention to capacity management, which is one of the keys to success in a high-fixed-cost environment. So, somewhat paradoxically, variable-cost pricing makes sense when you are running a fixed-cost mainframe.

But today’s service bureaus have increasingly moved away from mainframes to Unix or Windows NT-based servers. These systems cost much less per unit of capacity than mainframes, except perhaps at the highest end of the capacity scale. More important, the smallest individual systems are much cheaper than the smallest mainframes. This means it now does make sense to think of dedicating a single machine to an individual client. At the same time, clients are increasingly moving away from large periodic batch updates to smaller, more frequent updates–for example, daily instead of monthly. This removes the periodic spike in capacity demand that was the other major reason to use shared rather than dedicated machines to handle each client’s work.

But there’s more to this story than technology. At the same time that traditional marketing service bureaus are moving away from volume-based pricing, the most exciting new variation of the service bureau model is the application service provider or ASP. And guess what? Most ASP charges are based on volume.

The difference isn’t due to technology: nearly all ASPs run Unix or NT-based servers. And it isn’t due to usage patterns either: most ASP systems provide frequent if not real-time data access; few do large infrequent batch updates. Nor is there any fundamental difference in customer goals: people hire ASPs for the same reasons they hire traditional service bureaus, to get sophisticated systems running faster and more reliably than they could it themselves.

It appears the reason is a bit more subtle. Many ASP implementations are for operational systems, such as accounting or human resources management, or for production-oriented marketing applications such as outbound email campaigns or

Web site data analysis. This is a fairly sharp contrast to the marketing databases supported by traditional service bureaus: the exact use of these systems is often not understood when they are created and is expected to change over time.

In other words, the unstructured nature of a traditional marketing database makes it particularly suited to fixed cost pricing. In the quasi-operational world of ASP systems, each transaction has a fairly clear value: the marketer presumably can judge whether each piece of email is worth the incremental cost of sending it; the accounting people are comfortable with paying a little extra for each additional journal entry. But the value of a particular marketing analysis is just about impossible to predict, so there is no way for a marketer to justify the incremental cost of conducting it. This makes unit-based pricing particularly uncomfortable. Even worse, if the marketer does discover some valuable new application that involves much more intensive use of the database, volume-based pricing penalizes this success by driving up costs sharply. This is especially irksome because the marketer knows full well that the unit prices charged by the vendor are much higher than the true incremental costs of the added processing volume, so much of the cost increase is simply higher profit for the vendor.

The fixed price approach lets the marketer make judgements about how to allocate limited resources without facing the risk of sudden and unexpected changes in cost. This is a much more congenial environment for the experimentation and evolution that are the object of most conventional marketing databases. And, of course, if the marketer does find an application that significantly increases capacity requirements, there is still the ability to add hardware and support services in relatively small increments. So flexibility is retained.

Now that the distinction between structured and unstructured processing has been made, older….er, more experienced observers will also recognize that the structured processing done by ASPs resembles the tasks that service bureaus provided in the days before marketing databases: things like merge/purge and postal standardization. Of course, the service bureaus charged for these on a per unit basis, and they usually still do.

In short, while the switch from mainframe to server-based technology has something to do with service bureaus’ change from volume-based to fixed pricing, it is not only reason. Marketers who are evaluating vendor pricing schemes, or vendors who are designing such schemes, should also consider the nature of the task at hand. Volume-based pricing makes the most sense when the task is highly structured and well defined. Fixed pricing–that is, buying a bucket of capacity to be applied as the user pleases–makes more sense when the tasks and their values are less well understood.

* * *

David M. Raab is a Principal at Raab Associates Inc., a consultancy specializing in marketing technology and analytics. He can be reached at draab@raabassociates.com.

One of the major challenges in building a customer-centric database has always been matching records from different systems that refer to the same customer. The necessary technologies–name and address standardization, postal costing, matching and householding–are fairly mature and quite familiar to firms with a long history of database marketing. But they are new to firms without this background and require a level of implementation skill that most will not have available in-house. Nor it is easy to add such skills, because experienced technicians are rare and because even if you hired one, they wouldn’t get enough practice at most companies to maintain their expertise. As a result, many firms either hire consultants to set up their systems or farm out the whole process to a service bureau.

Both of these approaches have drawbacks. Consultants can be expensive–though they are a bargain compared with the cost of failure–and may not train in-house staff to run the system after they leave. Service bureaus are even more expensive, and typically require days or even weeks to complete an update cycle. Perhaps worst in today’s time-pressed environment, it can take months to set up, test and refine a satisfactory in-house or bureau-based matching process.

Over the past few years, a new alternative has emerged. This involves using an external matching service on a real-time basis: that is, sending each new record to be processed as it is entered, and then loading the result directly into an in-house system. Vendors including Acxiom, Experian, Sagent, iMarket and Dun & Bradstreet all offer some version of this approach. While not necessarily cheaper than conventional methods, it offers near-immediate deployment, better quality than most firms could achieve themselves, and the option to simultaneously enhance the new record with external descriptive data.

External matching uses basically the same technology as conventional matching–that is, records are parsed into name and address components, standardized against postal and other reference tables, and then matched against other standardized records. Parsing and standardizing a single record takes just a fraction of a second, and poses no particular technical challenge. But matching has traditionally been a batch process that involved sorting the entire set of records to bring similar records together and then comparing neighboring records to each other. This won’t work with a real-time process, because each new record would have to be inserted into the sorted file as it was added, so future records could match against it. Moreover, maintaining a separate file for each client would be expensive and involve a time-consuming setup process.

Instead, external matching systems use a standard reference table of all individuals and households or businesses in the U.S. (or whatever market is being served). Such files, compiled from a variety of sources, are readily available to large service bureaus. Once an incoming record is standardized, it can be matched directly against this file regardless of what other records are in a particular company’s existing database and without any need to insert new records (except the handful that don’t match the reference table itself–which may or may not be worth adding). Each record in the reference file is assigned a unique, permanent ID. This means that when a match is found, the system merely appends this ID to the new record and returns it to the company, where it is easy enough to check whether a record with that same ID is already present in the company’s database. Thus, external real-time matching can be highly efficient from both a processing and storage point of view.

The reference table also lets the vendor pre-associate demographics and other enhancement data with each record, making it easy to return that data with the matched record itself. Similarly, the vendor can predetermine which individuals belong to the same household, so no on-the-fly processing is needed to return a household ID. The vendor can also use the permanent ID to link old and new addresses for the same individual, so if an old address is presented it can return the new address as well. Because the same reference table is used by many different clients, the vendor can afford to update it frequently, thereby providing even low-volume marketers with the most current information.

Unfortunately, assigning a standard ID to each indvidual also raises significant privacy issues. Since all match requests are processed against the same reference table, it would be easy enough for the vendor to keep track of which names have been presented by which sources–creating an unprecedentedly complete activity profile. Achieving the same result with conventional technology requires a massive matching job of many individual lists against each other, something that only very large marketers, compilers or name rental cooperatives can afford. The same technology would also let marketers subscribe to information about specified individuals–that is, be notified when a particular individual moves, makes a significant purchase, is reported to have an income change, or simply shows up on somebody else’s list. Again, conventional technology could only do this through periodic batch processes rather than continously.

The vendors of these services have policies that are designed to prevent what they consider privacy abuses, although some consumers might have different standards. The vendors also have strong economic motives to ensure that list owners do not use the standard IDs to share data without the vendor’s participation, and that list owners can use their services without fear their information will be used by others. So there are some safeguards built into the system. Whether these are ultimately deemed adequate remains to be seen.

In short, external matching provides an attractive alternative to conventional techniques in many situations. Marketers should be sensitive to privacy issues but take advantage when appropriate of its significant benefits.

* * *

David M. Raab is a Principal at Raab Associates Inc., a consultancy specializing in marketing technology and analytics. He can be reached at draab@raabassociates.com.