David Raab Article Archive

Specialized Marketing Databases
David M. Raab
DM News
June, 1993
.

One of the life’s lesser mysteries is that, for all the talk about marketing databases, the actual number of installations by marketing database software vendors is quite small–perhaps under two hundred, depending on who you include. What’s everyone else using? Or is less going on than it seems?

The short answer is that many companies do in fact have some kind of a marketing database, but most are home-grown affairs built on standard database engines like DB2, Oracle or dBASE. Even service bureau systems tend to be built by the bureau, rather than purchased from the outside.

But there is another reason that general-purpose marketing database software has found a fairly small market. This is the growing number of marketing software packages that target specific industries.

MarketEXPERT (Retail Marketing Systems, 203-656-3411) is a good example of the type. The system offers the basic data gathering and reporting functions of a marketing database, but with capabilities specifically tailored to retailers. In fact, the existing ten clients are all grocery chains (totaling about 500 to 1,000 stores). In part, this is because of the specialized work needed to link to the point of sale systems used by grocery stores.

(Retail Marketing Systems is different from Retail Target Marketing Systems, whose Archer retailing marketing database was reviewed in this column on March 22, 1993.)

MarketEXPERT takes a flexible approach to one of the toughest problems facing retail marketing databases: the sheer volume of transactions. The system accepts each purchase by each customer directly from the point of sale system, typically on a daily basis. Then, to keep the volume manageable, it summarizes the transactions according to user-defined rules.

The key to MarketEXPERT is this summarization process, which allows each transaction to fall into one or more “behavior groups”. These groups serve multiple functions: product line or departmental reporting; tracking purchases of specific products to measure advertising effectiveness; flagging buyers of special-interest items such as low-salt or baby products; or monitoring purchases of specific brands. Any single item can be part of up to 32 behavior groups, and the total number of active groups is unlimited.

The dollar volume and number of items from each transaction is added to the weekly total for all the applicable behavior groups, which also store a transaction count. Carefully defining the groups is critical because once the system has posted information to the groups, it discards the transaction detail. The system also stores weekly and lifetime totals for each customer, and stores demographic and special interest information at the customer level.

Customers are typically loaded into the system, either manually or by batch file, when they sign up for a courtesy card or preferred-buyer program. The system allows multiple IDs per individual and lets individuals be grouped by household. Linkages among records can be built automatically using retailer-defined rules or by having the operator search for matches during data entry. For sophisticated matching efforts, MarketEXPERT relies on third-party duplicate identification software such as Postalsoft.

Retail Marketing Systems also has arrangements with R.L. Polk to do geocoding and demographic enhancement and to import prospect names with specified characteristics.

MarketEXPERT provides a number of standard reports, including summary and details by behavior group and by household. The system does not have a custom report writer, but plans to add one before the end of the year, along with the ability to do profitability calculations and to use third-party query/reporting packages.

As befits an industry-specific package, MarketEXPERT compensates for its lack of generalized data manipulation with a very powerful promotion system aimed especially retailers’ needs. The system lets users define customer lists based on behavior and demographics and then create different offers (say, coupons for various products in various dollar amounts), along with the customer qualifications associated with each offer and the priorities among offers. It then makes a single pass of the database to assign each customer to a minimum and/or maximum number of offers.

The result is two files: one with offers, and the other with a list of customers and the offers that apply to each. The files can be sent to a lettershop to create mailings that send the desired promotions to the desired customers.

MarketEXPERT runs on the Unix operating system, using the Informix C-ISAM database engine. A single 486 PC can handle between 10 and 15 stores with about 150,000 customers; larger installations can run multiprocessor NCR hardware. A version running the Oracle database engine has just been released: because Oracle requires much more powerful hardware than C-ISAM for the same size file, this is aimed at smaller installations.

Installation costs about $9,000 plus $2,500 per store for a six-month license. The license fee drops 25% per year, and there is a discount for multiple stores. The module to load prospect files from R.L. Polk costs extra.

MarketEXPERT was first installed in 1992. The largest current database has about 2.5 million customers from 350 stores, while most are in the 100,000 to 1 million range. Performance is determined by the file size and hardware, but typically the system operates by loading data and doing major promotion selections overnight. Simple reports might be run in minutes, while a complicated analysis might take several hours. Unix allows several functions to execute simultaneously, although a large query in the background will slow performance for other users on the system.

Harvest Marketing System (Worth Information, 612-934-0575) belongs to the largest group of specialized marketing databases: those aimed at financial institutions. Harvest, which has been available since 1989, is the only financial marketing database to use a non-proprietary relational database engine (Oracle). This allows great flexibility, including about 500 data fields per account record (compared to less than 100 for most competitors) and the ability to store an unlimited amount of historical data. It also means that third-party reporting and query tools can read the files directly, without the need to create ASCII extracts.

But Harvest pays a severe price in performance. Where systems with flat-file database engines can pass several million accounts per minute, Harvest on a PC server can take up to a minute to return queries against a 50,000 account file. Queries with subtotals or against larger files take even longer, although the exact performance is hard to predict. The larger amount of data also means that Harvest takes 10 to 15 times as much storage as competing systems.

(For reviews of other financial systems, see the October 19 and October 26, 1992 editions of this column.)

Performance has not been a problem for Harvest’s 30 existing users, who are mostly small community banks with about 50,000 accounts. Worth runs the updates for these clients and then transfers the completed files to the clients’ PCs.

To expand its market, Worth has recently introduced an option that allows institutions to run their own updates. Clients can then place the resulting files on very powerful computers–either mainframes or multi-processor file servers–that give adequate performance with large files on Oracle. In one recent benchmark (running a $500,000 server with 6 processors, 30 disk drives and 320 megabytes of RAM), 50 simultaneous users querying an 8 gigabyte, 3.5 million customer database got responses in no more than a few minutes.

Performance aside, Harvest offers an impressive facility that combines query definition and report writing. In a single screen, the user can choose fields from any data table; define which records to select through logical conditions such as equals, greater than, less than, includes, and like; determine whether to report at the account, customer or household level; and specify summary measures, such as count, sum, average, minimum/maximum, and standard deviation, for each field. A second screen then allows adding headers and footers, selecting a sort sequence, defining breaks and specifying subtotals and other summary measures on each break.

This is more efficient and easier to use than many other systems, but has its limits. Anything other than a columnar report–such as cross tabs or free form reports, graphics, mapping files, scoring or calculations–must be done outside the system, by exporting query results in an ASCII file. Nor can users apply spreadsheet-style calculations to their files or reports.

The system does apply fixed profitability formulas, using the client’s own cost figures, during the update. Results are stored at the account, customer and household levels.

Harvest also uses its query function to drive promotion selections. Once a list is defined in the first query screen, the user can immediately move into promotion definition, where the promotion can be assigned a name, start date and end date, list of included products, mailing list, custom salutation field, and percentage to set aside as a control group. The user can then review individual records on the list, to mark some for exclusion or to enter custom salutations (for example, “Bill” instead of the “William”). Any salutation change is stored permanently.

Once the list is set, the system will automatically create a tracking file of the selected names, and can create word processing merge files with customer-level data. At present, account-level data cannot be directly exported in word processing formats, although it could be placed in an ASCII file via the report writer. The system’s standard reports will show the number of responses to each promotion (defined as any sale of a product offered in the promotion to someone on the promotion list). Standard reports will also compare performance of the test vs. control group.

The newest version of Harvest allows direct editing of individual records (for example, to update a name or address). Other than this, the file cannot be modified between updates. This should change by the end of the year, when Worth plans to allow clients to make daily updates of account balances and to update changed records only.

Pricing of Harvest depends on whether the update is done by the client or by Worth, and on the number of users and processors in the server. A minimum system–up to 25,000 accounts running on a single-user 486 PC where Worth does monthly updates–could cost about $20,000 to install and $8,400 for a year of processing. An institution with a million accounts might buy a license for 16 users and a several-processor server, which would cost about $200,000 plus 15% per year in maintenance.

Target MCIF (Customer Potential Management, 800-332-2631) is really two different financial marketing database systems.

The original Target MCIF, introduced in 1988, was built on a proprietary relational database technology that yielded typical query performance: between 10,000 and 50,000 accounts per minute on a 386 PC. Most of this system’s 30 installations are small institutions in the 50,000 account range, although the largest has 280,000 accounts. Updates are done on CPM’s computers and the data is fixed between updates.

Target MCIF has a powerful menu-driven query function and numerous standard reports, but no user-defined reports or calculations. Names selected with the query function can be assigned to promotions and saved as tracking files for response analysis. There is no matrix mail function, although CPM can apply selection or scoring schemes during the update and then place tags indicating segment and subsegment on the Target MCIF database. Files can be exported in several word processing formats or flat ASCII for custom reporting, graphics, mapping and other uses.

The original Target MCIF costs about $7,500 to license and install. Update fees are based on frequency and the number of accounts: the annual cost would range from $7,800 for 50,000 accounts updated quarterly, to $19,200 for 100,000 accounts updated monthly. Training is done by video tape.

Like Worth, CPM has recently moved to expand its market by packaging its file creation software for in-house use. This version of Target MCIF runs on a Unix platform and creates files that can then be loaded into any relational database the client chooses. From that point, the client is able to manipulate the files directly using home-grown or third-party tools for querying, report generation, segmentation, scoring, graphics, word processing and other functions. When it’s time to rebuild the file with fresh data, the client again runs Target MCIF.

In addition to the data import, standardization, householding and file export functions that are part of the file building process, the new Target MCIF has modules for profitability scoring and for neural-network-based predictive response modeling. These modules are run at the time of the update, and the results embedded in the resulting file.

Pricing of the new system is based on hardware platform. A system on a 486 PC will cost about $55,000 for the license and installation, plus about $5,500 each for profit scoring and response modeling. A multi-processor system able to handle a million accounts might cost about $150,000, while a mainframe-class installation could cost as much as $500,000. Annual maintenance is 15% of the license fee.

* * *
David M. Raab is a Principal at Raab Associates Inc., a consultancy specializing in marketing technology and analytics. He can be reached at draab@raabassociates.com.