David Raab Article Archive

Unibase Direct D.Mark
Cogent Banking Systems AnswerBase
Decision Software, Inc. TopDog
David Raab
DM News
December, 1993

Once upon a time, PC-based marketing database systems fell into two neat groups: very fast systems using proprietary technology to handle millions of records, and much slower systems using off-the-shelf database engines for smaller files. Regardless of the technology, these systems were rigidly tuned to specific data structures, screens and functions, which the developers alone could change. Most also used fixed files that could only incorporate new data via complex loading procedures, which were typically run monthly.

That was then. Today, a new generation of PC-based marketing systems uses off-the-shelf database engines with performance that rivals yesterday’s proprietary products. These systems allow on-line data entry and let technically competent users make substantial changes. Some will even incorporate entire new user-created files automatically.

D.Mark (Unibase Direct, 201-285-1700) is one of these new systems. D.Mark is written in FoxPro, a blindingly-fast database that uses a dialect of the standard dBASE programming language. D.Mark systems routinely manage four gigabyte databases with 10 million prospect records and one million customer records. On these, it will return complex queries in five to ten seconds and run a cross tabulation in five to ten minutes.

The key to this performance is FoxPro itself. Fox uses a technology that it extremely fast when querying against indexed fields–so D.Mark systems routinely are built with every data field indexed (except fields that are not ordinarily analyzed, such as names). These indexes take up about as much space as the data itself, but D.Mark compensates through using Stacker data compression software which cuts storage requirements about by half. Interestingly, Stacker does not slow the system down: although it adds a little time to decompress the data, it saves an equal amount because only half as much data must be loaded from the disk.

The one fly in FoxPro’s ointment is that performance suffers dramatically when the system tries to work with more than one file in a relational fashion. D.Mark’s creators have compensated for this by building procedures that preprocess related files independently and then merge the results. For example, a penetration report comparing customers to prospects by Zip Code would first summarize the customer file by Zip, then summarize the prospect file by Zip, and then merge the two summaries. This is much faster than linking the customer and prospect files and doing the work in a single pass.

Unfortunately, this type of preprocessing requires careful programming. D.Mark includes standard penetration and cross tab functions with this capability built in, and its developers can set up other procedures for specific customers when their system is installed. But other, unplanned analysis can be quite slow indeed.

Despite this limitation, D.Mark offers its customers a large amount of control. The system is written in standard FoxPro, and Unibase includes the programs with each license–so competent users could modify the system themselves or hire outside programmers to do it for them. It is also possible to read or update the files outside of the system, since D.Mark uses standard FoxPro files in the familiar dBASE ‘DBF’ format.

Typically, though, D.Mark files are built with a custom data loading program that is part of the system. The actual process can be executed either at Unibase or by the client. Unibase will also perform any preliminary processing, such as identifying duplicate or related records, geocoding or gender coding, since these are not part of D.Mark’s capabilities.

Once the file is built, D.Mark users can employ a menu-driven query generator to make selections, do calculations or generate extract files. This is a simplified version of the standard FoxPro query generator, easier to use but limited to a single file. To make it more useful in multi-file situations, Unibase will customize the query generator so that related files appear to be a single file: for example, SIC descriptions look like they are part of the customer record, even though they are actually stored in a separate table.

The generator creates optimized FoxPro SQL (Structured Query Language) statements. This gives it the ability to do calculations, via an expression builder, and to store the results in either an existing database field or a new, temporary file. For more elaborate output, the system can also use the standard FoxPro report writer, which has both a “quick report” mode that lists selected fields from the chosen records, and a sophisticated mode that can handle multiple lines per record, calculations, logical expressions, precise formatting, subtotals, page breaks, and other niceties.

Queries and report formats can be saved for later reuse or modification. The system lacks batch processing or macro capabilities, however.

D.Mark has a useful function to generate random file samples for test mailings or other purposes. It also supports grouping based on Zip Codes (to assign sales territories, for example) and on variables such as user-defined sets of SIC codes. However, the system lacks direct marketing facilities such as single-pass selection of multiple complex file segments, and does not build a true promotion history file when selections are made. These functions could be custom coded if a user wanted them.

Like many other marketing database systems, D.Mark relies on data exports to third-party software for graphics, mapping, telemarketing and form letters. Data can be exported in ASCII, dBASE, Lotus or Excel formats. Exports run at 50,000 to 100,000 records per minute, depending on the size of the records and proportion of the file selected.

D.Mark software (including source code) plus installation and user training costs $15,000 for a stand-alone version or $30,000 for a local area network license, plus $5,000 to $2,500 per workstation depending on quantities. The initial data load and customization typically costs $15,000 to $30,000 more and takes four to six weeks. Annual maintenance is $1,000 per workstation and includes software upgrades, unlimited toll-paid support (9 a.m. to 6 p.m. Eastern time), and remote access by modem.

The system was introduced in 1992 and now has 12 clients, all divisions or subsidiaries of telecommunications companies who use it for business-to-business marketing. This is a result of the backgrounds of the developers rather than anything inherent in the system, however. Standard hardware is a stand-alone 486 PC running DOS, although a Windows version is being tested and the system has been run in local area network configurations. Unibase itself is a mid-sized service bureau that provides data processing, telephone-based lead generation and market research, and data entry.

AnswerBase (Cogent Banking Systems, 201-379-7979) was also introduced in 1992 and also uses an industry-standard development environment. In this case, the tool is Advanced Revelation, a “fourth generation language” that can use files built either in its own relational database or the more common Sybase/SQL Server. (Sybase and SQL Server are the same product, sold under different names by Sybase and Microsoft respectively.) This allows AnswerBase to run on either a standard PC network or in true “client/server” mode. All existing installations are network-based, although one client is reported to be considering client/server.

AnswerBase is designed to provide small to mid-size banks with the functions of a traditional “Marketing Customer Information File” (MCIF) system–that is, the ability to bring together all activity related to a customer or a household, and then to analyze the data and develop promotions based on it. The system originated with a bank that needed help with evaluating its loan portfolio, so in addition to standard MCIF functions it also includes an unusual ability to continuously reevaluate the quality of specific loans, and to make the information immediately available to individual loan officers.

In keeping with its mission of helping individual bank officers, AnswerBase also includes “contact management” capabilities such as callback dates, text notes and result codes for conversations. These are rarely found in conventional MCIFs.

In fact, AnswerBase is extremely flexible compared with a traditional MCIF. Since it is written in standard Advanced Revelation, users can change the file structures, screens and functions as well as the data. Cogent, the largest Advanced Revelation trainer in the country, encourages its clients’ technical staff to learn to make these changes. AnswerBase is a relatively new product for Cogent, which supplies several other types of financial systems to clients in all industries, as well as undertaking custom software development on PCs and midrange computers.

Of course, any self-respecting MCIF must prepare the data–that is, standardize the addresses and link related accounts–as well as use it. AnswerBase does this on the client’s PC, using a combination of third-party address processing software and its own algorithms to identify accounts belonging to the same individual and household. The system relies primarily on the Social Security number, 11-digit Zip Code and last name, and is sensitive enough to adjust for minor variations in address or name spellings. The process is tuned for each client, and can later be modified by the client’s technical staff.

Once the file is built, queries can be generated either by typing them directly in SQL or by using the standard Advanced Revelation menu-based query generator. Actual performance depends on the nature of the query: on a 100,000 record file, a simple query against an indexed field would execute in less than a second, while a query complex query against non-indexed fields could take as long as ten minutes. This is slower than many other PC-based systems, but adequate for the relatively small files of AnswerBase clients. One reason the system takes longer is that each query automatically generates a list of the included records. This ultimately saves time since subsequent operations can be run against that list rather than the entire file. The query statement itself can also be named and saved for reuse or modification.

Although queries can be of unlimited complexity, AnswerBase also lacks sophisticated direct mail management functions such as single-pass selection of multiple segments and sampling to assign test vs. control groups.

AnswerBase does allow the user to perform calculations against the files, using the Advanced Revelation programming language. Results can be stored either on the main file itself, or in a temporary file for ‘what if’ analysis. There are no built-in formulas for complex calculations such as profitability, but Cogent will incorporate specific formulas for individual customers as requested.

The system contains over 130 standard reports, targeted at banking applications, plus the ability to use an enhanced version of the Advanced Revelation report writer for ad hoc reporting. There are no built-in graphics, mapping or word processing, but the system can export files in specialized formats including Lotus 1-2-3, Excel, Microsoft Word, WordPerfect, MultiMate, Harvard Graphics, MapInfo and Strategic Mapping. Other formats will be added as customer request them.

Prices for AnswerBase depend on the size of the bank. A site license (unlimited users on a single server) costs $60,000 for institutions up to $250 million in assets, $70,000 for institutions between $250 million and $400 million, and an additional $10,000 for each further increment of $150 million. Alternately, smaller institutions can pay $5,000 for installation plus $2,000 per user per module. The system has three modules: loan administration, deposit management, and the general Bank Marketing System. Cogent is also working on a loan origination module.

Customization, installation and training are included in the purchase price; after 90 days, the company charges 15% of the purchase price for annual maintenance including upgrades, toll-paid phone support from 7 a.m. to 9 p.m. Eastern time, and remote access by modem. It takes about two weeks to set up a new system once the input files are received.

AnswerBase is currently used by seven banks, with over 100 users on PC-based local area networks.

TopDog (Decision Software Inc., 301-445-2481) resembles traditional marketing database systems in having programs (written in Visual Basic and Visual C++) that cannot be changed by the client. However, TopDog is vastly more flexible than conventional systems, both because it uses a standard database engine (again, Sybase/SQL Server) and, more important, because it automatically adjusts to changes in the files presented to the system.

Technically, TopDog adjusts to file changes by directly reading the Sybase/SQL Server table catalog. In practical terms, this means users can add a new file or a new field to an existing file with standard Sybase/SQL Server tools, and have the new versions immediately available for analysis or reporting through TopDog. It also means that, even though TopDog itself cannot update individual records, data can be changed outside of the system and be accessed via TopDog without a new data load.

TopDog does have its own data loading module, which takes information from various types of files (ASCII, mainframe, etc.) and puts them into Sybase/SQL Server tables. This is a template-driven utility so impressive that Decision Software has sometimes sold it as a stand-alone product. The data loader can also be customized to do summaries and to speed the load by doing incremental updates and selective overwriting rather than mass replacement of data. However, TopDog cannot perform address standardization, duplicate identification, householding, geocoding, gender coding or similar list preparation functions. These must all be done before the files are presented for loading.

The real strengths of TopDog are its graphical interface, query and reporting capabilities. Queries can be generated either by typing an SQL statement or by using a menu-driven “Assistant”; output can include calculated “expressions” as well as field contents. Speed is really determined by the capabilities of Sybase/SQL Server, and depends on the indexes available and query complexity. A simple count on an indexed field in a three-million record database might take 45 seconds on a 486 PC server, while a very complex query with subtotals and comparisons might run at about 200,000 records per minute–or up to five hours on the largest existing TopDog database, which has 50 million records and five gigabytes of data.

As a practical matter, TopDog users with very large databases build small sample files to speed results. To further improve productivity, the system uses Microsoft Windows’ multi-tasking capability to submit queries and then let the user do other work while a query is executed. The system lets users check the status of active queries while they are being processed.

Results of queries can be stored in temporary files, which are themselves accessible through TopDog reporting mechanisms. The system includes built-in functions for counts (e.g., the number of clients in New Jersey), frequency analysis (e.g., the number of clients in New Jersey by Zip Code), and cross tabulations (the number of clients in New Jersey by Zip Code by income level). In addition, users can “browse” the details of the selected records, view information in all files related to a single customer, and export data in ASCII fixed or delimited formats.

TopDog has no built-in telemarketing, mapping or word processing capability, and integrated graphics are limited to pie and bar charts of the frequency distribution report. But Windows allows it to easily integrate with external spreadsheet, reporting and word processing packages. Decision Software is working to add a mapping module that will incorporate demographic and boundary data.

The company is also working on a campaign tracking module that will allow single-pass selection of multiple, unduplicated segments, each with its own key code, and sampling within each segment to reach a specified quantity. The resulting file can be used to generate a set of mailing labels and be stored as a promotion history record. This feature is scheduled for release in February 1994.

A TopDog license costs $18,000 for unlimited users on a single server; total cost is about $30,000 for the license, set-up, installation, and training. It takes about a month to set up a new customer. Maintenance costs $3,000 per year, and includes upgrades plus unlimited toll-paid telephone support from 9 a.m. to 5 p.m. Eastern time. Technical support does not currently include remote access to user systems, but since Decision Software usually loads the client data itself, it typically has copies the client files available.

TopDog can run on any network that supports Sybase/SQL Server, using a PC running Windows as a workstation. Of the three current installation, two have UNIX servers–although the 50 million record site actually runs on a PC. TopDog was introduced in 1992.

* * *