David M. Raab
DM News
November, 1994
It’s easy to get lost in cyberspace and overawed by massive new hardware, but most direct marketers still manage conventional lists on conventional computers. Here are two new systems that can help.
IPRO (Anchor Computer, Inc., 800-452-2357) is developed by a mainframe-based service bureau that has embraced the capabilities of new technologies. The system is designed for service bureaus or in-house mailers with up to millions of records. It replicates traditional service bureau capabilities including list import and reformatting, merge/purge, selections, output file creation and on-going maintenance updates. But it implements them in current technologies–including client/server architecture, relational database structures and graphical user interfaces. Small installations can run on PC hardware, while larger files can use Unix servers.
IPRO has basic file import capabilities, allowing a user to accept either xBASE or ASCII text files. But Anchor expects most users to first process their files through a related reformatting system that runs on Unix, Windows or OS/2. This system will parse unstructured files, identify name and address information through table look-ups and logic, perform address standardization, and place the resulting output in a standard format with a specific field for each type of information. The resulting file, in xBASE or ASCII format, can then easily be processed through merge/purge, updates and other standard routines in IPRO itself. IPRO uses a xBASE-compatible database management system called CodeBase, which provides very fast indexing and update capabilities. It can also access files in other standard formats such as Oracle and Sybase via ODBC. The screens themselves are written in the PowerBuilder application development tool.
Once files are imported to IPRO, they can be defined as related tables with a graphical interface that lets the user visually link files on common fields. A location screen allows the user to find a specific record via near or exact match, while a browse screen then allows the user to see the underlying data. The browse can show data from multiple files (such as company, individual and transaction tables) relating to a single entity.
The user can also tell the system which fields to index, and can define specific editing rules for each field in a file. These rules are selected from a table of choices, which can be expanded if needed through custom programming. The rules apply to both on-line editing and batch updates of a file. Batch updates can compare new and existing records to implement choices such as placing a transaction total on a customer record, replacing old values with new values, appending new traransactions while retaining the old ones, and keeping the higher or lower value in a specific field. More complex calculations, such as a scoring algorithm, could be added. The system will eventually allow users to create custom on-line data entry screens for purposes such as customer service or order entry, although this is not available in the initial release.
IPRO includes a very sophisticated merge/purge capability with the same logic as Anchor’s mainframe-based system. End-users can select either business or consumer processing, with five levels of tightness for each. Sophisticated operators can also manipulate the underlying parameters that control specific processes, such as the elements used in matching, types of comparisons, the range of records compared, number of passes through the file, and treatment of rural vs. urban areas. The merge/purge can allocate matches by priority, random selection or randomly within priority groups–also mainframe-class capabilities. It includes standard reports showing matches made with their reasons, as well as counts of matches among lists.
The system has a multi-cell selection screen that allows the user to define individual segments, create random samples, assign key codes, and then select multiple cells in a single pass of the file. Output can be in the form of a xBASE file, fixed length ASCII records, or a print image to drive large mainframe printers. Later processing–such as bar coding, postal sorting, label creation or personalized letters–will be done with third-party software that Anchor can integrate with the system. Users can create batch jobs that incorporate several of these processing steps, and will be able to launch multiple batch jobs in operating systems–such as Windows NT, Unix or OS/2–that allow true multi-tasking.
In addition to merge/purge reports, IPRO includes standard update control reports and basic counts such as state and SCF. The system includes a third-party report writer, CodeReporter, which will allow users to create their own reports as needed.
At present, IPRO will run on Windows, Windows NT and OS/2 workstations, with PC or Unix servers.
Pricing for IPRO is expected to start at around $75,000, and will vary based on the selected options and hardware and software platform. The system is currently operating at one installation, and is slated for official release in December.
LISTman (Direct Marketing Software, 800-210-9157) is intended to give in-house operations or small service bureaus the ability to do sophisticated list management without the continuous involvement of technical personnel. The PC-based system does this by providing a framework for user-defined menus, screens, “methods”, functions and macros, which can be linked together as needed for a particular installation.
For example, a user wishing to create merge/purge can call upon a library of standard comparison functions, including exact, phonetic, string and statistical matching routines, plus others the user might create. One or more functions can be embedded in a “method”, which applies the functions to specific database fields or groups of fields. A method can also use multiple keys to identify sets of records that are potential matches, and then apply a second, more detailed set of comparisons to records within each set. Once a method is created, it appears as a selection option on a deduplication screen, and can be applied by selecting the option.
Things can be made still simpler, if desired. The selection itself can be made into a menu option, or embedded in a “macro” that strings together several activities. Macros can be submitted to a scheduler to execute at a specified time or to run repeatedly at regular intervals. In network installations with several servers, the scheduler can automatically assign a job to whichever server is available at the selected time.
The ability to create complex procedures and then hide the details from the user runs throughout LISTman. In addition to deduplication, users can create methods to merge data from multiple files and to modify or calculate on existing files. These methods can also be selected from lists and applied to specific files as needed.
The system also allows the user to create screens to view and edit data directly, and lets the user define complex validation rules for each field on a screen. These rules can check for specific values and control the format of the data, or apply more complex logic that might depend on the value already placed in another field (for example, verifying a city name based on the Zip), perform calculations on data from several fields, or modify the contents of another field. The validation rules can access data in multiple files, although a screen can only display records from one file at a time. Different validation rules may apply to the same file in different screens–which increases flexibility but also requires care to ensure consistency.
A LISTman installation begins with a standard set of menus and functions. These can be modified as needed, or set up as separate environments (different files, menus, procedures, etc.) for different clients. Actual lists can be created via update screens or, more likely, imported from xBASE or ASCII fixed or delimited file formats. The system uses special logic to simplify the import process, even attempting to identify the boundaries of fields in a fixed-format ASCII record (a difficult task, since there are no explicit delimiters between fields). It uses standard field names, which the user can customize if desired, to enforce consistency among files within the system. A “parser” for U.S. addresses, used to reorganize data from files where information has not been entered consistently, is under development and expected to be available by the end of this year.
The system includes several capabilities designed to help manage a service bureaus or large-volume production operation. It can assign up to five unique numbers to records on a list, to identify initial source, individual names, and other information which will be tracked through later processing. A central list registry keeps tabs on all the lists in the system, and generates an automated history (in English) of each process the list is put through. The registry can also be expanded to track list rental usage, response rates, and other information.
System logs also record which operators perform which tasks, when jobs are run, the indexes available for each file, and the causes of system failures. Additional functions work to detect corrupted indexes and data, and correct them when possible. The files are in xBASE format running the FoxPro database engine, while programs are written in Clipper, a popular xBASE development language noted for speed and power.
LISTman file selections are managed through a point-and-shoot interface with logical, string, date, and other functions. Users can also write Clipper programs for more complex selects. Selection statements cannot be saved, but they can be embedded in a macro that will recreate them on demand. Selections can access multiple files and can include true random or Nth samples. Output can be in xBASE or ASCII formats.
Reports in LISTman are produced by a third-party report writer called Bandit, which has been integrated with the list register and other elements of the system. The system includes basic reports on duplicates and merge/purge outputs, but the user is generally expected to create custom reports as required. Personalized letters, postal sorting, barcoding, Zip+4 coding and other standard processes are also expected to be handled by third-party software packages. Files can be exported to these packages, they can be called from system menus, or they can be linked more tightly through custom programming.
LISTman was originally introduced in Australia, where it has about 24 installations. The system has two major North American customers, in Canada and the Carribean, and was offically released in the U.S. in October, 1994. Support for all clients has been provided from Australia, although the company plans to open a San Francisco office for U.S. customers.
U.S. prices have initially been set at $10,000 for a single user, $15,000 for two users and $20,000 for three or more users, plus 15% for maintenance and support. The software currently runs on DOS-based PC systems, with a Windows version (written with the VisualObjects development tool) due in early 1995. The Windows version will allow client/server operation on databases other than FoxPro.
David M. Raab is a Principal at Raab Associates Inc., a consultancy specializing in marketing technology and analytics. He can be reached at draab@raabassociates.com.
Leave a Reply
You must be logged in to post a comment.