2007 Jan 01
Memetrics Memetrics xOs
by David M. Raab
DM News
January, help 2007
It’s a safe bet that few readers of this column care deeply about the technical differences among multivariate testing methodologies. Taguchi, salve Optimal Design, and Discrete Choice Models each have strengths and weaknesses, but all are ways to quickly and efficiently identify optimal combinations of marketing treatments. They can’t ignore the underlying technology totally, since it can affect important practical issues such as scalability and flexibility to handle unexpected needs. But, over all, users evaluate testing systems like as any other software: by looking at what it would be like to use them, without too much concern for what goes on under the hood. What ultimately matters is the result: better performing promotions.

Memetrics xOs (Memetrics, 415.513.5120, www.memetrics.com) is a multivariate testing system based on discrete choice models. This approach measures consumer preferences by asking them to choose among simulated versions of a complete offer, each having a different value combination. (Think of product design: attributes might be size, price, color, etc.; consumers would be asked to choose among sample products with different values for each.) Discrete choice models have proven more effective at determining the actual impact of each attribute and value than asking about single attributes or values in isolation.

This is all heady stuff and there are Nobel Prizes involved. But for projects such as Web page optimization, the practical result is similar to other multivariate testing methods: each page is divided into one or more zones (attributes), such as message, image, and offer, and these are assigned multiple test values. If all possible value combinations have been tested—something called a “full factorial” design—the system will identify the best-performing combination as optimal and identify any relationships (“interactions”) among values. If only some combinations have been tested, the system ignores any interactions, identifies the best-performing value for each attribute, and proposes the combination of best-performing values as optimal. Testing only some combinations is a typical multivariate approach that yields faster results from smaller, simpler tests. It requires proper test design, which, like other multivariate testing systems, Memetrics does automatically.

Either way, the system also builds a “choice model” that can estimate the results for any combination of values. This can be particularly helpful if the user is interested in multiple outcomes—say, gross revenue, profit margin and number of orders—and wants to balance them against each other rather than maximizing just one. The Enterprise version of xOs lets track several outcomes and either model them separately or combine them into a single measure and model that. Enterprise users can also assign an offer cost and selection value to each outcome and combine these into a target measure. Outcomes can be based information captured during an interaction or imported from external sources such as an order processing system. These features are not available in the simpler Express version of xOS, which tracks only one outcome.

Setting up a test in Memetrics begins with defining the attributes, values, outcomes and proportion of traffic to be tested. Attribute values can be defined with a name, Internet address (URL), or by uploading the actual content to the Memetrics server. A sample size calculator helps users determine the number of attributes and values to test based on traffic volume, time available, conversion rates, expected response variations, and target confidence level.

Once the elements are specified, xOs can generate a block of Javascript that identifies the test and its attributes. The user then embeds the Javascript in the Web page to be tested. The Javascript calls the Memetrics server each time the page is displayed, allowing Memetrics to assign each visitor to a test, control or default group and present the appropriate content. Memetrics uses persistent cookies to identify site visitors so it can ensure consistent treatment when they return.

The Memetrics Javascript can also capture information from the user’s URL, such as the search query that led them to the page. This is stored on the Memetrics server and used to analyze results by visitor segment. Javascript for the same test can be embedded in several pages, allowing consistent treatments and tracking of results such as registration or purchases.

xOs Express is limited to the Javascript approach. Enterprise can also use techniques such as .NET, PHP and JSP to communicate with Web servers or other interaction systems such as call centers. This enables Enterprise to manage tests across multiple channels. Enterprise, but not Express, can also use real-time filters to limit tests to predefined customer segments. These filters can access data provided by the interaction system or read from other sources such as a customer database.

After a test is complete, both systems let users define segments based on whatever visitor data is available. Users can test alternate segmentation approaches to find the best results. xOs can build one model against the entire test universe or build separate models for each segment. Reports for each model show the effect of each value and its statistical significance. Users can accept the system’s choice of optimal values or select their own, and then deploy this combination as a default. Either way, the system will show the expected results for the specified combination.

There is no automated adjustment of default values as customer behavior changes over time. Memetrics argues that humans should examine each test result and make conscious decisions about what to do next. A typical Web page test runs two to four weeks and evaluates five or six attributes, each with multiple values.

The default values will be shown to all visitors outside of a test sample and are also displayed if the Memetrics server is unavailable. The default contents are also viewed by Web search engines.

Memetrics was founded in 1999 and has more than 30 clients. Its original product was Enterprise, which is priced at $150,000 per year plus consulting. It can run in-house or be hosted by Memetrics. Express is a hosted service that was introduced in 2006. Price begins at $40,000 per year and is based on volume.

* * *
David M. Raab is a Principal at Raab Associates Inc., a consultancy specializing in marketing technology and analytics. He can be reached at draab@raabassociates.com.

Leave a Reply

You must be logged in to post a comment.