Making Sense of Scanner Data

When retailers began using checkout scanners, manufacturers assumed that they would be able to use this rich new source of sales data to significantly improve the effectiveness of their marketing and promotion activities. Rather than roll out one-size-fits-all campaigns, they’d be able to tailor programs to key retail accounts—for instance, all Safeway stores in Denver. That would mean much higher returns on marketing investments. But the promise of detailed scanner data has gone unfulfilled. Most manufacturers spend a lot of money to get detailed data on product sales, but they lack the statistical tools necessary to relate the sales numbers to promotional activities at the account level. In other words, they can’t isolate the effects of changes in pricing, features, or displays on sales. Attempts to apply traditional statistical methods, such as least-squares regression, routinely produce erroneous estimates. In one test, a traditional model was used to analyze the impact of putting Kraft sliced cheese on display at 77 accounts. The model indicated that, for many accounts, the displays either depressed sales or increased sales by a factor of 20 or more—obviously impossible results. Such clearly flawed estimates rendered all the model’s outputs unreliable. Most manufacturers spend a lot of money on product sales data but still can’t determine the effects of changes in pricing, features, or displays on sales. One possible solution would be to create individual statistical models for each retail account. But in addition to being prohibitively time-consuming and expensive, such an approach also produces unreliable estimates. Some promotional efforts are simply not used enough at some accounts to generate the depth of data necessary to produce statistically valid results. In addition, external events, such as unusual weather or a competitor’s promotion, may skew the sales figures for a particular promotion. No amount of custom modeling can get around the problem of too little information. But now there’s hope. A new type of statistical technique, called Bayesian shrinkage models, can greatly increase the reliability of estimates of the impact on store sales of promotions. Combining information from many accounts, these models “shrink” regression estimates toward the average, removing those that are implausible. The amount of shrinkage is governed by both the amount of information available at each account and the variability in the estimates across accounts. The more sparse the data or the more consistent the estimates, the more the technique pushes an estimate toward the mean, reducing the possibility that one bad number will undermine an entire analysis. Statistical software packages from companies like SAS and SPSS are beginning to include the ability to run Bayesian shrinkage models. Using the models isn’t easy—it requires a deep understanding of statistical methods—but the software certainly makes the technique practicable for many companies. Not only is it much cheaper than trying to develop customized models, but it also produces far more reliable results. Bayesian shrinkage may finally make it possible for manufacturers to reap the full potential of scanner data.

Making Sense of Scanner Data

Partner Center