We use cookies and other similar technologies (Cookies) to enhance your experience and to provide you with relevant content and ads. By using our website, you are agreeing to the use of Cookies. You can change your settings at any time. Cookie Policy.

Best Practices for Mining Retail Transactions

Originally published November 8, 2005

In last month’s article, I talked about how to navigate the many obstacles that arise when attempting to analyze transactional level data. In this third and final article in the series, I will discuss the key business insight that retailers should obtain from their transaction data asset.

Most retailers are driven by a common fear of being pushed out of the market by larger competitors. They subsequently recognize that they are likely to meet this fate unless they can connect with the needs and behaviors of their customers. Clearly, some of the largest industry players have achieved their status through exactly these behaviors. This means understanding the needs of their customers and doing whatever they could to meet those needs.

While customer surveys and focus groups can be helpful, customers truly vote with both their feet and their dollars. The transaction detail in your data warehouse, often called the “T-log,” is a (nearly) perfect record of the decisions your customers make, and how those decisions affect your business. Coupled with other data in your data warehouse, you have much of the information you need to understand the behavior of your customers. You can view the analysis of this behavior from two perspectives: the “customer-centric” and the “visit-centric” perspective.

“Customer-centric” Retailing

The holy grail of retail customer analysis would be to know everyone who visits your stores (or other channel), why they came, whether they found what they wanted, what they chose to buy and how they felt about the whole process. That information is difficult to find, but it constitutes a “customer-centric” analysis of customer behavior. Through loyalty programs and other means of tying the summary information about a transaction to a particular customer, many retailers can determine the following:

  • Number of visits/purchases by each customer
  • Which of your stores or channels they bought through
  • Basket metrics for each transaction, such as item count, sales amount, etc
  • Responsiveness to promotions—did they use a coupon or special promotion code?
  • How demographics (about customers and locations) affects behavior
  • Characteristics that describe profitable (or high-volume) and unprofitable (or low-volume) customers over the lifetime of the relationship

With this information, much of which is contained in the Transaction Summary, Transaction Detail and Transaction Tender components of a retail data warehouse, a retailer can adopt customer-centric retailing practices. These practices drive everything from how promotions are constructed to how stores are stocked, decorated and placed to actions, which can be used to improve relationships with desirable customers.

Best Buy’s store segmentation strategy is a notable example of this approach. The company has created store motifs for two customer segments, “Barry’s” and “Jill’s.” Whereas “Barry’s” are upscale professional men, “Jill’s” are mothers with discretionary income, who generally shop for family gifts. Besides this “segment store” approach, Best Buy has integrated its multiple channels, allowing people who shop bestbuy.com to complete purchases at the store. In addition, the company honors lower prices found on the site at the in-store checkout for customers with documentation. These actions, as well as the overall ability to segment customers and align the business behind “profitable” relationships, are largely driven by data warehousing technology and its wise usage.

“Visit-Centric” Retailing

The customer-centric strategy, of course, does not work for all retailers. In fact, a “visit-centric” approach to customer analysis might make more sense for many retailers. Examples of this are stores who service less discretionary purchases or retailers without loyalty programs, or an ability to tie customers to transactions. In these situations, the retailer’s effectiveness can come down to three simple things: how conveniently the store is located, whether it has products in stock at reasonable prices and how comfortable the shopping and purchasing experience is. Another reason to consider a visit-centric approach is an inability to identify individual customers and match them with transactions in the store.

The analysis of transaction-level detail, and the relentless optimization of operations are essential to the visit-centric strategy. While a store cannot change its location (though the location should be planned carefully), it can certainly be aggressive about ensuring item availability, pricing and the checkout process. These stores should also use promotions wisely, which can influence things like basket contents and repeat visits.

One might wonder how in-stock positions could be improved through examining POS data. Clearly, some queries to the inventory systems, or the inventory portions of the data warehouse, are necessary to become proactive in addressing stockouts. However, out-of-stock situations are often compounded when one of a highly sticky pair of products is not available. This can be a visit killer, where the shopper decides that because a particular item is out of stock he or she must visit another store anyway. Subsequently, the retailer loses more than just the sale of the individual item that is out of stock.

Through product affinity analysis, a retailer can better determine which products sell together during a particular visit. In-stock positions can also be viewed in groups. Thus, if one of a group of related products is out of stock, then all products in the group could be considered out of stock as well, or at least they will be impacted by the absence of the other product. In addition, retailers can use the stickiness of products to each other to plan better promotions and modify store layouts. Affinity analysis is complicated, and only a few business intelligence tools can actually perform it. Getting into detailed affinity analysis requires a tool with data mining and statistical capability, but the analysis should be done directly against transaction detail information.

In terms of pricing, the transaction log (obviously) thoroughly details how individual products are priced and sold at the store level. It also shows how those characteristics change over time. While this is moderately interesting at the chain level, what really influences customer behavior is how item prices compare to other local stores. A best practice is to complement POS data in the warehouse with competitive shopping information.

Many retailers regularly shop their competitors’ stores and obtain pricing for a common basket of items. Not only can the data warehouse tell a retailer which items should be in that competitive basket, it should also be where the competitor pricing data is stored and analyzed. With this approach, a retailer can create a competitive index by store to see how its pricing policies position itself against competition and, naturally, facilitate market-based price changes in response to competition.

Finally, the transaction log can provide a great deal of information about the checkout process. This occurs because the data model typically captures exact times for scanning each item, as well as all of the other transaction’s summary information. The fact that the chronological sequence of a transaction can be captured allows retailers to identify:

  • How long customers waited--length of transaction in seconds
  • How busy the store was – time between end of prior transaction and start of next transaction
  • How cashier errors and mis-keys affected transaction--delays between item scans
  • How tender types (check, cash, credit/debit) influence checkout time

From these and other metrics, retailers can spot training or performance deficiencies in stores or employees, plan optimal labor forecasting for their most costly employee category (cashiers) and adopt better checkout policies regarding tenders. With the introduction of self-checkout in some stores, comparing performance of regular versus self-checkout lanes might suggest changes in the mix of the two. Combined with affinity-driven in-stock and pricing optimization, retailers can best execute on their visit-centric retailing strategy.

In summary, there are clearly many benefits that can be derived from analyzing transaction information in a retail data warehouse. There are a number of technical challenges to address, such as the volume of the information and the difficulty of analyzing it with today’s business intelligence tools. However, the truly best-in-class retailers have been doing it for years. Success in the retail industry depends upon making effective use of this information to enhance your relationships with customers.

  • Dan RossDan Ross
    Dan is the Managing Partner of the Retail Practice at Claraview, a strategy and technology consultancy that helps leading companies and government agencies use business intelligence to achieve competitive advantage and operational excellence. Claraview clients realize measurable results: faster time to decision, improved information quality and greater strategic insight. Dan is a frequent contributor to business intelligence literature, writing on topics spanning technical approaches and business impact, and the Claraview Retail Practice serves some of the world's most advanced users of retail data warehouses.

    Editor's note: More retail articles, resources, news and events are available in the BeyeNETWORK's Retail Channel. Be sure to visit today!

Recent articles by Dan Ross



Want to post a comment? Login or become a member today!

Be the first to comment!