We use cookies and other similar technologies (Cookies) to enhance your experience and to provide you with relevant content and ads. By using our website, you are agreeing to the use of Cookies. You can change your settings at any time. Cookie Policy.


Predictive Models: A Marketer's Dilemma

Originally published November 5, 2013

This article explores the dilemma frequently faced by marketers in selecting the most appropriate analytics model to launch effective campaigns. It also delves into best practices that precede as well as succeed the building of alternative predictive models.

It examines how using predictive model outputs affects marketing decisions and how a business user or marketer can read and interpret them. A retail business, either brick-and-mortar or online, will easily fit into the framework of discussion in this article, even though the concepts can also be applied where there is a need for analyzing the customer propensity to buy, rent, lease, transact, etc.

Customer campaigns are one of the top expenses in marketing any product or service. The frequency and share of such spends are some of the most expensive items in the marketing budget. With most traditional brick-and-mortar companies taking their businesses online, more and more consumers rely on the Internet for product research before making a purchase. This trend is prominent in consumer durable goods, apparel, grocery, etc., and in service areas such as beauty, health care and rental.

With advancements in all spheres of technology, customer analytics as a service is increasingly within the reach of mom-and-pop boutiques, huge retail chains and corporations alike.

One of the core analytics techniques that is used in campaign planning and management is predictive analytics. Even though there are multiple ways of predicting customer responses for a given campaign like logistic regression, Bayesian statistics, CHAID, etc., business heads and entrepreneurs face the dilemma of how to best use the results provided by these models and understand how to use them in deploying their future campaigns.

Let us explore some of the dilemmas faced by marketers, not just in interpreting the predictive model outputs, but also the intangible components that are critical for optimizing campaign plans.

Predicting Model Outputs

Almost all traditional prediction models require comprehensive historical data, based on mathematical models that predict customer behavior. To be effective, the historical data used in analysis needs to be at least 2 full series cycles. If the minimum time interval considered in the analysis is a month, at least 24 months of history and preferably 36 months would be required for accurate predictions.

Let us consider a binary option trading predictive model like a “customer’s propensity to rent in the next 30 days.” This model has a yes or no decision to make, for example, in predicting whether or not a customer will return in the next 30 days. For building such models, the data set we require will be of the following form:
  • A unique indicator for identifying each customer
  • For each customer identified, a set of characteristics like the customer demography, past buying pattern, etc.
  • The actual response behavior of the customer. In our case, this would be the customer response – yes or no.
The formulation of the final equation for the predictive model will look as follows:
Response by the customer ~ Independent variables that describe the customer demography
+ Independent variables for past customer buying pattern

Model Selection Dilemma



Table 1: Model Comparison

A marketer responsible for rolling out campaigns must decide between the above models. The process for such a decision is not straightforward and is highly contextual. It is driven by the specific business objectives the company sets.

In model 1, the sensitivity is 38.08%, meaning that if we take 10 customers that the prediction model has identified as return customers, we can be confident that only 4 (rounding off 38.08%) of them are identified correctly and that the remaining 6 customers might not return (or be identified incorrectly as return customers by the model).

Now consider model 2, a sensitivity of 71.90% means out of 10 customers predicted by the model as returning customers, 7 are predicted correctly and 3 are incorrect.

If a campaign is launched targeted at customers who are more likely to return, model 1 is 38% effective while model 2 is 71% accurate. Model 2 might be preferred over model 1 in this context.

Now, let us look at the specificity of the model.

In model 1, a specificity of 94.7% means, that out of 10 customers that the model has predicted as non-returnees, the prediction accuracy of the model is close to 95%, while a 91.59% specificity for model 2 means that model 2 would predict 9 out of 10 non-return customers correctly.

If a campaign is targeted at customers who are unlikely to return, then model 1 with a 94.7% effectiveness might be a better option compared to model 2 with 91.59% effectiveness.

The difference in sensitivity between models 1 and 2 is so wide that we are able to make a binary decision in model selection. But there are other areas to be considered as well.

For instance, to what extent do the explanatory variables used in the model explain the outcome? Does the relationship between the two as explained by the model make any business sense? The evaluation of the contributing variables and causation effect on the outcome is subject to judgment and requires the analyst to possess domain expertise. That brings us to the art part of model evaluation.

The Art Behind Predictive Model Selection

For models to be effective, business objectives need to be spelled out as a precise and specific statement. For example, what business objective is the model expected to achieve for a specific segment of customers?

Common areas that will dictate the model selection process include:
  1. Business objective: What is the pressing business driver that led to the initiation of the model build? Is the objective targeted at bringing back inactive customers, improving business with cross-sell customers or enhancing the service for gold-class customers?

  2. Spend per customer: If a specific campaign has to be rolled out, what will be the marketing dollars to be spent per individual customer? The spend per customer for different campaigns needs to be known or estimated upfront.

  3. Return per customer: For every marketing dollar spent, what is the estimated return per customer? This area becomes contextual as the return will heavily depend on the demography of the customer segment.

  4. Customer base: This represents the segments within the customer base targeted with the intended campaign(s).

  5. Marketing budgets and the preference of allocation of these budgets: Organizations don’t have unlimited marketing budgets for campaigns. When there is a constraint on the overall budget, decisions need to be made about what campaigns take precedence over others.
If the customer base is too large (a million plus), a more accurate model needs to be considered. Even a small variation in accuracy of the model means a large difference in ineffective marketing spends for targeting customers.

If the target customer base is relatively small, the business can maximize the number of customers, provided the cost benefit is substantial, typically a strategy with higher value cross-sell / up-sell customer.

Focused Targeting

Meaningful understanding of customers comes not only with the understanding of their demography, but also with analyzing their past behavior. A wide set of well thought through metrics to measure the past behavioral traits of customers is the key to a successful prediction model.

If short-term monetary return maximization is the primary business objective, given certain budgetary constraints, it will be easy to see that the investment money will go into top x deciles and the bottom (n-x) deciles will be omitted from the campaign.

But most of the time, a company’s marketing objective will cover more than short-term return maximization such as retention of new customers, enhanced customer experience, up sell, cross sell, etc.

The overall objective will take the shape of optimizing the budget allocation across campaigns with both tangible and intangible results rather than maximizing short-term monetary returns.

Most businesses also need to focus on the long-term benefits to the business – for example, in a wristwatch or apparel industry, the brand affinity needs to be built among other groups such as teenagers who might not be currently the top contributing segment to the business. But at some point, they will occupy that spot, and it will be more difficult to build brand loyalty from scratch. Similarly, if a segment is not among the current top contributors, it cannot be immediately written off. It needs to be carefully considered in campaigns for longer term returns.

Conclusion

While statistical models can deliver predictive capabilities to launch marketing campaigns, in isolation these outputs can give companies a tactical edge. The longer term business decisions that are strategic in nature, however, need to factor in other aspects of the business that cannot be delivered by a single model alone. In most cases, a judgment by a domain expert combined with analysis of the dimensions that are counterintuitive to what the model suggests also need to be factored in for the long-term success of the business.

 

Comments

Want to post a comment? Login or become a member today!

Be the first to comment!