Applying Uplift Modeling to Increase Customer Satisfaction and Streamline Marketing Efforts

Summary

The Marketing Data Science team focuses on relationships with Mercari’s customers (including potential customers). Our goal is to create a better user experience and improve our business.

The Marketing Data Science team designs wide variety of campaigns to increase customer engagement. Different customers have varied levels of interest in different campaigns. The main task of the Marketing Data Science team is figuring out how to inform each customer of the right campaign to increase customer satisfaction and further streamline marketing efforts.

Purpose

Here, we will introduce Uplift Modeling, which is used to predict how customers will respond to a campaign, both when they do and do not decide to participate. The differences between the scenarios are called “Uplift scores”. A user with a high Uplift score is likely to be interested in the campaign, and is therefore sent a notification. Conversely, a user with a low Uplift score is likely to have a low reaction or even a negative reaction upon being notified, and thus no notification is sent in order to avoid inconveniencing the user. Matching customers with appropriate campaigns enables us to ensure a superior user experience along with efficient campaign budgeting.

Description

Marketers generally divide their customers into four categories according to the impact of marketing communications (campaigns/promotions) [1].

• “Do Not Disturb (DND)” customers have a strong negative reaction to marketing communications. Such customers buy if no action is taken, but do not buy if action is taken. Approaching such customers not only wastes marketing resources, but also gives the customers a negative impression.
• “Lost cause” users will not purchase products whether they are contacted or not. Marketing is ineffective in such cases, and marketing resources are therefore wasted.
• “Organic” users buy no matter what, whether they are contacted or not. Campaigns have no effect and there is thus no need to use the budget.
• “Persuadable” users always respond positively to marketing communication. Such users buy only when contacted (or buy more and earlier).

This causal relationship is called “Uplift” and is predicted as the difference between two probabilities. Here, 𝑌 is a flag determining the target action, and 𝑊 is a flag indicating communication.

Uplift values can be calculated for each category using these flags.

Mercari uses a variety of types of marketing communication. We therefore aim to focus only on persuadables in each campaign. The Uplift score of a persuadable differs from people in the other three categories, so we predict the Uplift score of the user to distinguish whether they are persuadable or not. “Uplift Modeling” is a predictive modeling method that aims to describe and estimate the causal effects of a given measure at the individual level [2]. It is used in the digital advertising industry to target marketing efforts to users most efficiently (persuadables) [3].

Estimating Uplift scores is both a causal inference problem and a machine learning problem. It is a causal inference problem because the difference between two outcomes that are mutually exclusive for a given individual must be estimated. The challenge for Uplift Modeling is to answer questions regarding opposing facts, namely whether or not the customer should receive campaign notifications. Uplift Modeling is also a matter of machine learning, because different models must be trained and the model that yields the most reliable prediction according to multiple conditions must be selected. Doing so requires a correct cross-validation strategy in addition to feature engineering [4].

The Marketing Data Science team is involved in both research and development of Uplift Modeling. The team is also experimenting with new types of Uplift Modeling to further improve performance. The model is also being applied to each of Mercari's services to validate performance in real-world scenarios and improve user experience and the business impact of campaigns.

Reference

[1] the basic tutorials of scikit-uplift
[2] Diemert, Eustache, et al. "A large scale benchmark for uplift modeling." KDD. 2018.
[3] Radcliffe, Nicholas. "Using control groups to target on predicted lift: Building and assessing uplift model." Direct Marketing Analytics Journal (2007): 14-21.
[4] Gutierrez, Pierre, and Jean-Yves Gérardy. "Causal inference and uplift modelling: A review of the literature." International Conference on Predictive Applications and APIs. PMLR, 2017.