Projects / Which customers to call in a bank telemarketing campaign

Which customers to call in a bank telemarketing campaign

Course project, Michigan Ross MBA. Seven classifiers in R, including a stacked model, each with a call cutoff chosen for profit. Not client or employer work.

R scriptModels, cutoffs and the test of the result

Situation	A bank sells term deposits by phone. A sale earns $5 net, and a call with no sale costs $1.
Complication	Only 11% of calls end in a sale, so a campaign that calls all customers loses money.
Question	Which customers must the bank call, and which model finds them best?
Answer	Call the 16% of customers with a sale probability of 17% or more. Profit is twice the profit at the usual 50% cutoff, and a plain logistic regression beats six other models.

Profit is sales less wasted calls; the model ranks the customers and the cutoff decides how many to call

Profit of the campaign$2,610 on the test customers
- Sales: $5 each
  - Model that ranks the customersThe lever: seven models tested
    Does a complex model find more buyers?
    No. A plain logistic regression is ahead of the six other models.
  - Customers who will buyFixed: 11% of all customers
- Calls with no sale: $1 each
  - Call cutoffThe lever: the lowest probability that gets a call
    Which cutoff gives the most profit?
    17%. The break-even probability: 1 sale in 6 calls.
  - Customers called16% of customers at this cutoff
    Do these calls find most of the buyers?
    Yes. 62% of all sales.

Lever what the decision changes Limit what the answer must respect Fixed outside the decision

Profit from the test customers

$2,610

10,298 customers; $1,275 with the usual 50% cutoff

Customers to call

16%

they give 62% of all sales; 42% of the calls end in a sale

Profit if the bank calls everyone

-$3,338

only 11.3% of calls end in a sale

The cutoff matters more than the model: it adds $1,335 to the best model, and the top six models are within $294 of each other

Logisticcall if the probability is 17% or more$2,610$1,275 at the 50% cutoff
calls 16% of customers

Neural networkcall if the probability is 13% or more$2,474$1,262 at the 50% cutoff
calls 18% of customers

Stackedcall if the probability is 7% or more$2,446$1,252 at the 50% cutoff
calls 13% of customers

Decision treecall if the probability is 6% or more$2,415$1,325 at the 50% cutoff
calls 19% of customers

kNNcall if the probability is 13% or more$2,340$698 at the 50% cutoff
calls 18% of customers

Random forestcall if the probability is 26% or more$2,316$1,645 at the 50% cutoff
calls 15% of customers

SVMcall if the probability is 10% or more$1,149$1,149 at the 50% cutoff
calls 4% of customers

A sale earns $5 net and a call with no sale costs $1, so a call pays when the probability of a sale is above 1 in 6 (17%). Each cutoff was chosen on validation data and applied once to the test customers.

The model

Decision

Call or do not call each customer, from data that the bank has before the call.

Objective

The highest profit: $5 for each sale, less $1 for each call with no sale.

Method

Six classifiers and a stacked model. Data split: 50% to train, 25% to choose the cutoffs and fit the stacked model, 25% to test.

The plain logistic regression has the highest profit. In 2,000 resamples of the test customers, it is ahead of each other model more than 99% of the time.
The stacked model does not add profit here. The six models make similar errors, so there is little for a second level to correct.
The cutoff of the logistic regression (17%) is the same as the break-even probability. Its probabilities are well calibrated; those of the SVM are not.
Call duration is not a model input. It is known only after the call.