Where to set the approval cutoff for a consumer lender: technical document

Baseline: what did the booked loans give to the lender?

The data gives the total payments, the date of the last payment and the recoveries of each loan. It does not give each monthly payment. The code makes the monthly cash flows with these rules (as-06):

• The borrower pays the installment each month until the month of the last payment.
• The borrower pays the rest of the total payments in the month of the last payment. For a loan that the borrower pays early, this is the prepayment.
• The recoveries, minus the collection fee, come 12 months after the last payment.

The NPV uses three costs. The cost of funds is 3% for each year (as-01). The servicing cost is 1% of each payment (as-02). The acquisition cost is $300 for each loan (as-03).

The table gives the baseline by issue year: what the booked loans gave to the lender. It contains only booked loans, so it shows the result of the champion policy of Lending Club.

How to read: each bar is the realized NPV of all booked loans of one issue year, after all costs (as-01 to as-03). The labels under the bars give the default rate and the mean interest rate.

So what: the 2015 loans lost money. The default rate went up from 12.3% to 14.9%, and the mean interest rate went down from 13.4% to 11.3%. This is the complication of Step 0 in dollars.

Branch 4: business value

The question: at the same risk, does the challenger approve more applicants and add NPV? Two tests answer it.

• Approval at the champion bad rate. How many applicants can the challenger approve, at the bad rate of the champion?
• Swap-set. At the same number of approvals, which loans does the challenger change, and are the new loans better?

At the champion bad rate, the challenger approves 85.7% of applicants, against 84.5% for the champion (+1.2 points).

How the swap-set works. The test gives both policies the same number of approvals: the champion approval share of 84.5%. Each 2015 applicant then falls into one of four groups:

Both policies approve the same number of applicants. Thus the swap-in group and the swap-out group have the same size. Only these two groups change the result.

The champion declines the sub-grades D1 and worse. Lending Club booked all these loans, so we know the real outcome of each swap-in loan. This is a form of reject inference with real outcomes. The limit: the data has no applicant that Lending Club declined. Step 7 examines this limit.

The two policies do not agree on 24,426 loans in each group (8.6% of the 2015 loans). The swap-in loans have a default rate of 23.6%. The swap-out loans have a default rate of 25.1%. With the same number of approvals, the challenger adds $8.3M of realized NPV.

The swap-in loans also have a higher interest rate (17.1% against 12.7%). The challenger finds applicants in lower grades with less risk than their grade shows. Lending Club gave these applicants a high interest rate.

The champion is an assumption (as-04). The table below repeats the approval test for three champion approval rates: 80%, 85% and 90%. For each one, the challenger must reach the bad rate of that champion.

The smallest gain of the three champions is +0.7 points. Thus the result does not depend on the exact champion assumption.

Rollout plan

The back-test on the 2015 loans is not proof for the future. The economy, the applicants and the prices change. A controlled test gives the proof before a full rollout. The plan has five parts.

1. Test design. Send a random 10% of applications to the challenger cutoff (rule A). The other 90% stay with the champion. Random allocation makes the two arms equal in all other ways, so the difference comes from the policy. Also approve a small random sample below both cutoffs, as the decision above says.

2. Main KPI. The approval rate of each arm, and its default rate at month 12. The 12-month default rate gives an early estimate of the lifetime default rate (Part 2: Loss forecast). The final KPI is the realized NPV of each arm at month 36.

3. Guardrails. Check these values each month:

• The PSI of the score, against the 2015 applicants. A value below 0.1 is stable.
• The mean PD of each monthly vintage against its actual default rate.
• The approval rate of each arm, against the plan.

4. Kill criteria. Stop the test in either of two cases:

• The 12-month default rate of the challenger arm is more than 1 point above the champion arm.
• The PSI of the score is more than 0.25.

5. Top risks.

Decision at the end. If the challenger arm keeps the bad rate and adds NPV, roll out the challenger. Use the result of the pricing test to set the final cutoff.

What a reviewer should challenge

These limits can change the result most. Each one has a control.

Other limits have a small effect. The dates of the cash flows are estimates (as-06), but the totals are actual values. The recoveries of young vintages are not complete. The Excel model uses the mean loan of each policy, so its NPV differs a little from the notebook.

Next steps

1. Business case deck (6–8 slides): the challenger cutoff with the pricing test as one recommendation.
2. Model risk review of the scorecard: the table of points, the reason code text and the GBM benchmark.
3. Design of the pricing test for the grades with a negative NPV, with the method of case 03.
4. Start the rollout plan of Step 7. It has a champion/challenger test on 10% of applications and a test below the cutoff. It also checks the PSI and the vintage curves each month.