Risk-based Collections: Judgmental Vs Statistical Models

Throughout the years I have worked in credit and collections, either doing credit analysis related to commercial lending decisions or identifying the elements that should be weighed when reviewing trade credit lines. I have seen many different scoring models used for many different purposes including judgmental scoring models that focus on the use of credit bureau data and statistical behavior models that focus on the use of payment history.

Scoring models help credit professionals determine the credit worthiness of prospects, help evaluate existing customer credit worthiness, and even help prioritize collection activities so that the company can get paid faster. While most people agree that the genesis of credit scoring was in the consumer mortgage business, over the years the models have proved to be an extremely valuable tool in the commercial credit world.

Even though it is not the first thing most of us think of when discussing credit scoring, prioritizing collection activity (also known as risk-based collections) is a natural and sensible use of scoring. In my experience, statistical behavior modeling produces the best scores for prioritizing collections. Creating the statistical models to produce credit scores is not simple. However, it is actually easy to use the resulting scores. Risk scores allow you to group customers into risk categories, which can then be used in conjunction with an associated value called “cash at risk” to prioritize collection activities. Collectors can prioritize their activities for those customers with not only the highest probability of delinquency or loss but also with the largest “cash at risk” values. The scores are also predictive, which means they are typically forecasting future behavior – behavior that can be changed with proper collection activity.

A statistical model based on payment behavior is significantly different than the judgmental scoring many of us do when making new customer decisions or evaluating the credit limits of existing customers. Since most of us are more familiar with judgmental scoring versus statistical scoring, I thought I would share five comparisons between the two that lead me to recommend statistical scoring for risk-based collections:

1. Imagine that you are sitting around with five credit professionals and you start talking about the factors that should be weighed when building a judgmental model. If you were even able to come to a consensus, do you think you would all be able to agree on the importance of each factor? Probably not. In fact, you would come up with 3 or 4 different models with different weights assigned to similar factors. That is the nature of judgmental scoring. The model is based on each individual’s experience and judgment. With statistical-based scoring models, once the factors to be included in the model have been determined by various statistical tests, the weights are assigned by statistical software used for model development. This provides you with one best-fitting model.
2. If the judgmental model is not performing well, it is extremely difficult to determine which factor(s) and weight(s) need to be adjusted since there are so many. With statistical-based scoring models, it is a straight forward process to determine which variables are causing the problem in order to adjust the model because the model is relying on existing payment behavior.
3. Building statistical models is actually easier than building judgmental models. With judgmental models, you must answer these types of questions: How many and which variables should I use? What should the variable weights be? What should be my cut-off point between a good risk and a bad risk account, etc.? Typically this is an iterative process. Once the model is built, there is not a way of knowing how effective it is because there is not a standard procedure for evaluating it. Alternatively, a statistical-based model uses existing payment behavior and can be built by companies that offer statistical modeling services in far less time than building and fine tuning a judgmental model.
4. Judgmental models are rarely, if ever, validated. After a judgmental model is prepared for use, the developers typically do not go back in time and say, “If we had this model six months ago how well would it have predicted the next six months?” In statistical-based modeling there is always an element that validates the output before using it – comparing the models’ performance with a previous well-performing portfolio. It is the validation process that tells you how good your model is and helps you determine whether it is adequate for your needs.
5. Most importantly, judgmental scoring models cannot quantify risk. They are essentially ranking models where the company with the highest score is considered the lowest risk. But, the score can’t tell you the probability or odds that a given company will pay its bill within any particular time period. Statistical-based modeling does this as a matter of course. It is this property of statistical-based models that allow them to be the basis for optimum allocation of credit and collection personnel and collection prioritization.

Are you only using judgmental models and are they working? Have you tried incorporating statistical modeling into your risk-based collection strategies? Stay tuned for Part II when I discuss how to manage credit lines. I’d like to hear from you.