Company ⇒ Industry Definitions
Industry Definitions
Validation Analysis
We score your portfolio back in time, anywhere from 6-18 months prior, to test how well our model works. We run your data through our model to see how our predictions compare to your actual results. Validation Analysis is also used to determine if a current model has maintained its predictiveness.Champion Challenger Test
We compare our score against your current score or your collection strategy.Judgmental/Rules-Based Model
The variables and weights in the scorecard are assigned by the person(s) who are developing the model based on the past experience and judgment of this person. Judgmental/Rules-Based Models do not quantify risk and are not usually validated.Statistical-Based Model
Statistical models are automated decision systems that are empirically derived and validated based on credit or recovery performance of accounts in a specific portfolio. Statistical analysis of historical data used in the model development determines which data elements are the most predictive and the model weights the data elements accordingly based on this analysis. Typically, credit and collection scoring models are designed to predict an unacceptable level of payment risk of delinquency, write-off or loss, whereas recovery scoring models are designed to predict the probability of recovering money and identifying who will pay. The output of the model is a probability and is often converted into a score for ease of implementation and interpretation.Some Differences Between Judgmental/Rules-Based and Statistical-Based Systems
1. If you put 10 credit/collection managers in a room and got them to agree on the most important factors for evaluating a company or person’s credit or recovery – not an easy task - and then asked them to assign the weights they think are the best for evaluating credit or recovery, you would come up with 10 different models. The factor weights would be biased based upon each individual’s past experience and judgment. In a statistical-based system, once the factors to be included in the model have been determined by various statistical tests, the weights are assigned by the statistical software used for that purpose. There will be one best fitting model.2. If the judgmental model, for what ever reason, is not performing as well as hoped, it is extremely difficult to determine which factor(s) and weight(s) need to be adjusted. In a statistical-based system, it is a straight forward process to determine which variables are causing the problem and fix the model.
3. Judgmental models are rarely, if ever, validated. After the model is determined the developers do not go back in time and say, “if we had this model six months ago how well would it have predicted the next six months?” In a statistical-based system you always validate your work. It’s the validation process that tells you how good your model is and helps you determine whether it’s adequate for the purpose it has been developed for.
4. Judgmental systems are not easy to build. And, this is probably the main reason that once in place, they are not frequently changed. Now, because of the availability of sophisticated and relatively inexpensive software, we can submit that if the necessary history is available, a statistical-based model can be developed in far less time than a judgmental model that uses a wide range of input variables.
5. Most importantly, judgmental systems can not quantify risk or recovery. They are essentially ranking systems where the company with the highest score is considered the best. However, the score can’t tell you the probability or odds that a given company or person will pay its bill within any particular time period or if in recovery who will pay and how much, which statistical-based systems do as a matter of course.
Days Sales Outstanding (DSO)
A measure of the average number of days that a company takes to collect revenue after a sale has been made. A low DSO number means that it takes a company fewer days to collect its accounts receivable. A high DSO number shows that a company is selling its product to customers on credit and taking longer to collect money.Days Sales Outstanding is calculated as:
| Ending Total Accounts Receivable x Number of Days in Period Analyzed |
| Credit Sales for Period Analyzed |
If the Period Analyzed is a quarter then Number of Days = 91, if the Period is a year then Number of days = 365.
Statistical Measures
- Development Versus Holdout Sample
Does the model work on a sample different than the one used to develop the model? Prefer Forward / Out of Time Sample. - KS Statistic: Kolmogorov - Smirnov
What is the maximum separation between the cumulative distribution of GOOD’s & BAD’s by Score? Historical Industry Standard - Predictive Index: “Power Stat”
How close is the estimated model to the “prefect” model? Replacing the KS as the Industry Standard. - Predicted Probability Versus Actual Probability
Specific to Logistic Probability based model. Applicable to calculating Expected Value ($) of BAD. - Divergence
Similar to KS. Measures mean score of GOODs versus BADs. Secondary statistic not widely used.
There are two key differences in our method that distinguish it from the standard industry practice:
(1) We look for behavior differences in the drivers of choice, not behavior differences in choice. Understanding behavior differences driven by the same choice drivers allows us to keep to have a better understanding of choice before the choice is made. Standard industry methods are designed to identify different groups AFTER the choice has been made.
(2) We look for behavior differences in the drivers of choice at the individual choice driver level and holistically, simultaneously accounting for the choice drivers as a group in their total influence on choice.
Difference in response to an individual choice driver may be significant in a statistical sense but it is more likely to be significant in a business sense if the differences are sustained across several choice drivers taken together.
The two key differences we provide in our segmentation analysis are a source of competitive advantage in your predictive choice modeling tools.
BACK TO TOP
