Optimal Binning using MDLP with Monotonicity Constraints

Performs supervised discretization of continuous numerical variables using the Minimum Description Length Principle (MDLP) algorithm, enhanced with optional monotonicity constraints on the Weight of Evidence (WoE). This method is particularly suitable for creating interpretable bins for logistic regression models in domains like credit scoring.

Usage

ob_numerical_fast_mdlp(
  feature,
  target,
  min_bins = 2L,
  max_bins = 5L,
  bin_cutoff = 0.05,
  max_n_prebins = 100L,
  convergence_threshold = 1e-06,
  max_iterations = 1000L,
  force_monotonicity = TRUE
)

Arguments

feature: A numeric vector representing the continuous predictor variable. Rows whose value is missing (NA/NaN) are excluded from the fit silently, so the bin counts sum to the number of non-missing rows; -Inf and +Inf are kept as extreme values of the first and last bin and never become a cutpoint.
target: An integer vector of binary outcomes (0/1) corresponding to each observation in feature. Must have the same length as feature. A missing value in target is an error.
min_bins: Integer. The minimum number of bins to produce. Must be $\ge$ 2. Defaults to 2.
max_bins: Integer. The maximum number of bins to produce. Must be $\ge$ min_bins. Defaults to 5.
bin_cutoff: Numeric. Currently unused in this implementation (reserved for future versions). Defaults to 0.05.
max_n_prebins: Integer. Currently unused in this implementation (reserved for future versions). Defaults to 100.
convergence_threshold: Numeric. The threshold for determining convergence during the iterative monotonicity enforcement process. Defaults to 1e-6.
max_iterations: Integer. Safety limit for the maximum number of iterations in the monotonicity enforcement phase. Defaults to 1000.
force_monotonicity: Logical. If TRUE, the algorithm enforces a strict monotonic relationship (increasing or decreasing) between the bin indices and their Weight of Evidence (WoE) values. Defaults to TRUE.

Value

A list containing the binning results:

id: Integer vector of bin identifiers.
bin: Character vector of bin labels in interval notation.
woe: Numeric vector of Weight of Evidence for each bin.
iv: Numeric vector of Information Value contribution per bin.
count: Integer vector of total observations per bin.
count_pos: Integer vector of positive cases.
count_neg: Integer vector of negative cases.
cutpoints: Numeric vector of upper boundaries (excluding Inf).
converged: Logical indicating if the monotonicity enforcement converged.
iterations: Integer count of iterations in monotonicity phase.

Details

This function implements a sophisticated hybrid approach combining the classic MDLP algorithm with modern monotonicity constraints.

Algorithm Pipeline:

Data Preparation: Removes NA values and sorts the data by feature value.
MDLP Discretization (Fayyad & Irani, 1993):
- Recursively evaluates the binary splits of the sorted data. Only boundary points (cuts that do not fall between two values holding observations of a single, common class) are candidates: the entropy-minimising cut is always one of them (Fayyad & Irani, 1992).
- For each candidate split, calculates the Information Gain (IG).
- Applies the MDLP stopping criterion to the best split $T$ of a set $S$ into $S_1$ and $S_2$: $$IG > \frac{\log_2(N-1) + \Delta}{N}$$ where $N$ is the number of samples in $S$ and $\Delta = \log_2(3^k - 2) - [k E(S) - k_1 E(S_1) - k_2 E(S_2)]$, with $k$, $k_1$, $k_2$ the number of classes present in $S$, $S_1$ and $S_2$.
- Only accepts splits that significantly reduce entropy beyond what would be expected by chance, balancing model fit with complexity.
Constraint Enforcement:
- Max Bins: Accepted splits are applied best-first, in decreasing order of the entropy reduction they achieve, and at most max_bins - 1 of them are kept. When max_bins is not binding this yields exactly the MDLP partition.
- Min Bins: When MDLP accepts fewer splits, additional ones are placed between distinct values, spread evenly over the distinct values and then over the largest runs of observations.
- Monotonicity (if enabled): Iteratively merges adjacent bins with the most similar WoE values until a strictly increasing or decreasing trend is achieved across all bins, or only min_bins bins remain.

Technical Notes:

The algorithm uses Laplace smoothing ($\alpha = 0.5$) when calculating WoE to prevent $\log(0)$ errors for bins with pure class distributions.
When all feature values are identical a single bin is returned, with a warning. When the feature has fewer distinct values than min_bins, each distinct value becomes its own bin and a warning reports that min_bins could not be met: a bin boundary can only fall between two different values.
The monotonicity enforcement phase is iterative and uses the convergence_threshold to determine when changes in WoE become negligible.

References

Fayyad, U. M., & Irani, K. B. (1993). Multi-interval discretization of continuous-valued attributes for classification learning. Proceedings of the 13th International Joint Conference on Artificial Intelligence, 1022-1029.

Fayyad, U. M., & Irani, K. B. (1992). On the handling of continuous-valued attributes in decision tree generation. Machine Learning, 8, 87-102.

Kurgan, L. A., & Musilek, P. (2006). A survey of techniques. IEEE Transactions on Knowledge and Data Engineering, 18(5), 673-689.

Garcia, S., Luengo, J., & Herrera, F. (2013). Data preprocessing in data mining. Springer Science & Business Media.

Examples

# Example: Standard usage with monotonicity
set.seed(123)
feature <- rnorm(1000)
target <- rbinom(1000, 1, plogis(2 * feature)) # Positive relationship

result <- ob_numerical_fast_mdlp(feature, target,
  min_bins = 3,
  max_bins = 6,
  force_monotonicity = TRUE
)

print(result$bin)
#> [1] "(-Inf;-0.506334]"     "(-0.506334;0.299594]" "(0.299594;1.214589]" 
#> [4] "(1.214589;+Inf]"     
print(result$woe) # Should show a monotonic trend
#> [1] -2.1085564 -0.2907929  1.3818394  3.8182297

# Example: Disabling monotonicity for exploratory analysis
result_no_mono <- ob_numerical_fast_mdlp(feature, target,
  min_bins = 3,
  max_bins = 6,
  force_monotonicity = FALSE
)

print(result_no_mono$woe) # May show non-monotonic patterns
#> [1] -2.1085564 -0.2907929  1.3818394  3.8182297