Skip to contents

Changelog

Source: NEWS.md

OptimalBinningWoE 1.13.6

Two scorecard reporting defects fixed at the origin (2026-09-03)

Both were long known and patched downstream by fisiv; they now live where the defect is.

  • .ob_score_metrics() ignored the scaling direction. The AUC was always mirrored (1 - AUC_event), which is right for a higher_is_safer score and reports the complement for higher_is_riskier – AUC 0.05 next to a KS of 0.78 in the 01_Model_Summary sheet. obwoe_scorecard() now passes scaling$direction and the mirror applies only when the score is high-for-safe. Gini follows. KS was never affected.

  • 07_Score_Gains$mean_score was hard-coded NA. obwoe_report() now fills it from the frozen training bands (new band_breaks element of the obwoe_scorecard object; recomputed from the training score for objects saved by earlier versions). Rows are matched by band label, so a label that does not reproduce yields NA rather than a shifted number.

OptimalBinningWoE 1.13.5

The SQL test evaluator now reads literals the way a database does (2026-08-31)

Single-defect release, and the defect is in the test suite rather than in the package. The CRAN checks for 1.13.4 were OK on all thirteen regular flavours; the additional noLD check – x86_64 Linux with R-devel configured --disable-long-double – reported two test failures.

  • tests/testthat/test-obwoe-sql.R failed on noLD at the boundary check “boundaries hold for cut points that need full double precision”: the observation sitting exactly on the first cut point was reported in bin 2 rather than bin 1, twice.

    The generated SQL was correct. obwoe_sql() wrote the cut point -1.3003598122031599 with all seventeen significant digits it needs, and any conforming reader – every database this SQL targets included – parses that string back to exactly the fitted double.

    The fault was in the miniature CASE evaluator the tests use to score the generated SQL. It read each literal back with as.numeric(), and as.numeric() is not a conforming reader on a build without long double. R accumulates the decimal digits of a string in LDOUBLE; where that is no wider than a double, a literal carrying more than fifteen digits can come back a few ULP from the double it names. There -1.3003598122031599 was read one ULP low, so the observation equal to the cut point failed x <= -1.3003598122031599 and fell through to the next branch. The evaluator, not the SQL, put it in the wrong bin.

    The evaluator now decodes a literal without trusting as.numeric() with anything it cannot be shown to parse exactly. Digits below 2^53 with at most 22 decimals are read as m / 10^nd, one correctly rounded division of two exactly representable doubles. Otherwise sprintf() – which hands the rendering to the C library, correctly rounded on every platform – confirms as.numeric()’s answer by printing it back, and where it does not print back, the neighbouring doubles are bracketed around the literal by exact decimal comparison and the nearer of the two wins.

    This also removes a smaller inaccuracy in the other direction: where LDOUBLE is 80 bits, as on x86_64, as.numeric() can double-round a sixteen-digit literal and disagree with the database over a literal the package considers valid.

  • A regression test was added for the evaluator’s reader itself, over 3,600 values spanning 1e-9 to 1e9: the literal obwoe_sql() writes must decode back to the identical double. The boundary tests are only as trustworthy as that reader, and on noLD this is what fails first if it regresses.

No package code changed, so no generated SQL, no fitted binning and no API changed with this release.

OptimalBinningWoE 1.13.4

CRAN release: 2026-08-26

SQL literals on platforms without an extended long double (2026-08-25)

Single-defect release. The CRAN checks for 1.13.3 failed on r-release-macos-arm64 and r-oldrel-macos-arm64 only; every other flavour, macOS on x86_64 included, was clean.

  • obwoe_sql() could write a cut point one bit away from the fitted value on Apple silicon, moving any observation that sits exactly on that boundary into the next bin. The generated SQL was internally consistent and gave no warning; it simply scored those rows with the wrong bin’s Weight of Evidence.

    The literal writer searched for the shortest decimal string that parses back to the identical double, checking each candidate with as.numeric() before emitting it. That check is not sound on aarch64. R accumulates the decimal digits of a string in LDOUBLE, which on aarch64 macOS is no wider than a double, so beyond about fifteen digits as.numeric() can land one bit from the nearest double – rejecting a literal that does round trip and accepting one that does not. With every candidate rejected, the search fell through to a fallback that wrote fewer digits than the value needs: the cut point -0.13964785691628961 came out as -0.13964785691629, which is the smaller number, so an observation equal to the cut point failed x <= -0.13964785691629 and fell one bin up.

    The search now decides for itself. A candidate with nd decimals is checked as m / 10^nd, where m is its digits read as an integer: while m is below 2^53 and nd is at most 22, both operands are exact, so IEEE 754 gives the correctly rounded quotient – the nearest double to the candidate – on every platform R runs on. Candidates outside those bounds are not judged at all; the value falls back to seventeen significant digits, the width that identifies a double uniquely. as.numeric() still has to agree before a candidate is accepted, not to decide the question but so that a literal R itself reads back as a different double is never written into an audit artifact.

    Cut points that are exact in binary – integers, halves, quarters – still read short. About five per cent of values now carry one more digit than they did in 1.13.3, being those the new check declines to judge.

  • digits now means decimal places, as documented. It rounded to the requested number of places and then formatted the result with R’s default seven significant digits, so digits = 8 on 1234.5678901234 emitted 1234.568 rather than 1234.56789012. Values needing at most seven significant digits – the common case, and the one the test suite covered – were unaffected.

macos-latest is back in the GitHub Actions check matrix. It had been disabled while infer had no ARM64 binary; that binary is back, and the architecture is the one this release repairs.

Three regression tests were added to tests/testthat/test-obwoe-sql.R: a bulk round trip over 3,500 values, a boundary check on cut points taken from continuous data, and a diagnostic that spells out a one-bit disagreement between the SQL and obwoe_apply(), which waldo reports only as “don’t know how to show the difference”. The existing boundary test used integer-valued cut points, which any literal writer renders exactly, and so could not catch this.

OptimalBinningWoE 1.13.3

CRAN release: 2026-08-23

Audit fixes (2026-08-21)

Bug-fix release from a third internal audit, covering the WoE/IV return contract, the converged flag and the max_bins constraint across all 37 algorithm/type combinations. Every item below was reproduced before the fix and re-verified after it. The regression suite gained tests/testthat/test-audit-regressions.R, which fails on 1.13.2.

Behavior changes (read before upgrading)

  • mdlp (numerical), gmb and fetb (categorical) now honour max_bins. All three stopped on their own criterion and never re-checked the cap, so max_bins = 5 returned 18, 11 and 10 bins respectively – silently, with no warning. All three roxygen blocks already documented max_bins as a hard constraint, so the code was wrong, not the documentation.

    These three algorithms now produce different bins. For mdlp the returned partition is no longer the unconstrained MDL optimum when the cap binds: merging continues past the MDL stopping point, each step taking the pair with the smallest increase in MDL cost. min_bins is never violated to satisfy max_bins.

  • obwoe_apply() now refuses multiclass models instead of scoring them wrongly. A multinomial fit carries a bins x classes WoE matrix; the lookup linear-indexed it column-major and returned class 1’s WoE for every row, discarding the other classes with no error and no warning. It now stops with an actionable message. The per-class matrix is still in $results for callers that want to handle it themselves.

  • dmiv now reports a numeric total_iv where summary$total_iv was previously NA for every feature. Code that tested for that NA will see a number.

Corrected values

  • The categorical sketch engine computed every WoE against the wrong marginal. It passed (total_neg, total_pos) to helpers whose signature is (total_pos, total_neg). On a random 8-category feature with no real signal, the reported IV was 10.4021 against a true value of 0.0043 – wrong by three orders of magnitude, and wrong in the direction that makes a useless variable look like the strongest predictor in the model. WoE deviated from log((pos_i/TP)/(neg_i/TN)) by up to 2.7721; it is now within 0.0002.

    The identical defect was fixed in the numerical twin some releases ago and never replicated here. A numerical-vs-categorical parity test now covers every algorithm that has both variants, so this class of divergence cannot recur silently.

  • dmiv returns iv and total_iv alongside the divergence measures it already reported. IV is computed from the smoothed class distributions rather than from woe, because the default bin_method = "woe1" is Zeng’s log-odds ln((pos + 0.5)/(neg + 0.5)), which differs from standard WoE by the constant ln(TP/TN); deriving IV from it would be wrong.

  • cm (categorical) exposes total_iv at top level, like the other 15 categorical engines, instead of only inside metadata. metadata is unchanged, so existing callers keep working.

Fixed crashes

  • sab (categorical) no longer aborts with unordered_map::at when a category contains zero events – a routine situation in credit data. The positive-count map only gained a key for categories with at least one event, but was read with .at() in seven places.

  • obwoe_scorecard() and the cutoff table now reject a missing target with a clear message naming the sample and the count, instead of failing deep inside with missing value where TRUE/FALSE needed. The development frame was already checked; samples passed through validation = were not. The NA is rejected at the boundary rather than swept up with na.rm = TRUE, which would have silently charged every unknown-outcome row to the non-events and corrupted KS and AUC.

converged now means the same thing everywhere

The flag was effectively inverted in several engines: initialised false and set true only on a degenerate shortcut or an in-loop tolerance test, while the normal successful exit – reaching the bin-count target – set nothing. So ordinary well-binned features reported FALSE and only degenerate ones reported TRUE.

  • dmiv and bb (numerical) report converged on reaching the bin-count target, matching the categorical dmiv, which already did.
  • ir (numerical) reports converged for the exact binning it produces when a feature has two or fewer distinct values. Every binary 0/1 feature previously reported FALSE despite a correct result.
  • dp, gmb, mba, sketch, cm and dmiv (categorical) report converged on every successful termination path, including the fast paths that bypassed the flag entirely.

The intended contract is now documented in src/common/bin_structures.h: converged == true means the algorithm reached a valid stopping state (tolerance met, monotonicity achieved, or the bin-count target reached); false means it exhausted max_iterations.

Three algorithms were quadratic in the number of rows

lpdb, ldb and numerical udt scaled as n^2.00, n^2.00 and n^2.30. A single variable with 10^6 rows would have taken lpdb roughly 72 minutes; measured against jedi at the same size they were 2,772x the median algorithm’s cost. Nothing warned about it.

Two distinct causes, neither of them inherent to the methods:

  • ldb and lpdb estimated the density with a naive double loop, evaluating the Gaussian kernel of every observation against every other one. The same defect had been written twice, in two files, which is how it survived. It is replaced by the standard linear-binning estimator – the one R’s own density() uses – which now lives once in src/common/optimal_binning_common.h so the two cannot diverge again.

  • udt rescanned every observation once per candidate split, allocating two vectors and recomputing the parent entropy each time, giving O(u x n). Information gain depends only on integer counts, so a single sweep carrying running totals produces the identical value.

Measured at n = 50,000, against a build of the previous revision:

algorithm before after speedup
ldb 10.740s 0.010s 1074x
lpdb 10.725s 0.011s 975x
udt 7.354s 0.019s 387x

All three now scale linearly and land within a factor of two of jedi, the package default. At n = 400,000 they take 0.085s, 0.087s and 0.170s against jedi’s 0.103s – sizes the previous code could not reach at all.

Results. udt and ldb are bit-identical to the previous revision: udt by construction, and ldb’s local-minimum search resolves the grid estimate to the same cut points. Both are pinned by a new regression test.

lpdb changes. It differentiates the density twice to find inflection points, and finite differences taken between adjacent observations are not the same thing as finite differences on a properly sampled curve. Its critical points are now located on the estimation grid. On German Credit the partitions generally improve – duration goes from 2 bins and IV 0.0923 to 5 bins and IV 0.2635, age from 2 bins and 0.0628 to 4 bins and 0.0781 – and no variable tested got materially worse. Anyone with a fitted lpdb model should expect different cut points.

Also removed OBN_LPDB::local_polynomial_density(), which no longer had a caller and never did local polynomial regression despite its name.

Fixed: categorical dmiv ignored max_bins

  • ob_categorical_dmiv() returned L - 1 bins for an L-category feature, whatever max_bins was set to. The merge loop compared the cost of the best available merge against convergence_threshold and broke out when it had barely moved. With many similarly sized categories the second-cheapest merge costs exactly what the cheapest one did, so the test fired on the second iteration, after a single merge, and nothing re-imposed the cap afterwards. Reported as converged = TRUE, with no warning.

    Reproduced at n = 60,000 with max_bins of 3, 5 and 8 alike: 40 levels returned 31 bins, 60 returned 50, 120 returned 119 and 300 returned 299. All nine divergence_method choices and both bin_method choices were affected. The roxygen already documented max_bins as a hard constraint, so the code was wrong, not the documentation.

    The loop now records the convergence and keeps merging by the same criterion — the pair with the lowest divergence, i.e. the most similar pair — until the cap is met (src/OBC_DMIV_v5.cpp). This is the remedy already applied to fetb, whose loop had the same shape. The merge ordering is unchanged, and min_bins is never violated to satisfy max_bins.

    dmiv produces different bins wherever the cap used to be abandoned; on inputs where it was reached anyway, results are byte-identical to the previous release — verified across all 16 categorical engines and 13 variables of the bundled German Credit data, 208 combinations, with the RNG stream fixed so the stochastic engines are comparable. The numerical dmiv never had the defect and is untouched.

    A new regression test asserts that every categorical engine honours max_bins on a high-cardinality feature, at two values of bin_cutoff and two of max_bins.

  • ?ob_categorical_dmiv described convergence_threshold as stopping the merging. It records convergence; max_bins is a hard constraint and merging continues until the bin count meets it.

  • find_most_similar_bins() now seeds its best pair with the first mergeable one instead of {0, 0}. The search only replaces that seed on a strictly smaller divergence, so a distance matrix that was entirely double::max – or that held a NaN, against which every comparison is false – would have returned a pair naming the same bin twice, and merging a bin with itself then erasing the duplicate would have dropped its observations. No input reaching that state was found, so this closes a defensive gap rather than a demonstrated defect.

Fixed: categorical mba read past the end of its bin vector

  • ob_categorical_mba() performed an out-of-range read while reducing the pre-bins. The list of candidate bins to merge is built before any merging starts, and every merge erases a bin, so an index taken later in that list could point past the end of the shrunken vector. The existing guard only clamped the indices passed to the merge itself, not the read that selects a merge partner.

    This is undefined behaviour, not a wrong number: under a checked standard library it aborts the R session, and without one it reads foreign memory and carries on. Reproduced with 60 categories, max_n_prebins = 20 and n = 60,000, at either the default bin_cutoff or a smaller one.

    Stale indices are now skipped. The guard fires only where the previous code was already out of range, so results are byte-identical on every input that worked before — verified across all 16 categorical engines and 13 variables of the bundled German Credit data. mba rejoins the regression test that asserts every categorical engine’s bins account for every observation.

Fixed: categorical ivb and gmb dropped observations

  • ob_categorical_ivb() and ob_categorical_gmb() discarded the categories that did not fit within max_n_prebins instead of pooling them, so those observations left the binning entirely. The vector of bins was resized, and everything past the cap was destroyed along with its counts. Nothing signalled it: no warning, error = FALSE, converged = TRUE, and a total_iv reported as if it described the whole sample.

    Default settings hide the defect — bin_cutoff = 0.05 lets at most 20 categories survive the rare-category merge, which is exactly the default max_n_prebins — but a smaller cutoff reaches it. With 60 levels and bin_cutoff = 0.005, both engines lost 39,306 of 60,000 rows (65.5%); dp and jedi accounted for every row on the same input.

    Both now fold the excess into the smallest retained bin (src/OBC_IVB_v5.cpp, src/OBC_GMB_v5.cpp). Which categories are kept as separate identities is unchanged — still the max_n_prebins most frequent — so results on the path where the cap never bound are byte-identical to the previous release: verified across all 16 categorical engines and 13 variables of the bundled German Credit data, 208 combinations, with the RNG stream fixed so the stochastic engines are comparable.

    A new regression test asserts that every categorical engine’s bins account for every observation.

Documentation

  • New vignette, Algorithm Reference: the 37 Binning Engines. Reference documentation for all 28 algorithms across their 37 algorithm/feature-type combinations, written from a reading of the C++ implementation of every engine and a check of that reading against the literature each one invokes. Entries are organised by the mechanism the code actually uses — exact dynamic programming, recursive entropy partitioning, statistical merging, isotonic regression, density estimation, divergence, streaming, metaheuristic, and greedy IV merging — which groups the engines differently from their names.

    Two rules governed it: a claim appears only if it traces to a specific line of the shipped source or to a publication verified to exist, and the name is never taken as evidence of the mechanism. Where neither could be established, the vignette says so — it closes with a section listing what could not be verified, including the provenance of the Information Value interpretation bands and whether Kerber’s ChiMerge uses the continuity correction this package implements.

    Substantive findings it documents: obwoe() does not forward algorithm-specific parameters and drops them silently, demonstrated live; mdlp does not implement the Fayyad–Irani criterion it cites while fast_mdlp does; fetb computes a hypergeometric point probability, not a Fisher exact-test p-value; bb neither branches nor bounds and numerical dp builds no dynamic-programming table, while the package’s only real DP sits inside sketch behind an n <= 50 threshold; the numerical sketch departs from the KLL construction it cites in both compactor capacity and compaction rule, so the quoted error bound does not transfer; and dmiv’s default bin_method = "woe1" is a per-bin log-odds rather than standard WoE.

    It also records parameters that are accepted and never read (polynomial_degree in lpdb; max_n_prebins in fast_mdlp, numerical sketch, categorical fetb and sab), convergence_threshold being inert in five numerical engines, and three citation errors in the shipped documentation.

  • max_n_prebins is documented as the modelling decision it is. For numerical features, pre-binning runs before any algorithm sees the data, so the default of 20 quantile cells can smear a heavy tail and lose the signal in it before optimization begins – silently, with converged = TRUE. Benchmarks on two open datasets (76,020 x 369 and 590,540 x 454, five-fold held-out IV) move the median held-out IV of heavy-tailed numerical predictors by +129% and +39% when the parameter is raised from 20 to 200, with the bin count essentially unchanged.

    The default is deliberately not changed. The same experiment shows the effect runs both ways: on the larger benchmark, raising it cost 15% of held-out IV on the twenty-five strongest predictors while inflating IV on weak ones, and no cheap rule separated the two cases. ?control.obwoe now says so, and recommends per-variable tuning against held-out data.

  • min_bins is documented as frequently binding. It reads as a safety floor but often sets the partition, because several algorithms stop merging on their own criterion well before max_bins. On German Credit amount with max_bins = 5, the default returns two bins and an IV of 0.0016 while min_bins = 4 returns 0.0824 – same data, same algorithm.

  • Corrected the 1.13.0 entry on binary size. It described -Os, -fvisibility=hidden, -ffunction-sections, -Wl,--gc-sections and a cleanup script, none of which are in the package: the visibility and section flags hid Rcpp symbols and broke the build on every platform and were reverted in b74e95b. The entry is corrected rather than deleted so the flags are not reinstated by someone reading the old claim.

Smaller items

  • An unmeasured IV is no longer reported as an IV of zero. With no finite IV anywhere, summary() printed Total IV: 0.0000 and IV Range: [Inf, -Inf], asserting “no predictive power” about features that were never measured; print() on a step_obwoe recipe printed total IV=0.0000. Both report NA with a note. obwoe_gains() failed with attempt to select less than one element in get1index and plot(type = "iv") with need finite 'xlim' values; both now say what is actually wrong.

  • sketch (numerical) returns a bin label for a constant feature, so summary$n_bins is 1 instead of NA.

  • udt (numerical) keeps WoE and IV finite with laplace_smoothing = 0. A bin holding no events gave woe = -Inf and iv = Inf; both distributions are now floored with the package-wide epsilon, so the parameter stays usable. Reachable only through ob_numerical_udt() directly – obwoe() never forwards the argument.

OptimalBinningWoE 1.13.2

The fitted object now records how it was fitted

  • obwoe() returns the control it used, alongside the effective min_bins, max_bins and algorithm. call was never a substitute: it records only the arguments the caller typed, never the defaults that actually applied, so a saved model could not say what produced it.

  • A custom bin_separator now works end to end. Everything that has to split grouped categories out of a bin label reads the separator from the model instead of assuming the package default: obwoe_apply(), the points table, the points SQL and the WoE SQL emitted into the workbook.

    This was a silent corruption, not a cosmetic gap. Fitting with control.obwoe(bin_separator = "||") and applying the model back to its own training data put 754 of 900 rows on the na_woe fallback, because the labels were split on "%;%" and never matched. The generated SQL was worse: a||e||c came out as g IN ('a', '|', '|', 'e', '|', '|', 'c') – broken SQL, no warning. Both are exact now.

  • Models saved by earlier versions keep working. They carry no control element, so the separator falls back to the package default rather than erroring on a missing field.

Nothing changes for the default configuration: "%;%" remains the default everywhere, so only the custom-separator path – which was broken – behaves differently.

OptimalBinningWoE 1.13.1

Audit fixes (2026-08-20)

Bug-fix release addressing an internal code audit of 1.13.0. Every item below either changes a computed value that was previously wrong, or removes an API surface the author never intended to publish. See the pull request for the full item-by-item breakdown.

API changes

  • fit_logistic_regression() is no longer exported. It is still used internally (renamed to .ob_fit_logistic_regression()) to implement engine = "obwoe" in obwoe_scorecard(), which is unaffected.

  • ob_gains_table() and ob_gains_table_feature() are renamed to obwoe_gains_score() and obwoe_gains_variable() respectively, to sit alongside the rest of the obwoe_* family. Behavior is unchanged; only the names moved. obwoe_gains() (the higher-level, plot-producing function) is a different function and was not renamed.

Behavior changes (read before upgrading)

  • obwoe_apply() now routes a missing categorical value to the fitted “missing” bin’s WoE when the binning built one, instead of always using na_woe. Previously obwoe_apply() ignored any missing-value bin learned during training and always returned na_woe for NA, while the generated deployment SQL (obwoe_sql(), null_to_na_bin = TRUE by default) routed IS NULL to that bin’s WoE — so R and the SQL scored the same missing value differently. na_woe is now only a fallback for variables where no missing-value bin exists. If you fit a model with real NAs in a categorical predictor, the WoE obwoe_apply()/predict() return for NA may change in this version; it now matches the SQL and the bin actually fitted. See ?obwoe_apply and ?obwoe_sql.

  • obwoe_scorecard(..., drop_negative = TRUE) now actually removes a single negative-coefficient variable instead of silently keeping it. Previously the removal loop stopped one variable too early whenever exactly one variable had a negative coefficient, so that case fell through to a warning instead of either fixing the model or raising stop(). It now keeps removing negative-coefficient variables until either none remain or only one variable is left (in which case, if it is still negative, obwoe_scorecard() stops with an error, as documented).

  • ob_cutpoints_num()’s bins are now right-closed (a, b], and both ob_cutpoints_num() and ob_cutpoints_cat() return the pieces (id, and cutpoints for the numerical version) ob_apply_woe_num()/ ob_apply_woe_cat() require. Previously ob_cutpoints_num() built left-closed [a, b) bins — the opposite of ob_apply_woe_num()’s include_upper_bound = TRUE default — so a value sitting exactly on a cutpoint could get a different, often sign-flipped, WoE depending on whether it went through the fit or the apply side; and neither manual cutpoint function’s result could be handed to its matching apply function at all, because both lacked the id element (and ob_cutpoints_num() the top-level cutpoints) the apply side requires. ob_cutpoints_cat()’s emitted bin labels also now use "%;%" (matching ob_apply_woe_cat()’s default separator and the main pipeline) instead of echoing the "+"-joined input verbatim; the "+" input format is unchanged. If you use ob_cutpoints_num()/ob_cutpoints_cat() and parse their bin labels yourself, check the new format. See ?ob_cutpoints_num and ?ob_cutpoints_cat.

Fixes

  • Gains tables sorted by sort_by = "bin" (including the score bands in obwoe_scorecard()) were ordered lexicographically instead of by the bins’ natural (level) order, silently understating KS by as much as a third on real data.
  • step_obwoe(algorithm = "auto") could resolve to the multiclass algorithm for a genuinely binary outcome whenever the outcome factor declared an unobserved third level.
  • print() on an unprepped step_obwoe recipe step errored when algorithm = tune::tune().
  • obwoe_gains(use_column = "woe", n_groups = k) returned NA WoE and zero total IV for every bin.
  • plot.obwoe_gains(type = "cumulative") assumed every bin held an equal share of the population; the x-axis now reflects each bin’s real size.
  • The advertised algorithm count was corrected from “36 (20 numerical, 16 categorical)” to the actual 37 (21 numerical, 16 categorical) — the numerical ir algorithm was implemented, documented and tested, but was never counted.

OptimalBinningWoE 1.13.0

New features (2026-08-20)

obwoe_scorecard() — the pipeline as one artefact

Runs the whole origination workflow in a single call — stratified split, binning, screening by Information Value and correlation, model fitting, PDO scaling — and returns an object that also writes itself out as an .xlsx model document. The point is not convenience: each stage records why it did what it did, so the workbook is reviewable evidence rather than a set of numbers.

  • The binning sees the training rows only. Binning is supervised: cut points and WoE are both chosen against the target, so fitting them on the full base before splitting leaks the hold-out into the transformation. The split therefore happens first, and the binning is fitted once, on the training rows, and applied everywhere else. Measured on German Credit, the difference is a hold-out AUC of 0.768 fitted the leaky way against 0.709 fitted correctly — the leak buys six points of AUC that do not exist.

  • A negative WoE coefficient is a fault, not a result. The WoE already carries the direction of risk, so a negative slope means the model is reversing a variable to compensate for another. Such variables are dropped one at a time, worst first, and the model refitted, with each removal recorded.

  • The points table is fixed, not per applicant. Points follow the Siddiqi allocation, points_ij = Offset/k - Factor(beta_j WoE_ij + alpha/k), so a bin is worth the same number of points to everyone. Rounding each bin once costs at most k/2 points against the unrounded model score; that drift is measured on every sample and reported rather than hidden by re-apportioning the rounding per row, which would make the same bin worth different points to different applicants.

  • The deployment SQL reproduces the R score exactly. Both the WoE form and the integer-points form are generated, the total computed in an outer SELECT over a subquery — referring to a select-list alias within the same SELECT is a MySQL extension that ANSI SQL, SQLite, PostgreSQL, SQL Server and Oracle all reject. Verified against live SQLite on all 1000 German Credit rows: maximum difference 0, unseen categories included.

  • A value in no fitted bin scores a defined fallback. It is counted and warned about, per sample and per variable, but it still produces a score — both in R and in SQL, from one and the same na_woe figure. A single unseen category cannot void an application.

  • Thirteen sheets. Model summary, scorecard, coefficients with standard errors, bin statistics, screening funnel with a reason per rejected variable, correlations before and after pruning, score gains, PSI between samples, cut-off strategy, SQL in both forms, and a reproducibility record.

  • Engines are a three-function contract — fit, link, coef — with glm, the package’s own C++ L-BFGS logistic regression, and glmnet registered, and custom engines accepted. coef() returning NULL declares the model non-additive, and the pipeline then produces no points table rather than fabricating one from a model that has no per-variable decomposition. A missing engine package is an error by default, not a silent substitution: a workbook documenting a model the analyst did not ask for is worse than a call that fails.

Supporting functions

  • obwoe_scale() and obwoe_score() implement PDO scaling, Factor = PDO/ln 2 and Offset = Score0 - Factor ln(Odds0), in both score directions. Doubling the good:bad odds moves the score by exactly one PDO at every point of the range.
  • obwoe_prune() removes redundant variables iteratively, dropping the worse-ranked member of the strongest surviving pair and recomputing, rather than resolving all pairs against the original matrix at once.
  • obwoe_psi() computes the Population Stability Index between two samples, for both binned and continuous inputs, using interior quantiles so that a merely shifted distribution does not produce a degenerate band.
  • obwoe_report() writes the workbook for an already-fitted scorecard.
  • predict() methods for "score", "card", "link", "prob" and "woe", reading only the binning, the coefficients and the scaling.

Fixed during review

  • The screening funnel named the wrong stage. Every variable that left the pipeline after Information Value screening was labelled corr_pruned, including those dropped by the negative-coefficient check and those that turned out constant after the WoE transform. The funnel is the sheet a reviewer reads to learn why a variable is absent, so a mislabelled rejection is a false statement in a model document. stage now distinguishes in_model, sign_rejected, corr_pruned, constant_woe and screened_out, each assigned by the step that actually rejected the variable.

  • The cut-off strategy sheet ignored the score direction. It always approved applicants scoring at or above the cut, which is right under the default scale but exactly inverted under direction = "higher_is_riskier" — the sheet then approved the worst applicants and reported an approved bad rate above the rejected one, at every one of its twenty cut-offs. The approval rule now follows the direction of the scale.

  • file is validated before the pipeline runs. A missing openxlsx, a non-existent directory or an unwritable one used to surface only at the final write, after the binning, the screening and the fit had all been computed and were about to be discarded. The check now happens up front: the failure arrives in about 0.04 s instead of after the whole run.

Dependencies

openxlsx and glmnet are added to Suggests. Neither is needed unless a workbook is written or engine = "glmnet" is requested.

OptimalBinningWoE 1.12.0

New features (2026-08-19)

Two additions close the gap between a fitted binning and a deployed scorecard: deciding which variables deserve to enter the model, and shipping the accepted transformation to the database where the data lives.

obwoe_select() — automated variable screening

Screens every binned variable of an obwoe model against the two criteria that govern variable admission in credit risk practice — predictive strength, graded with the Siddiqi (2006) Information Value bands, and guaranteed rank ordering, measured by monotonicity of the bin event rate — and returns a verdict for each.

  • Nothing is ever dropped. A base with 500 candidate variables yields 500 rows. Each carries a selected flag, a machine-readable reason listing every rule it violated, and a reason_desc in plain language, so the automatic verdict can be reviewed, overridden, or replaced by the analyst’s own triage.

  • Two levels of detail. detail = "summary" gives one row per variable with its headline metrics; detail = "full" expands to one row per variable and optimised bin, carrying the complete gains table of ob_gains_table() (31 metrics: WoE, IV, lift, cumulative KS, precision, recall, F1, KL and JS divergence, …) together with the interval bounds of numerical bins and the merged category lists of categorical ones.

  • Metrics that describe the deployed score. ks, auc and gini are computed on the WoE actually applied to the data, with bins ranked by that WoE and merged when tied. auc uses the tie-corrected Mann-Whitney form, which reproduces the rank-based AUC of the WoE-transformed observations to machine precision. ks is therefore the true KS of the transformed variable even when the binning is not monotonic.

  • An honest treatment of ordering. For numerical variables the bin sequence is intrinsic, so monotonicity is a genuine constraint. For nominal categories the sequence is a free relabelling, so the default require_monotonic = "numeric" applies the constraint only where it means something; "all" and "none" are available.

  • Rejects suspicious variables by default. iv_max = 0.50 excludes the Siddiqi Suspicious band, where a single-variable IV is far more often a symptom of target leakage than of a dominant predictor. On the Statlog German Credit benchmark this is exactly what happens to the checking-account status (IV = 0.67).

  • Further gates for min_bins, max_bins, minimum bin population share, bins with no events or no non-events, and a top_n cut by IV, KS, Gini or AUC. An Excellent/Good/Fair/Rejected quality tier summarises how cleanly the algorithm categorised each variable.

  • Returns a data.table when that package is installed and an identical data.frame otherwise; the table is assembled column-wise in a single pass, so 500 variables are screened in about a second.

obwoe_sql() — SQL code generation

Translates a fitted binning into executable SQL, so the WoE transformation runs inside the database with no round trip through R.

  • Exact interval semantics. Numerical bins are half-open on the right, (lower, upper], exactly as obwoe_apply() assigns them. Boundaries come from the fitted cutpoints vector, never parsed back from bin labels, whose formatting varies between algorithms. By default each branch states both of its bounds (WHEN x > 7 AND x <= 10 THEN ...) so it is correct in isolation; explicit_bounds = FALSE emits the shorter cascading form.

  • Cut points that survive the round trip. Literals are written as the shortest fixed-notation decimal that parses back to the identical IEEE 754 double, and never in scientific notation. Rounding a boundary such as 4049.5 would silently move observations between bins, so exactness is the default; digits makes rounding an explicit choice.

  • NULL cannot leak. In SQL, NULL <= 5 is NULL, not FALSE, so a missing value matches no comparison and would fall through to ELSE. Every generated expression opens with an explicit WHEN <col> IS NULL branch, and when the binner folded training missings into a category bin, null_to_na_bin = TRUE routes database NULLs to that same bin.

  • Escaping that holds. Single quotes are doubled per ANSI SQL; on MySQL, MariaDB and the Hive family backslashes are doubled too. Category names are matched byte for byte, whitespace included. Identifiers are quoted with the dialect’s own delimiters and, by default, only when the name needs it or collides with a reserved word — which keeps generated code readable on case-folding engines such as Oracle and Snowflake.

  • 14 dialects (ansi, postgres, mysql, mariadb, sqlserver, oracle, spark, hive, databricks, bigquery, snowflake, redshift, duckdb, sqlite), four assembly styles (select, case, cte, view), four output modes (woe, bin, index, both), an audit header recording package version and algorithm, and direct file output.

  • Accepts an obwoe object, a prepped step_obwoe() step, or a prepped recipe containing one, and composes with obwoe_select() through features = sel$feature[sel$selected].

Validation

  • Real benchmark data is bundled. inst/extdata/germancredit.csv.gz holds the Statlog (German Credit) dataset from the UCI Machine Learning Repository (1000 applications, 7 numerical and 13 categorical attributes, CC BY 4.0), in its labelled form — category names carrying spaces, slashes and colons make it a realistic test bed for SQL escaping.

  • obwoe_select() reproduces the published Information Values of that benchmark to four decimals for the 14 attributes the optimiser keeps ungrouped, and its IV, KS, AUC and Gini match an independent computation made from the raw observations to 1e-10.

  • The generated SQL is validated by parsing and executing it: the test suite carries a small CASE interpreter that reads the emitted text the way an engine would, and the resulting bin assignment is checked against the counts the binning algorithm itself reported — across five algorithms and all 20 German Credit variables. During development the same statements were additionally run against a live SQLite engine, whose results were identical. Edge cases covered include observations sitting exactly on a cut point, cut points with no exact binary representation, magnitudes from 1e-9 to 1e9, single-bin variables, degenerate bins, reserved-word column names, and categories containing quotes, backslashes, tabs, accented characters and significant whitespace.

Bug fixes

Both defects below were uncovered while validating the two new functions against real data, and both are pinned by tests in tests/testthat/test-regression-audit.R that fail on 1.11.0 — 31 assertions in total.

  • obwoe_apply() and bake.step_obwoe() silently failed to score any category carrying leading or trailing whitespace. Both rebuilt their category-to-bin lookup with trimws(strsplit(bin_label, "%;%", fixed = TRUE)[[1]]). The binning engines join the original category strings with the separator and add no padding — verified here for all 15 categorical algorithms — so the split pieces are already the categories byte for byte, and the trimming turned a category such as " N/A " or "PENDING " into a key no observation could match. Those rows were assigned bin = NA and woe = na_woe, i.e. scored as unseen, with no warning: on a base whose codes come from a CHAR(n) column or a hand-maintained code table, an entire segment could be dropped from the model without a trace. The trimming is gone; categories are now matched exactly, as obwoe_sql() already did.

  • obwoe_gains() reported the same lift for every bin. The column was built with ifelse(overall_rate > 0, df$pos_rate / overall_rate, 0); ifelse() returns a result shaped like its test, and that test is a scalar, so the expression collapsed to a length-one vector that R then recycled across the table. Every gains table with more than one bin reported the first bin’s lift throughout, and plot(type = "lift") drew a flat line. The R gains table now agrees with the independent C++ engine (ob_gains_table()) to machine precision on lift, woe, iv and ks.

  • ob_numerical_ir() reported counts that did not describe its own bins, and bins that were not monotonic. applyIsotonicRegression() ran the Pool Adjacent Violators algorithm over the bin event rates and then overwrote each bin with count_pos <- round(fitted_rate * count). Pooling adjacent violators means those bins form one block; keeping them separate and back-solving synthetic counts produced two defects at once. First, count_pos and count_neg no longer described the observations falling between the reported cutpoints, so WoE, IV, KS and every gains table derived from them referred to a distribution that does not exist — on the German Credit amount attribute the reported and observed event rates differed in four of six bins. Second, because of the rounding the reported bins were not even monotonic, which is the one property an isotonic binner exists to guarantee.

    PAVA blocks are now merged into single bins. This reproduces the isotonic fit exactly — with the bin counts as weights, a block’s pooled rate is sum(count_pos) / sum(count) over that block — while count, count_pos and count_neg stay equal to what was observed. Features whose event rate is already monotone are unaffected and return bit-identical results; where PAVA had violators to pool, the binning is now coarser and genuinely rank-ordering. min_bins becomes a target rather than a guarantee for this algorithm, since monotonicity cannot always be attained at that resolution.

Documentation

  • The README is a third of its former length. It now covers what the package is, the four steps of a run, how to choose an algorithm and where to read more; the extended worked examples moved into the vignettes, where an analyst can study them properly. Algorithm selection is presented as a decision graph (rendered by GitHub) with the family reference table kept below it.

  • Two vignettes replace the single kitchen-sink one.

    Optimal Binning and Weight of Evidence: A Practical Guide is the working reference: what WoE and IV measure and why monotonicity of the event rate and of the WoE are the same statement, reading bin and gains tables, screening a base with obwoe_select(), how the algorithm families differ, applying the transformation, exporting SQL, and preprocessing.

    An Industrial Scorecard Pipeline is new and runs an origination scorecard end to end: a wide synthetic base with the pathologies that matter (missing fields, rare dealer codes, near-duplicate vendor variables, pure noise and a leaky post-booking field), screening at scale, redundancy pruning with obcorr() in the WoE space, a recipes pipeline around step_obwoe(), logistic regression with the coefficient sign check, PDO scorecard points, out-of-time validation with gains and PSI, deployment through obwoe_sql(), tuning with tidymodels, and a governance checklist.

  • Both vignettes run on the bundled German Credit benchmark or on data they generate, so they build from recipes alone. The previous vignette required the Suggested scorecard package and failed to build without it; scorecard and pROC are no longer used anywhere and have been dropped from Suggests.

Notes

  • data.table is used when installed, for rbindlist() and for the returned table type, and is declared in Suggests only: the package has no new hard dependency and behaves identically without it, as the test suite verifies.

OptimalBinningWoE 1.11.0

C++ Engine — Runtime Audit (2026-08-12)

Follow-up to the 1.10.0 static audit, this time driven by instrumented builds (-fsanitize=address,undefined), a degenerate-input stress harness, and a golden-output regression suite covering all 37 exported algorithms (~3,200 result comparisons). All fixes below are covered by new tests in tests/testthat/test-regression-audit.R, each of which fails on 1.10.0.

Bug Fixes — crashes and hangs

  • ob_categorical_ivb crashed the R session (segmentation fault, not a catchable error) for any feature with no more categories than max_bins — with the default max_bins = 5 this meant every 2-, 3-, 4- or 5-level predictor, i.e. sex, marital status, region, education. In perform_binning() the ncat <= max_bins fast path skipped initialize_dp_structures(), the only place stats_cache was created, while result assembly dereferenced it unconditionally. The cache is now built before the branch.

  • ob_numerical_ir, ob_numerical_jedi, ob_numerical_jedi_mwoe hung forever whenever min_bins exceeded the number of distinct feature values — a routine situation when one min_bins is applied across a whole feature set. The “ensure at least min_bins” loops called a split routine that silently declines to split unbounded intervals (and, in JEDI, ran before counts existed, so it always targeted the unsplittable (-Inf, e1] bin), making no progress and never terminating. Since these loops contained no R_CheckUserInterrupt(), the hang could not even be interrupted with Ctrl-C. The loops now consider only splittable bins and stop as soon as no progress is possible.

Reproducibility

  • ob_categorical_sab is now reproducible under set.seed(). It was seeded from std::random_device, so identical input returned a different binning on every call with no way to control it — unusable for auditable or regulated models, and unreliable on some MinGW toolchains. The simulated-annealing search is now seeded from R’s own RNG stream. This changes ob_categorical_sab results, which were previously random and therefore had no stable baseline to preserve.

Output consistency

  • ob_numerical_sketch now returns the bin label field (plus total_iv) like every other numerical algorithm. Its absence broke generic consumers, including this package’s own test helper. bin_lower and bin_upper are retained, so the change is purely additive; no numeric output changed.

  • The ob_numerical_sketch test disabled since 1.0.7 for a segfault has been re-enabled: the underlying MergeCache defect was removed in an earlier round, and the case was re-verified clean under -fsanitize=address,undefined at n = 500/1000/2000/5000.

Parallelism (obcorr) — CRAN policy and determinism

  • obcorr() no longer seizes every core. With the default threads = 0 it called omp_set_num_threads(std::thread::hardware_concurrency()), taking all available cores. CRAN Repository Policy requires a package never to use more than two cores simultaneously by default, since the check farm is a shared resource; this was an archival risk. The default is now at most 2 threads, honouring any lower limit already set through OMP_NUM_THREADS, and capped by omp_get_num_procs(). An explicit positive threads is still respected.

  • obcorr() returned its rows in a non-deterministic order. Per-thread result buffers were spliced together inside an omp critical block, so the row order depended on thread scheduling — two runs on the same data with the same thread count could return the pairs in different orders. Any caller using head(), positional indexing, or a positional join saw different numbers between runs. Each iteration now writes to its own slot in a pre-sized vector, which is deterministic, independent of the thread count, and removes the critical section. Correlation values are unchanged; only the row order is now stable.

Build hygiene

  • Removed eight stale Dropbox conflict copies (*Cópia em conflito*.cpp/.h) from src/. R CMD build would have shipped and compiled them, producing duplicate symbols. Guards added to .gitignore and .Rbuildignore.

Interval convention — standardised on (a, b] (changes numeric output)

The package previously disagreed with itself about what a bin is. Bin labels advertised (a;b], but many algorithms assigned observations as [a;b); ob_apply_woe_num() had its two binary searches swapped, so both settings of include_upper_bound did the opposite of what the argument documents; and several equal-frequency pre-binners split runs of tied values, so their reported counts could not be derived from their own reported cutpoints under any convention. A value landing exactly on a cutpoint — routine for integer, rounded or currency features — could be scored into a different bin than the one it was trained in.

Measured across 63 algorithm/dataset combinations, 31 violated the documented (a, b] convention before this release; 0 do now.

  • ob_apply_woe_num(): include_upper_bound = TRUE now really means (a, b] (lower_bound search) and FALSE really means [a, b) (upper_bound). The two were previously exchanged.
  • ob_numerical_dp, _fetb, _ldb, _oslp: bin lookup switched from upper_bound to lower_bound, so boundary values stay in the bin below.
  • ob_numerical_bb, _dmiv: dropped an + EPSILON added to the search key, which silently turned the documented upper >= value test into a strict one.
  • ob_numerical_sketch: interval test changed from [lower, upper) to (lower, upper] (first bin remains closed on the left, so the minimum is included).
  • ob_numerical_mob, _mdlp, _mrblp: equal-frequency pre-binning is now tie-aware — a run of identical values is never split across two pre-bins — and boundaries are the last value in the bin rather than the first value of the next. Their labels changed from [a;b) to (a;b] to match.
  • ob_numerical_fast_mdlp: the force_min_bins() fallback picked a split at a raw index midpoint, which could land inside a run of tied observations. It now moves the split to a genuine value boundary, as the MDL recursion already did.

ob_numerical_cm silently discarded observations. Its equal-frequency pre-binner, on hitting a tie that straddled a bin boundary, advanced past the tied records without ever assigning them to a bin. On tied or discrete features this dropped a large share of the data — 26% on an integer feature and 40% on a coarse one in testing — and the reported WoE/IV were computed from the surviving subset. The tied records are now absorbed into the preceding bin.

Known issues (not yet fixed)

  • ob_categorical_dp and ob_categorical_fetb report WoE = IV = 0 for perfectly separating bins instead of applying smoothing.
  • Return fields are not uniform across algorithms: event_rate is present in only 13 of 37, total_iv in 28 of 37, and iv is missing from both _dmiv variants.
  • ob_apply_woe_num() does not support the multinomial *_jedi_mwoe variants, whose output carries per-class counts rather than count_pos / count_neg.

OptimalBinningWoE 1.10.0

C++ Engine — Comprehensive Audit & Hardening (2026-05-17)

This release is the result of a full static audit of the C++ engine covering all 36 binning algorithms. No public R API was changed.

Bug Fixes

  • OB_LogisticRegression — Replaced exact det != 0 singularity guard with a threshold-based check (|det| > 1e-10 × ‖H‖); replaced hessian.inverse() with Eigen::LDLT decomposition for numerical stability; added .cwiseMax(0.0) before sqrt() to prevent NaN standard errors from near-zero diagonal entries.
  • OBN_MDLP — monotonicity direction bug — is_monotonic() and enforce_monotonicity() previously hardcoded ascending direction, causing unnecessary merges on negatively-correlated features. Both now auto-detect the dominant trend via Welford’s slope algorithm before checking/enforcing monotonicity.
  • OBC_DP — DP backtracking out-of-bounds — Added guard before static_cast<size_t>(prev_j) in backtrack_optimal_bins(); invalid predecessor index now raises a descriptive runtime error instead of silent undefined behaviour.
  • OBN_DP — push before validate — Target value validation in optimal_binning_numerical_dp() now fires before the value is appended to target_vec, preventing insertion of invalid data.
  • OBN_MDLP — log2(0) in MDL cost — Guard added for the single-bin case where log2(k-1) would evaluate to log2(0) = -Inf.
  • NumericalBin constructor invariant — The 7-arg constructor now derives count = count_pos + count_neg regardless of the c argument, enforcing the count == total() invariant at construction time.

Performance Improvements

  • OBC_DP — DP outer loop removed — The deterministic DP in perform_dynamic_programming() was wrapped in a redundant max_iterations outer loop (default 1000). Removing it yields up to 1000× speedup for the categorical DP algorithm.
  • OBC_DP::ensure_max_prebins() — O(m² log m) full re-sort per merge step replaced with O(m log m + m²) std::lower_bound + insert.
  • OBN_MDLP::apply_mdl_merging() — O(k³) full-vector copy per candidate merge eliminated; MDL delta is now computed analytically from bin statistics in O(k²) per outer step.
  • OBN_BB::quantile() — Per-call sort-copy O(n_prebins × n log n) eliminated; prebinning() sorts once and passes the sorted vector to a stateless quantile().
  • monotonicity_utils.h — Welford index allocation — Removed unnecessary std::vector<double> indices(n) heap allocation; loop index cast directly to double.
  • OBN_DP — Pearson correlation instability — Replaced naive two-pass Pearson formula (catastrophic cancellation risk) with detect_trend_from_correlation() using Welford’s online algorithm.

CRAN / ODR Safety

  • safe_math.h — All 6 functions changed from constexpr to inline; std::log, std::exp, std::abs and std::isfinite are not guaranteed constexpr in C++11/14, risking compilation failure on SOLARIS/Studio.
  • chi_square_utils.h — CHI_SQUARE_CRITICAL_VALUES namespace-scope const replaced with a function returning a static local instance (one shared copy per process, C++11 thread-safe init).
  • entropy_utils.h — ENTROPY_LUT (~81 KB) replaced with entropy_lut_instance() returning a static local; eliminates one copy per translation unit.
  • OBC_CM_v5 — Duplicate ChiSquareCache class (global namespace) removed; file now uses OptimalBinning::ChiSquareCache from chi_square_utils.h.
  • 35 .cpp files — Duplicate using namespace Rcpp appearing before #include "common/" headers removed, preventing potential name-resolution ordering issues.

Code Quality

  • OBC_DP — Dead commented-out code blocks (// struct CategoryStats, // Local CategoricalBin definition removed) deleted.
  • OBN_DP — Local variable total_count renamed to rare_total to fix shadowing of the class member with the same name.
  • OBN_IR — [[Rcpp::plugins(cpp17)]] standardised to cpp11 for consistency with the rest of the package.
  • OBC_DP — Auto-detection of monotonicity direction (monotonic_trend = "auto") implemented in compute_and_sort_event_rates() via detect_trend_welford_woe().

OptimalBinningWoE 1.0.9

  • CRAN Fix (2026-03-14) - Replaced Rf_error with Rcpp::stop:
    • Fixed C++ Exception Handling: Addressed an issue reported by @Enchufa2 regarding the usage of ::Rf_error inside catch(...) blocks. Updated all instances to use Rcpp::stop to ensure proper C++ stack unwinding and avoid memory leaks.
    • Affected Files: src/OBN_LPDB_v5.cpp, src/OBN_EWB_v5.cpp, src/OBN_KMB_v5.cpp, src/OBN_LDB_v5.cpp, src/OBN_MBLP_v5.cpp.

OptimalBinningWoE 1.0.8

CRAN release: 2026-01-29

  • CRAN Fix (2026-01-28) - LTO/ODR Compliance:
    • Fixed One Definition Rule (ODR) violations: Wrapped internal helper classes IVCache and CumulativeStatsCache in anonymous namespaces within OBC_GMB_v5.cpp, OBC_IVB_v5.cpp, and OBC_JEDI_v5.cpp. This resolves Link-Time Optimization (LTO) warnings/errors on CRAN checks.

OptimalBinningWoE 1.0.7

  • UBSAN Investigation Fix (2026-01-27) - Addressing persistent memory safety errors:

    • Temporarily disabled ob_categorical_sketch tests: The sketch-based categorical binning algorithm is under investigation for persistent UBSAN memory errors that appear to be related to cache invalidation timing in GitHub Actions CI environment.

    • Removed MergeCache class from OBC_Sketch_v5.cpp: Completely removed the caching mechanism and implemented on-the-fly divergence calculation to eliminate potential memory corruption sources.

  • Affected Files:

    • src/OBC_Sketch_v5.cpp: MergeCache class removed, divergence calculated on-the-fly
    • tests/testthat/test-categorical-all.R: Sketch tests temporarily commented out

  • No API Changes: Fully backward compatible with v1.0.6.

OptimalBinningWoE 1.0.6

  • CRAN Fix (2026-01-26) - Resolving AddressSanitizer memory safety errors:

    • Fixed heap-buffer-overflow in OBN_CM_v5.cpp: The calculate_inconsistency_rate() function was accessing bins[j-1] when j=0 and bins.size()==1, causing invalid memory access. Restructured bin-finding loop to avoid negative index access.

    • Fixed uninitialized bool in OBC_MBA_v5.cpp: The MergeCache::enabled member was not explicitly initialized, causing “load of value 128, which is not a valid value for type ‘bool’” runtime error. Added explicit bool enabled = false initialization.

  • Affected Files:

    • src/OBN_CM_v5.cpp (lines 863-887): Safe bin-finding logic
    • src/OBC_MBA_v5.cpp (line 26): Explicit bool initialization

  • No API Changes: Fully backward compatible with v1.0.5.

OptimalBinningWoE 1.0.5

  • CRAN Fix (2026-01-25) - Resolving ERROR on macOS platforms during vignette re-build:

    • Fixed obwoe_apply() “breaks are not unique” error: Enhanced cutpoint deduplication logic to properly handle cases where sort(unique(cutpoints)) reduces the number of intervals. When the deduplicated cutpoint count doesn’t match the original bin count, the function now uses a fallback mapping with dynamically generated interval labels and mean WoE values, avoiding the cut.default() error.

    • This addresses the vignette build failure reported on r-release-macos-arm64, r-release-macos-x86_64, r-oldrel-macos-arm64, and r-oldrel-macos-x86_64 platforms.

  • Internal Changes:

    • Added interval count validation after cutpoint deduplication (R/obwoe.R)
    • Fallback to mean WoE when bin/interval mismatch occurs
    • Dynamic interval label generation for edge cases

OptimalBinningWoE 1.0.4

  • CRITICAL CRAN Fixes (2026-01-24) - Addressing ERROR and NOTE on macOS platforms:

    • Fixed macOS vignette ERROR: Added comprehensive validation for duplicate cutpoints in obwoe_apply() and bake.step_obwoe(). The R base cut() function now receives guaranteed unique, sorted breaks, preventing the "'breaks' are not unique" error that was causing vignette build failures on macOS platforms.

    • Attempted to reduce the package binary size with size optimization flags (-Os, -fvisibility=hidden, -ffunction-sections, -fdata-sections), the -Wl,--gc-sections linker flag and a cleanup script for symbol stripping.

      This was reverted and no longer describes the package. -fvisibility=hidden and -Wl,--gc-sections hid Rcpp symbols and broke the build on every platform, so they were removed in commit b74e95b; -Os and the cleanup script did not survive either. src/Makevars now sets only the OpenMP and BLAS/LAPACK flags. The entry is corrected here rather than deleted so that nobody reinstates flags that are already known to break the build.

      The size itself is not a problem: the shared object is large only because it carries debug symbols. Measured on 1.13.3, .debug_info alone is 33Mb against 2.2Mb of .text, and stripping takes the library from 70.4Mb to 2.6Mb. R CMD check --as-cran reports the installed size as INFO, not as a NOTE.

  • Internal Changes:

    • Added src/common/cutpoints_validator.h - new C++ utility header with validate_cutpoints() function to ensure cutpoint uniqueness across all numerical binning algorithms. Uses floating-point tolerance (1e-10) for safe duplicate detection.

    • Modified get_cutpoints() in src/OBN_MOB_v5.cpp (line 180) to apply validation before returning cutpoints.

    • Modified update_cutpoints() in src/OBN_UBSD_v5.cpp (line 874) to apply validation before storing cutpoints.

    • Added R-level validation in obwoe_apply() (R/obwoe.R, line 1550): cutpoints are now sorted and deduplicated using sort(unique(cutpoints)) before constructing breaks vector.

    • Added R-level validation in bake.step_obwoe() (R/step_obwoe.R, line 789): same deduplication logic for recipes integration.

    • Enhanced vignette robustness (vignettes/introduction.Rmd): Added try-catch error handling in scorecard workflow to prevent build failures on edge-case data distributions.

  • Affected Algorithms: All 21 numerical binning algorithms now validate cutpoints to prevent duplicate breaks:

    • Monotonic Optimal Binning (MOB)
    • Dynamic Programming (DP)
    • Chi-Merge (CM)
    • Unsupervised Binning with Standard Deviation (UBSD)
    • And 17 other numerical algorithms

  • No API Changes: Fully backward compatible with v1.0.3. All existing code will continue to work without modification.

OptimalBinningWoE 1.0.3

CRAN release: 2026-01-23

  • Critical Bug Fixes - KLL Sketch Algorithm (2026-01-20):
    • Fixed iterator invalidation in KLLSketch::compact_level() - the compactors.push_back() call was invalidating references to vector elements, causing crashes with datasets larger than ~200 observations.
    • Fixed parameter order bug in calculate_metrics() calls - swapped (total_good, total_bad) to correct order (total_pos, total_neg), fixing incorrect WoE calculations.
    • Fixed half-open interval logic in bin assignment - added explicit closed interval [lower, upper] check for the last bin to ensure boundary values are correctly assigned.
    • Fixed merge direction logic in enforce_bin_cutoff() - corrected iterator invalidation when merging bins by always erasing the higher-indexed bin.
    • Added bounds safety checks in DP optimization - ensured k >= 2 and k < n to prevent undefined behavior with edge cases.
    • Added underflow guard in compaction loop - check for compactor.size() < 2 before iteration.
    • Added input validation for non-finite values (Inf, NaN) in sketch updates.
    • Improved documentation in ob_numerical_sketch() with clearer parameter descriptions and simplified examples.
    • Replaced special_codes parameter with max_n_prebins for consistency with other algorithms.
  • CRAN Reviewer Feedback (2026-01-17):
    • Removed single quotes from author names (Siddiqi, Navas-Palencia) in DESCRIPTION.
    • Removed commented-out code from examples in obwoe_apply.
    • Replaced all \dontrun{} with \donttest{} in 12 function examples.
    • Added proper par() restoration in examples and vignettes.

OptimalBinningWoE 1.0.2

  • CRAN Resubmission:
    • Updated inst/WORDLIST to include technical terms and author names (MILP, Navas, Palencia) to resolve spelling notes.
    • Fixed README.md links for CONTRIBUTING.md and CODE_OF_CONDUCT.md to use absolute GitHub URLs, ensuring compliance with CRAN URI checks for ignored files.
    • Added Language: en-US to DESCRIPTION metadata.

OptimalBinningWoE 1.0.1

  • CRAN Preparation: Comprehensive updates for CRAN submission compliance.
  • Documentation:
    • Enhanced README.Rmd with detailed algorithm descriptions, tidymodels integration examples, and performance metrics.
    • Added CODE_OF_CONDUCT.md (Contributor Covenant v2.1) and CONTRIBUTING.md guidelines.
    • Added inst/WORDLIST for spell checking.
  • Metadata:
    • Updated DESCRIPTION with corrected fields (Authors, BugReports, Depends, References).
    • Added cran-comments.md for submission notes.

OptimalBinningWoE 1.0.0

Initial Release

OptimalBinningWoE is a high-performance R package for optimal binning and Weight of Evidence (WoE) transformation, designed for credit scoring and predictive modeling.

Key Features

  • Comprehensive Algorithm Suite: Implementation of 36 binning algorithms:
    • 20 Numerical Algorithms: Including MDLP (Minimum Description Length Principle), JEDI (Joint Entropy-Driven Information), MOB (Monotonic Optimal Binning), Sketch (KLL/Count-Min for large data), and more.
    • 16 Categorical Algorithms: Including ChiMerge, Fisher’s Exact Test Binning (FETB), SBLP (Similarity-Based LP), JEDI-MWoE (Multinomial WoE), and others.
  • High Performance: Core algorithms are implemented in C++ using Rcpp and RcppEigen for maximum efficiency and scalability.
  • Unified Interface:
  • tidymodels Integration:
    • step_obwoe(): A complete recipes step for integrating optimal binning into machine learning pipelines.
    • Supports tune() for hyperparameter optimization of binning parameters (algorithm, min_bins, etc.).
  • Multinomial Support:
    • Dedicated algorithms like JEDI-MWoE for handling multi-class target variables.
  • Robust Preprocessing:
    • ob_preprocess(): Utilities for missing value handling and outlier detection/treatment (IQR, Z-score, Grubbs).
  • Advanced Metrics:
    • ob_gains_table(): Computation of detailed gains tables including IV, WoE, KS, Gini, Lift, Precision, Recall, KL Divergence, and Jensen-Shannon Divergence.
  • Visualization:
    • S3 plot() methods for visualizing binning results and WoE patterns.

usage