Fix warnings emitted by the application scorecard full suite

[cachafla]

Pull Request Description

What and why?

Running application_scorecard_full_suite.ipynb on validmind 2.13.12 prints roughly 600 warning lines. This removes all of them except one template mismatch (see below).

Warning	Count	Fix
scorecardpy FutureWarning / UserWarning (pandas deprecations inside woebin, woebin_ply, condition_fun)	~480	lending_club: warnings.catch_warnings() around the two scorecardpy calls. Third-party code, unmaintained upstream.
TypeError: custom_recall() got an unexpected keyword argument 'needs_proba' plus sklearn "Scoring failed" / "non-finite test scores"	~85	HyperParametersTuning: make_scorer(..., response_method="predict_proba"). needs_proba was removed in scikit-learn 1.6, so the recall scorer had been silently failing. response_method exists from 1.4, so pyproject.toml now requires scikit-learn>=1.4 (base) and >=1.4,<1.8 (extras).
SettingWithCopyWarning in WeakspotsDiagnosis	12	.copy() the column slices before assigning the bin column.
FutureWarning: The default of observed=False is deprecated in ScoreProbabilityAlignment	1	groupby(..., observed=True). Bins come from qcut, so no empty categories are dropped.
Failed to generate description for HyperParametersTuning: Object of type DataFrame is not JSON serializable	1	generate_description: round-trip params through NumpyEncoder before the request. The demo config passes fit_params={"eval_set": [(x_test, y_test)]}.

Not addressed: Config key 'validmind.model_validation.ModelMetadata' does not match a test_id in the template. The demo config is shared by several notebooks and that test exists in other templates; the model's documentation template on the platform decides this.

How to test

• uv run python -m unittest tests.test_test_descriptions tests.unit_tests.model_validation.sklearn.test_WeakspotsDiagnosis — includes the new test_generate_description_params_are_json_serializable.
• _create_scoring_dict(["roc_auc", "recall"], ..., 0.3) fed to GridSearchCV on scikit-learn 1.7.2 returns finite recall and roc_auc scores.
• lending_club.feature_engineering on a 5k-row sample of the offline dataset emits zero scorecardpy warnings in the parent process.
• Rerun the notebook on JupyterHub and compare the output against the previous run.

What needs special review?

• The scikit-learn floor. >=1.4 matches the library's Python 3.9 minimum; the lock refresh only changed the three specifier lines.
• scorecardpy bins in a multiprocessing pool. On Linux the workers fork and inherit the suppression; on macOS they spawn fresh and a few warnings can still leak from workers.

Dependencies, breaking changes, and deployment notes

Adds a lower bound scikit-learn>=1.4 (January 2024). Environments on older scikit-learn will now fail at install time rather than with a TypeError in HyperParametersTuning.

Release notes

• Fixed HyperParametersTuning recall scoring on scikit-learn 1.6 and later; recall scores were reported as non-finite.
• Fixed AI description generation for tests whose params contain DataFrames, such as HyperParametersTuning with fit_params.
• Removed pandas and scikit-learn deprecation warnings from WeakspotsDiagnosis, ScoreProbabilityAlignment and the lending_club demo dataset.
• The library now requires scikit-learn 1.4 or newer.

Checklist

• What and why
• Screenshots or videos (Frontend)
• How to test
• What needs special review
• Dependencies, breaking changes, and deployment notes
• Labels applied
• PR linked to Shortcut
• Unit tests added (Backend)
• Tested locally
• Documentation updated (if required)
• Environment variable additions/changes documented (if required)

🤖 Generated with Claude Code

[github-actions]

Pull requests must include at least one of the required labels: internal, highlight, enhancement, bug, deprecation, documentation. Except for internal, pull requests must also include a description in the release notes section.

[github-actions]

Pull requests must include a description in the release notes section.

[github-actions]

Pull requests must include a description in the release notes section.

[github-actions]

Pull requests must include a description in the release notes section.

[gtagle]

✔️