Skip to content

fix(deps): resolve pip-audit failures for dnspython, tornado and soupsieve - #156

Merged
tkislan merged 2 commits into
mainfrom
tk/fix-pip-audit-2026-11-19
Nov 19, 2026
Merged

tkislan merged 2 commits into
mainfrom
tk/fix-pip-audit-2026-11-19

Conversation

@tkislan

@tkislan tkislan commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Audit - Production and Audit - Server fail on main with six advisories in dnspython, tornado and soupsieve. This PR raises the floors of all three:

  • dnspython: 2.9.2 on Python 3.11+.
  • tornado: 6.6.1 on Python 3.11+.
  • soupsieve: 4.1.2 on Python 3.11.5+, which covers every 3.11 kernel bundle. It is 2.10 below 3.11.5.

The audit's ignore list is unchanged.

Audit

  • CI runs pip-audit in the audit job of .github/workflows/ci.yml (around line 452).
  • It uses Python 3.11 and Poetry 2.2.0 with poetry-plugin-export, and exports three requirement sets: --without dev (Production), --extras server (Server) and --only dev (Dev).
  • pip-audit runs with --require-hashes --disable-pip --vulnerability-service pypi and the ignore-vulns list.
  • On main at 0b0d0ab, Production and Server report the same six advisories. Dev is clean.
  • With this PR, Production and Server report "No known vulnerabilities found, 3 ignored", the pre-existing pyarrow ignores. Dev reports "No known vulnerabilities found".

Precedent

dnspython

-    "dnspython>=2.9.0,<3.0; python_version >= '3.11'",  # 2.9.0 fixes CVE-2026-114605 (transitive via pymongo)
+    "dnspython>=2.9.2,<3.0; python_version >= '3.11'",  # 2.9.2 fixes CVE-2026-128089 (transitive via pymongo)
  • Lock: only the dnspython 2.9.0 block on Python >= 3.11 changes, to 2.9.2. No other package moves, and pymongo allows >=2.7.0,<3.0.0.
  • Verified:
    • The advisory is gone from both audits.
    • 2.9.1 → 2.9.2 changes only the WKS parser.
    • pymongo mongodb+srv lookups return the same hosts on 2.9.0 and 2.9.2.

tornado

  • Advisory: GHSA-5cjw-f85j-cv6r / CVE-2026-126347 (low), an open redirect through triple-slash paths in RedirectHandler. Affects <= 6.6.0.
  • Path: ipykernel, jupyter-client, jupyter-server (server extra) and terminado all depend on tornado, and the toolkit imports it in deepnote_core/widget_assets.py.
  • Fix: the floor on Python >= 3.11 goes from 6.6 to 6.6.1, the only fixed release, following fix(deps): resolve pip-audit failures for tornado and pyasn1 #146. Python 3.10 keeps 6.5.10; 6.5.x has no backport.
-    "tornado>=6.6,<7; python_version >= '3.11'",  # 6.6 fixes CVE-2026-115927, CVE-2026-116577, CVE-2026-116578
+    "tornado>=6.6.1,<7; python_version >= '3.11'",  # 6.6.1 fixes CVE-2026-126347
  • Lock: only the tornado 6.6 block on Python >= 3.11 changes, to 6.6.1. No other package moves.
  • Verified:
    • The advisory is gone from both audits.
    • tests/unit/test_widget_assets.py passes.
    • 6.6.1's other changes are a RedirectHandler fix that restores the 6.5 behaviour, and a StaticFileHandler fix that applies only on Windows.

soupsieve

Advisories on the locked 2.9.2:

Advisory Severity Problem Fixed in
GHSA-mmhr-3c5m-f64f / CVE-2026-126190 high descendant combinator, polynomial time 4.0
GHSA-6655-6qjp-jx7x / CVE-2026-126191 high ReDoS in selector compilation 3.0.2
GHSA-7g44-g9rc-x53w / CVE-2026-126193 medium uncontrolled recursion in the CSS parser 3.0.2
GHSA-4j63-6c4x-wfg4 / CVE-2026-126192 medium :has(a ~ b), quadratic time 2.10

Path: nbconvert → beautifulsoup4 4.12.3 (requires soupsieve >1.2) → soupsieve. pyproject.toml pinned >=2.8.4,<3 in #118.

Fix: soupsieve 3.0.2 and later require CPython 3.11.5. Their selector parser relies on possessive quantifiers in re, which are broken before 3.11.5; 3.0 and 3.0.1 were yanked for that reason. The constraint is therefore split at python_full_version 3.11.5:

  • Python >= 3.11.5 gets soupsieve 4.1.2.
    • 4.1.2 fixes all four advisories above and CVE-2026-126189.
    • It also fixes four newer advisories that so far exist only in soupsieve's repository: GHSA-f3mv-fm3p-593v, GHSA-98qm-p6jx-5wj2 (high), GHSA-gcq2-p324-qj2v (high) and GHSA-3f94-v2f2-4x3g. They are not yet in GitHub's global advisory database or in OSV.
    • The floor is 4.1.2 instead of 4.0 so that the audit doesn't fail again once those advisories reach its data.
  • Python < 3.11.5 gets soupsieve 2.10, which fixes CVE-2026-126192.
-    "soupsieve>=2.8.4,<3",
+    "soupsieve>=4.1.2,<5; python_full_version >= '3.11.5'",  # 4.1.2 fixes CVE-2026-126189..126193 and 4 newer advisories, such as GHSA-gcq2-p324-qj2v (transitive via beautifulsoup4)
+    "soupsieve>=2.10,<3; python_full_version < '3.11.5'",  # 2.10 fixes CVE-2026-126192; soupsieve 3.0.2+ requires Python 3.11.5

Kernel bundles: a bundle is frozen with pip freeze on the builder's Python (dockerfiles/builder/bundle.sh). The installer then picks a bundle by major.minor version only (installer/module/downloader.py, helper.py get_current_python_version). The pinned builder image ships CPython 3.11.16, so the 3.11 bundle gets soupsieve 4.1.2 and is installed on every 3.11 runtime. Deepnote's 3.11 runtime image is on CPython 3.11.5 or newer. A runtime on CPython 3.11.0–3.11.4 using this bundle would get a soupsieve that rejects every selector; BeautifulSoup(...).select("p") raises SelectorSyntaxError on 3.11.4.

Lock: the single soupsieve 2.9.2 block becomes two blocks, 2.10 for python_full_version < "3.11.5" and 4.1.2 for python_full_version >= "3.11.5". beautifulsoup4 stays at 4.12.3. Poetry's relock also reorders the and terms in the markers of jeepney, ptyprocess, pywin32-ctypes and secretstorage. The old and new markers evaluate the same in every environment, so those blocks were not copied.

Behaviour change on Python 3.11.5+: soupsieve 4 changes how some selectors in user notebooks behave.

  • A :has() that contains combinator chains, such as div:has(section p), now raises SelectorSyntaxError. Passing flags=soupsieve.NOSTRICT restores the old result.
  • <template> contents are no longer matched.

Nothing in the toolkit uses bs4 or soupsieve directly.

Verified:

  • The soupsieve advisories are gone from both audits.
  • bs4 4.12.3's CSS API and nbconvert's HTML export work with 2.10 on Python 3.10, and with 4.1.2 on 3.11.17, 3.12, 3.13 and 3.14.
  • CI's wheel install check, which installs without the lock and imports the toolkit, passes on 3.10 (soupsieve 2.10) and on 3.11.17, 3.12 and 3.14 (4.1.2). On 3.11.4, pip resolves soupsieve to 2.10.

Decisions

Checks

  • poetry check --lock (Poetry 2.2.0) passes.
  • The three CI audit commands pass, run with the ignore-vulns list parsed from ci.yml (unchanged from main):
    • Production: "No known vulnerabilities found, 3 ignored".
    • Server: "No known vulnerabilities found, 3 ignored".
    • Dev: "No known vulnerabilities found".
  • The lock blocks copied for dnspython, soupsieve and tornado, and the content-hash, are byte-identical to a Poetry 2.2.0 relock of this pyproject.toml. The soupsieve wheel and sdist hashes match PyPI.
  • Unit tests on Python 3.11.17, installed from the new lock with the server extra and the dev group (tornado 6.6.1, dnspython 2.9.2, soupsieve 4.1.2): 1464 passed and 46 failed. main with its own lock has the same 46 failures on Python 3.11.17 with the same extra and group. They are pyspark tests that fail with PYTHON_VERSION_MISMATCH, because the Spark workers start Python 3.13 under the 3.11 driver, and test_databricks_connector_dialect_alias_is_registered.

Not fixed

…sieve

- dnspython 2.9.2 fixes CVE-2026-128089 and tornado 6.6.1 fixes
  CVE-2026-126347. Both need Python 3.11, so the floors are raised on the
  >= 3.11 lines and 3.10 keeps dnspython 2.8.0 and tornado 6.5.10.
- soupsieve 4.1.2 fixes CVE-2026-126189..126193 and four newer advisories
  published only in soupsieve's repo. soupsieve 3.0.2+ requires CPython
  3.11.5 and fails every selector below it. The 3.11 kernel bundle is
  frozen on the builder's latest 3.11 but installed on any 3.11 patch
  release, so 4.1.2 is limited to Python >= 3.12. Python < 3.12 moves to
  2.10, which fixes CVE-2026-126192, and the audit ignores the three CVEs
  2.10 still carries.

Only the dnspython, soupsieve and tornado blocks and the content-hash
were spliced into the lock, since a Poetry 2.2.0 relock also drops about
1,200 unrelated older-Python lines.
@coderabbitai

coderabbitai Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The dnspython and tornado minimum versions increase for Python 3.11 and newer. The soupsieve constraint is split by Python full version: Python 3.11.5 and newer use >=4.1.2,<5, and earlier versions use >=2.10,<3.

Estimated code review effort: 1 (Trivial) | ~5 minutes

Possibly related PRs

Suggested reviewers: mfranczel

Merge Risk: Minimal · up to 5c420

This PR raises dependency minimums to address security advisories for dnspython, tornado and soupsieve. No in-repo code relies on the changed soupsieve selector behavior, so merge risk is minimal. Python 3.10 environments remain exposed to some advisories, as the PR states.

Pre-merge checks: Passed 5

Check name Status Explanation
Title check Passed Title is specific and matches the changeset: it names the dependency floor fixes for dnspython, tornado and soupsieve, which the pyproject.toml changes implement. It follows the fix(deps) convention u…
Description check Passed Description is detailed and on-topic. It covers the dnspython, tornado and soupsieve floor changes, the Python-version splits, the lock update, advisories, verification steps, and known unfixed Python…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Updates Docs Passed The check is about documenting implemented features. This PR adds no feature. The changed-file inventory has two files: poetry.lock and pyproject.toml. The changes raise dependency floors for dnsp…
Full details: Title check

Explanation

Title is specific and matches the changeset: it names the dependency floor fixes for dnspython, tornado and soupsieve, which the pyproject.toml changes implement. It follows the fix(deps) convention used in the referenced precedent PRs.

Full details: Description check

Explanation

Description is detailed and on-topic. It covers the dnspython, tornado and soupsieve floor changes, the Python-version splits, the lock update, advisories, verification steps, and known unfixed Python 3.10 exposure.

Full details: Updates Docs

Explanation

The check is about documenting implemented features. This PR adds no feature. The changed-file inventory has two files: poetry.lock and pyproject.toml. The changes raise dependency floors for dnspython, tornado and soupsieve, and split soupsieve by python_full_version. The PR description says nothing about user-facing behavior in the toolkit. It says the toolkit does not use bs4 or soupsieve directly. No documentation update is needed for this change. The check is inapplicable, so it passes. Note: the PR does describe soupsieve behavior changes for user notebooks (:has() strictness, &lt;template&gt; matching). Those changes come from the upstream library, not from a toolkit feature. The checked repository contains no docs changes for them. Reviewers may still want a release note. The check does not require one.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Essentials
  • Run ID: fcbc8554-4483-4c64-91fe-b30375ed7017
Commits

Reviewing files that changed from the base of the PR and between 0b0d0ab and 5c420bb.

Files ignored due to path filters: 1

  • poetry.lock is excluded by !**/*.lock
Files selected for processing: 1

  • pyproject.toml

Included review availability: This review used your included allowance. 4 included reviews remain after this review. Your included PR review attempts over the past 7 days set your current allowance at 5 reviews per hour.

@github-actions

github-actions Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

📦 Python package built successfully!

  • Version: 2.9.1.dev5+dfbe5fa
  • Wheel: deepnote_toolkit-2.9.1.dev5+dfbe5fa-py3-none-any.whl
  • Install:
    pip install "deepnote-toolkit @ https://deepnote-staging-runtime-artifactory.s3.amazonaws.com/deepnote-toolkit-packages/2.9.1.dev5%2Bdfbe5fa/deepnote_toolkit-2.9.1.dev5%2Bdfbe5fa-py3-none-any.whl"

@codecov

codecov Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 78.43%. Comparing base (0b0d0ab) to head (5c420bb).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #156   +/-   ##
=======================================
  Coverage   78.43%   78.43%           
=======================================
  Files         121      121           
  Lines        6920     6920           
  Branches     1005     1005           
=======================================
  Hits         5428     5428           
  Misses       1182     1182           
  Partials      310      310           
Flag Coverage Δ
combined 78.43% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

@deepnote-bot

deepnote-bot commented Nov 19, 2026 •

Copy link
Copy Markdown

🚀 Review App Deployment Started

📝 Description 🌐 Link / Info
🌍 Review application ra-156
🔑 Sign-in URL Click to sign-in
📊 Application logs View logs
🔄 Actions Click to redeploy
🚀 ArgoCD deployment View deployment
⏰ Last deployed 2026-11-19 10:02:07 (UTC)
📜 Deployed commit 3ffa0c5ac742f8771958cce91c5b37726ae5bb36
🛠️ Toolkit version dfbe5fa

Deepnote's 3.11 runtime image is on CPython 3.11.5 or newer, and the
bundle builder's 3.11 is 3.11.16, so the 3.11 kernel bundle can take
soupsieve 4.1.2 instead of 2.10. The split moves from Python 3.12 to
3.11.5, the first release whose re module has working possessive
quantifiers, which soupsieve 3.0.2+ requires.

With 3.11 on 4.1.2, the audit no longer needs to ignore
CVE-2026-126190, CVE-2026-126191 and CVE-2026-126193, so ci.yml is back
to main's version.

Only the soupsieve blocks and the content-hash change in the lock.
@tkislan
tkislan marked this pull request as ready for review November 19, 2026 10:36
@tkislan
tkislan requested a review from a team as a code owner November 19, 2026 10:36
@tkislan
tkislan requested review from m1so and mfranczel November 19, 2026 10:36
@tkislan
tkislan merged commit cca84cb into main Nov 19, 2026
45 checks passed
@tkislan
tkislan deleted the tk/fix-pip-audit-2026-11-19 branch November 19, 2026 10:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants