Resolve open Dependabot alerts by upgrading vulnerable dependencies

AGENTS.md

@@ -64,7 +64,7 @@ steerability/
 │   │   └── common/             # building blocks: drivers, processors, scorers, values, criteria, kv_cache,
 │   │                            # candidates
 │   └── structural_control/      # load_checkpoint/ and load_lora/ (artifact loaders; frozen forms of trained
-│       └── wrappers/            # structural controls); wrappers/: trl/ (sft, dpo, ppo, grpo, apo) and mergekit/
+│       └── wrappers/            # structural controls); wrappers/: trl/ (sft, dpo, grpo, apo) and mergekit/
 ├── spipe/                       # .spipe serialization: SPipe, manifest format, value codec,
 │                                # content-addressed artifact store, freeze orchestration
 ├── backends/                    # huggingface/ (HFBackend, ExclusiveSession); vllm/ (VLLMBackend, VLLMServeBackend)
@@ -100,15 +100,13 @@ Optional extras, in three tiers:
   stays inside trl's supported vLLM range
 - workflows: `eval`, the Inspect AI evaluation stack plus matplotlib and seaborn for `evaluation/plotting.py`
 - method-specific: `merging` (MergeKit)
-- `all`: `eval`
+- `all`: `eval` and `merging`
 
 Contributor tooling lives in `[dependency-groups]`: `dev` (pytest, pre-commit, the plugin core, and the `notebooks`
 group), `notebooks` (notebook, ipywidgets, textstat, nltk), and `docs` (site tooling). `uv sync` installs `dev` by
 default; add `--group docs` or `--group notebooks` as needed.
 
-`merging` cannot share an environment with `eval` (MergeKit pins an older pydantic than Inspect requires), so it stays
-out of `all` and `vllm`; `pyproject.toml` declares these as `[tool.uv] conflicts`. The optional-module-to-extra mapping
-lives in `OPTIONAL_MODULE_EXTRAS` (`steerability/utils/optional.py`).
+The optional-module-to-extra mapping lives in `OPTIONAL_MODULE_EXTRAS` (`steerability/utils/optional.py`).
 
 Hugging Face access uses the standard mechanism: `hf auth login` once, or `HF_TOKEN=hf_***` in the environment. Some
 models (e.g. `meta-llama/*`) are gated; the account behind the token needs access on the model's Hub page. Never commit
@@ -202,8 +200,7 @@ The registered names at the time of writing:
   `deal`, `dexperts`, `phased_decoding`, `rad`, `routed_decoding`, `sasa`, `search_decoding`, `stopping_rules`,
   `value_guidance`
 - structural: `load_checkpoint`, `load_lora` (artifact loaders, also the frozen forms of trained structural controls
-  in a `.spipe`), `mergekit`, `sft`, `dpo`, `ppo`, `grpo`, `apo` (MergeKit and TRL wrappers; `ppo` is registered
-  only when the installed TRL provides `trl.experimental.ppo`)
+  in a `.spipe`), `mergekit`, `sft`, `dpo`, `grpo`, `apo` (MergeKit and TRL wrappers)
 
 ### Pipeline semantics
 

docs/concepts/controls.md

@@ -105,7 +105,7 @@ around existing libraries. The toolkit implements:
     - *Description*: model merging via MergeKit[@goddard-etal-2024-arcees], combining multiple checkpoints with strategies such as linear interpolation, SLERP, and TIES from a YAML/dict config.
     - *Backends*: HF, vLLM (the merged checkpoint is served).
 - `TRL` ([API reference](../reference/algorithms/structural_control/trl_wrapper.md), [notebook](../examples/notebooks/algorithms/wrappers/trl.ipynb))
-    - *Description*: weight-level training via Hugging Face TRL[@vonwerra2022trl], exposing SFT, DPO, APO, PPO, and GRPO trainers, with optional LoRA/PEFT and a post-training merge. The `ppo` control is registered only when the installed TRL provides `trl.experimental.ppo` (discovery otherwise skips it with a logged hint). Since `training_args` is forwarded verbatim to the installed TRL config, a key the config does not declare raises an error at control construction. A `target_modules` list of module-name suffixes is scoped at steer time to the decoder stack of the resolved model layout, so a multimodal wrapper's vision and audio towers are not adapted; a regex targets other modules.
+    - *Description*: weight-level training via Hugging Face TRL[@vonwerra2022trl], exposing SFT, DPO, APO, and GRPO trainers, with optional LoRA/PEFT and a post-training merge. Since `training_args` is forwarded verbatim to the installed TRL config, a key the config does not declare raises an error at control construction. A `target_modules` list of module-name suffixes is scoped at steer time to the decoder stack of the resolved model layout, so a multimodal wrapper's vision and audio towers are not adapted; a regex targets other modules.
     - *Backends*: HF, vLLM (serves the steer-time artifact, a checkpoint or LoRA adapter; a merged LoRA run needs `merged_output_dir`).
 
 

docs/home/installation.md

@@ -45,15 +45,13 @@ grouped in three tiers:
 - Backends: `vllm`, the vLLM execution backends (offline engine or server) plus the `vllm_hook_plugins` core. This
   extra pulls in `trl[vllm]` such that the resolved vLLM version stays inside TRL's supported range.
 - Workflows: `eval`, the Inspect AI evaluation stack and the plotting utilities in `evaluation/plotting.py`.
-- Method-specific: `merging`, the MergeKit structural control, isolated because MergeKit pins an older pydantic than
-  Inspect requires.
+- Method-specific: `merging`, the MergeKit structural control.
 
 Constrained decoding on the Hugging Face backend uses xgrammar, which is a core dependency; on vLLM backends the
 constraint lowers to native structured outputs.
 
-The umbrella `all` extra currently installs `eval`; it is the stable name for every extra that can share one
-environment. Install `merging` and `vllm` by name, e.g., `uv pip install '.[vllm]'`. Note that `merging` cannot share
-an environment with `eval`; `pyproject.toml` declares this as a `[tool.uv]` conflict.
+The umbrella `all` extra installs `eval` and `merging`; it is the stable name for every extra that can share one
+environment on every platform. Install `vllm` by name, e.g., `uv pip install '.[vllm]'`.
 
 Contributors install with `uv sync --extra all`, which creates the environment, installs the toolkit in editable mode,
 and adds the `dev` dependency group (pytest, pre-commit, notebook tooling). Add `--group docs` to build the

pyproject.toml

@@ -51,9 +51,9 @@ dependencies = [
 # optional extras: vllm, eval, merging
 [project.optional-dependencies]
 vllm = [
-    "vllm>=0.26.0,<1.0.0",
+    "vllm>=0.32.0,<1.0.0",
     # keeps the resolved vLLM inside trl's supported range
-    "trl[vllm]",
+    "trl[vllm]>=1.17.0",
     "vllm-hook-plugins @ git+https://github.com/IBM/vLLM-Hook.git@steerability-challenge#subdirectory=vllm_hook_plugins",
 ]
 eval = [
@@ -63,20 +63,18 @@ eval = [
     "seaborn>=0.13.0,<0.14.0",
     "tqdm>=4.66.5,<5.0.0",
 ]
-# pinned main commit: 0.1.4 on PyPI never calls `model_rebuild()` on its torch-typed
-# pydantic models and fails at merge time
 merging = [
-    "mergekit @ git+https://github.com/arcee-ai/mergekit.git@a6e402884ba9bc30da7f23e8304a35f19485de95",
+    "mergekit>=0.1.5,<0.2.0",
 ]
 # every extra that coexists in one environment on every platform
 all = [
-    "steerability[eval]",
+    "steerability[eval,merging]",
 ]
 
 # contributor tooling; `uv sync` installs `dev` by default
 [dependency-groups]
 dev = [
-    "pytest>=8.3.2,<9.0.0",
+    "pytest>=9.0.3,<10.0.0",
     "pre-commit>=4.3.0",
     "vllm-hook-plugins @ git+https://github.com/IBM/vLLM-Hook.git@steerability-challenge#subdirectory=vllm_hook_plugins",
     { include-group = "notebooks" },
@@ -107,13 +105,6 @@ docs = [
 
 # tools
 [tool.uv]
-# mergekit pins pydantic~=2.10, which cannot import under the pydantic>=2.13 that inspect-ai
-# requires; its safetensors pin (~=0.5) conflicts with vllm's (>=0.6.2)
-conflicts = [
-    [{ extra = "merging" }, { extra = "eval" }],
-    [{ extra = "merging" }, { extra = "all" }],
-    [{ extra = "merging" }, { extra = "vllm" }],
-]
 # aiohttp is a transitive dependency; releases before 3.14.4 have unbounded memory/CPU use
 # (GHSA-2g87-pp6c-29x5)
 constraint-dependencies = ["aiohttp>=3.14.4"]

steerability/algorithms/input_control/cpo/args.py

@@ -47,11 +47,8 @@ class CPOArgs(BaseArgs):
     )
 
     trust_remote_code: bool = field(
-        default=True,
-        metadata={"help": (
-            "Trust remote code when loading `embedding_model`. Defaults to True for the default "
-            "encoder, which ships custom modeling code. Set False for standard encoders."
-        )},
+        default=False,
+        metadata={"help": "Trust remote code when loading `embedding_model`."},
     )
 
     pca_query_dim: int = field(

steerability/algorithms/input_control/cpo/control.py

@@ -95,7 +95,7 @@ class CPO(InputControl):
     offline_data: list[dict] | None = None
     n_prompts_per_query: int = 200
     embedding_model: str = "nomic-ai/nomic-embed-text-v1.5"
-    trust_remote_code: bool = True
+    trust_remote_code: bool = False
     pca_query_dim: int = 40
     pca_prompt_dim: int = 15
     prompt_lm: Any = None

steerability/algorithms/input_control/cpo/utils/embeddings.py

@@ -23,7 +23,7 @@ def __init__(
         self,
         model_name_or_path: str,
         device: str | torch.device | None = None,
-        trust_remote_code: bool = True,
+        trust_remote_code: bool = False,
     ) -> None:
         self.tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, trust_remote_code=trust_remote_code)
         self.model = AutoModel.from_pretrained(model_name_or_path, trust_remote_code=trust_remote_code)

steerability/algorithms/output_control/constrained_decoding/utils/automaton.py

@@ -1,7 +1,7 @@
 """Client-side automaton compilation for declarative constraints, over xgrammar.
 
-Json-schema constraints compile compact (`any_whitespace=False`) so the grammar matches the
-whitespace policy applied on every venue.
+Json-schema constraints compile compact (`any_whitespace=False` with `","` and `":"` separators) so the
+grammar matches the whitespace policy applied on every venue.
 """
 from __future__ import annotations
 
@@ -364,7 +364,7 @@ def compile_constraint_automaton(source: ConstraintSource, tokenizer: PreTrained
     if source.kind == "json_schema":
         schema = source.value if isinstance(source.value, str) else json.dumps(dict(source.value))
         # compile compact on every venue rather than inheriting each backend's whitespace default
-        compiled = compiler.compile_json_schema(schema, any_whitespace=False)
+        compiled = compiler.compile_json_schema(schema, any_whitespace=False, separators=(",", ":"))
     elif source.kind == "regex":
         compiled = compiler.compile_regex(source.value)
     elif source.kind == "grammar":

steerability/algorithms/structural_control/wrappers/trl/__init__.py

@@ -6,7 +6,6 @@
 - **SFT (Supervised Fine-Tuning)**: Standard supervised learning to fine-tune language models on demonstration data
 - **DPO (Direct Preference Optimization)**: Trains models directly on preference data without requiring a separate reward model
 - **APO (Anchored Preference Optimization)**: A variant of DPO that uses an anchor model to improve training stability and performance
-- **PPO (Proximal Policy Optimization)**: Reinforcement learning against a sequence-classification reward model (and value model)
 - **GRPO (Group-Relative Policy Optimization)**: Critic-free reinforcement learning that takes a callable reward function (no reward model and no value model); used by PRewrite for its metric-in-the-loop reward
 
 For documentation information, please refer to the [TRL page](https://huggingface.co/docs/trl/index).

steerability/algorithms/structural_control/wrappers/trl/args.py

@@ -49,7 +49,7 @@ class TRLArgs(BaseArgs):
     logging_steps: int = 10
     report_to: str | None = None
     seed: int | None = None
-    resume_from_checkpoint: str | None = None  # checkpoint directory passed to trainer.train(); unsupported by PPO
+    resume_from_checkpoint: str | None = None  # checkpoint directory passed to trainer.train()
 
     # PEFT knobs
     use_peft: bool = False

steerability/algorithms/structural_control/wrappers/trl/base_mixin.py

@@ -22,7 +22,7 @@ def resolve_config_kwargs(config_cls: type, training_args: dict[str, Any]) -> di
     does not declare would otherwise be passed to a constructor that does not accept it.
 
     Args:
-        config_cls: A TRL config dataclass (`DPOConfig`, `SFTConfig`, `GRPOConfig`, `PPOConfig`).
+        config_cls: A TRL config dataclass (`DPOConfig`, `SFTConfig`, `GRPOConfig`).
         training_args: The composed training arguments.
 
     Returns: