feat(scorers): validate model parameters

[Cedric / ViaDézo1er (viadezo1er)]

Validate scorer parameters against shared and custom model capabilities

• resolve project custom models before org models and the shared catalog
• cache model metadata for 24 hours with miss refresh and stale fallback
• normalize provider-specific parameter names

It's not very elegant because this is copying the parameter validation from the backend instead of exposing it in the backend and making a call to check it with bt. However, the backend pr isn't ready yet and this allows bt scorers create to be functionnal in the meantime.

When giving an incorrect name, there's a warning but the scorer is created:

❯ bt scorers create "Demo answer correctness" \
  --slug demo-answer-correctness --refresh-models \
  --description "Synthetic scorer used to demonstrate bt scorers create" \
  --model gpt-5.4-nanoawesrdfg \
  --messages @/tmp/bt-scorer-messages.json \
  --choice-scores '{"PASS":1,"FAIL":0}' \
  --use-cot=false \
  --pass-threshold 0.5 \
  --metadata 'purpose: synthetic-demo' \
  --if-exists replace --json --temperature 1 --reasoning-effort none|jq
! Model 'gpt-5.4-nanoawesrdfg' was not found in the shared catalog or configured custom models; checked only basic value ranges. Check the model ID or pass --refresh-models if it was recently configured.
{
  "found_existing": true,
  "id": "6681c294-0ab3-489b-b979-33351add071d",
  "ignored": false,
  "project_id": "b667bd4c-7e55-49e0-a370-f6526e6bbccd",
  "slug": "demo-answer-correctness",
  "version": "1000197689226740483"
}

Some parameters can be checked regardless of model:

❯ bt scorers create "Demo answer correctness" \
  --slug demo-answer-correctness \
  --description "Synthetic scorer used to demonstrate bt scorers create" \
  --model gpt-5.4-nanoawesrdfg \
  --messages @/tmp/bt-scorer-messages.json \
  --choice-scores '{"PASS":1,"FAIL":0}' \
  --use-cot=false \
  --pass-threshold 0.5 \
  --metadata 'purpose: synthetic-demo' \
  --if-exists replace --json --temperature 1000 --reasoning-effort none|jq
{
  "error": {
    "message": "--temperature must be between 0 and 2"
  }
}

Some parameters are model-specific:

❯ bt scorers create "Demo answer correctness" \
  --slug demo-answer-correctness --refresh-models \
  --description "Synthetic scorer used to demonstrate bt scorers create" \
  --model gpt-5.4-nano \
  --messages @/tmp/bt-scorer-messages.json \
  --choice-scores '{"PASS":1,"FAIL":0}' \
  --use-cot=false \
  --pass-threshold 0.5 \
  --metadata 'purpose: synthetic-demo' \
  --if-exists replace --json --temperature 1 --reasoning-effort high|jq
{
  "error": {
    "message": "--temperature is not supported by model 'gpt-5.4-nano' unless reasoning effort is 'none'; pass `--reasoning-effort none` or omit `--temperature`"
  }
}

[github-actions]

Latest downloadable build artifacts for this PR commit aec769b7c40a:

• Workflow run: https://github.com/braintrustdata/bt/actions/runs/31857602981
• Download all artifacts (GitHub CLI): gh run download 31857602981 --repo braintrustdata/bt
• Installers are published from main automatically. To publish one for a PR branch, run release-canary manually via workflow_dispatch.

Available artifact names

• artifacts-build-global
• artifacts-build-local-x86_64-apple-darwin
• artifacts-build-local-aarch64-pc-windows-msvc
• artifacts-build-local-x86_64-pc-windows-msvc
• artifacts-build-local-x86_64-unknown-linux-musl
• artifacts-build-local-x86_64-unknown-linux-gnu
• artifacts-build-local-aarch64-apple-darwin
• artifacts-build-local-aarch64-unknown-linux-gnu
• artifacts-plan-dist-manifest
• cargo-dist-cache