Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 0 additions & 47 deletions .github/hooks/deval-hook.cjs

This file was deleted.

36 changes: 0 additions & 36 deletions .github/hooks/deval.json

This file was deleted.

62 changes: 0 additions & 62 deletions .github/workflows/ci.yml

This file was deleted.

16 changes: 16 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,22 @@ All notable changes to the `deval` extension are documented here.

Check [Keep a Changelog](http://keepachangelog.com/) for recommendations on how to structure this file.

## [Unreleased]

### Added

- `@deval` is now a real tool-calling agent instead of a chat forwarder. It runs an agentic loop (`src/agent/loop.ts`) against the Chat model with three read-only tools — `deval_readFile`, `deval_listFiles`, `deval_grep` — declared in `package.json` and registered via `vscode.lm`.
- `src/events/event-writer.ts` — writes agent-loop runs as JSONL captures in the format `event-log.ts` reads, so `DeepEval: Evaluate Captured Agent Run` scores the agent's trajectory with no extra wiring.
- `benchmarks/agent-loop-benchmark.test.ts` — deterministic suite driving the loop with a scripted model through `observe()`, asserting with `ContainsAllMetric` and `ToolCorrectnessMetric`.
- Unit tests for the loop core (fake model/tool ports) and the capture writer.

### Removed

- The reference agent (`src/agent-harness.ts`, `benchmarks/todo-benchmark.test.ts`) — the `@deval` agent loop is the system under test now. The `AgentRun` contract moved into `src/agent/loop.ts`.
- The Local-harness capture pipeline: `src/capture-hook.ts`, `.github/hooks/`, and the `DeepEval: Install Capture Hook` command. Captures are written automatically by the agent loop.
- The `DeepEval: Scaffold Sample Benchmark` command.
- The GitHub Actions CI workflow; the README documents the local verification commands instead.

## [0.1.0]

First complete starter release.
Expand Down
Loading