Harden GhostTrace evidence accuracy and release gates

.github/workflows/ci.yml

@@ -0,0 +1,37 @@
+name: GhostTrace quality gates
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+
+permissions:
+  contents: read
+
+jobs:
+  verify:
+    runs-on: ubuntu-latest
+    timeout-minutes: 10
+    steps:
+      - uses: actions/checkout@v5
+      - uses: actions/setup-python@v6
+        with:
+          python-version: "3.11"
+      - name: Install local evidence extractors
+        run: |
+          sudo apt-get update
+          sudo apt-get install --yes poppler-utils tesseract-ocr
+      - name: Compile
+        run: python -m compileall -q code
+      - name: Test
+        run: python -m unittest discover -s code/tests -v
+      - name: Generate and validate all decisions
+        run: |
+          python code/main.py run
+          python code/main.py validate
+      - name: Evaluate public sample
+        run: python code/evaluation/main.py
+      - name: Verify submission package
+        run: |
+          python code/package_submission.py
+          unzip -t code.zip

.gitignore

@@ -21,4 +21,6 @@ code.zip
 output.csv
 *.log
 !log.txt
+!dataset/evidence/logs/
+!dataset/evidence/logs/*.log
 

README.md

@@ -105,15 +105,15 @@ On the 32 participant-visible labeled cases:
 
 | Metric | Result |
 | --- | ---: |
-| Case status accuracy | 90.625% |
-| Exact row / exact sequence accuracy | 18.750% |
-| Step-field accuracy | 66.062% |
-| Step precision | 62.346% |
-| Step recall | 52.332% |
+| Case status accuracy | 100.000% |
+| Exact row / exact sequence accuracy | 50.000% |
+| Step-field accuracy | 78.792% |
+| Step precision | 79.651% |
+| Step recall | 70.984% |
 
-The three status misses are deliberately fail-closed cases where the visible evidence does not provide a subject-to-resource link. GhostTrace does not use the public labels as prediction shortcuts. Full methodology and limitations are documented in [`code/evaluation/evaluation_report.md`](code/evaluation/evaluation_report.md).
+GhostTrace reaches every participant-visible sample verdict without reading labels at prediction time. Remaining exact-sequence gaps are concentrated in canonical resource links that are not present in cutoff-eligible participant evidence; the engine does not invent those links. Full methodology and limitations are documented in [`code/evaluation/evaluation_report.md`](code/evaluation/evaluation_report.md).
 
-The final full-data run currently produces 142 `COMPLETE`, 366 `INCOMPLETE`, and 10 `NOT_APPLICABLE` decisions. Its `output.csv` passes the strict validator with 518/518 rows. Step count is bounded after parser hardening (median 5, 95th percentile 12, maximum 14), preventing service instance IDs from being promoted to user resources.
+The final full-data run currently produces 143 `COMPLETE`, 365 `INCOMPLETE`, and 10 `NOT_APPLICABLE` decisions. Its `output.csv` passes the strict validator with 518/518 rows. Step count is bounded after parser hardening (median 5, 95th percentile 12, maximum 14), preventing service instance IDs from being promoted to user resources.
 
 ## Security and privacy posture
 

code/evaluation/evaluation_report.md

@@ -16,21 +16,21 @@ Environment: Python 3.14.0, deterministic local execution, 32 public sample case
 
 | Level | Definition | Result |
 | --- | --- | ---: |
-| Case accuracy | Correct `deletion_status` | 29/32 = **90.625%** |
-| Row accuracy | Status and complete ordered step array exactly match | 6/32 = **18.750%** |
-| Field accuracy | Micro accuracy across seven aligned step fields; missing/extra steps count as seven misses | **66.062%** |
-| Sequence accuracy | Exact ordered `(phase, system, resource, operation, state)` sequence | 6/32 = **18.750%** |
-| Step precision | Exact predicted step objects also present in target | **62.346%** |
-| Step recall | Exact target step objects recovered | **52.332%** |
+| Case accuracy | Correct `deletion_status` | 32/32 = **100.000%** |
+| Row accuracy | Status and complete ordered step array exactly match | 16/32 = **50.000%** |
+| Field accuracy | Micro accuracy across seven aligned step fields; missing/extra steps count as seven misses | **78.792%** |
+| Sequence accuracy | Exact ordered `(phase, system, resource, operation, state)` sequence | 16/32 = **50.000%** |
+| Step precision | Exact predicted step objects also present in target | **79.651%** |
+| Step recall | Exact target step objects recovered | **70.984%** |
 
-Status accuracy is the primary readiness gate because it measures whether GhostTrace reaches the correct deletion conclusion. Exact sequence is lower because the participant-visible evidence does not consistently expose every subject-to-resource edge implied by the sample plan. The runtime intentionally refuses to recover those edges from labels.
+Status accuracy is the primary readiness gate because it measures whether GhostTrace reaches the correct deletion conclusion. The accuracy-hardening release raises verdict accuracy from 90.625% to 100%, exact rows from 18.750% to 50%, precision from 62.346% to 79.651%, and recall from 52.332% to 70.984%. Exact sequence remains lower because the participant-visible evidence does not consistently expose every subject-to-resource edge implied by the sample plan. The runtime intentionally refuses to recover those edges from labels.
 
 ## Full-dataset contract evaluation
 
 The production command emitted 518/518 canonically sorted rows and passed every validator rule:
 
 - Exact header and one row per request.
-- 142 `COMPLETE`, 366 `INCOMPLETE`, 10 `NOT_APPLICABLE`.
+- 143 `COMPLETE`, 365 `INCOMPLETE`, 10 `NOT_APPLICABLE`.
 - Valid JSON arrays and consecutive step numbering.
 - Legal status, phase, operation, and state combinations.
 - Evidence on `COMPLETE` verification and `INCOMPLETE` findings.
@@ -70,10 +70,15 @@ Generate the entire 518-row file in a temporary directory and run the strict val
 3. **Trace grouping ambiguity.** Earlier grouping searched every token in a trace for resource-like values. It now accepts only the explicit resource field while still combining identity and terminal-state lines from the same trace.
 4. **Non-deletion coverage.** The intent gate was expanded to cover multilingual requests and explicit negation/limited-scope boundaries, then checked against all ten public non-deletion cases.
 5. **Silent uncertainty.** Evidence-free authorized requests now fail closed as `INCOMPLETE` with a warning rather than being declared complete.
+6. **Policy-category ambiguity.** Support attachments, legacy contact indexes, and marketing contacts formerly inherited policy from only their resource prefix. Source semantics now distinguish delete, redact, and anonymize requirements.
+7. **Ambiguous OCR ownership.** An OCR table window could contain adjacent users. An eligible authoritative catalog link now overrides free-form OCR ownership.
+8. **Workflow/readback correlation.** Unique workflow targets can now be joined to independent cutoff-eligible absence or anonymization proof. The join is rejected when concurrent targets make attribution ambiguous.
+9. **Negative account proof.** A uniquely attributed account workflow can use a later, cutoff-eligible complete account snapshot as absence proof; activity success alone is never treated as proof.
+10. **Evidence integrity.** The validator now rejects missing finding citations and evidence IDs absent from every participant-visible evidence namespace.
 
 ## Remaining limitations
 
-Three public status misses (`case_01`, `case_02`, `case_04`) have canonical `COMPLETE` answers but no participant-visible, cutoff-eligible subject-to-resource connection that GhostTrace can substantiate. The system returns `INCOMPLETE` with no invented steps. This is an intentional audit-safety choice.
+All 32 public statuses now match. Sixteen exact-sequence gaps remain where the canonical plan includes additional resource relationships that are absent or ambiguous in cutoff-eligible participant evidence. For example, `account_004` appears in the canonical output but nowhere in the participant dataset outside that label. GhostTrace does not hardcode case IDs or synthesize such identifiers from naming coincidences; this preserves hidden-set generalization and audit integrity.
 
 OCR quality depends on the locally installed Tesseract build. Missing `pdftotext` or Tesseract is reported in diagnostics; the run does not silently pretend those files were parsed. Free-form OCR observations carry reduced confidence.
 
@@ -86,4 +91,4 @@ python3 code/main.py run
 python3 code/main.py validate
 ```
 
-Final regression result: **9/9 tests pass**.
+Final regression result: **17/17 tests pass**. The suite includes a manifest-completeness and extraction-health gate so a checkout cannot silently omit participant evidence.

code/ghosttrace/engine.py

@@ -104,6 +104,20 @@ def audit(self, case: Case) -> Decision:
             for observation in self.evidence.eligible(case.audit_at)
             if self.evidence.matches(observation, aliases, case.audit_at)
         ]
+        workflow_account = self.evidence.unambiguous_account_resource(case, aliases)
+        if workflow_account and not any(
+            item.resource_id == workflow_account for item in observations
+        ):
+            absence_proof = self.evidence.verified_account_readback(
+                workflow_account, case, aliases
+            ) or self.evidence.prove_account_absence(workflow_account, case)
+            if absence_proof is not None:
+                observations.append(absence_proof)
+        anonymization_proof = self.evidence.verified_unique_anonymization(case, aliases)
+        if anonymization_proof is not None and not any(
+            item.resource_id == anonymization_proof.resource_id for item in observations
+        ):
+            observations.append(anonymization_proof)
         observations = latest_by_resource(observations)
         observations = [item for item in observations if item.category in ACTION_BY_CATEGORY]
 
@@ -112,7 +126,24 @@ def audit(self, case: Case) -> Decision:
         # to the investigator UI through the EvidenceIndex.
         violations = [item for item in observations if not self._compliant(item)]
         status = "INCOMPLETE" if violations else "COMPLETE"
-        steps = self._steps(observations, status)
+        planned_verifications: list[Observation] = []
+        if (
+            status == "INCOMPLETE"
+            and workflow_account
+            and not any(item.resource_id == workflow_account for item in observations)
+        ):
+            planned_verifications.append(
+                Observation(
+                    system_id="account_service",
+                    resource_id=workflow_account,
+                    category="USER_PROFILE",
+                    state="ABSENT",
+                    observed_at=case.audit_at,
+                    evidence_id="",
+                    confidence=0.75,
+                )
+            )
+        steps = self._steps(observations, status, planned_verifications)
         warnings: list[str] = []
         if not observations:
             status = "INCOMPLETE"
@@ -133,7 +164,12 @@ def _compliant(observation: Observation) -> bool:
         action = ACTION_BY_CATEGORY[observation.category]
         return observation.state == TARGET_STATE[action]
 
-    def _steps(self, observations: list[Observation], status: str) -> list[Step]:
+    def _steps(
+        self,
+        observations: list[Observation],
+        status: str,
+        planned_verifications: list[Observation] | None = None,
+    ) -> list[Step]:
         raw: list[tuple[str, str, str, str, str, str]] = []
         if status == "INCOMPLETE":
             for observation in observations:
@@ -157,7 +193,11 @@ def _steps(self, observations: list[Observation], status: str) -> list[Step]:
                 raw.append(("FINDING", observation.system_id, observation.resource_id, finding[0], finding[1], observation.evidence_id))
                 raw.append(("REMEDIATION", observation.system_id, observation.resource_id, remediation[0], remediation[1], ""))
 
-        for observation in observations:
+        verification_observations = [
+            *observations,
+            *(planned_verifications or []),
+        ]
+        for observation in verification_observations:
             action = ACTION_BY_CATEGORY[observation.category]
             operation = {
                 "DELETE": "VERIFY_RECORD_ABSENT",