Structured expected/actual evidence on violations (#69)

docs/architecture/IMPLEMENTATION_STATUS.md

@@ -1060,3 +1060,37 @@ All under `rules/`, all illustrative (`Contoso.*` namespace, `illustrative: true
   `.claude` worktree double-discovery are all fixed, and `codeguard validate` against this repo
   completes end-to-end. Only the unproven multi-solution `(ProjectName, FullName)` edge case noted
   in gotcha #6 remains.
+
+### Post-v1 addition: structured expected/actual/evidence on violations (#69)
+
+`AssertionOutcome` gained optional `Expected`, `Actual` and `Evidence` (`IReadOnlyList<string>?`), all
+defaulting to `null`, plus a `Failure(message, expected, actual, evidence)` overload. `Violation` and
+`AnalyzerViolation` gained the same optional fields, and `RuleEvaluator.CreateViolation` copies them
+from the outcome. Existing constructors and call sites still compile, and `Message` text is unchanged.
+
+- Populated by the assertions in `CodeGuard.Evaluation.Assertions` wherever there is a natural value.
+  Examples: `must_inherit_from` gives the pattern as `expected`, the actual base type as `actual`, and
+  the type as evidence. The dependency assertions (`must_not_depend_on`, `must_only_depend_on`) give
+  the offending types or dependency names as evidence. The naming and namespace assertions give the
+  pattern as `expected` and the name as `actual`. The count and existence assertions give the expected
+  and actual counts, with the matched candidates as evidence.
+- Quantifiers aggregate nested outcomes. `expected` is kept only when every nested failure agrees on it.
+  `evidence` is the failing matches, each qualified by its nested evidence (`<match>: <item>`).
+- Evidence is capped at 50 entries (`CandidateDescriptor.MaxEvidenceEntries`, `CandidateDescriptor.CapEvidence`).
+  When truncated, the last entry is `...and N more`. The same cap applies to the list-valued `actual` values
+  (`ListOrNone`, `must_not_depend_on`'s matched names). `must_not_exist` materialises at most 51 candidates,
+  so its failure reports `more than 50 match(es)` and ends the evidence with `...and more`, since the total
+  is not counted. `must_any_match` makes one pass over its matches, evaluating each nested assertion at most
+  once per match. It stops at the first satisfying match, and within a match at the first failing assertion.
+- Helpers `CandidateDescriptor.ListOrNone` and `CandidateDescriptor.CommonValue` are internal to the
+  evaluation assembly. Assertions with no sensible structured value leave the fields `null`. These are
+  file/directory existence
+  checks, `must_have_modifier`/`must_not_have_modifier` (`actual` is absent), `must_depend_on`
+  (`actual` is absent), `must_use_package_version` with a missing package, and every "can only be
+  evaluated against X" misapplication failure. The file and directory existence checks give `expected`
+  only. The content checks give the pattern as `expected` and the file as evidence, with no `actual`.
+- Surfaced in the console reporter as indented `expected:`/`actual:`/`evidence:` lines, only when
+  present. The JSON v1 contract gains optional `expected`, `actual` and `evidence` properties on each
+  violation (`schemaVersion` stays `1.0`, see `RESULT_CONTRACT.md`). In SARIF they go into the result
+  property bag under the keys `expected`, `actual` and `evidence`. A `Metadata` key with the same name
+  takes precedence. The HTML reporter shows them under the message.

docs/architecture/RESULT_CONTRACT.md

@@ -60,15 +60,33 @@ object has `additionalProperties: false`, so a consumer can rely on the set of k
 }
 ```
 
-Optional fields (`file`, `line`, `column`, `endLine`, `endColumn`, `metadata`, `symbol`, `project`,
-`remediation`, and the optional fields of `evaluationErrors` and `analysisWarnings`) are omitted when
-unknown, not written as `null`.
+Optional fields (`file`, `line`, `column`, `endLine`, `endColumn`, `metadata`, `expected`, `actual`,
+`evidence`, `symbol`, `project`, `remediation`, and the optional fields of `evaluationErrors` and
+`analysisWarnings`) are omitted when unknown, not written as `null`.
 
 Per violation, `endLine` and `endColumn` give the end of the source span when the producer knows it
 (currently Roslyn-backed rules). `endColumn` is 1-based and exclusive, i.e. the column just past the last
 character, matching SARIF `region.endColumn`. `metadata` is an optional string-to-string bag for
 producer-specific details; no built-in rule populates it yet.
 
+Per violation, `expected`, `actual` and `evidence` are optional structured evidence (issue #69), so an
+agent can act on a failure without parsing `message`. They are present only when the failing assertion
+has a natural value for them, and `message` is unchanged either way:
+
+- `expected` (string): what the rule required, such as the required base type pattern or name regex.
+- `actual` (string): what was found instead, such as the actual base type or name. `(none)` when a
+  required member or relationship is absent.
+- `evidence` (array of strings): the offending references that caused the failure, such as type
+  full names, dependency names, file paths or project names. For quantifiers (`must_all_match`,
+  `must_any_match`, `must_none_match`), each entry is qualified by the match it came from
+  (`<match>: <item>`).
+
+`evidence` is capped at 50 entries. When a failure has more, the list holds the first 50 followed by a
+final entry `...and N more`, where N is the number omitted. The cap also bounds the evidence-like parts of
+`actual` for list-valued assertions. `expected` and `actual` are not otherwise limited.
+
+The fields are additive; `schemaVersion` stays `1.0`.
+
 ## Top-level fields
 
 | Field | Type | Meaning |

src/CodeGuard.Core/Evaluation/RuleEvaluator.cs

@@ -90,7 +90,7 @@
         var rulesEvaluated = enabledRules.Count;
         var status = evaluationErrors.Count > 0
             ? ValidationStatus.PartiallyEvaluated
             : violations.Count == 0 ? ValidationStatus.Passed : ValidationStatus.Failed;
 
         // Not a rule result - a signal about the AnalysisModel itself (e.g. a project MSBuildWorkspace
         // couldn't fully load, recorded on RepositoryModel.Diagnostics by the provider that built
@@ -134,7 +134,7 @@
     /// whether it's enabled for whole-repo <see cref="Evaluate"/> runs. Does not catch exceptions; callers that
     /// need per-rule isolation (like <see cref="Evaluate"/>) handle that themselves.
     /// </summary>
     public IReadOnlyList<Violation> EvaluateRule(RuleDefinition rule, RepositoryModel model, CancellationToken cancellationToken = default)
     {
         var violations = new List<Violation>();
 
@@ -167,7 +167,10 @@
                 rule.Remediation,
                 rule.Documentation,
                 EndLine: analyzerViolation.EndLine,
-                EndColumn: analyzerViolation.EndColumn));
+                EndColumn: analyzerViolation.EndColumn,
+                Expected: analyzerViolation.Expected,
+                Actual: analyzerViolation.Actual,
+                Evidence: analyzerViolation.Evidence));
         }
     }
 
@@ -217,7 +220,10 @@
             rule.Remediation,
             rule.Documentation,
             EndLine: endLine,
-            EndColumn: endColumn);
+            EndColumn: endColumn,
+            Expected: outcome.Expected,
+            Actual: outcome.Actual,
+            Evidence: outcome.Evidence);
     }
 
     private static (string? File, int? Line, int? Column, int? EndLine, int? EndColumn, string? Symbol, string? Project) ExtractLocation(object candidate) =>

src/CodeGuard.Core/Results/ValidationResult.cs

@@ -37,7 +37,10 @@ public sealed record Violation(
     IReadOnlyList<string> DocumentationReferences,
     int? EndLine = null,
     int? EndColumn = null,
-    IReadOnlyDictionary<string, string>? Metadata = null);
+    IReadOnlyDictionary<string, string>? Metadata = null,
+    string? Expected = null,
+    string? Actual = null,
+    IReadOnlyList<string>? Evidence = null);
 
 /// <summary>A rule whose selector/assertion/analyzer threw instead of producing a pass/fail result.</summary>
 public sealed record RuleEvaluationError(

src/CodeGuard.Evaluation/Assertions/CandidateDescriptor.cs

@@ -25,4 +25,44 @@ internal static class CandidateDescriptor
         DirectoryModel directory => directory.RelativePath,
         _ => candidate.ToString() ?? candidate.GetType().Name
     };
+
+    /// <summary>
+    /// Upper bound on the number of entries in any Evidence (or evidence-like Actual) list, so a
+    /// failure over thousands of matches stays a bounded payload for reporters and agents.
+    /// </summary>
+    public const int MaxEvidenceEntries = 50;
+
+    /// <summary>
+    /// Returns at most <paramref name="max"/> items. When there are more, the final entry is
+    /// "...and N more", where N is the number of items omitted.
+    /// </summary>
+    public static List<string> CapEvidence(IEnumerable<string> items, int max = MaxEvidenceEntries)
+    {
+        var all = items.ToList();
+        if (all.Count <= max)
+        {
+            return all;
+        }
+
+        var capped = all.Take(max).ToList();
+        capped.Add($"...and {all.Count - max} more");
+        return capped;
+    }
+
+    /// <summary>Joins names for an Actual value, or "(none)" when empty. Bounded like evidence.</summary>
+    public static string ListOrNone(IEnumerable<string> items)
+    {
+        var list = CapEvidence(items);
+        return list.Count == 0 ? "(none)" : string.Join(", ", list);
+    }
+
+    /// <summary>
+    /// Aggregates nested outcome values for a quantifier: returns the value only when every nested
+    /// outcome agrees on it (including agreeing that it is null), otherwise null.
+    /// </summary>
+    public static string? CommonValue(IEnumerable<string?> values)
+    {
+        var distinct = values.Distinct().ToList();
+        return distinct.Count == 1 ? distinct[0] : null;
+    }
 }

src/CodeGuard.Evaluation/Assertions/MustAllMatchAssertion.cs

@@ -25,16 +25,46 @@ public sealed class MustAllMatchAssertion(
     public AssertionOutcome Evaluate(object candidate, RepositoryModel model)
     {
         var selector = selectorFactory(SelectorTemplateResolver.Resolve(selectorTemplate, candidate));
-        var failures = selector.SelectCandidates(model)
-            .SelectMany(match => assertions
+        var failedMatches = selector.SelectCandidates(model)
+            .Select(match => (Match: match, Failures: assertions
                 .Select(assertion => assertion.Evaluate(match, model))
                 .Where(outcome => !outcome.Passed)
-                .Select(outcome => $"{CandidateDescriptor.Describe(match)}: {outcome.Message}"))
+                .ToList()))
+            .Where(entry => entry.Failures.Count > 0)
             .ToList();
 
-        return failures.Count == 0
-            ? AssertionOutcome.Success()
-            : AssertionOutcome.Failure(
-                $"Not every match for a '{selector.Kind}' selector satisfied every nested assertion: {string.Join("; ", failures)}");
+        if (failedMatches.Count == 0)
+        {
+            return AssertionOutcome.Success();
+        }
+
+        var failures = failedMatches
+            .SelectMany(entry => entry.Failures.Select(outcome => $"{CandidateDescriptor.Describe(entry.Match)}: {outcome.Message}"))
+            .ToList();
+
+        // Evidence is per failing match: the match's description, qualified by each nested outcome's
+        // own evidence when it has any, so the offending references stay traceable to their match.
+        var evidence = new List<string>();
+        foreach (var entry in failedMatches)
+        {
+            var description = CandidateDescriptor.Describe(entry.Match);
+            var nested = entry.Failures.SelectMany(outcome => outcome.Evidence ?? []).ToList();
+            if (nested.Count == 0)
+            {
+                evidence.Add(description);
+            }
+            else
+            {
+                evidence.AddRange(nested.Select(item => $"{description}: {item}"));
+            }
+        }
+
+        var nestedFailures = failedMatches.SelectMany(entry => entry.Failures).ToList();
+
+        return AssertionOutcome.Failure(
+            $"Not every match for a '{selector.Kind}' selector satisfied every nested assertion: {string.Join("; ", failures)}",
+            CandidateDescriptor.CommonValue(nestedFailures.Select(outcome => outcome.Expected)),
+            $"{failedMatches.Count} match(es) failed a nested assertion",
+            CandidateDescriptor.CapEvidence(evidence));
     }
 }

src/CodeGuard.Evaluation/Assertions/MustAnyMatchAssertion.cs

@@ -22,11 +22,42 @@ public AssertionOutcome Evaluate(object candidate, RepositoryModel model)
     {
         var selector = selectorFactory(SelectorTemplateResolver.Resolve(selectorTemplate, candidate));
         var matches = selector.SelectCandidates(model).ToList();
-        var satisfied = matches.Any(match => assertions.All(assertion => assertion.Evaluate(match, model).Passed));
 
-        return satisfied
-            ? AssertionOutcome.Success()
-            : AssertionOutcome.Failure(
-                $"Expected at least one match for a '{selector.Kind}' selector to satisfy every nested assertion, but none of the {matches.Count} match(es) did.");
+        // Single pass: each (match, assertion) pair is evaluated at most once. Within a match, evaluation
+        // stops at the first failing assertion (as assertions.All does), and the search stops at the first
+        // fully satisfying match, since no failure data is needed then. The failure outcomes collected here
+        // are the ones reported, so nothing is re-evaluated to build the failure message or evidence.
+        var nestedFailures = new List<AssertionOutcome>(matches.Count);
+        foreach (var match in matches)
+        {
+            var failure = FirstFailure(match, model);
+            if (failure is null)
+            {
+                return AssertionOutcome.Success();
+            }
+
+            nestedFailures.Add(failure);
+        }
+
+        return AssertionOutcome.Failure(
+            $"Expected at least one match for a '{selector.Kind}' selector to satisfy every nested assertion, but none of the {matches.Count} match(es) did.",
+            CandidateDescriptor.CommonValue(nestedFailures.Select(outcome => outcome.Expected)),
+            $"0 of {matches.Count} match(es) satisfied every nested assertion",
+            matches.Count == 0 ? null : CandidateDescriptor.CapEvidence(matches.Select(CandidateDescriptor.Describe)));
+    }
+
+    /// <summary>Returns the first nested outcome that fails for <paramref name="match"/>, or null when all pass.</summary>
+    private AssertionOutcome? FirstFailure(object match, RepositoryModel model)
+    {
+        foreach (var assertion in assertions)
+        {
+            var outcome = assertion.Evaluate(match, model);
+            if (!outcome.Passed)
+            {
+                return outcome;
+            }
+        }
+
+        return null;
     }
 }

src/CodeGuard.Evaluation/Assertions/MustBeInNamespaceAssertion.cs

@@ -16,6 +16,6 @@ public AssertionOutcome Evaluate(object candidate, RepositoryModel model)
 
         return GlobMatcher.IsMatch(type.Namespace, namespacePattern)
             ? AssertionOutcome.Success()
-            : AssertionOutcome.Failure($"'{type.FullName}' must be in a namespace matching '{namespacePattern}'.");
+            : AssertionOutcome.Failure($"'{type.FullName}' must be in a namespace matching '{namespacePattern}'.", namespacePattern, type.Namespace, [type.FullName]);
     }
 }

src/CodeGuard.Evaluation/Assertions/MustBeInProjectAssertion.cs

@@ -16,6 +16,6 @@ public AssertionOutcome Evaluate(object candidate, RepositoryModel model)
 
         return GlobMatcher.IsMatch(type.ProjectName, projectNamePattern)
             ? AssertionOutcome.Success()
-            : AssertionOutcome.Failure($"'{type.FullName}' must be in a project matching '{projectNamePattern}'.");
+            : AssertionOutcome.Failure($"'{type.FullName}' must be in a project matching '{projectNamePattern}'.", projectNamePattern, type.ProjectName, [type.FullName]);
     }
 }

src/CodeGuard.Evaluation/Assertions/MustDependOnAssertion.cs

@@ -26,6 +26,6 @@ public AssertionOutcome Evaluate(object candidate, RepositoryModel model)
 
         return hasMatch
             ? AssertionOutcome.Success()
-            : AssertionOutcome.Failure($"Project '{project.Name}' must depend on a type matching '{requiredTypePattern}'.");
+            : AssertionOutcome.Failure($"Project '{project.Name}' must depend on a type matching '{requiredTypePattern}'.", requiredTypePattern, null, [project.Name]);
     }
 }

src/CodeGuard.Evaluation/Assertions/MustExistAssertion.cs

@@ -14,6 +14,6 @@ public AssertionOutcome Evaluate(object candidate, RepositoryModel model)
         var selector = selectorFactory(SelectorTemplateResolver.Resolve(selectorTemplate, candidate));
         return selector.SelectCandidates(model).Any()
             ? AssertionOutcome.Success()
-            : AssertionOutcome.Failure($"Expected at least one match for a '{selector.Kind}' selector, but found none.");
+            : AssertionOutcome.Failure($"Expected at least one match for a '{selector.Kind}' selector, but found none.", "at least one match", "0 matches", null);
     }
 }

src/CodeGuard.Evaluation/Assertions/MustHaveAttributeAssertion.cs

@@ -20,6 +20,6 @@
 
         return match is not null
             ? AssertionOutcome.Success()
-            : AssertionOutcome.Failure($"Candidate must have attribute matching '{typePattern}'{(argument is null ? "" : $" with argument '{argument}'")}.");
+            : AssertionOutcome.Failure($"Candidate must have attribute matching '{typePattern}'{(argument is null ? "" : $" with argument '{argument}'")}.", argument is null ? typePattern : $"{typePattern} with argument '{argument}'", CandidateDescriptor.ListOrNone(attributes.Select(a => a.TypeName)), [CandidateDescriptor.Describe(candidate)]);
     }
 }

src/CodeGuard.Evaluation/Assertions/MustHaveConstructorAssertion.cs

@@ -17,6 +17,6 @@ public AssertionOutcome Evaluate(object candidate, RepositoryModel model)
         return type.Constructors.Any(c => allowedAccessibilities.Contains(c.Accessibility))
             ? AssertionOutcome.Success()
             : AssertionOutcome.Failure(
-                $"'{type.FullName}' must have a constructor with accessibility {string.Join(" or ", allowedAccessibilities)}.");
+                $"'{type.FullName}' must have a constructor with accessibility {string.Join(" or ", allowedAccessibilities)}.", string.Join(" or ", allowedAccessibilities), CandidateDescriptor.ListOrNone(type.Constructors.Select(c => c.Accessibility.ToString())), [type.FullName]);
     }
 }

src/CodeGuard.Evaluation/Assertions/MustHaveCountAssertion.cs

@@ -25,24 +25,29 @@ public sealed class MustHaveCountAssertion(
     public AssertionOutcome Evaluate(object candidate, RepositoryModel model)
     {
         var selector = selectorFactory(SelectorTemplateResolver.Resolve(selectorTemplate, candidate));
-        var count = selector.SelectCandidates(model).Count();
+        var matches = selector.SelectCandidates(model).ToList();
+        var count = matches.Count;
+        var evidence = matches.Count == 0 ? null : CandidateDescriptor.CapEvidence(matches.Select(CandidateDescriptor.Describe));
 
         if (exactly is not null && count != exactly)
         {
             return AssertionOutcome.Failure(
-                $"Expected exactly {exactly} match(es) for a '{selector.Kind}' selector, but found {count}.");
+                $"Expected exactly {exactly} match(es) for a '{selector.Kind}' selector, but found {count}.",
+                $"exactly {exactly}", count.ToString(), evidence);
         }
 
         if (min is not null && count < min)
         {
             return AssertionOutcome.Failure(
-                $"Expected at least {min} match(es) for a '{selector.Kind}' selector, but found {count}.");
+                $"Expected at least {min} match(es) for a '{selector.Kind}' selector, but found {count}.",
+                $"at least {min}", count.ToString(), evidence);
         }
 
         if (max is not null && count > max)
         {
             return AssertionOutcome.Failure(
-                $"Expected at most {max} match(es) for a '{selector.Kind}' selector, but found {count}.");
+                $"Expected at most {max} match(es) for a '{selector.Kind}' selector, but found {count}.",
+                $"at most {max}", count.ToString(), evidence);
         }
 
         return AssertionOutcome.Success();

src/CodeGuard.Evaluation/Assertions/MustHaveDirectoryAssertion.cs

@@ -16,6 +16,6 @@ public AssertionOutcome Evaluate(object candidate, RepositoryModel model)
 
         return repository.Directories.Any(d => GlobMatcher.IsMatch(d.RelativePath, path))
             ? AssertionOutcome.Success()
-            : AssertionOutcome.Failure($"Repository must have a directory at '{path}'.");
+            : AssertionOutcome.Failure($"Repository must have a directory at '{path}'.", path, null, null);
     }
 }

src/CodeGuard.Evaluation/Assertions/MustHaveFieldAssertion.cs

@@ -16,6 +16,6 @@ public AssertionOutcome Evaluate(object candidate, RepositoryModel model)
 
         return type.Fields.Any(f => GlobMatcher.IsMatch(f.Name, namePattern))
             ? AssertionOutcome.Success()
-            : AssertionOutcome.Failure($"'{type.FullName}' must have a field matching '{namePattern}'.");
+            : AssertionOutcome.Failure($"'{type.FullName}' must have a field matching '{namePattern}'.", namePattern, CandidateDescriptor.ListOrNone(type.Fields.Select(f => f.Name)), [type.FullName]);
     }
 }