Feat/ci list failed tests

CHANGELOG.md

@@ -9,6 +9,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/).
 
 ### Unreleased
 
+- **Failed tests with trace links in GitHub Actions job summary** — when running inside GitHub Actions and `tracing.enabled: true` is on, failed tests are automatically summarized in a Markdown table (test name, error message, and path to the HTML trace file) appended to `$GITHUB_STEP_SUMMARY`. Triage failures and locate trace paths directly from the Actions job summary without downloading artifact bundles; no-op outside Actions or when all tests pass. (#137)
+
 - **Bounded failure detail record** — every failed test now exports one optional-field `failureDetail` object with the failing locator, top application frame, last successful step, page URL/title, heal events, console errors, retry/history status, and representative same-cause context. The HTML failure panel and opt-in AI prompt reuse this record; raw errors and stack traces remain available. Text/list/URL limits and the exact AI data disclosure are documented. (#142)
 
 - **`Locator`: `waitFor(State)`, `allTextContents()`, and element `screenshot()`** — `waitFor(State)` (and `waitFor()`) waits for an element to reach a target lifecycle state (`VISIBLE`, `HIDDEN`, `ATTACHED`, `DETACHED`), retrying up to `timeouts.explicit`. `allTextContents()` returns the text content of all matched elements in a single pass without waiting if empty, re-resolving if an element goes stale during read. `screenshot()` returns the element's PNG bytes, and `screenshot(Path)` writes them to a file path; both terminal actions re-resolve on every poll to survive stale elements. (#135)

docs-site/docs/changelog.md

@@ -11,6 +11,8 @@ All notable changes to Selenium Boot are documented here.
 
 ## [Unreleased]
 
+- **Failed tests with trace links in GitHub Actions job summary** — when running inside GitHub Actions and `tracing.enabled: true` is on, failed tests are automatically summarized in a Markdown table (test name, error message, and path to the HTML trace file) appended to `$GITHUB_STEP_SUMMARY`. Triage failures and locate trace paths directly from the Actions job summary without downloading artifact bundles; no-op outside Actions or when all tests pass. (#137)
+
 - **Bounded failure detail record** — every failed test now exports one optional-field `failureDetail` object with the failing locator, top application frame, last successful step, page URL/title, heal events, console errors, retry/history status, and representative same-cause context. The HTML failure panel and opt-in AI prompt reuse this record; raw errors and stack traces remain available. Text/list/URL limits and the exact AI data disclosure are documented. (#142)
 
 - **`Locator`: `waitFor(State)`, `allTextContents()`, and element `screenshot()`** — `waitFor(State)` (and `waitFor()`) waits for an element to reach a target lifecycle state (`VISIBLE`, `HIDDEN`, `ATTACHED`, `DETACHED`), retrying up to `timeouts.explicit`. `allTextContents()` returns the text content of all matched elements in a single pass without waiting if empty, re-resolving if an element goes stale during read. `screenshot()` returns the element's PNG bytes, and `screenshot(Path)` writes them to a file path; both terminal actions re-resolve on every poll to survive stale elements. (#135)

docs-site/docs/ci/github-actions.md

@@ -59,11 +59,24 @@ When it detects a CI environment (GitHub Actions, Jenkins, CircleCI and others),
 - `execution.threadCount` is derived from the available CPU cores, capped at `execution.maxActiveSessions` — only when you left it at the default of `1` and `parallel` is not `none`.
 - Inside Docker or Kubernetes, Chrome and Firefox get the container flags they need (for Chrome: `--no-sandbox`, `--disable-dev-shm-usage`, `--disable-gpu`, `--window-size=1920,1080`).
 - JUnit XML is written to `target/surefire-reports/TEST-SeleniumBoot.xml` on every run.
+- When running in GitHub Actions and `tracing.enabled: true` is on, failed tests are summarized in a Markdown table (test name, error, and path to `target/traces/...`) appended to `$GITHUB_STEP_SUMMARY` so failures are visible without opening artifacts.
 
 Each override is printed to the console as `[Selenium Boot] CI override: …`.
 
 ---
 
+## Job summary with trace links
+
+When `tracing.enabled: true` is enabled in `selenium-boot.yml` and tests fail, Selenium Boot automatically appends a Markdown table to `$GITHUB_STEP_SUMMARY`:
+
+| Test | Error | Trace |
+| --- | --- | --- |
+| `com.example.LoginTest#testInvalidLogin` | Expected [Dashboard] but found [401 Unauthorized] | target/traces/LoginTest/testInvalidLogin-trace.html |
+
+This lists every failing test, its error message, and the path to its trace file directly in the GitHub Actions run summary page. If no tests fail or outside GitHub Actions, nothing is written.
+
+---
+
 ## Publish JUnit XML test results
 
 ```yaml

docs-site/docs/guides/trace-viewer.md

@@ -0,0 +1,63 @@
+---
+description: "Self-contained HTML trace viewer per test with clickable step timeline, per-step screenshots, failure snapshot, error details, and GitHub Actions summary integration."
+id: trace-viewer
+title: Trace Viewer
+sidebar_label: Trace Viewer
+sidebar_position: 9
+---
+
+# Trace Viewer
+
+Selenium Boot generates a standalone, self-contained HTML trace viewer per test with an interactive step timeline, per-step screenshots, final-state failure snapshot, and full error details — with zero external CDN dependencies.
+
+---
+
+## Enabling Trace Recording
+
+Trace recording is opt-in and configured in `selenium-boot.yml`:
+
+```yaml title="selenium-boot.yml"
+tracing:
+  enabled: true
+  captureOnPass: false  # set true to generate traces for passing tests too
+```
+
+When enabled, a self-contained HTML file is generated for each failed test at:
+
+```
+target/traces/{ClassName}/{testMethod}-trace.html
+```
+
+(Or `build/traces/...` in Gradle projects, honoring `-Dseleniumboot.reports.dir`).
+
+---
+
+## What the Trace Viewer captures
+
+Each trace file is an independent, single-file HTML document featuring:
+
+1. **Metadata header** — test method name, class, browser name, duration in milliseconds, status badge, and timestamp.
+2. **Interactive step timeline** — every named step recorded via `StepLogger.step(...)` appears in the timeline with its offset. Steps that captured screenshots display a camera icon (📷) and open the interactive screenshot panel on click.
+3. **Final-state screenshot** — full-browser screenshot captured automatically at the exact moment of test failure.
+4. **Failure details** — highlighted error message and full scrollable stack trace.
+5. **Zero CDN dependencies** — CSS styles and interactive JavaScript are completely inlined so traces can be viewed offline, archived, or shared without broken assets.
+
+---
+
+## Viewing traces
+
+### From the HTML report
+In the main Selenium Boot HTML report (`target/selenium-boot-report.html`), every failed test in the Failures tab features a **View Trace** link in its failure detail panel that opens the corresponding trace file.
+
+### Directly from the file system
+Open `target/traces/{ClassName}/{testMethod}-trace.html` in any modern web browser.
+
+### In GitHub Actions
+When running in GitHub Actions and `tracing.enabled: true` is on, failed tests are automatically summarized in a Markdown table appended to `$GITHUB_STEP_SUMMARY`:
+
+| Test | Error | Trace |
+| --- | --- | --- |
+| `com.example.CartTest.testCheckout` | `Timeout waiting for element` | `target/traces/CartTest/testCheckout-trace.html` |
+
+This lets teams triage failures and locate the corresponding HTML trace file directly on the Actions job summary page without downloading and extracting artifact archives.
+

docs-site/sidebars.js

@@ -34,6 +34,7 @@ const sidebars = {
         'guides/retry',
         'guides/screenshots',
         'guides/step-logging',
+        'guides/trace-viewer',
         'guides/browser-lifecycle',
         'guides/parallel',
         'guides/smart-locator',

src/main/java/com/seleniumboot/ci/CiEnvironmentDetector.java

@@ -31,6 +31,13 @@ public static boolean isCI() {
                 || isEnvSet("BITBUCKET_BUILD_NUMBER");
     }
 
+    /**
+     * Returns true if running inside GitHub Actions.
+     */
+    public static boolean isGitHubActions() {
+        return isEnvSet("GITHUB_ACTIONS");
+    }
+
     /**
      * Returns a human-readable name for the detected CI provider.
      */

src/main/java/com/seleniumboot/ci/GitHubActionsSummary.java

@@ -0,0 +1,210 @@
+package com.seleniumboot.ci;
+
+import com.seleniumboot.config.SeleniumBootConfig;
+import com.seleniumboot.internal.SeleniumBootContext;
+import com.seleniumboot.metrics.TestTiming;
+import com.seleniumboot.reporting.ReportPaths;
+
+import java.io.File;
+import java.io.FileWriter;
+import java.io.IOException;
+import java.nio.charset.StandardCharsets;
+import java.util.Collection;
+import java.util.List;
+
+/**
+ * Appends a Markdown summary table of failed tests with their error message and trace links
+ * to GitHub Actions' {@code $GITHUB_STEP_SUMMARY}.
+ *
+ * <p>Activated when running in GitHub Actions, {@code tracing.enabled: true} is configured,
+ * and at least one test failed. Outside GitHub Actions or when no tests failed, this is a no-op.
+ */
+public final class GitHubActionsSummary {
+
+    private GitHubActionsSummary() {}
+
+    /**
+     * Builds the Markdown table string for any failed tests in {@code timings}.
+     * Returns an empty string if {@code timings} is null or contains no failed tests.
+     */
+    public static String buildMarkdown(Collection<TestTiming> timings) {
+        if (timings == null || timings.isEmpty()) {
+            return "";
+        }
+
+        List<TestTiming> failed = timings.stream()
+                .filter(t -> "FAILED".equalsIgnoreCase(t.getStatus()))
+                .toList();
+
+        if (failed.isEmpty()) {
+            return "";
+        }
+
+        String baseDir = ReportPaths.baseDir();
+
+        StringBuilder sb = new StringBuilder();
+        sb.append("### Failed Tests\n\n");
+        sb.append("| Test | Error | Trace |\n");
+        sb.append("| --- | --- | --- |\n");
+
+        for (TestTiming t : failed) {
+            String testId = escapeTableCell(t.getTestId() != null ? t.getTestId() : "Unknown");
+            String error = escapeTableCell(t.getErrorMessage() != null ? t.getErrorMessage() : "—");
+            String tracePath = resolveTracePath(t, baseDir);
+
+            sb.append("| ").append(testId)
+              .append(" | ").append(error)
+              .append(" | ").append(tracePath)
+              .append(" |\n");
+        }
+
+        return sb.toString();
+    }
+
+    /**
+     * Appends failed tests with trace links to {@code summaryFile} when tracing is enabled
+     * and at least one test failed.
+     *
+     * @return {@code true} if summary content was appended; {@code false} otherwise.
+     */
+    public static boolean appendSummary(SeleniumBootConfig config,
+                                        Collection<TestTiming> timings,
+                                        File summaryFile) {
+        if (config == null) {
+            return false;
+        }
+        SeleniumBootConfig.Tracing tracing = config.getTracing();
+        if (tracing == null || !tracing.isEnabled()) {
+            return false;
+        }
+        if (timings == null || timings.isEmpty() || summaryFile == null) {
+            return false;
+        }
+
+        boolean hasFailures = timings.stream()
+                .anyMatch(t -> "FAILED".equalsIgnoreCase(t.getStatus()));
+        if (!hasFailures) {
+            return false;
+        }
+
+        String markdown = buildMarkdown(timings);
+        if (markdown.isEmpty()) {
+            return false;
+        }
+
+        try {
+            File parent = summaryFile.getParentFile();
+            if (parent != null && !parent.exists()) {
+                parent.mkdirs();
+            }
+
+            boolean prependNewline = summaryFile.exists() && summaryFile.length() > 0;
+            String content = (prependNewline ? "\n" : "") + markdown;
+
+            try (FileWriter writer = new FileWriter(summaryFile, StandardCharsets.UTF_8, true)) {
+                writer.write(content);
+            }
+
+            System.out.println("[Selenium Boot] Appended failed tests to GitHub Actions summary → "
+                    + summaryFile.getPath());
+            return true;
+        } catch (IOException e) {
+            System.err.println("[Selenium Boot] Failed to write to $GITHUB_STEP_SUMMARY: "
+                    + e.getMessage());
+            return false;
+        }
+    }
+
+    /**
+     * Exports failed tests to {@code $GITHUB_STEP_SUMMARY} if running in GitHub Actions.
+     *
+     * @return {@code true} if written; {@code false} if skipped or failed.
+     */
+    public static boolean export(SeleniumBootConfig config, Collection<TestTiming> timings) {
+        String testOverride = System.getProperty("seleniumboot.github.step.summary");
+        boolean isActions = CiEnvironmentDetector.isGitHubActions()
+                || (testOverride != null && !testOverride.isBlank());
+        if (!isActions) {
+            return false;
+        }
+
+        String summaryPath = (testOverride != null && !testOverride.isBlank())
+                ? testOverride.trim()
+                : System.getenv("GITHUB_STEP_SUMMARY");
+        if (summaryPath == null || summaryPath.isBlank()) {
+            return false;
+        }
+
+        return appendSummary(config, timings, new File(summaryPath));
+    }
+
+    /**
+     * Overload using the globally configured {@link SeleniumBootConfig}.
+     */
+    public static boolean export(Collection<TestTiming> timings) {
+        SeleniumBootConfig config = null;
+        try {
+            config = SeleniumBootContext.getConfig();
+        } catch (Exception ignored) {}
+        return export(config, timings);
+    }
+
+    static String resolveTracePath(TestTiming timing, String baseDir) {
+        String base = (baseDir != null && !baseDir.isBlank())
+                ? baseDir.replace('\\', '/').trim()
+                : "target";
+        if (base.endsWith("/")) {
+            base = base.substring(0, base.length() - 1);
+        }
+
+        String path = timing.getTracePath();
+        if (path != null && !path.isBlank()) {
+            String normalized = path.replace('\\', '/').trim();
+            if (normalized.startsWith("/")) {
+                normalized = normalized.substring(1);
+            }
+            if (normalized.startsWith(base + "/")) {
+                return normalized;
+            }
+            return base + "/" + normalized;
+        }
+
+        String className = timing.getTestClassName() != null && !timing.getTestClassName().isBlank()
+                ? timing.getTestClassName()
+                : "UnknownClass";
+        String testId = timing.getTestId();
+        String methodName = extractMethodName(testId);
+        return base + "/traces/" + className + "/" + sanitize(methodName) + "-trace.html";
+    }
+
+    static String extractMethodName(String testId) {
+        if (testId == null || testId.isBlank()) {
+            return "unknown";
+        }
+        int hashIdx = testId.indexOf('#');
+        if (hashIdx >= 0 && hashIdx < testId.length() - 1) {
+            return testId.substring(hashIdx + 1);
+        }
+        int dotIdx = testId.lastIndexOf('.');
+        if (dotIdx >= 0 && dotIdx < testId.length() - 1) {
+            return testId.substring(dotIdx + 1);
+        }
+        return testId;
+    }
+
+    static String escapeTableCell(String value) {
+        if (value == null || value.isBlank()) {
+            return "—";
+        }
+        return value.replace("\r\n", " ")
+                    .replace('\n', ' ')
+                    .replace('\r', ' ')
+                    .replace("|", "\\|")
+                    .trim();
+    }
+
+    private static String sanitize(String s) {
+        return s == null ? "unknown" : s.replaceAll("[^a-zA-Z0-9_\\-]", "_");
+    }
+}
+

src/main/java/com/seleniumboot/junit5/SeleniumBootLauncherListener.java

@@ -2,6 +2,7 @@
 
 import com.seleniumboot.api.SeleniumBootApi;
 import com.seleniumboot.ci.BuildThresholdEnforcer;
+import com.seleniumboot.ci.GitHubActionsSummary;
 import com.seleniumboot.config.SeleniumBootConfig;
 import com.seleniumboot.flakiness.FlakinessAnalyzer;
 import com.seleniumboot.hooks.HookRegistry;
@@ -37,12 +38,14 @@ public void testPlanExecutionFinished(TestPlan testPlan) {
 
             JUnitXmlReporter.export(ExecutionMetrics.getTimings(), System.currentTimeMillis());
 
+            SeleniumBootConfig config = SeleniumBootContext.getConfig();
+            GitHubActionsSummary.export(config, ExecutionMetrics.getTimings());
+
             ReportAdapterRegistry.generateAll();
             PreConditionRunner.clearAll();
             HookRegistry.onSuiteEnd();
             TestManagementReporter.getInstance().onSuiteEnd();
 
-            SeleniumBootConfig config = SeleniumBootContext.getConfig();
             if (config != null) {
                 BuildThresholdEnforcer.enforce(config, ExecutionMetrics.getTimings());
             }

src/main/java/com/seleniumboot/listeners/SuiteExecutionListener.java

@@ -1,6 +1,7 @@
 package com.seleniumboot.listeners;
 
 import com.seleniumboot.ci.BuildThresholdEnforcer;
+import com.seleniumboot.ci.GitHubActionsSummary;
 import com.seleniumboot.flakiness.FlakinessAnalyzer;
 import com.seleniumboot.healing.HealLog;
 import com.seleniumboot.config.SeleniumBootConfig;
@@ -99,6 +100,10 @@ public void onFinish(ISuite suite) {
                 suite.getAllInvokedMethods().size() > 0
                         ? System.currentTimeMillis() : 0L);
 
+        // GitHub Actions job summary (appends failed tests with trace links)
+        SeleniumBootConfig config = SeleniumBootContext.getConfig();
+        GitHubActionsSummary.export(config, ExecutionMetrics.getTimings());
+
         ReportAdapterRegistry.generateAll();
         PreConditionRunner.clearAll();
         DriverManager.quitAllSuiteDrivers(); // per-suite lifecycle — quits all kept-alive drivers
@@ -108,7 +113,6 @@ public void onFinish(ISuite suite) {
         PluginRegistry.unloadAll();
 
         // Build quality gates — must run last so all metrics are recorded
-        SeleniumBootConfig config = SeleniumBootContext.getConfig();
         BuildThresholdEnforcer.enforce(config, ExecutionMetrics.getTimings());
     }
 }