Skip to content

Instrument application with PostHog - #28

Merged
Pratyay360 merged 1 commit into
mainfrom
posthog/wizard-01a1ee412b42
Nov 18, 2026
Merged

Pratyay360 merged 1 commit into
mainfrom
posthog/wizard-01a1ee412b42

Conversation

@posthog

@posthog posthog Bot commented Nov 18, 2026

Copy link
Copy Markdown
Contributor

PostHog setup report — LightCMS

PostHog is configured for LightCMS browser and server analytics, authenticated product events, error tracking, AI Observability, Logs, and a starter dashboard; live delivery and command validation remain unconfirmed.

Initialization and SDKs

PostHog initializes once in src/hooks.client.ts through posthog-js, using PUBLIC_POSTHOG_PROJECT_TOKEN and PUBLIC_POSTHOG_HOST from SvelteKit’s public dynamic environment. The browser sends directly to the configured host; no reverse proxy was added. Server-side analytics use the posthog-node singleton in src/lib/server/posthog.ts, with per-request safe flushing, a 2-second timeout, and zero retries.

The run recorded both public environment variables as present in .env and documented in .env.example. The final review removed the legacy inline browser loader in src/app.html and corrected the server module to use $env/dynamic/public, leaving one browser initialization path. Static review confirmed that paths.relative: false is configured for session replay.

PostHog SDK installation

The SvelteKit application already declares both required PostHog SDKs in package.json: posthog-js for browser analytics and posthog-node for server-side capture. Bun is the project package manager. No dependency changes were needed.

Events instrumented

The event plan records 16 existing authenticated server-action events. Source review confirmed each listed call is on its successful action path and uses the authenticated Better Auth user ID. The run did not observe any event arrive in PostHog.

Event What it measures File
github_installation_started Authenticated user begins the GitHub App installation flow. src/routes/github/install/+server.ts
github_installation_completed Authenticated user completes and authorizes a GitHub App installation. src/routes/github/setup/+server.ts
collection_created Authenticated user creates a repository-root content collection. src/routes/cms/+page.server.ts
collection_initialized Authenticated user initializes a configured collection. src/routes/cms/[collection]/+page.server.ts
collection_folder_created Authenticated user creates a folder within a collection. src/routes/cms/[collection]/+page.server.ts
collection_folder_deleted Authenticated user deletes a folder within a collection. src/routes/cms/[collection]/+page.server.ts
collection_entry_created Authenticated user creates a structured collection entry. src/routes/cms/[collection]/[slug]/+page.server.ts
collection_entry_updated Authenticated user saves changes to a structured collection entry. src/routes/cms/[collection]/[slug]/+page.server.ts
collection_entry_deleted Authenticated user deletes a structured collection entry. src/routes/cms/[collection]/[slug]/+page.server.ts
collection_entry_moved Authenticated user moves a structured entry; surface distinguishes list and editor flows. src/routes/cms/[collection]/[slug]/+page.server.ts
repository_folder_created Authenticated user creates a folder in the repository tree. src/routes/cms/tree/[...path]/+page.server.ts
repository_folder_deleted Authenticated user deletes a folder from the repository tree. src/routes/cms/tree/[...path]/+page.server.ts
repository_file_created Authenticated user creates a file in the repository tree. src/routes/cms/tree/[...path]/+page.server.ts
repository_file_saved Authenticated user saves content in the repository file editor. src/routes/cms/edit/[...path]/+page.server.ts
repository_file_deleted Authenticated user deletes a repository file. src/routes/cms/edit/[...path]/+page.server.ts
repository_file_moved Authenticated user moves or renames a repository file; surface distinguishes tree and editor flows. src/routes/cms/edit/[...path]/+page.server.ts

Product event capture

Recorded the core LightCMS product-event contract in .posthog-wizard-cache/.posthog-events.json. The existing server-side instrumentation captures successful GitHub installation, collection, collection-entry, repository file, and repository-folder lifecycle actions using the authenticated Better Auth user ID. Event properties are limited to non-PII operational context, such as the UI surface for move actions.

User identification

PostHog identity is established in the root Svelte layout using Better Auth’s stable user.id, with email and name sent only as person properties. This runs for authenticated page refreshes and protects direct account switching with a reset; the confirmed sign-out flow also resets PostHog. Server-side product events and error reporting use the authenticated user ID from request locals. Browser requests to the same app origin now include PostHog distinct-id and session-id tracing headers, improving client/server correlation.

Error tracking

Browser error tracking is configured centrally in src/hooks.client.ts: unhandled errors and unhandled promise rejections are captured, while console-error capture remains disabled to reduce noise. The existing SvelteKit handleError hook captures framework-routed errors with posthog.captureException(error). The run statically verified this configuration but did not start the app or observe an exception arrive in PostHog.

AI Observability

AI Observability is configured for production Vercel AI SDK streamText and transcribe calls through the existing OpenTelemetry bootstrap in src/lib/server/ai-observability.ts. Authenticated distinct ID, Better Auth session ID, and the cms_speech_transcription trace name are forwarded to the AI SDK transcription path through src/lib/server/transcription.ts and src/lib/server/speech-ai-sdk.ts. Existing manual $ai_generation capture covers server-side Wit.ai transcription and the optional direct-browser Wit.ai fallback. No tools are registered with these AI calls, so no tool spans are expected.

Prompt and completion content is captured and stored in PostHog. The effective configuration is privacyMode: false in src/lib/server/ai-observability.ts:32-36, with Vercel input/output recording enabled in src/lib/server/speech-ai-sdk.ts:136-145 and the existing text-generation telemetry block in src/lib/server/orpc/router.ts. Before handling prompts or outputs that may contain sensitive content, disable the relevant recordInputs and/or recordOutputs setting; redact or omit manually captured Wit.ai transcript properties at their capture sites, since Vercel privacy controls do not redact them. See the AI Observability privacy-mode documentation.

Static source review confirmed the intended call paths are instrumented. No build, type check, test, app run, paid model call, or delivery observation occurred, so trace delivery remains unconfirmed.

PostHog log capture

PostHog Logs is configured for both LightCMS runtimes. The browser PostHog singleton now identifies browser records as lightcms-web, while the existing dedicated server OpenTelemetry provider exports only the purpose-written CMS AI-generation start and finish records as lightcms. Existing application loggers and console output remain local and are not exported. Trigger an authenticated CMS editor AI generation and check PostHog Logs for cms_ai_content_generation; delivery still needs to be verified in the running app. To export existing server logs later, deliberately route reviewed logger records to the dedicated provider in src/lib/server/ai-observability.ts rather than attaching its exporter globally.

Starter analytics dashboard

Created Analytics basics (wizard) with five saved PostHog views: GitHub installation conversion, CMS activation to first entry, unique collection creators, structured-entry activity, and repository-file activity. Each insight uses the instrumented CMS event contract and is tagged (wizard) so it can be found and refined as production events arrive.

Verification status and build conflict

The integration was statically reviewed and two issues were corrected: the obsolete inline browser loader was removed from src/app.html, and the server PostHog module now reads the configured public environment values. Static review also confirmed a single browser initialization path, guarded server initialization, stable authenticated IDs, caught per-request flushes, timeout/retry settings, exception capture, tracing headers, default browser capture behavior, and replay-compatible paths configuration.

Verification incomplete: bun install could not run because the Bun executable is missing (/bin/bash: bun: command not found). Consequently, the run could not execute a fresh or frozen install, build, Svelte type check, lint, format, app startup, tests, or live delivery validation. This does not indicate a failed build; it is unconfirmed. The lockfile was recorded as resolving posthog-js@1.450.0, posthog-node@5.74.0, and @posthog/ai@8.20.0.

Before you merge

  • In an environment with Bun 1.4.x, verify src/hooks.client.ts and src/lib/server/posthog.ts compile by running bun install, the frozen-lockfile equivalent, bun run build, and bun run check; fix any type, lint, or format errors introduced by the integration. The run did not record exact line numbers for these blocks.
  • Run the test suite and update any affected mocks or fixtures for the instrumented server-action files listed in the event table. The run did not record individual call-site line numbers.
  • Confirm PUBLIC_POSTHOG_PROJECT_TOKEN and PUBLIC_POSTHOG_HOST are present in .env.example and in every deploy environment or bootstrap script; the run recorded the files but not line numbers.
  • If production browser bundles are minified, add source-map upload to CI so Error Tracking stack traces can de-minify; review src/hooks.client.ts (error-capture block; line number not recorded) and follow source-map setup guidance.
  • If a Content-Security-Policy is added or exists in deployment, load the app and check browser-console CSP violations against src/hooks.client.ts (PostHog init block; line number not recorded), because a blocked SDK can queue events without sending them.
  • Trigger an authenticated CMS editor AI action and a microphone dictation, then confirm new AI traces or $ai_generation events in PostHog. Review privacy settings first at src/lib/server/ai-observability.ts:32-36 and input/output capture at src/lib/server/speech-ai-sdk.ts:136-145.
  • Trigger authenticated CMS AI generation and confirm the cms_ai_content_generation start/finish records in PostHog Logs with service.name=lightcms; trigger a SvelteKit client error and confirm sveltekit.handleError with service.name=lightcms-web. Review src/hooks.client.ts and src/lib/server/ai-observability.ts (line numbers not recorded by the run).
  • Exercise a listed server action and confirm its event reaches PostHog and its dashboard tile receives data; inspect the file named for that event in the event table (the recorded plan does not include line numbers).
  • Verify the returning authenticated visitor path identifies the user in src/routes/+layout.svelte (line number not recorded) so returning sessions do not fragment onto anonymous distinct IDs.

@vercel

vercel Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
lightcms Ready Ready Preview Nov 18, 2026 10:53am UTC

@coderabbitai

coderabbitai Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

Important

Review skipped

This PR was authored by a bot without an assigned CodeRabbit review seat. Ask an organization administrator to assign a seat to the bot. See the seat-assignment guide. After the seat is assigned, request a review again.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Repository UI (base), Organization UI (inherited)
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 10854160-d2ba-402b-9ce7-22cc172ec5df

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@coderabbitai

coderabbitai Bot commented Nov 18, 2026

Copy link
Copy Markdown

Add Carrot credits or activate Agent usage billing to use Autofix

@Pratyay360
Pratyay360 merged commit 558357e into main Nov 18, 2026
6 of 8 checks passed

This branch was successfully deployed

1 active deployment
Preview — efdd339b Deployed Nov 18, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant