Repository navigation
Instrument application with PostHog - #28
Merged
Merged
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Important Review skippedThis PR was authored by a bot without an assigned CodeRabbit review seat. Ask an organization administrator to assign a seat to the bot. See the seat-assignment guide. After the seat is assigned, request a review again.
|
|
Add Carrot credits or activate Agent usage billing to use Autofix |
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PostHog setup report — LightCMS
PostHog is configured for LightCMS browser and server analytics, authenticated product events, error tracking, AI Observability, Logs, and a starter dashboard; live delivery and command validation remain unconfirmed.
Initialization and SDKs
PostHog initializes once in
src/hooks.client.tsthroughposthog-js, usingPUBLIC_POSTHOG_PROJECT_TOKENandPUBLIC_POSTHOG_HOSTfrom SvelteKit’s public dynamic environment. The browser sends directly to the configured host; no reverse proxy was added. Server-side analytics use theposthog-nodesingleton insrc/lib/server/posthog.ts, with per-request safe flushing, a 2-second timeout, and zero retries.The run recorded both public environment variables as present in
.envand documented in.env.example. The final review removed the legacy inline browser loader insrc/app.htmland corrected the server module to use$env/dynamic/public, leaving one browser initialization path. Static review confirmed thatpaths.relative: falseis configured for session replay.PostHog SDK installation
The SvelteKit application already declares both required PostHog SDKs in
package.json:posthog-jsfor browser analytics andposthog-nodefor server-side capture. Bun is the project package manager. No dependency changes were needed.Events instrumented
The event plan records 16 existing authenticated server-action events. Source review confirmed each listed call is on its successful action path and uses the authenticated Better Auth user ID. The run did not observe any event arrive in PostHog.
github_installation_startedsrc/routes/github/install/+server.tsgithub_installation_completedsrc/routes/github/setup/+server.tscollection_createdsrc/routes/cms/+page.server.tscollection_initializedsrc/routes/cms/[collection]/+page.server.tscollection_folder_createdsrc/routes/cms/[collection]/+page.server.tscollection_folder_deletedsrc/routes/cms/[collection]/+page.server.tscollection_entry_createdsrc/routes/cms/[collection]/[slug]/+page.server.tscollection_entry_updatedsrc/routes/cms/[collection]/[slug]/+page.server.tscollection_entry_deletedsrc/routes/cms/[collection]/[slug]/+page.server.tscollection_entry_movedsurfacedistinguishes list and editor flows.src/routes/cms/[collection]/[slug]/+page.server.tsrepository_folder_createdsrc/routes/cms/tree/[...path]/+page.server.tsrepository_folder_deletedsrc/routes/cms/tree/[...path]/+page.server.tsrepository_file_createdsrc/routes/cms/tree/[...path]/+page.server.tsrepository_file_savedsrc/routes/cms/edit/[...path]/+page.server.tsrepository_file_deletedsrc/routes/cms/edit/[...path]/+page.server.tsrepository_file_movedsurfacedistinguishes tree and editor flows.src/routes/cms/edit/[...path]/+page.server.tsProduct event capture
Recorded the core LightCMS product-event contract in
.posthog-wizard-cache/.posthog-events.json. The existing server-side instrumentation captures successful GitHub installation, collection, collection-entry, repository file, and repository-folder lifecycle actions using the authenticated Better Auth user ID. Event properties are limited to non-PII operational context, such as the UI surface for move actions.User identification
PostHog identity is established in the root Svelte layout using Better Auth’s stable
user.id, with email and name sent only as person properties. This runs for authenticated page refreshes and protects direct account switching with a reset; the confirmed sign-out flow also resets PostHog. Server-side product events and error reporting use the authenticated user ID from request locals. Browser requests to the same app origin now include PostHog distinct-id and session-id tracing headers, improving client/server correlation.Error tracking
Browser error tracking is configured centrally in
src/hooks.client.ts: unhandled errors and unhandled promise rejections are captured, while console-error capture remains disabled to reduce noise. The existing SvelteKithandleErrorhook captures framework-routed errors withposthog.captureException(error). The run statically verified this configuration but did not start the app or observe an exception arrive in PostHog.AI Observability
AI Observability is configured for production Vercel AI SDK
streamTextandtranscribecalls through the existing OpenTelemetry bootstrap insrc/lib/server/ai-observability.ts. Authenticated distinct ID, Better Auth session ID, and thecms_speech_transcriptiontrace name are forwarded to the AI SDK transcription path throughsrc/lib/server/transcription.tsandsrc/lib/server/speech-ai-sdk.ts. Existing manual$ai_generationcapture covers server-side Wit.ai transcription and the optional direct-browser Wit.ai fallback. No tools are registered with these AI calls, so no tool spans are expected.Prompt and completion content is captured and stored in PostHog. The effective configuration is
privacyMode: falseinsrc/lib/server/ai-observability.ts:32-36, with Vercel input/output recording enabled insrc/lib/server/speech-ai-sdk.ts:136-145and the existing text-generation telemetry block insrc/lib/server/orpc/router.ts. Before handling prompts or outputs that may contain sensitive content, disable the relevantrecordInputsand/orrecordOutputssetting; redact or omit manually captured Wit.ai transcript properties at their capture sites, since Vercel privacy controls do not redact them. See the AI Observability privacy-mode documentation.Static source review confirmed the intended call paths are instrumented. No build, type check, test, app run, paid model call, or delivery observation occurred, so trace delivery remains unconfirmed.
PostHog log capture
PostHog Logs is configured for both LightCMS runtimes. The browser PostHog singleton now identifies browser records as
lightcms-web, while the existing dedicated server OpenTelemetry provider exports only the purpose-written CMS AI-generation start and finish records aslightcms. Existing application loggers and console output remain local and are not exported. Trigger an authenticated CMS editor AI generation and check PostHog Logs forcms_ai_content_generation; delivery still needs to be verified in the running app. To export existing server logs later, deliberately route reviewed logger records to the dedicated provider insrc/lib/server/ai-observability.tsrather than attaching its exporter globally.Starter analytics dashboard
Created Analytics basics (wizard) with five saved PostHog views: GitHub installation conversion, CMS activation to first entry, unique collection creators, structured-entry activity, and repository-file activity. Each insight uses the instrumented CMS event contract and is tagged
(wizard)so it can be found and refined as production events arrive.Verification status and build conflict
The integration was statically reviewed and two issues were corrected: the obsolete inline browser loader was removed from
src/app.html, and the server PostHog module now reads the configured public environment values. Static review also confirmed a single browser initialization path, guarded server initialization, stable authenticated IDs, caught per-request flushes, timeout/retry settings, exception capture, tracing headers, default browser capture behavior, and replay-compatible paths configuration.Verification incomplete:
bun installcould not run because the Bun executable is missing (/bin/bash: bun: command not found). Consequently, the run could not execute a fresh or frozen install, build, Svelte type check, lint, format, app startup, tests, or live delivery validation. This does not indicate a failed build; it is unconfirmed. The lockfile was recorded as resolvingposthog-js@1.450.0,posthog-node@5.74.0, and@posthog/ai@8.20.0.Before you merge
src/hooks.client.tsandsrc/lib/server/posthog.tscompile by runningbun install, the frozen-lockfile equivalent,bun run build, andbun run check; fix any type, lint, or format errors introduced by the integration. The run did not record exact line numbers for these blocks.PUBLIC_POSTHOG_PROJECT_TOKENandPUBLIC_POSTHOG_HOSTare present in.env.exampleand in every deploy environment or bootstrap script; the run recorded the files but not line numbers.src/hooks.client.ts(error-capture block; line number not recorded) and follow source-map setup guidance.src/hooks.client.ts(PostHog init block; line number not recorded), because a blocked SDK can queue events without sending them.$ai_generationevents in PostHog. Review privacy settings first atsrc/lib/server/ai-observability.ts:32-36and input/output capture atsrc/lib/server/speech-ai-sdk.ts:136-145.cms_ai_content_generationstart/finish records in PostHog Logs withservice.name=lightcms; trigger a SvelteKit client error and confirmsveltekit.handleErrorwithservice.name=lightcms-web. Reviewsrc/hooks.client.tsandsrc/lib/server/ai-observability.ts(line numbers not recorded by the run).src/routes/+layout.svelte(line number not recorded) so returning sessions do not fragment onto anonymous distinct IDs.