Skip to content

chore: Add CI logic to start using typed analytics schema contract - #36941

Merged
jvbriones merged 7 commits into
mainfrom
feat/schema-first-approach
Sep 29, 2026
Merged

jvbriones merged 7 commits into
mainfrom
feat/schema-first-approach

Conversation

@jvbriones

@jvbriones jvbriones commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

Description

This PR introduces the first Mobile-side consumer flow for deployed typed analytics contracts, focused on CI validation and reproducibility.

It downloads the immutable private Mobile contract through the shared OIDC action, verifies its metadata and hashes, regenerates the public Quick Buy v1/v2 types, and detects drift in CI.

Overall changes

  • Download and consumes the deployed Mobile contract through a shared GitHub Action (github-tools)

  • Verifies release metadata, asset digest, semantic contract hash, source SHA,
    tracking-plan ID, and rule count.

  • Adds a pinned contract lock for reproducible local validation.

  • Publishes strict Quick Buy Amount Selected v1/v2 TypeScript types.

  • Validates that tracked public types match regenerated output in CI.

  • Preserves Mobile’s existing analytics runtime and dispatch behavior.

  • Adds runtime tests for event-name and version-context forwarding.

  • No production analytics call-site migration.

  • No changes to @metamask/analytics-controller.

  • No automatic contract-update PR yet.

  • No progressive runtime builder yet.

Public generated types

The generated Quick Buy facade is tracked publicly at:

  • app/util/analytics/generated/QuickBuyAmountSelected.ts
  • app/util/analytics/generated/QuickBuyAmountSelected.test-d.ts

It provides:

  • explicit v1 and v2 APIs;
  • required-property checks;
  • enum/value checks;
  • unknown-property rejection;
  • context.protocols.event_version.

The raw contract remains private, i.e. the metamask-mobile.json contract downloaded from segment-schema release is never committed.

Generator comparison

We compared two approaches against the same immutable Mobile contract and the
same Quick Buy Amount Selected v1/v2 fixture.

  • Release: analytics-contracts-67390efdfde3-8db9e9f80441
  • Contract hash: sha256:f75e241c1a832b22c80f9537765e51dac0ba380133f1ec90a6f3e7c77235c4c3
  • Fixture: Quick Buy Amount Selected v1/v2

Lightweight custom generator

The custom generator consumes the verified deployed Mobile contract and a
reviewed caller-property boundary. It generates:

  • separate v1 and v2 property interfaces;
  • required and optional property checks;
  • enum/string-literal validation;
  • unknown-property rejection;
  • explicit context.protocols.event_version;
  • compile-time fixtures for invalid calls;
  • a runtime wrapper test for event name and version forwarding.

Stock Typewriter

The Typewriter comparison used TypeScript with analytics-react-native and a
temporary local plan.json derived from the same pinned contract. Its output
was used only as reference material. Typewriter was included only for evaluation and should have been removed before the final merge; it is not a production dependency or the selected implementation.

Stock Typewriter:

  • retains only the highest version for a repeated event key, so the fixture
    produces v2 but loses the explicit v1 API;
  • emits a permissive [property: string]: any index signature;
  • does not emit the required event-version context;
  • does not provide the progressive builder API;
  • does not provide the exact caller-property boundary needed by this pilot.

Result

The lightweight generator is the better fit for this Mobile pilot because it
preserves event versions, enforces exact properties, and supplies the Segment
event-version context without adding a runtime dependency or requiring Segment
credentials.

This is not yet a final decision for the all-events generator. Typewriter’s
schema normalization and generation code may still provide reusable ideas.
The Typewriter dependency and comparison files are temporary spike material and
will be removed after review.

The temporary evaluation files are available in the initial spike commit:
comparison spike

The Typewriter comparison was used as a temporary design spike and has been
removed from the final implementation. The comparison showed that the narrow
generator is a better fit for this versioned Mobile pilot because it preserves
v1/v2, rejects unknown properties, and adds event-version context.

Security and privacy

  • The raw contract is never committed.
  • GitHub Actions uses short-lived OIDC access through the token exchange service.
  • Local contract and generated comparison output are written only under ignored
    temp/.
  • The committed lock contains release identifiers and hashes only.

Validation

Passed:

  • yarn analytics:contract:check:locked
  • yarn analytics:contract:compare
  • yarn jest scripts/analytics/typed-events --runInBand --coverage=false --forceExit
  • targeted TypeScript check
  • targeted ESLint
  • Prettier check
  • regenerated public facade matches tracked files exactly

Targeted checks passed.

Follow-up

  1. Remove the temporary Typewriter spike files in this PR before merge. DONE. ✅
  2. After merge, manually dispatch the workflow from main.
  3. Confirm OIDC access, contract verification, public-type drift checks, and
    Slack notification.
  4. Open and merge a follow-up PR integrating the generated facade into one
    production Quick Buy call.
  5. After step 4 is merged, add the trusted rolling-update workflow. It should:
    • detect a newer deployed contract release;
    • update contract.lock.json;
    • regenerate the public Quick Buy types;
    • run tests and drift checks;
    • open a reviewable bot PR.
  6. Add progressive event construction and runtime completeness validation.
  7. Expand typed coverage to additional event families.

Changelog

CHANGELOG entry:

Related issues

Fixes:

Manual testing steps

In description

Screenshots/Recordings

  • From local, proving the schema contract is downloaded and verified:
Screenshot 2026-09-29 at 15 42 37
  • Temporary files dowloaded and generated:
Screenshot 2026-09-29 at 15 45 19
  • The generated public facade exposes separate v1 and v2 property contracts, including version-specific fields and the required event-version context (what we have committed in this PR as POC app/util/analytics/generated/QuickBuyAmountSelected.ts):
Screenshot 2026-09-29 at 15 53 18

Before

After

Pre-merge author checklist

Performance checks (if applicable)

  • I've tested on Android
    • Ideally on a mid-range device; emulator is acceptable
  • I've tested with a power user scenario
    • Use these power-user SRPs to import wallets with many accounts and tokens
  • I've instrumented key operations with Sentry traces for production performance metrics

For performance guidelines and tooling, see the Performance Guide.

Pre-merge reviewer checklist

  • I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
  • I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.

Note

Low Risk
CI/tooling and generated types only; no production analytics path or runtime behavior changes.

Overview
Adds typed analytics contract validation: Mobile can download the immutable Segment schema release (metamask-mobile.json), verify hashes and metadata, regenerate a strict TypeScript facade, and fail when tracked types drift.

New yarn analytics:contract:check / check:locked (plus scripts/analytics/typed-events/*) handle release resolution, contract.lock.json, semantic hash checks, and a narrow generator for Quick Buy Amount Selected v1/v2 with exact properties and context.protocols.event_version. Committed pilot output lives under app/util/analytics/generated/ with runtime and compile-time tests.

A scheduled check-analytics-contract workflow uses the shared download action, runs the check, diffs regenerated files against the repo, writes a job summary, and posts Slack on failure. Docs describe the lifecycle; production analytics dispatch is unchanged (no call-site migration yet).

Reviewed by Cursor Bugbot for commit 0b54ef4. Bugbot is set up for automated code reviews on this repo. Configure here.

@github-actions

Copy link
Copy Markdown
Contributor

CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes.

@github-actions github-actions Bot added size-XL pr-not-ready-for-e2e Skip E2E and block merging. Remove this label once the PR is ready to run the E2E tests. labels Sep 28, 2026
@metamask-ci metamask-ci Bot added the team-qa QA team label Sep 28, 2026
@metamask-ci

metamask-ci Bot commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

PR template — items to address before "Ready for review"

Warnings — informational, address before merging:

  • Related issues section is empty. Add Fixes: #123 / Closes: <URL> / Refs: <Jira key>, or write a short rationale after the colon.
  • Pre-merge author checklist has unchecked items (e.g. "I've tested on Android"). Every box must be consciously checked — see docs/readme/ready-for-review.md.

See docs/readme/ready-for-review.md for the full Definition of Ready for Review.

@socket-security

socket-security Bot commented Sep 28, 2026 •

Copy link
Copy Markdown

No dependency changes detected. Learn more about Socket for GitHub.

👍 No dependency changes detected in pull request

@jvbriones jvbriones added no-changelog no-changelog Indicates no external facing user changes, therefore no changelog documentation needed skip-sonar-cloud Only used for bypassing sonar cloud when failures are not relevant to the changes. labels Sep 28, 2026
@jvbriones
jvbriones marked this pull request as ready for review September 28, 2026 14:04
@jvbriones
jvbriones requested a review from a team as a code owner September 28, 2026 14:04
@jvbriones jvbriones added the DO-NOT-MERGE Pull requests that should not be merged label Sep 28, 2026
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 28, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-28T14:08:53.821179Z f956d52 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f956d5228b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/check-analytics-contract.yml Outdated
@github-actions github-actions Bot added the risk:high AI analysis: high risk label Sep 28, 2026
@jvbriones jvbriones self-assigned this Sep 28, 2026
@jvbriones jvbriones changed the title chore(MMQA-1951): Use typed analytics schema contract - Evaluation chore(MMQA-1951): Use typed analytics schema contract - 1st Iteration Sep 28, 2026
@jvbriones jvbriones changed the title chore(MMQA-1951): Use typed analytics schema contract - 1st Iteration chore(MMQA-1951): Use typed analytics schema contract - 1st iteration Sep 28, 2026
@jvbriones jvbriones changed the title chore(MMQA-1951): Use typed analytics schema contract - 1st iteration chore(MMQA-1951): Add CI logic to start using typed analytics schema contract Sep 28, 2026
@jvbriones jvbriones changed the title chore(MMQA-1951): Add CI logic to start using typed analytics schema contract chore: Add CI logic to start using typed analytics schema contract Sep 28, 2026
@jvbriones jvbriones removed the skip-sonar-cloud Only used for bypassing sonar cloud when failures are not relevant to the changes. label Sep 28, 2026
Comment thread .github/workflows/check-analytics-contract.yml Outdated
Comment thread .github/workflows/check-analytics-contract.yml
@jvbriones jvbriones removed the DO-NOT-MERGE Pull requests that should not be merged label Sep 29, 2026
@jvbriones jvbriones added skip-e2e skip E2E test jobs and removed pr-not-ready-for-e2e Skip E2E and block merging. Remove this label once the PR is ready to run the E2E tests. labels Sep 29, 2026
@sonarqubecloud

Copy link
Copy Markdown

@NicolasMassart NicolasMassart left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before merge, I’d only ask that we finish the PR readiness housekeeping: complete the changelog/manual-testing/screenshots fields so they match the repository’s Ready for Review requirements, clean up the stale wording saying Typewriter is still included, and wait for the remaining policy checks to finish.

@jvbriones
jvbriones added this pull request to the merge queue Sep 29, 2026
Merged via the queue into main with commit f637c3f Sep 29, 2026
129 of 163 checks passed
@jvbriones
jvbriones deleted the feat/schema-first-approach branch September 29, 2026 15:10
@github-actions github-actions Bot locked and limited conversation to collaborators Sep 29, 2026
@metamask-ci metamask-ci Bot added the release-8.15.0 Issue or pull request that will be included in release 8.15.0 label Sep 29, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

no-changelog no-changelog Indicates no external facing user changes, therefore no changelog documentation needed release-8.15.0 Issue or pull request that will be included in release 8.15.0 risk:high AI analysis: high risk size-XL skip-e2e skip E2E test jobs team-qa QA team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants