Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 86 additions & 0 deletions .claude/skills/cross-repo-change/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
---
name: cross-repo-change
description: Make a change that spans the Privee server (MaxDac/Privee) and the Android app (MaxDac/PriveeApp, com.privee.app). Covers the shared contract (REST under /api/app, Phoenix channels, api_version, push payload, libsignal version), the safe order of work (server first and backward compatible, deploy, then app release, then F-Droid), app-side implementation in core/net, core/signal and app, tests, and compatibility with older app versions still installed from F-Droid. Use when a feature, fix or refactor needs both repos, when the server API or channel events change, when bumping api_version, or when asked how the app and server fit together.
---

# Change both repos

Privee is two repositories:
- **MaxDac/Privee**: the Phoenix server and the website, with a libsignal WASM client.
- **MaxDac/PriveeApp**: this Android app, with a libsignal Android client.

Deployment is owned separately by [MaxDac/PriveeDeploy](https://github.com/MaxDac/PriveeDeploy).
Privee runs CI only; merging a server PR does not deploy it.

The canonical contract and change order live on the server side:
- <https://github.com/MaxDac/Privee/blob/main/docs/cross-repo.md>: the contract, version rules and release order;
- <https://github.com/MaxDac/Privee/blob/main/docs/client-api.md>: endpoints and channel events;
- <https://github.com/MaxDac/Privee/blob/main/docs/e2e-encryption.md>: the protocol.

On the server, the matching skill is `api-change`.

This file exists twice and the two copies must stay byte-identical:
- `.claude/skills/cross-repo-change/SKILL.md`;
- `.github/skills/cross-repo-change/SKILL.md`.

`scripts/tests/test_skill_copies.py` enforces this.

## Why order matters

Server deployment is manual and can reach users before an app update. App updates reach users days or weeks later, and F-Droid builds lag further, so old app versions stay in use for a long time. Therefore:

1. **Server first, backward compatible.**
- Add new endpoints, events and fields.
- Keep old ones working.
- New request fields must be optional.
- Never change the meaning of an existing field.
2. **Deploy the server separately.** Follow the server's
[`deploy-privee` skill](https://github.com/MaxDac/Privee/blob/main/.github/skills/deploy-privee/SKILL.md).
Check successful server CI for the full source commit SHA, obtain explicit
production-deploy confirmation, then trigger PriveeDeploy's manual workflow:

```bash
gh workflow run deploy.yml -R MaxDac/PriveeDeploy -f ref=<full-source-sha>
```

For forks, use the owner's deploy repository and verify its `PRIVEE_REPO`
setting matches the server repository. Wait for deployment to succeed and
verify `GET <server>/api/app/info` before releasing an app that needs it.
3. **App change.** Use the new contract, and degrade gracefully on servers that do not have it yet: self-hosted instances update on their own schedule.
4. **App release** (`fdroid-release` skill), then F-Droid picks it up.
5. **Remove the old server behaviour** only after no supported app version uses it.

A breaking change needs an explicit migration plan, not just a new
`api_version`. The discovery response reports one integer and released apps
currently accept only `1`: changing it immediately locks those apps out.
First ship an app that accepts both versions and works with the old server.
Keep existing server behaviour working while users update, including F-Droid
users. Only then deploy the API-version change, with an agreed compatibility
window. Additive, backward-compatible changes keep API version `1`.

## App-side checklist

1. **Read the server PR and docs first.** Work in a separate clone of the server if you need to read code (`gh repo clone MaxDac/Privee`).
2. **REST:**
- edit `core/net/src/main/kotlin/com/privee/net/PriveeApi.kt`;
- add a MockWebServer test in `PriveeApiTest`;
- decode with `ignoreUnknownKeys`-style tolerance, so the server can add fields.
3. **Channels:**
- incoming events go in `PriveeSession.handle` (topic `session`) or in `ChatConversation` (topic `chat:<peer>`);
- calls go through `ServerCall` in `SignalClient`;
- unknown events must be ignored, not treated as errors.
4. **Protocol or key changes** go in `core/signal` and need tests in `SignalClientTest`. Both clients must agree, so check the website's behaviour and libsignal version (`libsignal-upgrade` skill).
5. **`api_version`:** update `ServerInfo.SUPPORTED_API_VERSIONS`, `ServerInfoTest`, and the "Choosing a server" section of the README.
6. **Push:** pushes are wake-ups only. Never add message content, sender names or keys to a push payload, on either side.
7. **Test against a local server.** Run the server's `main` (or the PR branch) and the debug app on an emulator at `http://10.0.2.2:4000`; see the `store-screenshots` skill, step 1.
8. **PR.**
- Use Conventional Commits, for example `feat(chat): …`, and link the server PR.
- The four required checks must pass.
- Do not touch `version.properties`.

## Invariants (both repos)

- The server only stores public key material, ciphertext and metadata it needs for routing. It never stores plaintext or private keys.
- Push payloads contain no content.
- Every API change stays compatible with released app versions; an API-version bump requires the staged migration above.
- Both repos are AGPL-3.0-only. The server exposes its source URL (`PRIVEE_SOURCE_URL`), and forks must keep doing so.
75 changes: 75 additions & 0 deletions .claude/skills/dependency-upgrade/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
---
name: dependency-upgrade
description: Upgrade Gradle, the Android Gradle Plugin, Kotlin, AndroidX/Compose, OkHttp, kotlinx, UnifiedPush or GitHub Actions in Privee for Android (com.privee.app) without breaking reproducible builds or F-Droid inclusion. Covers Dependabot PRs, the version catalogue, buildserver JDK/SDK/NDK constraints, the native-library allowlist, non-free dependency checks and the Reproducibility workflow. Use when asked to bump, update or upgrade a dependency, plugin, SDK level, Gradle wrapper or workflow action, or to review or merge a Dependabot PR. For libsignal use the libsignal-upgrade skill instead.
---

# Upgrade dependencies

Every published APK must be byte-identical when built by us and by F-Droid, and
must contain no proprietary code. Any dependency change can break either.

Background reading:
- [`docs/TECHNOLOGIES.md`](../../../docs/TECHNOLOGIES.md): the stack and why each part is there.
- [`docs/ARCHITECTURE.md`](../../../docs/ARCHITECTURE.md): the section on changes that affect reproducible builds.
- [`docs/FDROID_VALIDATION.md`](../../../docs/FDROID_VALIDATION.md): local buildserver builds and diffoscope triage.

This file exists twice and the two copies must stay byte-identical:
- `.claude/skills/dependency-upgrade/SKILL.md`;
- `.github/skills/dependency-upgrade/SKILL.md`.

`scripts/tests/test_skill_copies.py` enforces this.

## Where versions live

| What | File |
|---|---|
| Libraries and plugins | `gradle/libs.versions.toml` (single source of truth) |
| Gradle | `gradle/wrapper/gradle-wrapper.properties` (use `./gradlew wrapper --gradle-version X`; this also updates the wrapper jar) |
| SDK levels, JVM target | `app/build.gradle.kts`, `core/*/build.gradle.kts` |
| CI SDK packages and JDK | `.github/workflows/ci.yml` (`platforms;android-NN`, `build-tools;X`, Java 17) |
| Actions | `.github/workflows/*.yml`. `reproducibility.yml` and `release.yml` pin actions by SHA, so keep doing that. |
| Buildserver image | `scripts/fdroid-rb-docker.sh` (digest) |
| Release tooling | `PyYAML==6.0.3` in `ci.yml` |

Dependabot (`.github/dependabot.yml`) opens weekly PRs for Gradle dependencies and Actions.

## Procedure

1. **One logical change per PR.** Change one library, or one coupled group (for example AGP plus Gradle, or Kotlin plus Compose compiler), in each PR. Title: `build(deps): bump <name> from A to B`.
2. **Check that the license is free.**
- No Google Play Services, Firebase, GMS or proprietary SDKs.
- Check that new transitive dependencies are not proprietary: `./gradlew :app:dependencies --configuration releaseRuntimeClasspath`.
3. **Check the buildserver can build it.**
- F-Droid builds in `fdroidserver:buildserver-trixie` (Debian trixie, OpenJDK 21) with the SDK packages the recipe requests.
- AGP or Gradle bumps that need a newer JDK, or a `compileSdk` not yet in the buildserver, will fail there even if CI passes.
- If `compileSdk` or build-tools change, update `ci.yml` and the README requirements too.
4. **Build and test locally:**
```bash
./gradlew test :app:lintDebug :app:assembleDebug
python3 -m unittest discover -s scripts/tests # WSL on Windows
```
5. **Check the native libraries.**
- `scripts/verify_release_apk.py` allows exactly two `.so` files: `lib/arm64-v8a/libsignal_jni.so` and `lib/arm64-v8a/libandroidx.graphics.path.so`.
- If a bump adds a native library, either exclude it in `app/build.gradle.kts` (`packaging.jniLibs.excludes`) or stop and ask. A new prebuilt `.so` is usually not acceptable on F-Droid.
6. **Push the PR and wait for the four required checks:**
- `Android (test, lint, assemble)`;
- `Build (replay, release checks)`;
- `Build (fdroid build)`;
- `compare`.

If `compare` fails, download the diffoscope artifact and use the triage table in FDROID_VALIDATION.md.
7. **Do not release in the same PR.** Version bumps go through the release-bump PR (skill `fdroid-release`).

## Special cases

| Upgrade | Watch out for |
|---|---|
| AGP | DSL changes in `app/build.gradle.kts`; `packaging` and `jniLibs.keepDebugSymbols` behaviour; dex differences between JDKs (the buildserver uses JDK 21) |
| Gradle wrapper | Commit `gradle-wrapper.jar` and the properties file together; `validateDistributionUrl` stays `true` |
| Kotlin | Compose compiler and serialization plugins share `kotlin` in the catalogue |
| Compose BOM | `material-icons-core` is pinned outside the BOM (1.7.8), so leave it there |
| UnifiedPush connector | Re-test push end to end with a distributor (ntfy); check `PriveePushService` API changes |
| OkHttp | `PhoenixSocket` WebSocket behaviour and `mockwebserver3` tests |
| desugar_jdk_libs | Required by libsignal-android; keep `isCoreLibraryDesugaringEnabled` |
| Buildserver image digest | Take it from fdroiddata's CI; rerun the Reproducibility workflow (see FDROID_VALIDATION.md) |
| libsignal | Use the `libsignal-upgrade` skill |
2 changes: 2 additions & 0 deletions .claude/skills/fdroid-release/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ Background reading, if a step fails:
- [`docs/RELEASING.md`](../../../docs/RELEASING.md): Release workflow, one-time setup, recovery.
- [`docs/FDROID_VALIDATION.md`](../../../docs/FDROID_VALIDATION.md): local and buildserver builds, diffoscope.
- [`libsignal/README.md`](../../../libsignal/README.md): source build of `libsignal_jni.so`, re-pinning.
- [`docs/ARCHITECTURE.md`](../../../docs/ARCHITECTURE.md) and [`docs/TECHNOLOGIES.md`](../../../docs/TECHNOLOGIES.md): what changes affect reproducibility, and the stack.
- Related skills: `libsignal-upgrade`, `dependency-upgrade`, `store-screenshots` (refresh store images before a release), `cross-repo-change` (server changes must be deployed before the app release that needs them).

This file exists twice and the two copies must stay byte-identical:
- `.claude/skills/fdroid-release/SKILL.md` (Claude Code);
Expand Down
94 changes: 94 additions & 0 deletions .claude/skills/libsignal-upgrade/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
---
name: libsignal-upgrade
description: Upgrade libsignal in Privee for Android (com.privee.app) while keeping release builds reproducible and F-Droid-compatible. Covers the version catalogue, libsignal/source.lock.json (tag, commit, Rust toolchain, NDK, Cargo cfg, prune list), rebuilding libsignal_jni.so from source, re-pinning its SHA-256 from the F-Droid buildserver image, the Reproducibility workflow, and keeping the Privee website's libsignal WASM on the same version. Use when asked to bump, update or upgrade libsignal, change the NDK or Rust toolchain for libsignal, or when verifyLibsignal or the Reproducibility check fails on libsignal_jni.so.
---

# Upgrade libsignal

The app uses libsignal twice:
- in debug builds and CI, from Maven (`org.signal:libsignal-client` and `libsignal-android`);
- in release and F-Droid builds, built from source at a pinned commit (`-PlibsignalBuiltFromSource`).

Both must use the same version. That version must also match the libsignal WASM build used by the Privee website, because the two clients talk to each other.

Background reading:
- [`libsignal/README.md`](../../../libsignal/README.md): the source build and the bump checklist.
- [`docs/FDROID_VALIDATION.md`](../../../docs/FDROID_VALIDATION.md): buildserver builds and the diffoscope triage table.
- [`docs/TECHNOLOGIES.md`](../../../docs/TECHNOLOGIES.md): why libsignal is built from source.
- Server side: <https://github.com/MaxDac/Privee/blob/main/docs/cross-repo.md>, the libsignal alignment section.

This file exists twice and the two copies must stay byte-identical:
- `.claude/skills/libsignal-upgrade/SKILL.md`;
- `.github/skills/libsignal-upgrade/SKILL.md`.

`scripts/tests/test_skill_copies.py` enforces this.

## 0. Decide the target version

```bash
git ls-remote --tags https://github.com/signalapp/libsignal 'refs/tags/v*' | sort -t/ -k3 -V | tail -5
grep -n libsignal gradle/libs.versions.toml
```

Check which libsignal version the Privee website uses (see the cross-repo doc). Ship an upgrade together with the server, or after it. If the protocol changed between versions, read libsignal's release notes and Privee's `docs/e2e-encryption.md` before going ahead.

## 1. Bump the pins

1. In `gradle/libs.versions.toml`, set `libsignal = "X.Y.Z"`.
2. In `libsignal/source.lock.json`:
- set `source.tag` to `vX.Y.Z`;
- set `source.commit` to the full SHA from `git ls-remote https://github.com/signalapp/libsignal refs/tags/vX.Y.Z`. Take the peeled `^{}` SHA if the tag is annotated.
- set `rust.toolchain` to the contents of libsignal's `rust-toolchain` file at that tag;
- check `build.ndkRevision`, `build.rustCfg` and `build.features` against libsignal's `java/build_jni.sh` and `.cargo/config.toml` at that tag;
- check that every `preparation.keep` path still exists at that tag.
3. If the NDK revision changes, also update `ndk:` in `metadata/com.privee.app.yml` and in the matching block of `docs/FDROID.md`.

## 2. Build and test against Maven

```bash
./gradlew test :app:lintDebug :app:assembleDebug
```

This build uses the Maven artifacts. Fix any API changes in `core/signal`, and in `app` where it uses libsignal directly. Then run `SignalClientTest`.

## 3. Build from source locally (Linux or WSL)

```bash
NDK_ROOT=/path/to/android-ndk-r28c bash libsignal/scripts/build-libsignal.sh all
./gradlew -PlibsignalBuiltFromSource -PallowUnpinnedLibsignal :app:assembleRelease
```

`-PallowUnpinnedLibsignal` is for this step only. Never commit or use it in CI.

## 4. Re-pin the `.so` hash from the buildserver image

The hash that matters is the one F-Droid gets, so build in the pinned buildserver image:

```bash
bash scripts/fdroid-rb-docker.sh out
cat out/libsignal/SHA256SUMS
```

Write the `libsignal_jni.so` hash into `build.expectedSha256` in `libsignal/source.lock.json`. You can also open the PR first and take the hash from the Reproducibility workflow's artifacts.

## 5. Scanner and blobs

Run the F-Droid scanner over the pruned checkout (see FDROID_VALIDATION.md). If a new binary or blob appears, add it to `preparation.remove`. Never use `scanignore`.

## 6. PR

- Title: `build(deps): bump libsignal to X.Y.Z`.
- The four required checks must pass. The Reproducibility checks (`Build (replay, release checks)`, `Build (fdroid build)` and `compare`) prove both F-Droid builders agree.
- Mention the matching server or website libsignal version in the PR body.
- Do not change `version.properties` in this PR; releasing is a separate release-bump PR (skill `fdroid-release`).

## Troubleshooting

| Symptom | Cause | Fix |
|---|---|---|
| `verifyLibsignal` fails | `.so` hash differs from the pin | Expected after a bump: re-pin from the buildserver image (step 4) |
| Module fails: tag differs from catalogue | Catalogue and lock disagree | Make both `X.Y.Z` |
| `cargo` fails offline | `fetch` step did not run, or `Cargo.lock` changed | Run `build-libsignal.sh fetch` (or `all`) again |
| `compare` fails on `lib/arm64-v8a/libsignal_jni.so` | Toolchain, NDK or path remapping differs | Compare `PROVENANCE` and `SHA256SUMS` of both artifacts |
| `verify_release_apk.py` fails on natives | A new `.so` was packaged | Remove it or justify it; the allowed list is in the script |
| Website and app cannot talk | libsignal versions differ in a protocol-relevant way | Align versions with the server (cross-repo doc) |
Loading
Loading