Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@ Skill / 命令手册随 `skills/bailian-*/` 经 `npx skills add modelstudioai/cl
| Skill 文案 / 路由 | 改 SKILL 路由、安装约定、hand-off、hub/领域边界 | [docs/agents/skill-change.md](docs/agents/skill-change.md) |
| 错误文案变更 | 改 `BailianError` 的 message 或 hint | [docs/agents/error-hint-change.md](docs/agents/error-hint-change.md) |
| URL / 渠道变更 | 控制台域名 / 文档站 / 追踪参数 | [docs/agents/url-change.md](docs/agents/url-change.md) |
| 埋点变更 | 改 AEM 命令事件、后端渠道 header、User-Agent | [docs/agents/telemetry-change.md](docs/agents/telemetry-change.md) |
| 鉴权扩展 | 加 OAuth / SSO / 换 token 来源 | [docs/agents/auth-change.md](docs/agents/auth-change.md) |
| 配置项扩展 | 新 env var 或 `~/.bailian/config.json` 字段 | [docs/agents/config-add.md](docs/agents/config-add.md) |
| Profile / 激活 | 改命名 Profile、预设或 `active_config` | [docs/agents/config-profile-change.md](docs/agents/config-profile-change.md) |
Expand Down
165 changes: 165 additions & 0 deletions docs/agents/telemetry-change.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,165 @@
# 埋点变更

## 触发条件

- 调整 AEM 命令事件、事件字段或参数 allowlist
- 调整 `User-Agent`、`x-dashscope-source-config` 或其他后端渠道标识
- 新增鉴权域、请求网关或绕开统一 Client 的网络出口
- 排查命令量、成功率、版本、鉴权域或后端渠道数据不一致

## 当前数据流

三套鉴权对应三套请求域,但不代表三套网关使用相同的后端埋点。命令侧另有一套覆盖所有实际执行命令的 AEM 客户端事件,两者必须分开理解。

```text
命令进入 run
├─ telemetryStage
│ ├─ ~/.bailian/telemetry.jsonl
│ └─ AEM(pid=bailian-cli-node, event name=命令路径)
└─ authStage
├─ apiKey → DashScope / 模型域
├─ console → Bailian Console Gateway
├─ openapi → 阿里云 OpenAPI
└─ none → 无凭证域;本地命令也仍有 AEM 命令事件
```

### 1. 三套鉴权与埋点标识

| 命令声明 | 凭证 / 请求域 | 主要请求出口 | 后端埋点标识 | 前端埋点标识(AEM) |
| ----------------- | --------------------------------------------------- | ------------------------------------------------------------------------------------- | --------------------------------------------- | ------------------------------------------------ |
| `auth: "apiKey"` | API Key;DashScope / OpenAI-compatible 模型域 | `Client.request/requestJson`、`McpClient`、Managed Agent instrumented fetch、上传策略 | 有:`User-Agent`、`x-dashscope-source-config` | 有:`pid=bailian-cli-node`、`authMethod=apiKey` |
| `auth: "console"` | Console access token;Bailian Console Gateway | `callConsoleGateway()` → `/cli/api.json` | 无 | 有:`pid=bailian-cli-node`、`authMethod=console` |
| `auth: "openapi"` | AccessKey ID/Secret,可选 STS token;阿里云 OpenAPI | `Client.openApiJson()` | 有:`x-dashscope-source-config` | 有:`pid=bailian-cli-node`、`authMethod=openapi` |
| `auth: "none"` | 无凭证域 | 本地逻辑或命令自行管理的登录/配置流程 | 无 | 有:`pid=bailian-cli-node`、`authMethod=none` |

`authMethod` 记录的是命令声明的鉴权域,不是凭证来源。它不会区分 API Key 来自 flag、env 还是 config。
鉴权域是命令的准入门槛和主请求域,不保证命令内部只有一种网络出口;例如部分 `apiKey` 命令也可能读取匿名 Console 公共目录,Managed Agent 还可能访问其他 provider。

表中的后端埋点按该鉴权域的主要业务请求填写:

- Managed Agent 的 `User-Agent` 对所有 SDK 请求注入;`x-dashscope-source-config` 仅对阿里云 host 注入
- DashScope 上传策略 `getPolicy` 只有 `x-dashscope-source-config`,没有显式 CLI `User-Agent`
- OpenAPI 的 ACS 签名头,以及 Console Gateway 的 `product`、`action`、`api` 是鉴权或路由字段,不计为埋点标识

### 2. 后端渠道参数

当前 `x-dashscope-source-config` 结构为:

```json
{
"channel": "bailian-cli",
"tags": {
"t1": "public",
"t2": "bl 或 kscli",
"t3": "实际 CLI 版本"
}
}
```

- `t2` 取产品 `identity.binName`:完整 CLI 为 `bl`,Knowledge Studio CLI 为 `kscli`
- `t3` 取产品 `identity.version`,由产品入口的 `package.json` 注入
- `channel` 与 `t1` 是当前固定口径
- `User-Agent` 是独立标识:`bl` 为 `bailian-cli/<version>`,`kscli` 为 `knowledge-studio-cli/<version>`

source-config 只用于百炼 / DashScope API 侧消费,不发送到通用网络传输:

| 请求 | source-config |
| ------------------------------------ | ------------- |
| 模型 API、任务提交与轮询 | 有 |
| Bailian MCP / OpenAPI | 有 |
| DashScope 上传策略 `getPolicy` | 有 |
| OSS 文件上传 | 无 |
| 图片、视频、音频、转录结果下载 | 无 |
| npm / 二进制更新检查、Skill registry | 无 |

当前已知例外:Pipeline runtime 自建的 `Identity.version` 为 `0.0.0-dev`,因此 Pipeline 内部模型请求的 `t3` 不代表产品包版本;现阶段不纳入本轮收敛。

### 3. 全命令 AEM 客户端埋点

`packages/runtime/src/middleware.ts` 的 `telemetryStage` 包裹 `authStage` 与命令执行,因此成功、业务失败、网络失败和鉴权失败都会形成一次命令事件。事件名是空格连接的命令路径,例如 `text chat`。

以下情况不会形成命令事件,因为没有进入 middleware 的 `run`:

- 根帮助、子命令 `--help`、`--version`
- 未识别命令、参数解析失败、缺少必填参数
- `defineCommand.validate` 在 dispatch 阶段拒绝的请求

遥测默认开启;`DO_NOT_TRACK=1` 一票否决,配置文件 `telemetry: false` 也可关闭。关闭后本地和远端均不记录。

单条 `TrackingEvent` 当前包含:

- `command`、`timestamp`、`durationMs`、`success`
- `cliVersion`、`nodeVersion`、`os`
- `authMethod`
- 失败时的 `errorMessage`、`httpStatus`、`requestId`
- 安全 allowlist 过滤后的 `params`

参数默认不上传,只有 `packages/core/src/telemetry/tracker.ts` 的 `PARAM_ALLOWLIST` 中字段会进入事件。不得加入 prompt、凭证、文件路径、URL、账号/租户/工作空间 ID 或其他用户内容。

事件同时写入两处:

1. 本地 `~/.bailian/telemetry.jsonl`:权限 `0600`,超过 5 MB 后重建
2. AEM:`pid=bailian-cli-node`,源码运行自动使用 `env=dev`,npm 安装或编译二进制使用 `env=prod`

底层 Node tracker 还会附加公共设备字段:OS 类型/版本、Node 应用名与版本、平台,以及由本机网络标识计算的 MD5 `device_id`。

当前 AEM 事件没有 `binName` 或 `clientName` 产品维度,并且 `bl`、`kscli` 共用 `pid=bailian-cli-node`。两边相同路径的 `config show`、`config set`、`update` 无法仅凭当前事件稳定区分产品;Knowledge 命令虽然因路径映射不同而表现为 `knowledge chat` 与 `chat`,也不应把命令路径当作长期产品标识。后端 source-config 的 `t2` 已能区分 `bl/kscli`,但这个维度尚未进入 AEM 客户端事件。

AEM 映射:

| AEM 字段 | 内容 |
| ---------- | ----------------------------------------- |
| event name | 命令路径 |
| `et` | `EXP` |
| `ext` | 除 `command`、`params` 外的结构化事件字段 |
| `c1` | allowlist 参数 |
| `c2` | `success` / `failure` |
| `c3` | HTTP status |
| `c4` | 错误文案,最多 500 字符 |
| `c5` | request ID |

远端发送是 best-effort,不得阻塞命令或改变退出码。正常退出最多等待 1 秒,SIGINT 最多等待 500 ms。

## 必查清单

### A. 新增或调整命令

- [ ] `defineCommand({ auth })` 必须声明真实请求域;AEM 的 `authMethod` 直接读取该值
- [ ] 新命令进入 `run` 后自动有基础事件,不得在命令内重复发送同名事件
- [ ] 需要按产品分析 AEM 数据时,必须显式设计产品字段;不得从命令路径推断 `bl/kscli`
- [ ] 只有可枚举、数值或布尔等低风险字段才可加入 `PARAM_ALLOWLIST`
- [ ] 新增 console raw API flag 时只允许记录公开 API 名,不得记录请求 `data`

### B. 调整后端渠道参数

- [ ] 同时核对 `packages/core/src/client/http.ts`、`mcp.ts`、`instrumented-fetch.ts`、`client.ts` 与 `files/upload.ts`
- [ ] 产品身份必须来自 `Identity`;不得从命令路径、环境变量或 `process.argv` 猜测
- [ ] `bl` 与 `kscli` 必须分别验证 `binName`、`clientName`、`version`
- [ ] OSS、结果文件、npm、二进制和 Skill 下载不得为了业务渠道统计新增 source-config
- [ ] 改 URL / host 范围时同时执行 [URL / 渠道变更](url-change.md) 清单

### C. 调整 AEM 事件

- [ ] 更新 `TrackingEvent`、`createTrackingEvent()` 与 `buildRemoteAemOptions()` 的字段映射
- [ ] 本地 JSONL 与远端 AEM 必须基于同一结构化事件,不能维护两套字段口径
- [ ] 成功与失败均覆盖;遥测异常必须静默且不改变业务退出码
- [ ] 检查 `DO_NOT_TRACK=1` 与 `telemetry: false` 两个关闭入口
- [ ] 错误字段不得额外拼接 token、请求体、prompt 或本地路径

## 完成后自查

```sh
rg -n "trackingHeaders|x-dashscope-source-config|User-Agent" packages --glob '*.ts'
rg -n "trackCommandExecution|PARAM_ALLOWLIST|buildRemoteAemOptions" packages/core packages/runtime --glob '*.ts'
vp check
vp test packages/core/tests packages/commands/tests/e2e/auth.e2e.test.ts
```

## 常见漏点

- ✗ 只看 AEM 命令事件,误以为它能替代网关侧请求渠道统计
- ✗ 把 `authMethod` 当成实际凭证来源;它只是命令声明的鉴权域
- ✗ 新增 bypass `fetch` 后漏掉应由网关消费的 source-config,或把它发给 OSS / npm / 第三方下载地址
- ✗ 只改 `bl` 入口,导致 `kscli` 的产品名或版本标签错误
- ✗ 把帮助、版本或参数校验失败算进“全部命令”;这些路径当前没有进入 telemetry middleware
5 changes: 1 addition & 4 deletions packages/commands/src/commands/speech/recognize.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@ import {
type DashScopeASRRequest,
type DashScopeASRTaskResult,
type DashScopeAsyncResponse,
trackingHeaders,
stripUndefined,
taskPath,
speechRecognizePath,
Expand Down Expand Up @@ -201,9 +200,7 @@ async function handleAsyncMode(
}

// Fetch transcription JSON
const transRes = await fetch(subResult.transcription_url, {
headers: trackingHeaders(),
});
const transRes = await fetch(subResult.transcription_url);
if (!transRes.ok) {
throw new BailianError(
`Failed to download transcription: HTTP ${transRes.status}`,
Expand Down
7 changes: 5 additions & 2 deletions packages/core/src/client/client.ts
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,10 @@ export class Client {
/** Resolve a file arg: upload a local path to OSS (returns oss:// URL), or pass a URL through. */
uploadFile(source: string, model: string, opts: { signal?: AbortSignal } = {}): Promise<string> {
if (!isLocalFile(source)) return Promise.resolve(source);
return resolveFileUrl(source, this.requireApi().token, model, opts);
return resolveFileUrl(source, this.requireApi().token, model, {
...opts,
identity: this.deps.identity,
});
}

/**
Expand Down Expand Up @@ -233,7 +236,7 @@ export class Client {
const timeoutMs = this.deps.settings.timeout * 1000;
const res = await fetch(endpoint, {
method: opts.method,
headers: { ...headers, ...trackingHeaders() },
headers: { ...headers, ...trackingHeaders(this.deps.identity) },
body: bodyStr || undefined,
signal: AbortSignal.timeout(timeoutMs),
});
Expand Down
30 changes: 19 additions & 11 deletions packages/core/src/client/headers.ts
Original file line number Diff line number Diff line change
@@ -1,23 +1,31 @@
/**
* Shared HTTP request headers for all outgoing requests.
*
* Centralises the `x-dashscope-source-config` header so every fetch call
* (both via the central http client and the bypass paths) uses the
* same values from a single source of truth.
* Centralises the `x-dashscope-source-config` header so Bailian/DashScope API
* transports use the same product identity. Generic npm, OSS, and result-file
* transfers deliberately do not send this gateway-consumed metadata.
*/

import type { Identity } from "../config/schema.ts";

export const CHANNEL = "bailian-cli";

export const TAGS = { t1: "public", t2: "" };
export type TrackingIdentity = Pick<Identity, "binName" | "version">;

export const SOURCE_CONFIG = JSON.stringify({
channel: CHANNEL,
tags: TAGS,
});
export function sourceConfig(identity: TrackingIdentity): string {
return JSON.stringify({
channel: CHANNEL,
tags: {
t1: "public",
t2: identity.binName,
t3: identity.version,
},
});
}

/** Standard tracking headers required on every outbound request. */
export function trackingHeaders(): Record<string, string> {
/** Tracking headers for Bailian/DashScope API requests. */
export function trackingHeaders(identity: TrackingIdentity): Record<string, string> {
return {
"x-dashscope-source-config": SOURCE_CONFIG,
"x-dashscope-source-config": sourceConfig(identity),
};
}
6 changes: 3 additions & 3 deletions packages/core/src/client/http.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import { BailianError } from "../errors/base.ts";
import { ExitCode } from "../errors/codes.ts";
import { mapApiError } from "../errors/api.ts";
import { maskToken } from "../utils/token.ts";
import { SOURCE_CONFIG, trackingHeaders } from "./headers.ts";
import { sourceConfig, trackingHeaders } from "./headers.ts";

/** 传输层依赖:UA 用 identity,timeout/verbose 用 settings。凭证由调用方(Client)注头。 */
export interface HttpDeps {
Expand Down Expand Up @@ -39,7 +39,7 @@ export async function request(deps: HttpDeps, opts: RequestOpts): Promise<Respon

const headers: Record<string, string> = {
"User-Agent": `${deps.identity.clientName}/${deps.identity.version}`,
...trackingHeaders(),
...trackingHeaders(deps.identity),
...opts.headers,
};

Expand All @@ -59,7 +59,7 @@ export async function request(deps: HttpDeps, opts: RequestOpts): Promise<Respon
console.error(`> ${opts.method ?? "GET"} ${opts.url}`);
const auth = headers["Authorization"];
if (auth) console.error(`> Auth: ${maskToken(auth.replace(/^Bearer /, ""))}`);
console.error(`> x-dashscope-source-config: ${SOURCE_CONFIG}`);
console.error(`> x-dashscope-source-config: ${sourceConfig(deps.identity)}`);
}

const timeoutMs = (opts.timeout ?? deps.settings.timeout) * 1000;
Expand Down
2 changes: 1 addition & 1 deletion packages/core/src/client/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ export {
type ImageInputStyle,
type ImageSizeProfile,
} from "./image-routes.ts";
export { CHANNEL, SOURCE_CONFIG, TAGS, trackingHeaders } from "./headers.ts";
export { CHANNEL, sourceConfig, trackingHeaders, type TrackingIdentity } from "./headers.ts";
export type { HttpDeps, RequestOpts } from "./http.ts";
export { request, requestJson } from "./http.ts";
export { createInstrumentedFetch, type FetchImplementation } from "./instrumented-fetch.ts";
Expand Down
2 changes: 1 addition & 1 deletion packages/core/src/client/instrumented-fetch.ts
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ export function createInstrumentedFetch(deps: HttpDeps): FetchImplementation {
headers.set("User-Agent", `${deps.identity.clientName}/${deps.identity.version}`);
}
if (isAlibabaCloudHost(url)) {
for (const [name, value] of Object.entries(trackingHeaders())) {
for (const [name, value] of Object.entries(trackingHeaders(deps.identity))) {
headers.set(name, value);
}
}
Expand Down
2 changes: 1 addition & 1 deletion packages/core/src/client/mcp.ts
Original file line number Diff line number Diff line change
Expand Up @@ -148,7 +148,7 @@ export class McpClient {
"Content-Type": "application/json",
Accept: "application/json, text/event-stream",
"User-Agent": `${this.deps.identity.clientName}/${this.deps.identity.version}`,
...trackingHeaders(),
...trackingHeaders(this.deps.identity),
};

if (this.authToken) {
Expand Down
23 changes: 14 additions & 9 deletions packages/core/src/files/upload.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import { existsSync, readFileSync, statSync } from "fs";
import { basename, extname } from "path";
import { BailianError } from "../errors/base.ts";
import { ExitCode } from "../errors/codes.ts";
import { trackingHeaders } from "../client/headers.ts";
import { trackingHeaders, type TrackingIdentity } from "../client/headers.ts";
import { REGIONS } from "../config/schema.ts";

// Pinned to cn region; thread baseUrl through if overseas upload becomes a requirement.
Expand All @@ -36,6 +36,7 @@ interface UploadPolicyResponse {
async function getUploadPolicy(
apiKey: string,
model: string,
identity: TrackingIdentity,
signal?: AbortSignal,
): Promise<UploadPolicy> {
const url = `${UPLOAD_API}?action=getPolicy&model=${encodeURIComponent(model)}`;
Expand All @@ -44,7 +45,7 @@ async function getUploadPolicy(
headers: {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
...trackingHeaders(),
...trackingHeaders(identity),
},
signal: policySignal.signal,
}).finally(policySignal.cleanup);
Expand Down Expand Up @@ -87,9 +88,6 @@ async function uploadToOSS(
const uploadSignal = combineWithTimeout(120_000, signal);
const res = await fetch(policy.upload_host, {
method: "POST",
headers: {
...trackingHeaders(),
},
body: form,
signal: uploadSignal.signal,
}).finally(uploadSignal.cleanup);
Expand All @@ -109,6 +107,7 @@ export interface UploadOptions {
apiKey: string;
model: string;
filePath: string;
identity: TrackingIdentity;
signal?: AbortSignal;
}

Expand Down Expand Up @@ -160,7 +159,7 @@ export function redactDataUri(input: string): string {
* The URL is valid for 48 hours.
*/
export async function uploadFile(opts: UploadOptions): Promise<string> {
const { apiKey, model, filePath, signal } = opts;
const { apiKey, model, filePath, identity, signal } = opts;

if (!existsSync(filePath)) {
throw new BailianError(`File not found: ${filePath}`, ExitCode.USAGE);
Expand All @@ -171,7 +170,7 @@ export async function uploadFile(opts: UploadOptions): Promise<string> {
throw new BailianError(`Not a file: ${filePath}`, ExitCode.USAGE);
}

const policy = await getUploadPolicy(apiKey, model, signal);
const policy = await getUploadPolicy(apiKey, model, identity, signal);
return uploadToOSS(policy, filePath, signal);
}

Expand All @@ -193,10 +192,16 @@ export async function resolveFileUrl(
input: string,
apiKey: string,
model: string,
opts: { signal?: AbortSignal } = {},
opts: { identity: TrackingIdentity; signal?: AbortSignal },
): Promise<string> {
if (!isLocalFile(input)) return input;
return uploadFile({ apiKey, model, filePath: input, signal: opts.signal });
return uploadFile({
apiKey,
model,
filePath: input,
identity: opts.identity,
signal: opts.signal,
});
}

function combineWithTimeout(
Expand Down
Loading