Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
91 changes: 91 additions & 0 deletions docs/docs/plugins/analytics.md
Original file line number Diff line number Diff line change
Expand Up @@ -282,6 +282,95 @@ On failure it emits an `error` event instead.

Editing `definitions.json` is picked up on the next request — no server restart is needed. A previously malformed file that you fix likewise starts working on the next request.

## Query tags

Every statement issued by the `/query` and `/metric` routes and by the `query` agent tool carries [query tags](https://docs.databricks.com/aws/en/sql/user/queries/query-tags). Tags are recorded in the `query_tags` column of `system.query.history`, so warehouse owners can attribute load and latency to the app and query that caused it.

AppKit adds these tags automatically:

| Tag | Value |
| ------------------ | ---------------------------------------------------------------------- |
| `appkit_app` | `DATABRICKS_APP_NAME` (omitted when unset, e.g. local development) |
| `appkit_query_key` | The query key (`/query`) or metric key (`/metric`) |
| `appkit_route` | `query`, `metric`, or `agent_tool` |
| `appkit_lane` | `sp` (service principal) or `obo` (on behalf of the user) |

The user's identity is never tagged. On-behalf-of statements already run as that user, so `system.query.history` attributes them.

### Custom tags

Add your own tags with the `queryTags` config, as a static map or a function evaluated per request:

```ts
analytics({
// static
queryTags: { team: "finance" },
});

analytics({
// per request: ctx has route, queryKey, lane, req, and the client's tags
queryTags: ({ req }) => {
const tenant = req?.header("x-tenant");
return tenant ? { tenant } : undefined;
},
});
```

The same query can run on several pages, so the client can also tag each request. A typical use is naming the page that issued it:

```tsx
useAnalyticsQuery("revenue", params, { queryTags: { page: "sales_overview" } });
useMetricView("orders", { measures: ["revenue"], queryTags: { page: "orders" } });
```

Client tags are untrusted input, so any user of the app can set them. How they're handled depends on the `queryTags` config:

- **No config, or a static map:** validated client tags are sent. On a key collision, the config wins.
- **A function:** its return value is the whole custom set. The function receives the validated client tags as `ctx.clientTags`, and they're only sent if it returns them. To keep a client's `page` tag but nothing else:

```ts
analytics({
queryTags: ({ clientTags, req }) => {
const tags: Record<string, string> = {};
if (clientTags.page) tags.page = clientTags.page;
const tenant = req?.header("x-tenant");
if (tenant) tags.tenant = tenant;
return tags;
},
});
```

Use a function whenever a tag feeds chargeback or governance, so a client can't fill in a key you left out. Set `clientQueryTags: false` to ignore client tags entirely.

**Rules:**

- Precedence on collisions: automatic `appkit_*` tags, then the `queryTags` config, then client tags. The `appkit_` prefix is reserved (in any letter case) and can't be set by config or the client.
- At most 20 tags are sent per statement, with automatic tags first.
- Keys must not contain `,` `:` `-` `/` `=` `.` or control characters, start with `@@`, or have leading or trailing spaces. Keys and values are limited to 128 characters; longer values are truncated. A `null` value records a key-only tag.
- Invalid tags are dropped, never failing the query. Invalid config tags log a warning.
- Tags are stored as plain text. Never put secrets or PII in them.
- Tags aren't part of the cache key. Cached results never reach the warehouse, so they're not tagged. Concurrent identical requests share one in-flight statement, which carries the tags of the request that arrived first. For per-page or per-request tags, that means only the request that actually ran the statement is recorded, and other requests served by that run go uncounted.

| Option | Type | Default | Description |
| ----------------- | ------------------------------ | ------- | ------------------------------------------------------------------ |
| `queryTags` | `QueryTagMap \| (ctx) => QueryTagMap` | — | Custom tags for every statement |
| `autoQueryTags` | `boolean` | `true` | Add the automatic `appkit_*` tags |
| `clientQueryTags` | `boolean` | `true` | Forward validated tags sent by `useAnalyticsQuery`/`useMetricView` |

To find slow pages, for example:

```sql
SELECT
query_tags['appkit_query_key'] AS query_key,
query_tags['page'] AS page,
count(*) AS runs,
percentile(total_duration_ms, 0.9) AS p90_ms
FROM system.query.history
WHERE query_tags['appkit_app'] = 'my-app'
GROUP BY ALL
ORDER BY p90_ms DESC;
```

## Frontend usage

### useAnalyticsQuery
Expand Down Expand Up @@ -316,6 +405,7 @@ const { data, loading, error } = useAnalyticsQuery(
| `format` | `"JSON" \| "ARROW"` | `"JSON"` | Response format |
| `maxParametersSize` | `number` | `102400` | Max serialized parameters size in bytes |
| `autoStart` | `boolean` | `true` | Start query on mount |
| `queryTags` | `AnalyticsQueryTags` | — | Query tags for this request (see [Query tags](#query-tags)) |

### Warehouse readiness

Expand Down Expand Up @@ -562,6 +652,7 @@ Time-series queries should explicitly order their selected time dimension ascend
| `orderBy` | `{field, direction?}[]` | no | Sort keys. `field` is narrowed to the measures/dimensions this call selected, so ordering by an unselected column is a type error. See [Deterministic results with `limit`](#deterministic-results-with-limit). |
| `limit` | `number` | no | Positive integer row cap. |
| `autoStart` | `boolean` | no | Start the metric query automatically. Defaults to `true`; set to `false` to defer it until the option becomes `true`. |
| `queryTags` | `AnalyticsQueryTags` | no | Query tags for this request (see [Query tags](#query-tags)). |

**Return type:**

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -320,6 +320,49 @@ describe("useAnalyticsQuery", () => {
expect(mockConnectSSE).toHaveBeenCalledTimes(1);
});

describe("queryTags", () => {
test("serializes queryTags into the POST body", () => {
renderHook(() =>
useAnalyticsQuery("q", null, { queryTags: { page: "overview" } }),
);

expect(JSON.parse(lastConnectArgs.payload)).toEqual({
parameters: null,
format: "JSON_ARRAY",
queryTags: { page: "overview" },
});
});

test("omits queryTags from the body when undefined", () => {
renderHook(() => useAnalyticsQuery("q", null));

expect(JSON.parse(lastConnectArgs.payload)).not.toHaveProperty(
"queryTags",
);
});

test("does not refetch when an equal queryTags object is passed on re-render", () => {
const { rerender } = renderHook(
({ page }) => useAnalyticsQuery("q", null, { queryTags: { page } }),
{ initialProps: { page: "overview" } },
);
rerender({ page: "overview" });

expect(mockConnectSSE).toHaveBeenCalledTimes(1);
});

test("issues separate requests for different queryTags", () => {
renderHook(() =>
useAnalyticsQuery("q", null, { queryTags: { page: "a" } }),
);
renderHook(() =>
useAnalyticsQuery("q", null, { queryTags: { page: "b" } }),
);

expect(mockConnectSSE).toHaveBeenCalledTimes(2);
});
});

describe("warehouse_status", () => {
test("surfaces warehouseStatus while waiting and clears loading on result", async () => {
let capturedOnMessage:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,20 @@ describe("useMetricView", () => {
});
});

test("serializes queryTags into the POST body when provided", () => {
renderHook(() =>
useMetricView("orders", {
measures: ["revenue"],
queryTags: { page: "overview" },
}),
);

expect(JSON.parse(lastConnectArgs.payload)).toEqual({
measures: ["revenue"],
queryTags: { page: "overview" },
});
});

test("omits orderBy from the body when undefined", () => {
renderHook(() =>
useMetricView("orders", {
Expand Down
1 change: 1 addition & 0 deletions packages/appkit-ui/src/react/hooks/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ export {
} from "../resource-status-indicator";
export type {
AnalyticsFormat,
AnalyticsQueryTags,
GrainsForSelectedTimeDims,
InferDimensionKeys,
InferMeasureKeys,
Expand Down
17 changes: 17 additions & 0 deletions packages/appkit-ui/src/react/hooks/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,18 @@ export interface TypedArrowTable<
// Query Options & Result Types
// ============================================================================

/**
* Query tags attached to the statement, recorded in
* `system.query.history.query_tags`. A `null` value records a key-only tag.
* The server validates them (keys must not contain `, : - / = .` or start with
* `appkit_`; keys/values are capped at 128 characters), drops invalid ones, and
* decides which to forward (server-configured tags win on collisions). Never
* put secrets or PII here.
*
* @example { page: "sales_overview" }
*/
export type AnalyticsQueryTags = Record<string, string | null>;

/** Options for configuring an analytics SSE query */
export interface UseAnalyticsQueryOptions<
F extends AnalyticsFormat = "JSON_ARRAY",
Expand All @@ -56,6 +68,9 @@ export interface UseAnalyticsQueryOptions<

/** Whether to automatically start the query when the hook is mounted. Default is true. */
autoStart?: boolean;

/** Query tags for this request, e.g. the page issuing it. See {@link AnalyticsQueryTags}. */
queryTags?: AnalyticsQueryTags;
}

/**
Expand Down Expand Up @@ -456,6 +471,8 @@ export type UseMetricViewOptions<
limit?: number;
/** Whether to automatically start the metric query. Default is true. */
autoStart?: boolean;
/** Query tags for this request, e.g. the page issuing it. See {@link AnalyticsQueryTags}. */
queryTags?: AnalyticsQueryTags;
} & MetricViewTimeOptions<K, D>;

export interface UseMetricViewResult<T = Record<string, unknown>[]> {
Expand Down
7 changes: 5 additions & 2 deletions packages/appkit-ui/src/react/hooks/use-analytics-query.ts
Original file line number Diff line number Diff line change
Expand Up @@ -125,12 +125,14 @@ export function useAnalyticsQuery<
}

const stableParameters = useStableParams(parameters);
const stableQueryTags = useStableParams(options?.queryTags);

const payload = useMemo(() => {
try {
const serialized = JSON.stringify({
parameters: stableParameters,
format,
queryTags: stableQueryTags,
});
const sizeInBytes = new Blob([serialized]).size;
if (sizeInBytes > maxParametersSize) {
Expand All @@ -144,10 +146,11 @@ export function useAnalyticsQuery<
console.error("useAnalyticsQuery: Failed to serialize parameters", error);
return null;
}
}, [stableParameters, format, maxParametersSize]);
}, [stableParameters, format, stableQueryTags, maxParametersSize]);

// Cache key shared across hook instances. `payload` already serializes
// `{ parameters, format }`, so identical requests collapse to one key.
// `{ parameters, format, queryTags }`, so identical requests collapse to one
// key.
// On a serialization failure (`payload === null`) the key stays unused: no
// request is retained and the store reports the stable idle snapshot.
const cacheKey = `${urlSuffix}::${payload}`;
Expand Down
3 changes: 3 additions & 0 deletions packages/appkit-ui/src/react/hooks/use-metric-view.ts
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,7 @@ export function useMetricView<
timeDimension?: unknown;
orderBy?: ReadonlyArray<unknown>;
limit?: number;
queryTags?: Record<string, string | null>;
} = { measures: options.measures };
if (options.dimensions !== undefined) body.dimensions = options.dimensions;
if (options.filter !== undefined) body.filter = options.filter;
Expand All @@ -127,6 +128,7 @@ export function useMetricView<
body.timeDimension = options.timeDimension;
if (options.orderBy !== undefined) body.orderBy = options.orderBy;
if (options.limit !== undefined) body.limit = options.limit;
if (options.queryTags !== undefined) body.queryTags = options.queryTags;
return JSON.stringify(body);
}, [
options.measures,
Expand All @@ -136,6 +138,7 @@ export function useMetricView<
options.timeDimension,
options.orderBy,
options.limit,
options.queryTags,
]);

// Detect shape changes: clear rows when metric/measures/dimensions change,
Expand Down
1 change: 1 addition & 0 deletions packages/appkit/src/connectors/sql-warehouse/client.ts
Original file line number Diff line number Diff line change
Expand Up @@ -313,6 +313,7 @@ export class SQLWarehouseConnector {
rowLimit: input.rowLimit,
onWaitTimeout:
input.onWaitTimeout || executeStatementDefaults.onWaitTimeout,
queryTags: input.queryTags,
};

span.addEvent("statement.submitting", {
Expand Down
28 changes: 28 additions & 0 deletions packages/appkit/src/connectors/tests/sql-warehouse.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -255,6 +255,34 @@ describe("SQLWarehouseConnector", () => {
});
});

describe("query tags", () => {
test("forwards queryTags to the statement execution request", async () => {
const connector = new SQLWarehouseConnector({ timeout: 5000 });
const executeStatement = vi.fn().mockResolvedValue({
statementId: "stmt-1",
status: { state: "SUCCEEDED" },
manifest: { format: "JSON_ARRAY", schema: { columns: [] } },
result: { dataArray: [] },
});
const mockWorkspaceClient = {
statementExecution: { executeStatement },
config: { host: "https://test.databricks.com" },
};
const queryTags = [{ key: "team", value: "eng" }, { key: "exp" }];

await connector.executeStatement(mockWorkspaceClient as any, {
statement: "SELECT 1",
warehouseId: "wh-1",
queryTags,
});

expect(executeStatement).toHaveBeenCalledWith(
expect.objectContaining({ queryTags }),
expect.anything(),
);
});
});

describe("ensureWarehouseRunning", () => {
let connector: SQLWarehouseConnector;

Expand Down
Loading
Loading