Skip to content

Repo sync for protected branch #2930

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
learn-build-service-prod merged 3 commits into from
Aug 18, 2026
Merged

Repo sync for protected branch #2930

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
9b643bf
Doc denisa changes (#7585)
denisa-ms Aug 18, 2026
be5b488
Merge pull request #7590 from MicrosoftDocs/main
learn-build-service-prod[bot] Aug 18, 2026
83e0bbf
Merging changes synced from https://github.com/MicrosoftDocs/dataexpl…
Aug 18, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
160 changes: 78 additions & 82 deletions data-explorer/kusto/management/materialized-views/materialized-views-monitoring.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ Monitor the health of materialized views in the following ways:
::: moniker-end
* Monitor the `IsHealthy` property by using [`.show materialized-view`](materialized-view-show-command.md#show-materialized-views).

* Inspect extent, hot-cache, size, and effective policy details by using [`.show materialized-view details`](materialized-view-show-details-command.md).

* Check for failures by using [`.show materialized-view failures`](materialized-view-show-failures-command.md#show-materialized-view-failures).

> [!NOTE]
Expand All @@ -28,101 +30,105 @@ Monitor the health of materialized views in the following ways:

## Troubleshooting unhealthy materialized views

If the `MaterializedViewAge` metric constantly increases, and the `MaterializedViewHealth` metric shows that the view is unhealthy, follow these recommendations to identify the root cause:
If the `MaterializedViewAgeSeconds` metric constantly increases and the `MaterializedViewHealth` metric shows that the view is unhealthy, use the following symptoms to identify the root cause.

:::moniker range="azure-data-explorer"

* Check the number of materialized views on the cluster, and the current capacity for materialized views:
### Not enough materialization concurrency

```kusto
.show capacity
| where Resource == "MaterializedView"
| project Resource, Total, Consumed
```
Use [`.show capacity`](../show-capacity-command.md) to compare the number of materialized views that can run concurrently with the number currently running:

**Output**
```kusto
.show capacity
| where Resource == "MaterializedView"
| project Resource, Total, Consumed
```

|Resource|Total|Consumed|
|---|---|---|
|MaterializedView|1|0|
| Resource | Total | Consumed |
|---|---|---|
| MaterializedView | 1 | 0 |

* The number of materialized views that can run concurrently depends on the capacity shown in the `Total` column. The `Consumed` column shows the number of materialized views currently running. If concurrency is limiting materialization, see [Increase available resources](materialized-views-optimization.md#increase-available-resources).
`Total` is the current concurrency limit, and `Consumed` is the number of materialized views currently running. If the limit is delaying materialization, increase the materialization concurrency only after evaluating the effect on other workloads. For more information, see [Increase available resources](materialized-views-optimization.md#increase-available-resources).
::: moniker-end

* Check if there are failures during the materialization process by using [.show materialized-view failures](materialized-view-show-failures-command.md#show-materialized-view-failures).
* If the error is permanent, the system automatically disables the materialized view. To check if it's disabled, use the [.show materialized-view](materialized-view-show-command.md) command and see if the value in the `IsEnabled` column is `false`. Then check the [Journal](../journal.md) for the disabled event by using the [.show journal](../journal.md#show-journal) command.
An example of a permanent failure is a source table schema change that makes it incompatible with the materialized view. For more information, see [.create materialized-view command](materialized-view-create.md#supported-properties).
* If the failure is transient, the system automatically retries the operation. However, the failure can delay the materialization and increase the age of the materialized view. This type of failure occurs, for example, when hitting memory limits or with a query time-out. See the following recommendations for more ways to troubleshoot transient failures.

* Analyze the materialization process by using the [.show commands-and-queries](../commands-and-queries.md) command. Replace *Databasename* and *ViewName* to filter for a specific view:

```kusto
.show commands-and-queries
| where Database == "DatabaseName" and ClientActivityId startswith "DN.MaterializedViews;ViewName;"
```

* Check the memory consumption in the `MemoryPeak` column to identify operations that failed because they reached memory limits, such as [runaway queries](../../concepts/runaway-queries.md). For remediation, see [Increase the materialization memory limit](materialized-views-optimization.md#increase-the-materialization-memory-limit).

* Check if the materialization process is hitting cold cache. The following example shows cache statistics over the past day for the materialized view, `ViewName`:

```kusto
.show commands-and-queries
| where ClientActivityId startswith "DN.MaterializedViews;ViewName"
| where StartedOn > ago(1d)
| extend HotCacheHits = tolong(CacheStatistics.Shards.Hot.HitBytes),
HotCacheMisses = tolong(CacheStatistics.Shards.Hot.MissBytes),
HotCacheRetrieved = tolong(CacheStatistics.Shards.Hot.RetrieveBytes),
ColdCacheHits = tolong(CacheStatistics.Shards.Cold.HitBytes),
ColdCacheMisses = tolong(CacheStatistics.Shards.Cold.MissBytes),
ColdCacheRetrieved = tolong(CacheStatistics.Shards.Cold.RetrieveBytes)
| summarize HotCacheHits = format_bytes(sum(HotCacheHits)),
HotCacheMisses = format_bytes(sum(HotCacheMisses)),
HotCacheRetrieved = format_bytes(sum(HotCacheRetrieved)),
ColdCacheHits =format_bytes(sum(ColdCacheHits)),
ColdCacheMisses = format_bytes(sum(ColdCacheMisses)),
ColdCacheRetrieved = format_bytes(sum(ColdCacheRetrieved))
```

**Output**

|HotCacheHits|HotCacheMisses|HotCacheRetrieved|ColdCacheHits|ColdCacheMisses|ColdCacheRetrieved|
|---|---|---|---|---|---|
|26 GB|0 Bytes|0 Bytes|1 GB|0 Bytes|866 MB|

* If the view isn't fully in the hot cache, materialization can experience disk misses, significantly slowing down the process.

* If cache misses are slowing materialization, see [Adjust caching policies](materialized-views-optimization.md#adjust-caching-policies).
* Check if the materialization is scanning old records by checking `ScannedExtentsStatistics` with the [.show queries](../show-queries-command.md) command. If the number of scanned extents is high and `MinDataScannedTime` is old, the cycle scans all or most of the *materialized* part to find intersections with the *delta*. For ways to reduce the amount of data scanned, see [Optimize materialized views](materialized-views-optimization.md).
:::moniker range="azure-data-explorer"
### Materialization failures

* Check whether there's enough ingestion capacity by verifying if either the [`MaterializedViewResult` metric](#materializedviewresult-metric) or [IngestionUtilization metric](/azure/data-explorer/monitor-data-explorer-reference#supported-metrics-for-microsoftkustoclusters) show `InsufficientCapacity` values. These values indicate that available ingestion capacity is limiting materialization. For remediation, see [Increase available resources](materialized-views-optimization.md#increase-available-resources).
::: moniker-end
:::moniker range="microsoft-fabric"
Use [`.show materialized-view failures`](materialized-view-show-failures-command.md#show-materialized-view-failures) to inspect failures.

* Check whether there's enough ingestion capacity by verifying if the [`MaterializedViewResult` metric](#materializedviewresult-metric) shows `InsufficientCapacity` values. These values indicate that available ingestion capacity is limiting materialization. For remediation, see [Increase available resources](materialized-views-optimization.md#increase-available-resources).
::: moniker-end
* For a permanent error, the system automatically disables the materialized view. Use [`.show materialized-view`](materialized-view-show-command.md) to check whether `IsEnabled` is `false`, and use [`.show journal`](../journal.md#show-journal) to find the disabled event. A source table schema change that makes the table incompatible with the materialized view is an example of a permanent error. For more information, see [.create materialized-view](materialized-view-create.md#supported-properties).
* For a transient error, the system automatically retries the operation. Repeated memory-limit or query-timeout failures delay materialization and increase the materialized view age. For ways to reduce transient failures, see [Optimize materialized views](materialized-views-optimization.md).

* If the materialized view is still unhealthy, the service might not have sufficient capacity or resources to materialize all data on time. See [Optimize materialized views](materialized-views-optimization.md) for remediation options.
### Materialization exceeds the memory limit

## MaterializedViewResult metric
Use [`.show commands-and-queries`](../commands-and-queries.md) to inspect `MemoryPeak`. Replace `DatabaseName` and `ViewName` with your values:

```kusto
.show commands-and-queries
| where Database == "DatabaseName" and ClientActivityId startswith "DN.MaterializedViews;ViewName;"
| project StartedOn, LastUpdatedOn, Duration, State, FailureReason,
TotalCpu, MemoryPeak
```

A failure that reaches the memory limit can appear as a [runaway query](../../concepts/runaway-queries.md). Raise the materialization memory limit or reduce the memory required by each cycle. For more information, see [Increase the materialization memory limit](materialized-views-optimization.md#increase-the-materialization-memory-limit).

### Materialization is hitting cold cache

Use the cache statistics from [`.show commands-and-queries`](../commands-and-queries.md) to determine whether materialization is retrieving data from the cold cache. The following query summarizes cache activity over the past day for `ViewName`:

The `MaterializedViewResult` metric provides information about the result of a materialization cycle. Use it to identify problems in the materialized view health status. The metric includes the `Database`, `MaterializedViewName`, and a `Result` dimension.
```kusto
.show commands-and-queries
| where ClientActivityId startswith "DN.MaterializedViews;ViewName"
| where StartedOn > ago(1d)
| extend HotCacheHits = tolong(CacheStatistics.Shards.Hot.HitBytes),
HotCacheMisses = tolong(CacheStatistics.Shards.Hot.MissBytes),
HotCacheRetrieved = tolong(CacheStatistics.Shards.Hot.RetrieveBytes),
ColdCacheHits = tolong(CacheStatistics.Shards.Cold.HitBytes),
ColdCacheMisses = tolong(CacheStatistics.Shards.Cold.MissBytes),
ColdCacheRetrieved = tolong(CacheStatistics.Shards.Cold.RetrieveBytes)
| summarize HotCacheHits = format_bytes(sum(HotCacheHits)),
HotCacheMisses = format_bytes(sum(HotCacheMisses)),
HotCacheRetrieved = format_bytes(sum(HotCacheRetrieved)),
ColdCacheHits = format_bytes(sum(ColdCacheHits)),
ColdCacheMisses = format_bytes(sum(ColdCacheMisses)),
ColdCacheRetrieved = format_bytes(sum(ColdCacheRetrieved))
```

| HotCacheHits | HotCacheMisses | HotCacheRetrieved | ColdCacheHits | ColdCacheMisses | ColdCacheRetrieved |
|---|---|---|---|---|---|
| 26 GB | 0 Bytes | 0 Bytes | 1 GB | 0 Bytes | 866 MB |

Cold-cache hits or retrieved bytes indicate that materialization is reading data outside the hot cache, which can significantly slow the process. Extend the caching policies to cover the data that materialization scans. For more information, see [Adjust caching policies](materialized-views-optimization.md#adjust-caching-policies).

The `Result` dimension can have one of the following values:
### Materialization scans old records

* **Success**: The materialization completed successfully.
Inspect `ScannedExtentsStatistics` with [`.show queries`](../show-queries-command.md). A high number of scanned extents and an old `MinDataScannedTime` indicate that the cycle is scanning all or most of the materialized part to find intersections with the delta. Reduce the scan by using an appropriate datetime group-by key, lookback period, or caching policy. For more information, see [Optimize materialized views](materialized-views-optimization.md).

* **SourceTableNotFound**: The source table of the materialized view was dropped, so the materialized view is automatically disabled.
### Not enough ingestion capacity

* **SourceTableSchemaChange**: The schema of the source table changed in a way that isn't compatible with the materialized view definition. Since the materialized view query no longer matches the materialized view schema, the materialized view is automatically disabled.
:::moniker range="azure-data-explorer"
* **InsufficientCapacity**: The instance doesn't have sufficient capacity to materialize the materialized view, due to a lack of [ingestion capacity](../capacity-policy.md#ingestion-capacity). Insufficient capacity failures can be transient, but recurring failures indicate that available ingestion capacity is limiting materialization.
Check whether the [`MaterializedViewResult` metric](#materializedviewresult-metric) or [IngestionUtilization metric](/azure/data-explorer/monitor-data-explorer-reference#supported-metrics-for-microsoftkustoclusters) has an `InsufficientCapacity` value.
::: moniker-end
:::moniker range="microsoft-fabric"
* **InsufficientCapacity**: The instance doesn't have sufficient capacity to materialize the materialized view, due to a lack of ingestion capacity. Insufficient capacity failures can be transient, but recurring failures indicate that available ingestion capacity is limiting materialization.
Check whether the [`MaterializedViewResult` metric](#materializedviewresult-metric) has an `InsufficientCapacity` value.
::: moniker-end

* **InsufficientResources:** The database doesn't have sufficient resources (memory) to materialize the materialized view. Insufficient resource errors can be transient, but recurring failures indicate that the database lacks sufficient memory for materialization. For remediation, see [Optimize materialized views](materialized-views-optimization.md).
Recurring `InsufficientCapacity` values indicate that available ingestion capacity is limiting materialization. Increase the resources available for materialization. For more information, see [Increase available resources](materialized-views-optimization.md#increase-available-resources).

### Materialized view remains unhealthy

If the preceding diagnostics don't identify a specific cause, the service might not have sufficient capacity or resources to materialize all data on time. Increase available resources or split a memory-intensive view only after applying less disruptive optimizations. For more information, see [Optimize materialized views](materialized-views-optimization.md).

## MaterializedViewResult metric

The `MaterializedViewResult` metric provides the result of each materialization cycle. Use it to identify problems with materialized view health. The metric includes the `Database`, `MaterializedViewName`, and `Result` dimensions.

| Value | Meaning | Action |
|---|---|---|
| `Success` | The materialization cycle completed successfully. | None. |
| `SourceTableNotFound` | The source table was dropped, so the materialized view is automatically disabled. | Restore the source table and [enable the materialized view](materialized-view-enable-disable.md). |
| `SourceTableSchemaChange` | The source table schema is incompatible with the materialized view definition, so the view is automatically disabled. | Make the source schema and materialized view query compatible, and then [enable the materialized view](materialized-view-enable-disable.md). |
| `InsufficientCapacity` | Available ingestion capacity is limiting materialization. The failure can be transient, but recurring values indicate a capacity issue. For Azure Data Explorer, see [Ingestion capacity](../capacity-policy.md#ingestion-capacity). | [Increase available resources](materialized-views-optimization.md#increase-available-resources). |
| `InsufficientResources` | Materialization exceeded the memory limit allowed for a single operation. | [Increase the materialization memory limit](materialized-views-optimization.md#increase-the-materialization-memory-limit) or reduce the memory required by each cycle. |

## Materialized views in follower databases

Expand All @@ -133,16 +139,6 @@ You can define materialized views in [follower databases](materialized-views-lim
::: moniker-end
* The [.show materialized-view failures command](materialized-view-show-failures-command.md) only works in the leader database.

## Track resource consumption

**Materialized views resource consumption:** Use the [`.show commands-and-queries`](../commands-and-queries.md) command to track the resources the materialized views materialization process consumes. To filter the records for a specific view, use the following query and replace `DatabaseName` and `ViewName` with your values:

```kusto
.show commands-and-queries
| where Database == "DatabaseName" and ClientActivityId startswith "DN.MaterializedViews;ViewName;"
| project StartedOn, LastUpdatedOn, Duration, State, FailureReason,
TotalCpu, MemoryPeak, CacheStatistics, ScannedExtentsStatistics
```

## Related content

Expand Down
Loading