DOCS-3028: Add release notes for the Fluentd to Fluent Bit migration - #3006
Draft
ctauchen wants to merge 6 commits into
Draft
DOCS-3028: Add release notes for the Fluentd to Fluent Bit migration#3006ctauchen wants to merge 6 commits into
ctauchen wants to merge 6 commits into
Conversation
…-6164) Calico Enterprise removes the fluentd log collector in favour of Fluent Bit (calico-fluent-bit in calico-system; tigera/operator#4910). Update the next-version Calico Enterprise and Calico Cloud docs to match: - Rewrite "Filter flow logs" / "Filter DNS logs": examples move from fluentd <filter> syntax to Fluent Bit YAML filter lists, the ConfigMap is renamed fluentd-filters -> fluent-bit-filters, and an upgrade note explains that old fluentd-syntax filters are not translated (the operator raises a tigera status warning naming the offending key). - Metrics pages: Fluent Bit's built-in Prometheus endpoint (port 2020, /api/v2/metrics/prometheus, fluentbit_* metrics); the fluentd buffer-space alert becomes fluentbit_output_chunk_available_capacity_percent plus a dropped-chunks alert on fluentbit_output_retries_failed_total. - BYO Prometheus: the fluent-bit tab needs no client TLS (plain HTTP behind the allow-calico-fluent-bit policy); ServiceMonitor sample is fluent-bit-metrics-service-monitor.yaml. - Sweep of remaining pages: tigera-fluentd namespace -> calico-system, fluentd-node -> calico-fluent-bit (labels, secrets, packet capture retrieval via calico-node, diags output, resource override examples now use calicoFluentBitDaemonSet, architecture/overview descriptions, cc-arch-diagram ports, operator checklist). Not touched: versioned docs, releases.json (historical), and the generated installation API reference (regenerates from the operator release). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The operator now folds a user-supplied CA from a splunk-ca ConfigMap into the log collector's trusted bundle, the same way syslog-ca works for syslog, so Splunk HEC endpoints with self-signed certificates can be used. Add the step to the Splunk archiving instructions and a release-note line. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Move the Fluentd filter migration into a Before you begin section on both filtering pages, so a reader who already has filters sees it before the task rather than after logs go missing. Name the command that surfaces the operator warning, and note that filters apply to Linux nodes only. List the four filters the log collector ships as an exhaustive set, and link each one, so a reader does not write a filter the image cannot load. Document the operational detail on the pages that own it: filesystem buffering and its per-destination caps in the log collection overview, the S3 object key layout and the syslog packet-size default in Archive logs, and the deprecated override field and container-name aliases in Configure resource requests and limits.
Use the ConfigMap and DaemonSet spellings Vale expects, and put the resource and field names in the private-CA steps into code formatting. The syslog step gets the same treatment as the new Splunk step beside it, so the two adjacent steps read the same way.
Document the fluentd -> Fluent Bit migration (tigera/operator#4910) in the next-version Calico Enterprise release notes: - New feature entry for the Fluent Bit log collector. - Deprecated: the fluentdDaemonSet override, superseded by calicoFluentBitDaemonSet. - Upgrade notes with the breaking changes: tigera-fluentd namespace/resource rename to calico-system, fluentd-filters -> fluent-bit-filters ConfigMap, fluentbit_* Prometheus metrics on port 2020, and the new S3 archive key layout, plus the non-breaking moves to filesystem buffering and the larger syslog packet-size default. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Give each upgrade note one action and one link, and move the mechanism behind it onto the page that documents it. A reader planning an upgrade needs to know what to do and where to read more, not how filesystem buffering is configured. Add the splunk-ca entry to the feature summary, and point the deprecation bullet at the page that shows how to move an override to the new field.
✅ Deploy Preview for calico-docs-preview-next ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
✅ Deploy Preview succeeded!Built without sensitive environment variables
To edit notification comments on pull requests, go to your Netlify project configuration. |
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
The archive storage docs introduce an audit_tsee/ directory name that conflicts with the audit_ee terminology used elsewhere and should be clarified or aligned to avoid misleading upgrade guidance.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Adds/updates Calico Enterprise (and related Calico Cloud) documentation for the Fluentd → Fluent Bit log collector migration, including release/upgrade notes and aligned operational guidance across filtering, metrics scraping, archiving, and troubleshooting.
Changes:
- Adds Calico Enterprise release notes + upgrade notes covering the Fluent Bit migration and related breaking changes/default changes.
- Updates multiple docs pages to rename Fluentd resources to Fluent Bit equivalents (namespaces, DaemonSets, secrets, metrics endpoints) and revise examples accordingly.
- Extends archiving/monitoring docs to reflect Fluent Bit behavior (filesystem buffering, Prometheus metrics over port 2020, S3 key layout, private-CA support for Splunk HEC).
File summaries
| File | Description |
|---|---|
| calico-enterprise/release-notes/index.mdx | Adds release notes and upgrade notes for Fluentd → Fluent Bit migration. |
| calico-enterprise/reference/component-resources/configure-resources.mdx | Updates LogCollector override examples/notes for calicoFluentBitDaemonSet and new names. |
| calico-enterprise/reference/clis/calicoctl/cluster/diags.mdx | Updates diags sample output to reflect calico-fluent-bit in calico-system. |
| calico-enterprise/reference/architecture/overview.mdx | Updates bundled component references from Fluentd to Fluent Bit; minor formatting tweaks. |
| calico-enterprise/operations/monitor/prometheus/byo-prometheus.mdx | Updates BYO Prometheus guidance for Fluent Bit metrics (HTTP :2020, new ServiceMonitor). |
| calico-enterprise/operations/monitor/metrics/elasticsearch-and-fluentd-metrics.mdx | Renames/updates alerting guidance for Fluent Bit metrics and new metrics/alerts. |
| calico-enterprise/operations/license-options.mdx | Updates license-expiration behavior wording to reference Fluent Bit. |
| calico-enterprise/operations/comms/index.mdx | Updates secret naming/ownership table for Fluent Bit TLS secret. |
| calico-enterprise/observability/review-unused-network-policies.mdx | Updates sample output for Fluent Bit network policy naming/namespace. |
| calico-enterprise/observability/packetcapture.mdx | Updates packet capture retrieval commands (namespace/label changes). |
| calico-enterprise/observability/kube-audit.mdx | Updates wording to reference Fluent Bit collector for Kubernetes audit logs. |
| calico-enterprise/observability/elastic/overview.mdx | Updates overview to Fluent Bit and documents filesystem buffering behavior/caps. |
| calico-enterprise/observability/elastic/flow/filtering.mdx | Migrates flow filtering docs from Fluentd syntax to Fluent Bit YAML and new ConfigMap name. |
| calico-enterprise/observability/elastic/dns/filtering-dns.mdx | Migrates DNS filtering docs from Fluentd syntax to Fluent Bit YAML and new ConfigMap name. |
| calico-enterprise/observability/elastic/archive-storage.mdx | Documents new S3 object key layout; updates syslog defaults; adds Splunk private-CA ConfigMap step. |
| calico-enterprise/_includes/content/_default-install.mdx | Updates default-install content to reference Fluent Bit. |
| calico-enterprise/_includes/components/ReqsSys.js | Updates requirements table wording for logs/storage. |
| calico-cloud/reference/component-resources/configure-resources.mdx | Cloud equivalent: updates LogCollector override examples/notes for calicoFluentBitDaemonSet. |
| calico-cloud/operations/monitor/prometheus/byo-prometheus.mdx | Cloud equivalent: updates BYO Prometheus guidance for Fluent Bit metrics and new ServiceMonitor. |
| calico-cloud/operations/monitor/metrics/elasticsearch-and-fluentd-metrics.mdx | Cloud equivalent: updates Fluent Bit alerting guidance/metrics. |
| calico-cloud/operations/comms/index.mdx | Cloud equivalent: updates secret naming/ownership table for Fluent Bit TLS secret. |
| calico-cloud/observability/packetcapture.mdx | Cloud equivalent: updates packet capture retrieval commands (namespace/label changes). |
| calico-cloud/observability/kube-audit.mdx | Cloud equivalent: updates wording to reference Fluent Bit collector for Kubernetes audit logs. |
| calico-cloud/observability/elastic/overview.mdx | Cloud equivalent: updates overview to Fluent Bit and documents filesystem buffering behavior/caps. |
| calico-cloud/observability/elastic/flow/filtering.mdx | Cloud equivalent: migrates flow filtering docs to Fluent Bit YAML and new ConfigMap name. |
| calico-cloud/observability/elastic/dns/filtering-dns.mdx | Cloud equivalent: migrates DNS filtering docs to Fluent Bit YAML and new ConfigMap name. |
| calico-cloud/observability/elastic/archive-storage.mdx | Cloud equivalent: documents new S3 object key layout; updates syslog defaults. |
| calico-cloud/get-started/operator-checklist.mdx | Updates checklist troubleshooting text and examples for Fluent Bit and other terminology fixes. |
| calico-cloud/get-started/cc-arch-diagram.mdx | Updates architecture diagram table entries/ports from Fluentd to Fluent Bit. |
| calico-cloud/_includes/components/ReqsSys.js | Cloud equivalent: updates requirements table wording for logs/storage. |
Review details
- Files reviewed: 30/30 changed files
- Comments generated: 6
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+56
to
+58
| ```yaml | ||
| - name: grep | ||
| exclude: qname \.cluster\.local$ |
| ### CalicoFluentBitDaemonSet. | ||
|
|
||
| To configure resource specification for the [FluentdDaemonSet](../../reference/installation/api.mdx#fluentddaemonset), patch the LogCollector CR using the below command: | ||
| To configure resource specification for the [CalicoFluentBitDaemonSet](../../reference/installation/api.mdx#logcollector), patch the LogCollector CR using the below command: |
Comment on lines
+84
to
+90
| ``` | ||
| <bucketPath>/flows/20260101_<uuid>.gz | ||
| <bucketPath>/dns/20260101_<uuid>.gz | ||
| <bucketPath>/l7/20260101_<uuid>.gz | ||
| <bucketPath>/audit_kube/20260101_<uuid>.gz | ||
| <bucketPath>/audit_tsee/20260101_<uuid>.gz | ||
| ``` |
|
|
||
| :::note | ||
|
|
||
| Skip this step if publicCA bundle is good enough to verify the server certificates. |
Comment on lines
+58
to
+60
| ```yaml | ||
| - name: grep | ||
| exclude: qname \.cluster\.local$ |
| ### CalicoFluentBitDaemonSet. | ||
|
|
||
| To configure resource specification for the [FluentdDaemonSet](../../reference/installation/api.mdx#fluentddaemonset), patch the LogCollector CR using the below command: | ||
| To configure resource specification for the [CalicoFluentBitDaemonSet](../../reference/installation/api.mdx#logcollector), patch the LogCollector CR using the below command: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release notes and upgrade notes for the Fluentd to Fluent Bit log collector migration in Calico Enterprise 3.24 EP3.
This is stacked on #3004 and is a draft until that merges. Only the last commit belongs to this PR, so review calico-enterprise/release-notes/index.mdx and ignore the rest of the diff. Once #3004 is in main, the diff here collapses to that one file and I will mark this ready for review.
The split follows the review I left on #2815: keep the documentation changes in one PR, and the release-specific content in another. Jiawei's release-notes commit is cherry-picked with his authorship intact, then reworked.
What the release notes say:
What changed from #2815: the mechanism behind each bullet moved onto the page that documents it, which is the change #3004 carries. Someone planning an upgrade needs to know what to do and where to read more, not how filesystem buffering is configured. The filter rewrite leads the list because it is the only item that needs action before the upgrade.
The feature is not in technology preview. Fluentd is removed outright in calico-private, so Fluent Bit is the only log collector in this release, and the entry stays under New features and enhancements.
Validation: Vale reports nothing on this file, and every relative link resolves to an existing page.
Preview: /calico-enterprise/next/release-notes/
Issue: https://tigera.atlassian.net/browse/DOCS-3028