Skip to content

Add Homebrew support for Fluent Package v6.1.0 on macOS - #1136

Open
ashie wants to merge 20 commits into
masterfrom
homebrew-formula
Open

ashie wants to merge 20 commits into
masterfrom
homebrew-formula

Conversation

@ashie

@ashie ashie commented Oct 25, 2026 •

Copy link
Copy Markdown
Member

Summary

This PR adds Homebrew support for installing Fluent Package v6.1.0 on Apple Silicon Macs.

The formula is maintained in this repository and installs the v6.1.0 release with its locked gem set. It integrates with brew services, keeps configuration, logs, and user-installed gems outside the Cellar, and includes CI coverage for installation, native extensions, service operation, and data preservation.

Installation

After this PR is merged into the default branch:

brew tap fluent/package-builder https://github.com/fluent/fluent-package-builder
brew install fluent/package-builder/fluent-package

The formula currently supports Apple Silicon Macs. Intel Macs are not supported.

Release plan

This provides the macOS Homebrew installation path for Fluent Package v6.1.0. It does not add a separate macOS binary package or a macOS-specific release tag. The formula points to the v6.1.0 source release; future Fluent Package releases will update the formula as part of their normal release cycle.

Validation

The formula has been validated locally on Apple Silicon with Homebrew, including installation, brew test, brew services, native gem loading, linkage, and preservation of configuration and user data.

GitHub Actions has not yet run for this PR. The workflow will run the Homebrew checks on macOS 15, and 26.

@ashie

ashie commented Oct 25, 2026

Copy link
Copy Markdown
Member Author

My proposal is to target the upcoming 6.1.0 release rather than 6.0.5. After 6.1.0 is released, I plan to rebase this work onto the 6.1.0 branch and update the formula for that release. I’d also like to add Homebrew installation instructions to the download site, so users can install directly from the tap without a separate Homebrew binary package.

ashie added 15 commits November 13, 2026 21:50
Adds HomebrewFormula/fluent-package.rb, a formula that installs the
gem set locked by a fluent-package release tag from local gem
archives, on Homebrew's ruby@3.4 with Homebrew's own libraries.
The four commands (fluentd, fluent-gem, fluent-cat, td) are wrappers
that keep configuration, logs and user-installed plugins outside the
Cellar so they survive brew upgrade/uninstall, and Name    Status User File
emacs   none
unbound none
manages fluentd through a launchd job.

homebrew/update_formula.rb regenerates the formula's gem resource
blocks from a release tag's fluent-package/Gemfile.lock, choosing
arm64-darwin .gem platform builds where published; its --check mode
guards against the formula drifting from the newest tag.

HomebrewFormula/README.md documents usage and formula maintenance.

Signed-off-by: Takuro Ashie <ashie@clear-code.com>
Assisted-by: Qwen-3.8-Flash-Next [OpenCode]
Installs on Apple Silicon Macs are documented in the root README and
in HomebrewFormula/README.md. The workflow runs brew style/audit/
install/test and a brew services round trip on a macos-14 runner,
plus a scheduled check that flags upstream releases whose gem set
the formula has not picked up yet; it never writes to the repository.

Signed-off-by: Takuro Ashie <ashie@clear-code.com>
Assisted-by: Qwen-3.8-Flash-Next [OpenCode]
Signed-off-by: Takuro Ashie <ashie@clear-code.com>
Assisted-by: GPT-6.1 Sol [Codex]
- cmetrics: add a checksummed Homebrew resource for the calyptia/cmetrics
  C sources and repack the gem (gem unpack + inreplace + gem build) so
  mini_portile reads the mirrored tarball via file:// instead of hitting
  GitHub during the build; `gem install --local` is then actually true
  end to end
- user gems live in var/fluent/gems/<ruby ABI>; entries of the old
  unversioned layout are migrated at install, gems from other ABIs are
  never loaded, and post_install explains how to recover them
- regenerate resources with nokogiri excluded (Windows-only Gemfile
  entry; nothing on macOS uses it)
- update_formula.rb: parse GIT lock entries and verify via the GitHub
  API (cached) that the locked revision of fluentd is exactly the tag of
  the gem the formula installs; annotate and abort on mismatch
- do not clobber user-edited etc/fluent/fluentd.conf on reinstall
  (Pathname#install moves over existing files)
- formula: sorbet sigil + class doc, service/caveats text, aligned
  write_env_script arguments (brew audit --strict)
- README/CI: match the ABI gem layout, the verified git sourcing and the
  offline guarantee; drop nokogiri from the native-load check

Signed-off-by: Takuro Ashie <ashie@clear-code.com>
Assisted-by: Qwen-3.8-Flash-Next [OpenCode]
Formula#etc is extended with Homebrew's InstallRenamed module, and the
extension propagates through `+` and `/`; the previous assumption that
Pathname#install would clobber an existing config was wrong. A temp-dir
reproduction with the real extended paths (Homebrew 7.0.10) shows the
standard conffile behaviour: no file -> installed, identical file ->
left alone, user-modified file -> kept, with the new default installed
as fluentd.conf.default. Reverting to the plain etc.install therefore
keeps user edits and follows Homebrew conventions instead of
hand-rolling an existence check.

Signed-off-by: Takuro Ashie <ashie@clear-code.com>
Assisted-by: Qwen-3.8-Flash-Next [OpenCode]
Moving everything found directly in var/fluent/gems into the current
ABI directory during install was wrong on several counts: it treated
native extensions built for an older Ruby as belonging to the new ABI,
it could collide with directories already present in the destination,
and the install step moved user data directly. Leave every pre-existing
entry where it is instead; GEM_HOME/GEM_PATH point at the ABI-specific
directory only, so those gems are kept but never loaded, and the
recovery path is a plain `fluent-gem install` (documented in the
README, in caveats and in the stale-gem notice, which now also lists
the pre-namespacing layout entries).

Signed-off-by: Takuro Ashie <ashie@clear-code.com>
Assisted-by: Qwen-3.8-Flash-Next [OpenCode]
Corrections found while re-checking the previous round's statements:

- fluentd git lock: what update_formula.rb verifies is that the locked
  revision equals the commit the release tag points at; the published
  .gem's content is not compared against that revision. README, the
  generator comments and the update instructions now say so.
- rdkafka: the 0.27.0 binary gem the formula installs embeds librdkafka
  2.14.0 (the version rdkafka 0.27.0's own version.rb targets), not
  2.8.0 as previously written; byte-identity with the .dmg's copy was
  never checked and is now stated as unverified.
- google-protobuf/Ruby: quote the checked gemspec range
  (required_ruby_version < 3.5.dev for the 4.32.1 arm64-darwin gem)
  instead of a bare assertion.
- offline build: record how it was verified (brew reinstall succeeds
  with the build-phase network blocked via Homebrew's standard
  HOMEBREW_FORMULA_BUILD_NETWORK=deny; the unpatched cmetrics gem fails
  that build with a codeload.github.com connection attempt), note that
  Homebrew 7 allows build-phase network by default, and list the exact
  upstream 0.3.3 strings the repack's inreplace calls depend on.
- CI: the workflow is defined but has not executed on GitHub; what ran
  is the local equivalents (listed). Persistence checks are same-version
  reinstall, not a cross-version upgrade.

Signed-off-by: Takuro Ashie <ashie@clear-code.com>
Assisted-by: Qwen-3.8-Flash-Next [OpenCode]
Signed-off-by: Takuro Ashie <ashie@clear-code.com>
Assisted-by: GPT-6.1 Sol [Codex]
Signed-off-by: Takuro Ashie <ashie@clear-code.com>
Assisted-by: GPT-6.1 Sol [Codex]
Homebrew's Homebrew/NoFileutilsRmrf cop includes only files under a
directory literally named Formula/ (or Casks/), so it did not apply
while the formula lived in HomebrewFormula/ but does now. rm_rf is the
FileUtils.rm_rf that Formula gets from include FileUtils; the safer
rm_r is equivalent here because the paths come from a Dir.glob of
paths that exist at that moment.

Signed-off-by: Takuro Ashie <ashie@clear-code.com>
Assisted-by: Qwen3.8-Flash-Next [OpenCode]
Homebrew discovers tap formulae only under a top-level Formula/ or
HomebrewFormula/ directory (or bare *.rb at the tap root), so after the
move to homebrew/Formula/ a `brew tap` of this repository exposed
nothing and the README's tap install instructions failed. Add the
conventional top-level Formula symlink to homebrew/Formula (the files
themselves stay under homebrew/), and note the constraint in
homebrew/README.md. Verified against a tap clone of this branch: the
formula is discovered ("1 formula") and style/audit inspect it; CI's
copied temp tap keeps working unchanged. Beware that without a
discoverable formula, `brew style`/`audit` of the tap name silently
inspect zero files and exit 0.

Signed-off-by: Takuro Ashie <ashie@clear-code.com>
Assisted-by: Qwen3.8-Flash-Next [OpenCode]
Signed-off-by: Takuro Ashie <ashie@clear-code.com>
…sing

A --cache dir populated before a gem release keeps the old
api/v1/versions JSON forever, so --check aborted with "gems not found
on rubygems.org" for cool.io 1.11.0, net-ssh 8.0.4, net-scp 4.1.2 and
fluentd 1.20.1, all published after the cache snapshot (the formula
itself carries the correct URLs and checksums). On a metadata miss,
resolve_gem now fetches the versions JSON again and refreshes the
cache entry; gems still absent upstream are reported as before.

Assisted-by: Qwen3.8-Flash-Next [OpenCode]
Signed-off-by: Takuro Ashie <ashie@clear-code.com>
v6.1.0 adds the native openssl 4.0.2 gem. Homebrew's ruby@3.4 now
depends on openssl@4 (its shipped openssl.bundle links opt/openssl@4),
so keeping depends_on openssl@3 made the gem build split-brained:
pkg-config supplied @3 link flags while @4 headers shadowed the include
path, and openssl_missing.h's compat shims then collided with @4's
ASN1 declarations (gem install aborted mid-reinstall).

Switch the dep to openssl@4 so headers, link flags and ruby@3.4's own
OpenSSL all agree. Verified locally: openssl 4.0.2 compiles and loads
against OpenSSL 4.1.0 (OpenSSL::OPENSSL_VERSION). README's rbconfig
claim corrected: rbconfig carries no OpenSSL flags; pkg-config does
the discovery.

Assisted-by: Qwen3.8-Flash-Next [OpenCode]
Signed-off-by: Takuro Ashie <ashie@clear-code.com>
Offline reinstall from v6.0.5 to v6.1.0 (brew test, linkage --test,
etc/var preservation, no scratch leftovers, cmetrics build output with
zero network references, openssl 4.0.2 gem compiling and loading
against openssl@4) verified; the cross-version install item can leave
the unverified list.

Assisted-by: Qwen3.8-Flash-Next [OpenCode]
Signed-off-by: Takuro Ashie <ashie@clear-code.com>
The v6.1.0 gem set added the openssl gem, but the native-extension
check never required it: a broken libexec would go unnoticed because
ruby@3.4 provides its own default openssl gem as a fallback. Require
openssl in that step and assert it loads from the formula's libexec;
the comparison uses realpaths because brew --prefix goes through the
opt/ symlink while gem paths resolve under Cellar/.

Also add `brew linkage --test` after install, and fix two README
gaps: `brew tap` only clones the default branch, so the tap commands
work once the formula is merged, and update_formula.rb re-fetches gem
metadata when a locked version is missing from the cached payload.

Assisted-by: Qwen3.8-Flash-Next [OpenCode]
Assisted-by: GPT-6.1 Sol [Codex]
Signed-off-by: Takuro Ashie <ashie@clear-code.com>
Assisted-by: GPT-6.1 Sol [Codex]
Signed-off-by: Takuro Ashie <ashie@clear-code.com>
@ashie ashie changed the title Review the Homebrew formula implementation and validation Add Homebrew support for Fluent Package v6.1.0 on macOS Nov 13, 2026
@ashie

ashie commented Nov 13, 2026

Copy link
Copy Markdown
Member Author

The corresponding documentation updates are also in review:

The download site will link to the Homebrew installation instructions, and the guide covers installation, service management, and uninstalling on Apple Silicon macOS.

Assisted-by: GPT-6.1 Sol [Codex]
Signed-off-by: Takuro Ashie <ashie@clear-code.com>
Assisted-by: GPT-6.1 Sol [Codex]
Signed-off-by: Takuro Ashie <ashie@clear-code.com>
@ashie
ashie marked this pull request as ready for review November 13, 2026 15:12
@ashie
ashie requested review from Watson1978 and kenhys November 13, 2026 15:14
Comment thread homebrew/Formula/fluent-package.rb Resolved
Comment thread homebrew/Formula/fluent-package.rb Resolved
Comment thread homebrew/Formula/fluent-package.rb Resolved
Cache the Ruby ABI lookup, clarify the installer caveat, and pass string paths to Process.spawn in the formula test.

Assisted-by: GPT-6.1 Sol [Codex]
Signed-off-by: Takuro Ashie <ashie@clear-code.com>
@ashie

ashie commented Nov 16, 2026

Copy link
Copy Markdown
Member Author

Thanks for your review!
I've fixed them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants