Homebrew and One-Line Installers for My Rust CLI: Lessons Learned

Have you ever run brew install sometool or curl https://sometool.dev | bash to install software? Did you ever want to set up one for your own software? Yes? If so, you are in the right spot.

Throughout this article, I will share how I set up a Homebrew tap and a curl|bash installer for my Rust project. Along with this will also be the closely related topics of:

• GitHub releases
• cargo binstall and its GitHub Actions applications
• GitHub attestations

As a reminder, this is a series of posts I made about packaging my Rust CLI:

• Part 1: PyPI, NPM, and GitHub Actions
• Part 2: Linux and macOS (this post)
• Part 3: FreeBSD, Nix, Guix
• Part 4: Windows

One of the reasons why I wrote this post was to share what I learned writing celq. It is a small tool that can query JSON, TOML, and YAML from the command-line. Go check it out!

GitHub Releases

Before we dive into setting up a Homebrew tap and the installer script, we need to talk about the prerequisite: hosting the pre-built binaries.

For this blog post, I will use GitHub releases to host the binaries. This is mostly out of convenience. GitHub offers a very generous limits of free bandwidth for downloading binaries hosted with them. Feel free to host your binaries elsewhere (e.g. GitLab, S3). As long as you have a predictable URL based on the version of your asset, you should be fine.

To generate the pre-built binaries for a Rust CLI, a high-level overview is:

1. Install the Rust toolchain for the target
2. Run cargo build --release
3. Possibly compress and rename the binary from step 2
4. Run gh release upload $version $asset

The steps can vary depending on your project. For example, you might need to install additional tools for step 1. You can also choose to not compress the binary in step 3.

With regards to compression, it is fairly standard to compress Linux and macOS binaries with .tar.gz and Windows binaries with .zip. Nothing is stopping you from using other compression schemes like zstd if you want, but from looking around most people try to stick with what is already available in systems by default.

For celq, I used GitHub Actions to perform steps 1 to 4. You can check release_github.yml if you are curious. This is again mostly out of convenience. GitHub Actions was the easiest way for me to get a Windows machine.

cargo binstall

Before we jump into Homebrew, let’s collect a freebie: cargo binstall. It lets users install celq with cargo binstall celq.

I found this method valuable for the particular reason: combined with install-action, it makes it trivial to use Rust CLIs on GitHub Actions! Look at this:

- uses: taiki-e/install-action@v2
  with:
    tool: celq

cargo binstall works in this clever way. First, it queries crates.io to get metadata about your crate. Then, it infers the download link for the pre-built binary and downloads it. Afterward, it puts the binary in $HOME/.cargo/bin which should be in the PATH if users have cargo installed.

You can refer to cargo binstall’s documentation for tweaking your crate if the defaults don’t work. I needed to use overrides for celq because I customized the asset names. In general, configuration was straightforward and it just required a [package.metadata.binstall] section in Cargo.toml.

Homebrew

I think a responsible thing to say for this section is: check the dist tool documentation after reading the blog post. There is prior art to this, dist is the most mindless way of setting things up.

With that being said, I did not use dist because I wanted to know how things worked. This made me learn about some limitations from dist that I will share with you, such as installing from source.

Setting up Homebrew starts with setting up a Homebrew tap. A tap is just a GitHub repository with the homebrew- prefix. For example, get-celq/homebrew-tap is the tap I set up.

When installing things from it, the commands look like:

brew install get-celq/tap/celq

If the repository name was homebrew-rustcli, the command would use get-celq/rustcli/$formula instead. It is possible to host taps outside of GitHub by telling users to run brew tap $repo $url first.

It is also possible to distribute your binary through homebrew-core. That is the default channel when running brew install $formula, but the bar for submitting software there is of course higher. Overall, I appreciate Homebrew’s support for third-party channels.

Inside the tap, there are formulas in the Formula folder. Formulas are written in Ruby, so for celq there is Formula/celq.rb. Apart from metadata, I think I can split the formula into three parts: URL to assets, installation step, and tests.

Tying this back to the GitHub releases section, the formulas contain link to assets and the SHA256 checksum. To support multiple architectures and OSes, formulas use the OS.linux/OS.mac and Hardware::CPU.arm/Hardware::CPU.intel variables combined with Ruby if statements:

if OS.mac?
    if Hardware::CPU.arm?
      url "$GITHUB_REPO_URL/releases/download/$VERSION/$asset"
      sha256 "$HASH"
    end
end

To automate Homebrew releases, I simply fill in a formula template with the version and hashes. Afterward, I make a commit to the tap Git repository. You can check update-homebrew-formula.sh for the script that does it.

The rest of the formula is fixed. There’s the install section, which for me was quite simple:

def install
    bin.install "celq"
end

Still, you could leverage Ruby to write more complicated actions such as executing post-installation scripts.

Lastly, Homebrew also lets us run tests post installation. This is not a requirement, but I found it nice to be able to write a smoke test with test do to confirm that things work at runtime.

Homebrew from source plus bottles

One intriguing limitation from dist for me was that it didn’t support creating formulas that build from source. But what did that entail?

Upon looking at other formulas in homebrew-core for Rust CLIs such as the one for just, the answer became clearer.

If you list Rust as a build dependency, Homebrew can delegate the work to Cargo and install the binary in the right spot:

depends_on "rust" => :build

def install
  system "cargo", "install", "--locked", "--root", prefix, "--path", "."
end

For crates that don’t want to host pre-built binaries or crates that depend on system libraries, that option can be interesting.

Another thing I learned about while setting up the installation from source were Homebrew bottles. It turns out that homebrew-core handles pre-built binaries differently from how we discussed in the previous section!

Instead of modifying the url field based on the platform, formulas with bottles keep the source code in the url field and then generate binaries with brew bottle $formula. It looks like this:

bottle do
  root_url "$GITHUB_REPO_URL"
  rebuild 0
  sha256 cellar: :any, arm64_tahoe:   "$hash1"
  sha256 cellar: :any, arm64_sequoia: "$hash2"
  sha256 cellar: :any, x86_64_linux:  "$hash4"
end

I assume that dist chose the style without bottles because of simplicity. Formulas with bottles are a two-step process. First, you need to update the formula. Then, you need to run brew bottle after the formula is in the tap.

It’s easier to automate the case without bottles. Nevertheless, bottles unlock the benefits of pre-built binaries with the flexibility of allowing users to pass --build-from-source when required.

Making an installer script can be quite hard

Next, to install scripts. My original goal was to have one of those one-liners:

curl --proto '=https' --tlsv1.2 -sSf https://get-celq.github.io/install.sh | bash

Before I wrote my own script, I looked at uv’s script from their GitHub release. uv uses dist to generate their script.

To my surprise, I found around 2,000 lines of shell script. I can summarize my learnings as: it is more complicated than it looks! There are lots of details to think about. To list a few:

• Detecting glibc vs musl can be a lot of work
• Adding your binary to the $PATH is also a lot of work
• Handling shadowed bins, updaters, and more
• sh is barebones, you’ll need shellcheck to make sure you didn’t use a feature that is not portable

Even though I like uv and dist, I discarded the idea of re-implementing their strategy.

Copying a good existing script

My next best strategy was to research other Rust software that didn’t use dist and shipped with a script. I found two good candidates: just and cargo-binstall.

I learned some useful tricks by looking at their work. cargo-binstall had the benefit of being an installer, so of course its script uses cargo-binstall to install itself. I guess one way of writing an installer is downloading cargo-binstall and calling it! Although I chose not to take that option.

I ended up basing celq’s installer on the one written by Casey Rodarmor for just. Just’s script was considerably shorter. At 180 lines, it makes a few compromises. It does less things than the script generated by dist, but it still gets the job done.

The first compromise is perhaps the most controversial. How does just’s installer detect glibc or musl’s version? The answer is: it doesn’t! I did not realize that was an option, but if you browse just’s release page you will see that indeed they only ship musl binaries.

Choosing only musl binaries has some consequences, as the binaries compiled with x86_64-unknown-linux-gnu are smaller than the ones for x86_64-unknown-linux-musl. There are also other differences such as the musl vs glibc allocators, if you don’t use a custom allocator. But the amount of complexity avoided by only using musl in the installer made me adopt the same strategy.

The other simplification was with regards to modifying the $PATH environment variable. The just installer avoids this problem by requiring a --to argument. If you pass a location that is in the $PATH, good for you! Otherwise the installer will not intervene. This again cuts many lines of code.

I diverged a bit from just’s behavior by guessing default destinations that are generally in $PATH, although the --to option was still available. If the guessed location is not in the path, I just print a warning.

Again, all these small optimizations from Casey compound and make the just installer be less complex.

Attestations and checksums

To wrap things up, let’s discuss two small additions I made to celq’s installer compared to just’s. I saw uv using GitHub attestations and I wanted to play with them.

The first modification was a --verify-checksum flag:

curl --proto '=https' --tlsv1.2 -sSf https://get-celq.github.io/install.sh | \
    bash -s -- --verify-checksum

The concept is simple; I embedded the SHA256 checksums in the installer. It is a simpler check than attestations, but at least it helps with integrity checks to make sure the download was not corrupted. You can see template_install.sh at the repository, it was fairly easy to implement.

The second modification was the --verify-attestation flag. That flag relies on the GitHub CLI (gh). I thought about it because I saw some discussion online about how insecure curl|bash one-liners were. The idea behind it is to mitigate some of those concerns with attestations:

curl --proto '=https' --tlsv1.2 -sSf https://get-celq.github.io/install.sh > install.sh
gh attestation verify install.sh --repo IvanIsCoding/celq
bash install.sh --verify-attestation

Instead of directly piping the script to bash, we save the script to a file. Then, we use gh to verify that the script was indeed generated by me in a GitHub action. Once we establish that the installer is legitimate and from my repository, we run it with the --verify-attestation that applies the same logic to the pre-built binaries. If there is an issue checking that the binaries are from any place other than the ones built by my GitHub Actions, the script fails.

My release process was very similar to Homebrew’s: template the shell script and then make a commit to the GitHub pages repository to publish the script. release_github.yml contains the templating and the attestation generation.

What’s Next

If you found this blog post interesting, stay tuned for the next one. It will cover FreeBSD, Nix, and Guix. There’s cross-compiling Rust binaries to FreeBSD, writing Nix with two kinds of derivations, patching my crate to compile on Guix’s older Rust version, and more.