Skip to content

Instantly share code, notes, and snippets.

@aldoborrero

aldoborrero/Research.md

Created March 2, 2026 15:45
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save aldoborrero/4819eff7701c93d9aa9c9956b0e80d85 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save aldoborrero/4819eff7701c93d9aa9c9956b0e80d85 to your computer and use it in GitHub Desktop.
Download ZIP
How nix build go apps
Raw
Research.md

How buildGoModule Works in Nixpkgs

Overview

buildGoModule is the standard builder for Go projects in nixpkgs. It wraps stdenv.mkDerivation with Go-specific build phases and produces two derivations: a fixed-output derivation (FOD) that vendors/downloads dependencies, and a main derivation that compiles the Go binaries.

The older buildGoPackage (GOPATH-mode) was removed in October 2024. This document covers only buildGoModule.

File Map

File Role
pkgs/build-support/go/module.nix The entire buildGoModule implementation (413 lines)
pkgs/top-level/all-packages.nix:7806-7828 Wires versioned Go compilers to versioned builders
lib/customisation.nix:860-902 extendMkDerivation -- the abstraction buildGoModule is built on
lib/systems/default.nix:548-574 Maps Nix platform names to GOOS/GOARCH/GOARM
pkgs/development/compilers/go/1.24.nix Go 1.24 compiler
pkgs/development/compilers/go/1.25.nix Go 1.25 compiler (current default)
pkgs/development/compilers/go/1.26.nix Go 1.26 compiler (latest)
pkgs/development/compilers/go/go_no_vendor_checks-1.23.patch Nix-specific patch to bypass vendor consistency checks
pkgs/development/compilers/go/go-env-go_ldso.patch Nix-specific patch making the ELF interpreter configurable via GO_LDSO
doc/languages-frameworks/go.section.md User-facing documentation

How the Builder is Wired Up

In pkgs/top-level/all-packages.nix:7806-7828:

go = go_1_25;
buildGoModule = buildGo125Module;

go_latest = go_1_26;
buildGoLatestModule = buildGo126Module;

buildGo124Module = callPackage ../build-support/go/module.nix { go = buildPackages.go_1_24; };
buildGo125Module = callPackage ../build-support/go/module.nix { go = buildPackages.go_1_25; };
buildGo126Module = callPackage ../build-support/go/module.nix { go = buildPackages.go_1_26; };

Every versioned builder uses the same module.nix -- the only difference is which Go compiler is injected. The go comes from buildPackages (the build platform), which is critical for cross-compilation.

The extendMkDerivation Abstraction

buildGoModule is not a simple wrapper function. It uses lib.extendMkDerivation (lib/customisation.nix:860-902) to create a proper extension of stdenv.mkDerivation that:

  1. Preserves finalAttrs fixed-point semantics (so attributes can reference each other and overrideAttrs works correctly)
  2. Extends the derivation arguments with Go-specific logic
  3. Excludes internal attribute names (like overrideModAttrs) from being passed to the underlying mkDerivation
lib.extendMkDerivation {
  constructDrv = stdenv.mkDerivation;
  excludeDrvArgNames = [ "overrideModAttrs" ];
  extendDrvArgs = finalAttrs: { ... }@args: { ... };
}

The extendDrvArgs function receives finalAttrs (the self-referencing fixed point) and the user-provided arguments, then returns the actual derivation attributes with all the Go build phases defined inline as Nix strings.

The Two-Derivation Architecture

                    ┌──────────────────────┐
                    │    User's package    │
                    │   (buildGoModule)    │
                    └──────────┬───────────┘
                               │
              ┌────────────────┴────────────────┐
              │                                 │
   ┌──────────▼──────────┐           ┌──────────▼──────────┐
   │  goModules (FOD)    │           │  Main derivation    │
   │                     │           │                     │
   │ - Fetches deps      │ ─────────▶  - Compiles Go code  │
   │ - vendorHash pins   │  output   │ - Runs tests        │
   │   the output        │  is used  │ - Installs binaries │
   │ - Network access OK │  as input │ - No network access │
   └─────────────────────┘           └─────────────────────┘

Why two derivations?

Nix builds are sandboxed with no network access. Go modules need to be downloaded from the internet. The solution is a fixed-output derivation (FOD) whose content is pinned by vendorHash. Because the output hash is known in advance, Nix allows network access during its build. The main derivation then uses these pre-fetched dependencies with GOPROXY=off.

User-Facing Attributes

These are the Go-specific attributes accepted by buildGoModule (module.nix:17-70):

Attribute Default Purpose
vendorHash required SRI hash of the vendored dependencies FOD. Set to null if deps are already vendored in source.
modRoot "./" Directory containing go.mod/go.sum relative to src
proxyVendor false Use go mod download + module proxy instead of go mod vendor
deleteVendor false Delete existing vendor/ before fetching deps
goSum null Track go.sum changes to trigger rebuilds of goModules
allowGoReference false Allow the output to reference the Go compiler in the store
ldflags [] Go linker flags (e.g., -s -w, -X main.version=...)
GOFLAGS [] Additional Go build flags
tags (none) Go build tags
subPackages (none) Specific packages to build (e.g., ["cmd/foo" "cmd/bar"])
excludedPackages (none) Packages to exclude from the build
buildTestBinaries false Build test binaries instead of regular binaries
overrideModAttrs identity Function to override the goModules sub-derivation
modPostBuild (none) postBuild hook for the goModules derivation (not the main one)
modConfigurePhase (none) Override configurePhase for the goModules derivation
modBuildPhase (none) Override buildPhase for the goModules derivation
modInstallPhase (none) Override installPhase for the goModules derivation

The legacy vendorSha256 is explicitly rejected with an error at module.nix:30-31.

Phase-by-Phase Breakdown

1. goModules Derivation (the FOD)

Created only when vendorHash != null. When vendorHash == null, goModules is set to "" and skipped entirely (the source must already contain a vendor/ directory or have no external dependencies).

goModules: configurePhase (module.nix:124-131)

export GOCACHE=$TMPDIR/go-cache
export GOPATH="$TMPDIR/go"
cd "$modRoot"

Sets up temporary Go caches and changes to the module root.

goModules: buildPhase (module.nix:133-172)

The build phase has three stages:

Stage 1 -- Handle existing vendor directory:

  • If deleteVendor = true: removes vendor/, errors if it doesn't exist
  • If vendor/ still exists: errors with "please set vendorHash = null"

Stage 2 -- Fetch dependencies (two modes):

proxyVendor = false (default) proxyVendor = true
Runs go mod vendor Runs go mod download
Creates a vendor/ directory with all source code Populates $GOPATH/pkg/mod/cache/download
Simpler, most common Needed for C dependencies or case-insensitive filesystem conflicts

Stage 3: Creates vendor/ directory (mkdir -p) and runs postBuild.

goModules: installPhase (module.nix:174-196)

proxyVendor = false proxyVendor = true
cp -r vendor $out Removes sumdb/, copies $GOPATH/pkg/mod/cache/download to $out

Errors if the output is empty (tells user to set vendorHash = null).

goModules: Key Properties (module.nix:198-215)

outputHashMode = "recursive";
outputHash = finalAttrs.vendorHash;
outputHashAlgo = if finalAttrs.vendorHash == "" then "sha256" else null;
dontFixup = true;

The FOD inherits these attributes from the main derivation: src, prePatch, patches, patchFlags, postPatch, preBuild, sourceRoot, setSourceRoot, env. This is because patches may modify go.mod/go.sum.

It also has impureEnvVars for proxy settings (GIT_PROXY_COMMAND, SOCKS_SERVER, GOPROXY) since it needs network access.

2. Main Derivation: Environment Setup (module.nix:219-238)

Before any phase runs, these environment variables are set in env:

env = {
  GOOS = go.GOOS;       # e.g., "linux"
  GOARCH = go.GOARCH;   # e.g., "amd64"
  GO111MODULE = "on";
  GOTOOLCHAIN = "local";
  CGO_ENABLED = go.CGO_ENABLED;  # usually "1"
  GOFLAGS = ...;  # assembled from user flags + defaults
};

GOFLAGS assembly (module.nix:227-237):

  1. User-provided GOFLAGS
  2. -mod=vendor (unless proxyVendor = true)
  3. -trimpath (unless allowGoReference = true)

Warnings are emitted if the user manually sets -mod= or -trimpath.

ldflags (module.nix:243): -buildid= is appended automatically for reproducibility unless the user already set one.

3. Main Derivation: configurePhase (module.nix:245-275)

export GOCACHE=$TMPDIR/go-cache
export GOPATH="$TMPDIR/go"
export GOPROXY=off       # No network access
export GOSUMDB=off       # No checksum database
# Cross-compilation: set dynamic linker
if [ -f "$NIX_CC_FOR_TARGET/nix-support/dynamic-linker" ]; then
  export GO_LDSO=$(cat $NIX_CC_FOR_TARGET/nix-support/dynamic-linker)
fi
cd "$modRoot"

Then, if vendorHash != null (goModules was built):

proxyVendor = false proxyVendor = true
rm -rf vendor && cp -r "$goModules" vendor export GOPROXY="file://$goModules"

This is where the output of the goModules FOD is injected into the build.

4. Main Derivation: buildPhase (module.nix:277-365)

This is the most substantial phase. It defines two shell functions and then iterates over packages to build.

getGoDirs function (module.nix:322-330)

Discovers which packages to build:

getGoDirs() {
  local type="$1"
  if [ -n "$subPackages" ]; then
    echo "$subPackages" | sed "s,\(^\| \),\1./,g"
  else
    find . -type f -name \*$type.go -exec dirname {} \; \
      | grep -v "/vendor/" | sort --unique | grep -v "$exclude"
  fi
}
  • If subPackages is set: uses exactly those (prefixed with ./)
  • Otherwise: finds all directories with .go files, excluding vendor/, _-prefixed dirs, examples, Godeps, testdata, and anything in excludedPackages

buildGoDir function (module.nix:293-319)

Core compilation function:

buildGoDir() {
  local cmd="$1" dir="$2"
  declare -a flags
  flags+=(${tags:+-tags=$(concatStringsSep "," tags)})
  flags+=(${ldflags:+-ldflags="${ldflags[*]}"})
  flags+=("-p" "$NIX_BUILD_CORES")
  if [ "$cmd" = "test" ]; then
    flags+=(-vet=off)
    flags+=($checkFlags)
  fi
  # Run go $cmd, silently ignore "no Go files" errors
  if ! OUT="$(go $cmd "${flags[@]}" $dir 2>&1)"; then
    if ! echo "$OUT" | grep -qE '(no( buildable| non-test)?|build constraints exclude all) Go (source )?files'; then
      echo "$OUT" >&2
      return 1
    fi
  fi
}

Build loop (module.nix:335-348)

for pkg in $(getGoDirs ""); do
  # Normal mode:
  buildGoDir install "$pkg"        # go install
  # Or if buildTestBinaries = true:
  buildGoDir "test -c -o $GOPATH/bin/" "$pkg"  # go test -c
done

Cross-compilation normalization (module.nix:350-361)

When cross-compiling, Go places binaries in $GOPATH/bin/${GOOS}_${GOARCH}/. This block moves them up to $GOPATH/bin/ so the install phase works uniformly.

5. Main Derivation: checkPhase (module.nix:367-379)

# Remove -trimpath for tests (tests may reference test assets)
export GOFLAGS=${GOFLAGS//-trimpath/}
for pkg in $(getGoDirs test); do
  buildGoDir test "$pkg"
done
  • doCheck defaults to true (unless buildTestBinaries = true)
  • Reuses the same buildGoDir and getGoDirs functions from the build phase
  • The test argument to getGoDirs causes it to find *test.go files

6. Main Derivation: installPhase (module.nix:381-390)

mkdir -p $out
dir="$GOPATH/bin"
[ -e "$dir" ] && cp -r $dir $out

Simply copies compiled binaries from $GOPATH/bin to $out/bin.

Vendoring Modes Summary

vendorHash = "sha256-...";     vendorHash = "sha256-...";      vendorHash = null;
proxyVendor = false (default)  proxyVendor = true              (no FOD created)

  ┌─ goModules FOD ──┐          ┌─ goModules FOD ──┐            Source must have
  │ go mod vendor    │          │ go mod download  │            vendor/ already
  │ copies vendor/   │          │ copies mod cache │            or no deps
  └────────┬─────────┘          └────────┬─────────┘
           │                             │
  ┌─ Main build ─────┐          ┌─ Main build ─────┐          ┌─ Main build ─────┐
  │ cp goModules →   │          │ GOPROXY=file://  │          │ Uses existing    │
  │    vendor/       │          │    goModules     │          │   vendor/        │
  │ GOFLAGS=         │          │ (no -mod=vendor) │          │ GOFLAGS=         │
  │   -mod=vendor    │          │                  │          │   -mod=vendor    │
  └──────────────────┘          └──────────────────┘          └──────────────────┘

Platform Mapping

GOOS and GOARCH are derived from the Nix platform in lib/systems/default.nix:548-574:

go = {
  GOARCH = {
    "aarch64"     = "arm64";
    "arm"         = "arm";
    "i686"        = "386";
    "x86_64"      = "amd64";
    "riscv64"     = "riscv64";
    "wasm32"      = "wasm";
    # ... and more
  }.${final.parsed.cpu.name} or null;
  GOOS = if final.isWasi then "wasip1" else final.parsed.kernel.name;
  GOARM = ...;  # "5", "6", or "7" from cpu.version
};

Nix-Specific Patches to the Go Compiler

The Go compiler in nixpkgs carries several patches that make buildGoModule work:

go_no_vendor_checks patch

Adds an environment variable GO_NO_VENDOR_CHECKS=1 that bypasses Go's vendor consistency checks. Without this, Go would reject vendor directories that were constructed by Nix (copied from the FOD) because they may not exactly match what go mod vendor would produce on the same machine.

The patch modifies two locations in cmd/go/internal/modload/:

  • import.go: Allows importing vendored packages not listed in modules.txt
  • vendor.go: Skips vendor consistency checks when the env var is set and modules.txt is empty

go-env-go_ldso patch

Makes the ELF interpreter (dynamic linker) configurable via a GO_LDSO environment variable at link time. Normally Go hardcodes the dynamic linker path at compiler build time. This patch is essential for cross-compilation in Nix, where the build and target platforms have different dynamic linkers.

Used in the main configurePhase:

export GO_LDSO=$(cat $NIX_CC_FOR_TARGET/nix-support/dynamic-linker)

iana-etc, mailcap, tzdata patches

These patches prepend Nix store paths so Go programs can find system data files (/etc/protocols, /etc/services, MIME types, timezone data) on NixOS, where these files don't live at their traditional FHS locations.

Security: disallowedReferences

disallowedReferences = lib.optional (!finalAttrs.allowGoReference) go;

By default (allowGoReference = false), the Go compiler is added as a disallowed reference. This means Nix will fail the build if the output binary contains any store path pointing to the Go toolchain. Combined with -trimpath in GOFLAGS, this ensures Go binaries don't leak build-time paths.

Only set allowGoReference = true for programs tightly coupled with the compiler (e.g., gopls, delve).

overrideModAttrs

Every buildGoModule derivation exposes passthru.overrideModAttrs to allow overriding the goModules sub-derivation without affecting the main build:

myPackage.overrideAttrs {
  overrideModAttrs = finalAttrs: prevAttrs: {
    # Add extra native build inputs only to the goModules FOD
    nativeBuildInputs = prevAttrs.nativeBuildInputs ++ [ pkgs.mercurial ];
  };
}

This is implemented using lib.toExtension (module.nix:402) to canonicalize the function as an attribute overlay, and composed via lib.composeExtensions when chaining overrides.

Complete Build Flow (Sequence)

1. Nix evaluates buildGoModule { ... }
   │
   ├─ lib.extendMkDerivation merges Go-specific attrs with user attrs
   │
   ├─ If vendorHash != null:
   │   │
   │   ├─ 2. Build goModules FOD
   │   │   ├─ Apply patches (inherited from main derivation)
   │   │   ├─ configurePhase: set GOCACHE, GOPATH, cd modRoot
   │   │   ├─ buildPhase: optionally deleteVendor, then:
   │   │   │   ├─ proxyVendor=false: go mod vendor
   │   │   │   └─ proxyVendor=true:  go mod download
   │   │   ├─ installPhase: copy vendor/ or mod cache to $out
   │   │   └─ Hash output against vendorHash (FOD verification)
   │   │
   │   └─ 3. Build main derivation (goModules is now a store path)
   │
   └─ If vendorHash == null:
       │
       └─ 3. Build main derivation (no goModules, source has vendor/)

3. Main derivation
   ├─ configurePhase:
   │   ├─ Set GOCACHE, GOPATH, GOPROXY=off, GOSUMDB=off
   │   ├─ Set GO_LDSO for cross-compilation
   │   ├─ cd modRoot
   │   └─ Inject goModules: copy vendor/ or set GOPROXY=file://
   │
   ├─ buildPhase:
   │   ├─ Discover packages (getGoDirs)
   │   ├─ Build exclusion pattern
   │   └─ For each package: go install (or go test -c)
   │       with -tags, -ldflags, -p $NIX_BUILD_CORES
   │
   ├─ checkPhase (if doCheck=true):
   │   ├─ Remove -trimpath from GOFLAGS
   │   └─ For each test package: go test
   │
   ├─ installPhase:
   │   └─ cp $GOPATH/bin → $out/bin
   │
   └─ fixupPhase (standard):
       └─ Verify no disallowed references to Go compiler

Historical Note: buildGoPackage (Removed)

buildGoPackage was removed in October 2024. Key differences from buildGoModule:

Aspect buildGoModule buildGoPackage
Go module mode GO111MODULE=on GO111MODULE=off (GOPATH mode)
Dependencies go.mod/go.sum + vendorHash goDeps Nix expression listing each dependency
Source layout Source stays in place Source moved into $GOPATH/src/$goPackagePath
Required attribute vendorHash goPackagePath
doCheck default true false
Implementation lib.extendMkDerivation with finalAttrs Direct stdenv.mkDerivation with removeAttrs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment