git-namespace-vendoring.md

Git Namespace Vendoring

A method for vendoring git dependencies into the host repository using git namespaces, so that installation from git remotes resolves using the local git namespaces rather than over the network. This makes installs faster, eliminates dependency on upstream availability, protects against remote force-pushes rewriting history, and protects against link rot rendering a package unbuildable.

The Technique

Git namespaces allow us to treat a prefix under refs/namespaces/<n> as a remote repository root. This enables us to mirror a foreign repository into that prefix on our local repository: head, branches, tags, notes, replacements, and any other refs the remote exposes. The underlying objects (commits, trees, blobs) are deduplicated into the shared object store. Git invoked via git --namespace=<ns> or GIT_NAMESPACE=<ns> treats refs/namespaces/<ns>/ as the ref root for operations against remote URLs.

By configuring git's url.*.insteadOf rules to rewrite a git dependency's remote URL to an ext:: transport targeting a corresponding git namespace on the host repository, any invocation of git against the remote URL will operate on the local namespaced mirror instead. When the package manager invokes git against the original remote URL, branches, tags, and version constraints resolve exactly as they would against the upstream remote.

Critically, because insteadOf rules apply at the transport level, they affect every git invocation made during the install process, not just those involving direct dependencies. If any dependency has its own git+<proto>: dependency whose URL matches an existing insteadOf rule, it will be resolved against the local namespace. Transitive dependencies can be intentionally vendored by adding a corresponding namespace and insteadOf rule. Dependencies with remote URLs that don't match any rule resolve from their upstream as usual.

Setup and Installation

1. Fetch into a namespace

Add the dependency as a remote with a mirroring refspec * that maps all refs into the chosen git namespace refs/namespaces/<ns>:

git remote add wonderland 'https://example.com/alice/wonderland.git'
git config remote.wonderland.fetch '*:refs/namespaces/wonderland/*'
git config remote.wonderland.tagOpt --no-tags
git fetch wonderland

We intentionally do not use the +* force-update refspec so that git fetch fails when the remote has rewritten history.

--no-tags prevents upstream tags from polluting the local project's refs/tags/. Tags still get fetched, the * refspec places them under refs/namespaces/wonderland/refs/tags/* but they're isolated from the host repository.

2. Rewrite the URL

Configure git to redirect the upstream URL to the local namespace. We do this with a url.*.insteadOf rule that rewrites the remote to an ext:: transport.

git config --global protocol.ext.allow always
git config --global \
	"url.ext::git --namespace=wonderland %s $(pwd).insteadOf" \
	"https://example.com/alice/wonderland.git"

Warning: Because --global is used, protocol.ext.allow always permits git to execute arbitrary commands via ext:: URLs from any repository on this machine. This is acceptable for demonstrating the mechanism, but for real use you should scope the permission per invocation. See [[#Avoiding Global gitconfig]] below.

Both settings use --global because package managers clone dependencies in temporary directories outside the project tree, where the host repository's .git/config is not visible. For the same reason, $(pwd) bakes an absolute path into the rule at configuration time — the working directory at resolve time is unpredictable. This means moving or renaming the repository silently breaks resolution, and these config entries are machine-specific (don't share them via dotfiles or committed gitconfig includes).

Note

The rewrite config rule has the form:

url.<transport command>.insteadOf = <original URL>

where the transport command is:

ext::git --namespace=wonderland %s /absolute/path/to/pwd

• ext:: — the transport; run the following command instead of connecting to a remote.
• git --namespace=wonderland — scope all ref lookups to refs/namespaces/wonderland/.
• %s — placeholder replaced by the git service being requested (e.g. upload-pack).
• /home/user/myproject — absolute path to this repository.

3. Install via Package Manager

With the insteadOf rule in place, the package manager resolves the original URL and git redirects it locally:

pnpm add 'git+https://example.com/alice/wonderland.git#semver:^1.0.0'
pnpm add 'git+https://example.com/alice/wonderland.git#main'
pnpm add 'git+https://example.com/alice/wonderland.git#<commit>'

Semver ranges, branch names, and commit hashes all work. The package manager invokes git to resolve the specifier fragment and the insteadOf rule intercepts, so the entire install is local.

Supported Protocols

Package managers strip the git+ prefix before passing the URL to git. The insteadOf URL must match what git actually receives, not the package manager specifier.

Specifier	What git receives
git+https://example.com/alice/wonderland.git#fragment	https://example.com/alice/wonderland.git
git+ssh://git@example.com/alice/wonderland.git#fragment	ssh://git@example.com/alice/wonderland.git
ssh://git@example.com/alice/wonderland.git#fragment	ssh://git@example.com/alice/wonderland.git
git+file:///home/alice/wonderland.git#fragment	file:///home/alice/wonderland.git

Bare git:// and ssh:// are passed through unchanged.

The currently whitelisted protocols are git, git+http, git+https, git+ssh, git+file, git+rsync, git+ftp, ssh. An unrecognized protocol may be resolved by the package manager in unexpected ways, such as treating it as a path relative to the local filesystem.

Note

The package manager does not validate the URI after stripping the prefix, it passes whatever remains to git. Even a malformed URI like file:../invalid-uri (from git+file:../invalid-uri) is handed through, where insteadOf can intercept it before git tries to resolve it.

Updating Namespaced Dependencies

git fetch <remote> # or --all
pnpm update # will still use semver, branch, or commitish if configured

Avoiding Global gitconfig

The --global approach works but pollutes the global gitconfig and forces a fragile absolute path to the host repository. One alternative is to inject the rewrite via environment variables scoped to the package manager's process. For pnpm, a .pnpmfile.cjs can generate the config for every pnpm invocation:

// .pnpmfile.cjs
const root = require("path").resolve(__dirname);

function configure(env) {
	let base = +env.GIT_CONFIG_COUNT || 0;
	env[`GIT_CONFIG_KEY_${base}`] = "protocol.ext.allow";
	env[`GIT_CONFIG_VALUE_${base++}`] = "always";
	env[`GIT_CONFIG_KEY_${base}`] =
		`url.ext::git --namespace=wonderland %s ${root}.insteadOf`;
	env[`GIT_CONFIG_VALUE_${base++}`] =
		"https://example.com/alice/wonderland.git";
	env.GIT_CONFIG_COUNT = `${base}`;
	return env;
}
configure(process.env);

module.exports = {};

GIT_CONFIG_COUNT, GIT_CONFIG_KEY_*, and GIT_CONFIG_VALUE_* are git's environment-based config mechanism. They are scoped to the process, so they affect the package manager's child git processes without touching any config file. The same approach works from a wrapper script around any package manager.

Local .git/config cannot be used here because pnpm install switches to the store directory before it calls git; it clones into a store directory, where the local config no longer applies.

Scoping ext:: to git processes only

The environment variable approach exposes protocol.ext.allow to the entire process tree for all invocations of pnpm. A tighter alternative is to monkey-patch child_process.spawn in .pnpmfile.cjs so that the config is only injected into git invocations:

// .pnpmfile.cjs
const cp = require("child_process");
const root = require("path").resolve(__dirname);
const { spawn, spawnSync } = cp;

function configure(env) {
	let base = +env.GIT_CONFIG_COUNT || 0;
	env[`GIT_CONFIG_KEY_${base}`] = "protocol.ext.allow";
	env[`GIT_CONFIG_VALUE_${base++}`] = "always";
	env[`GIT_CONFIG_KEY_${base}`] =
		`url.ext::git --namespace=wonderland %s ${root}.insteadOf`;
	env[`GIT_CONFIG_VALUE_${base++}`] =
		"https://example.com/alice/wonderland.git";
	env.GIT_CONFIG_COUNT = `${base}`;
	return env;
}

const bind = (spawn) =>
	function (file, args, opts) {
		if (file === "git") {
			if (!Array.isArray(args)) [args, opts] = [[], args];
			opts = { ...opts, env: configure({ ...opts?.env }) };
		}
		return spawn.call(this, file, args, opts);
	};

cp.spawn = bind(cp.spawn);
cp.spawnSync = bind(cp.spawnSync);

module.exports = {};

This way non-git child processes never see protocol.ext.allow, and the absolute path to the host repository is always correct.