Nix dirty workspace experiments (effect-utils)

gist-report.md

Nix Dirty Workspace Experiments (effect-utils)

Scope and Goals

We need a fast, reliable, pure way to build Bun-based native CLIs from TypeScript that:

• works with dirty/uncommitted changes,
• avoids copying large trees like node_modules,
• works in flakes and devenv,
• keeps dirty builds close to clean build performance.

Environment

• macOS (aarch64-darwin)
• Determinate Nix
• repo: effect-utils

Key Constraints

• No --impure for the main path.
• Must support untracked changes during local iteration.
• Prefer pure evaluation and avoid large source copies.

High-level Conclusion

Nix-native source filtering (filesets/cleanSourceWith/sourceByRegex/nix-gitignore) still copies into the store and is slow on macOS (80–215s in tests). Pure eval cannot see arbitrary local paths. The rsync-staged “mini workspace” approach is the only method that is pure, includes dirty/untracked files, avoids large copies, and is fast (~0.16s).

Detailed Experiments (Commands + Timings)

All timings are real time.

1) rsync staging (current approach)

• Purpose: create a minimal “workspace” tree for a path-based flake without node_modules.
• Command (via helper):

  prepare_cli_flake "$WORKSPACE_ROOT"

• Result: success.
• Timing: ~0.16s (sync to cli-flake/workspace).

2) Nix-derived workspace (pure)

• Goal: build a minimal source tree via Nix derivation (filtered + cp -R).

Prefetch (needed to avoid first-time fetch cost)

nix flake prefetch --no-write-lock-file github:NixOS/nixpkgs/release-25.11

• Result (ulimit 8192): success
• Timing: 74.04s
• Default ulimit 256: failed (Too many open files)

Build

nix build --no-link --no-write-lock-file "path:/private/tmp/cli-workspace-experiment.tHhLQ5#cliWorkspace"

• Result: success
• Timing: 174.33s

3) Pure-eval path visibility (blocked)

These confirm why pure eval cannot build from dirty local paths without staging:

• builtins.getFlake on local path: or git+file:: fails (“unlocked flake reference”).
• lib.fileset.toSource on local paths: fails (“path does not exist”).

4) Impure approaches (dirty access)

All runs use --impure. These succeed, but are slow.

4.1) lib.fileset.toSource (full allowed tree)

nix eval --raw --impure --expr 'let flake = builtins.getFlake "path:/Users/schickling/Code/overengineeringstudio/workspace2/effect-utils"; pkgs = import flake.inputs.nixpkgs { system = "aarch64-darwin"; }; fs = pkgs.lib.fileset; root = /Users/schickling/Code/overengineeringstudio/workspace2/effect-utils; files = fs.unions [ /Users/schickling/Code/overengineeringstudio/workspace2/effect-utils/nix /Users/schickling/Code/overengineeringstudio/workspace2/effect-utils/scripts /Users/schickling/Code/overengineeringstudio/workspace2/effect-utils/patches /Users/schickling/Code/overengineeringstudio/workspace2/effect-utils/packages /Users/schickling/Code/overengineeringstudio/workspace2/effect-utils/dotdot.json /Users/schickling/Code/overengineeringstudio/workspace2/effect-utils/dotdot.json.genie.ts /Users/schickling/Code/overengineeringstudio/workspace2/effect-utils/tsconfig.all.json /Users/schickling/Code/overengineeringstudio/workspace2/effect-utils/tsconfig.all.json.genie.ts ]; in fs.toSource { root = root; fileset = files; }'

• Result: /nix/store/hskjqg45d7nl2qk87374nxb5bii704rz-source
• Timing: 206.27s

4.2) lib.sources.cleanSourceWith (dotdot only)

nix eval --raw --impure --expr 'let flake = builtins.getFlake "path:/Users/schickling/Code/overengineeringstudio/workspace2/effect-utils"; pkgs = import flake.inputs.nixpkgs { system = "aarch64-darwin"; }; lib = pkgs.lib; root = /Users/schickling/Code/overengineeringstudio/workspace2/effect-utils; prefixes = [ "packages/@overeng/dotdot" ]; src = lib.sources.cleanSourceWith { src = root; filter = path: type: let rel = lib.removePrefix (toString root + "/") (toString path); isUnder = prefix: rel == prefix || lib.hasPrefix (prefix + "/") rel; include = lib.any isUnder prefixes; in if type == "directory" then rel == "" || include else include; }; in src'

• Result: /nix/store/0ccnxa25whszw7mgbgyzdm4nqc0zwnm8-source
• Timing: 80.67s

4.3) lib.sources.cleanSourceWith (dotdot + utils)

nix eval --raw --impure --expr 'let flake = builtins.getFlake "path:/Users/schickling/Code/overengineeringstudio/workspace2/effect-utils"; pkgs = import flake.inputs.nixpkgs { system = "aarch64-darwin"; }; lib = pkgs.lib; root = /Users/schickling/Code/overengineeringstudio/workspace2/effect-utils; prefixes = [ "packages/@overeng/dotdot" "packages/@overeng/utils" ]; src = lib.sources.cleanSourceWith { src = root; filter = path: type: let rel = lib.removePrefix (toString root + "/") (toString path); isUnder = prefix: rel == prefix || lib.hasPrefix (prefix + "/") rel; include = lib.any isUnder prefixes; in if type == "directory" then rel == "" || include else include; }; in src'

• Result: /nix/store/0ccnxa25whszw7mgbgyzdm4nqc0zwnm8-source
• Timing: 111.94s

4.4) lib.sources.sourceByRegex

nix eval --raw --impure --expr 'let flake = builtins.getFlake "path:/Users/schickling/Code/overengineeringstudio/workspace2/effect-utils"; pkgs = import flake.inputs.nixpkgs { system = "aarch64-darwin"; }; lib = pkgs.lib; root = /Users/schickling/Code/overengineeringstudio/workspace2/effect-utils; in lib.sources.sourceByRegex root [ "^packages/@overeng/dotdot(/.*)?$" "^packages/@overeng/utils(/.*)?$" ]'

• Result: /nix/store/0ccnxa25whszw7mgbgyzdm4nqc0zwnm8-source
• Timing: 84.92s

4.5) builtins.path with custom filter

nix eval --raw --impure --expr 'let flake = builtins.getFlake "path:/Users/schickling/Code/overengineeringstudio/workspace2/effect-utils"; pkgs = import flake.inputs.nixpkgs { system = "aarch64-darwin"; }; lib = pkgs.lib; root = /Users/schickling/Code/overengineeringstudio/workspace2/effect-utils; prefixes = [ "packages/@overeng/dotdot" "packages/@overeng/utils" ]; isUnder = prefix: rel: rel == prefix || lib.hasPrefix (prefix + "/") rel; includePath = rel: lib.any (prefix: isUnder prefix rel) prefixes; filter = path: type: let rel = lib.removePrefix (toString root + "/") (toString path); in if type == "directory" then rel == "" || includePath rel else includePath rel; in builtins.path { path = root; filter = filter; }'

• Result: /nix/store/2hx25b866251sq7v044vdd5k29w7bky9-effect-utils
• Timing: 107.03s

4.6) pkgs.nix-gitignore allowlist

nix eval --raw --impure --expr 'let flake = builtins.getFlake "path:/Users/schickling/Code/overengineeringstudio/workspace2/effect-utils"; pkgs = import flake.inputs.nixpkgs { system = "aarch64-darwin"; }; root = /Users/schickling/Code/overengineeringstudio/workspace2/effect-utils; ignore = "*\\n!/packages/\\n!/packages/@overeng/\\n!/packages/@overeng/dotdot/\\n!/packages/@overeng/dotdot/**\\n!/packages/@overeng/utils/\\n!/packages/@overeng/utils/**\\n"; src = pkgs.nix-gitignore.gitignoreSourcePure ignore root; in src'

• Result: /nix/store/xpfv1srai0kry6jbi1vhlvgc07093pgk-effect-utils
• Timing: 215.20s

4.7) linkFarm composition

nix build --no-link --no-write-lock-file --impure --expr 'let flake = builtins.getFlake "path:/Users/schickling/Code/overengineeringstudio/workspace2/effect-utils"; pkgs = import flake.inputs.nixpkgs { system = "aarch64-darwin"; }; lib = pkgs.lib; root = /Users/schickling/Code/overengineeringstudio/workspace2/effect-utils; mkSrc = prefixes: lib.sources.cleanSourceWith { src = root; filter = path: type: let rel = lib.removePrefix (toString root + "/") (toString path); isUnder = prefix: rel == prefix || lib.hasPrefix (prefix + "/") rel; include = lib.any isUnder prefixes; in if type == "directory" then rel == "" || include else include; }; dotdot = mkSrc [ "packages/@overeng/dotdot" ]; utils = mkSrc [ "packages/@overeng/utils" ]; in pkgs.linkFarm "cli-src" [ { name = "packages/@overeng/dotdot"; path = dotdot; } { name = "packages/@overeng/utils"; path = utils; } ]'

• Result: built /nix/store/synnzr516nq2sjls7mhky04rm7bdlf7a-cli-src.drv
• Timing: 89.30s

Summary Table

Approach	Mode	Scope	Result	Real time
rsync staging	pure	minimal workspace	success	~0.16s
cli-workspace derivation	pure	minimal workspace	success	174.33s
fileset.toSource	impure	full allowed tree	success	206.27s
cleanSourceWith	impure	dotdot	success	80.67s
cleanSourceWith	impure	dotdot + utils	success	111.94s
sourceByRegex	impure	dotdot + utils	success	84.92s
builtins.path filter	impure	dotdot + utils	success	107.03s
nix-gitignore allowlist	impure	dotdot + utils	success	215.20s
linkFarm (filtered sources)	impure	dotdot + utils	success	89.30s

Notes

• Pure evaluation cannot access arbitrary local paths; staging is required for dirty changes.
• Impure access enables direct local source selection but is far too slow for iterative development on macOS.
• linkFarm helps slightly but doesn’t avoid the expensive source import step.

Conclusion

Given the constraints, the rsync-staged minimal workspace remains the only viable solution that is pure, includes dirty/untracked files, avoids large copies, and is fast.

research-findings-deep.md

Nix-native Approaches to a 'Dirty Workspace' in Pure Flakes

Handling Dirty (Uncommitted) Sources in Pure Flake Builds

In pure flake mode, Nix normally expects source inputs to be clean and fixed (usually a specific VCS commit). Uncommitted changes make the Git tree “dirty,” which flakes warn about. Best practices include:

• Commit or stage changes whenever possible. Flakes will only include files tracked by Git in the source; untracked files are ignored by default oai_citation:0‡tweag.io. Staging new files ensures they are picked up. This avoids the “Git tree … is dirty” warnings and ensures Nix sees all intended files.
• Use a local path: flake input for development. You can point a flake input to a directory on disk (e.g. inputs.myproj.url = "path:../myproj";). This allows using the current workspace without committing. Nix will treat it as an impure input (not locked), but it will still evaluate in pure mode using the current content oai_citation:1‡github.com. The flake will be marked dirty (and Nix refuses to update the lockfile for it oai_citation:2‡github.com), which is acceptable for iterative local builds.
• Avoid builtins.fetchGit for dirty worktrees in pure eval. While Nix flakes RFC allowed a “Git working tree” input, in practice builtins.fetchGit without a fixed rev is nearly unusable in pure mode – it demands a narHash and errors out if the hash isn’t provided oai_citation:3‡github.com oai_citation:4‡github.com. This makes it impractical to rely on fetchGit for uncommitted changes.
• Leverage nix flake archive for sharing a dirty state if needed. The nix flake archive command can snapshot your flake’s source and inputs. For example, it outputs JSON of the store paths for the flake and its inputs oai_citation:5‡discourse.nixos.org. This is useful in CI or caching: you can archive a dirty dev snapshot and push those paths to a binary cache so that others (or future builds) use the same exact source state without requiring a Git commit. It’s a way to materialize the dirty workspace into a content-addressed store snapshot.
• Keep builds pure: Even though you use an impure input for development convenience, everything inside the flake evaluation should remain pure. For example, avoid referencing absolute filesystem paths or environment variables in the Nix code. The goal is to localize impurity to the flake input (which points at your workspace), so that builds are still isolated and reproducible given that same source tree.

Note on Flake source filtering: If you use a path: input or build the flake from your working directory, Nix (>=2.18) will only add the Git-tracked files to the Nix store by default oai_citation:6‡tweag.io. This means things like untracked build artifacts, node_modules/, caches, etc. are automatically excluded from the build input (unless you explicitly add them). This behavior helps maintain purity and avoid accidental inclusion of large or secret files. If you do need to include an untracked file in the source (not recommended), you’d have to either stage it or use a custom source filter (described below) to include it.

Nix-native Alternatives to an Rsync Workflow

Your current approach uses rsync to create a trimmed copy of the repo for building. There are several Nix-native ways to achieve a similar result – filtering the source to only the needed files – without external scripting. Key alternatives include:

• Nixpkgs lib.fileset library (File Sets): This is a newer, high-level way to represent and filter file sets without immediately copying them into the store. You can define a file set by unioning, intersecting, or subtracting paths. For example, to include only specific directories and file types:

  let
    fs = lib.fileset;
    files = fs.union ./Makefile ./src;
    filtered = fs.fileFilter (file: file.ext == "c" || file.ext == "h") files;
  in 
  fs.toSource { 
    root = ./.; 
    fileset = filtered; 
  }

  Here filtered is a file set of only C source files (and the Makefile) in the project. The call to fs.toSource then imports those files into the store as a single source tree oai_citation:7‡discourse.nixos.org. Prior to toSource, the file set is just a descriptor and no files are copied. The file set library is designed for composability and performance: you can incrementally build up the set of files to include or exclude (with fs.union, fs.difference, etc.) and only at the end Nix will add the needed files to the store oai_citation:8‡discourse.nixos.org. This avoids multiple intermediate copies and has clear semantics. Performance: File sets aim to be as fast as or faster than manual filtering. They defer work until necessary and can optimize certain operations. (Under the hood, file sets use Nix’s built-in filtering in an efficient way.)

• cleanSourceWith and related helpers: Nixpkgs provides lib.cleanSourceWith to filter a source directory with a custom predicate. It’s essentially a wrapper over builtins.filterSource that allows composing multiple filters without extra store copies oai_citation:9‡ryantm.github.io. For example, you could do:

  src = lib.cleanSourceWith {
    src = ./.;
    filter = path: type:
      let rel = lib.removePrefix (toString ./. + "/") (toString path);
      in builtins.match "node_modules/*" rel == null; 
  };

  This (roughly) would exclude any path under node_modules from the source. You can chain cleanSourceWith calls or combine them (it internally ensures filters are lazy ANDed) oai_citation:10‡ryantm.github.io. Nixpkgs also includes convenience filters like cleanSource (which drops VCS dirs, editor backups, etc. by default) oai_citation:11‡ryantm.github.io and more targeted functions. One such convenience is lib.sources.sourceByRegex, which takes a path and a list of regexes, and includes only files matching any of those patterns oai_citation:12‡ryantm.github.io. For example:

  src = lib.sources.sourceByRegex ./my-subproject [ ".*\\.py$" "^database\\.sql$" ];

  would include only *.py files and a database.sql file from my-subproject oai_citation:13‡ryantm.github.io. These helpers wrap around cleanSourceWith (or filterSource) for common use cases. Performance: Filtering at build time will require Nix to traverse the file tree and copy matched files into the store. If the directory is large, this can be I/O intensive. The Nix builtins themselves are written in C++ and are reasonably efficient at copying, but on macOS the filesystem operations can still be slow (more on that below). The advantage of using these library functions is mostly correctness and convenience; performance-wise, they avoid some pitfalls (like multiple filtering passes) and let you be precise about what to include.

• builtins.path with a custom filter: This is the lower-level primitive that both of the above ultimately use. It allows you to import a path into the Nix store with a user-specified filtering function. For example, the following will add the current directory but only include certain files:

  builtins.path {
    path = ./.;
    filter = path: type: 
      /* your logic, return true to include path */ ;
  }

  Under the hood, builtins.path is what Nix calls to turn a path into a content-addressed store object oai_citation:14‡tweag.io. You provide a filter(path, type) -> bool function that decides which entries to keep oai_citation:15‡tweag.io. Using builtins.path directly gives maximum control (for instance, you could read an external ignore file or implement complex logic), but it’s a bit “hardcore.” The Nixpkgs lib functions (cleanSourceWith, file sets, etc.) are easier ways to achieve the same result in most cases oai_citation:16‡discourse.nixos.org. Performance of builtins.path filtering is generally the same as the above – it must walk the directory and apply the predicate. One advantage of doing this at eval time (as builtins.path does) is that the filtering happens once during evaluation rather than as a separate build step. However, the downside is that if the source is huge, your evaluation can become slow or even hit memory limits. File sets mitigate this by being lazier (they don’t evaluate the whole set until needed).

• nix flake archive for inputs caching: While not an alternative filter, flake archive is a useful command in flakes context (especially CI). As mentioned, it lists the store paths of the flake’s current source and inputs oai_citation:17‡discourse.nixos.org. You can use this to prefetch and cache those. For example, running nix flake archive --json | jq -r '.path,(.inputs|to_entries[].value.path)' | cachix push mycache will push the current source and all inputs to a binary cache. Next time, nix build can grab them from the cache instead of refetching or copying. This doesn’t change how you filter the source, but it can eliminate the cost of copying large sources on each build (since they’ll already be in the store/cache). It’s particularly helpful for macOS, where copying to the store can be slow – caching the store paths avoids repeatedly re-copying the same content.

In summary, the above Nix-native approaches let you replace the manual rsync step with Nix expressions that produce a minimal source tree for the build. For example, instead of rsyncing specific directories to cli-flake/workspace, you could define a file set that includes exactly those directories and use fs.toSource to get a filtered src. This source can then be passed to your build (e.g. a derivation that runs Bun). The result is functionally similar to the rsync approach – superfluous files are left behind – but done within Nix’s declarative framework.

Performance on macOS: It’s worth noting that some users have observed slower source filtering on macOS. One reason is the Nix store operations on Darwin can be less efficient (APFS filesystem and security overhead). In fact, a recent Nix 2.19 regression made store imports extremely slow on macOS (operations that took seconds on 2.18 took minutes on 2.19) oai_citation:18‡github.com. Ensure you’re on an up-to-date Nix if you encountered abnormally slow copies. In general, filtering a large tree will be I/O-bound (reading many small files from disk). If your project has thousands of files, the initial import to the store can take time (though subsequent builds will reuse the cached store path until files change).

Avoiding Large File Copies While Preserving Purity

One of the goals is to avoid repeatedly copying large volumes of data (like node_modules or build outputs) into the Nix store, while still keeping builds pure. Here are some patterns to achieve that:

• Exclude what you don’t need. The simplest win is to not include giant or irrelevant directories in the derivation input. Your rsync script already does this (e.g. excluding .git, .direnv, node_modules, dist, caches, etc.) oai_citation:19‡file-gwbnqbxbzxxxtfueqpeye5. In Nix, you should do the same via source filters. For example, Nixpkgs’ default filters drop VCS folders and backup files oai_citation:20‡ryantm.github.io. By pruning these, you reduce the amount of data Nix has to hash and copy. This also improves cache re-use: if a file isn’t in the store, changing it won’t invalidate anything. The Tweag article on source filtering emphasizes that copying the entire directory means any file change forces a rebuild, even something like reformatting Nix code or updating a readme oai_citation:21‡tweag.io. Selective inclusion avoids that cascade.

• Split the source into parts (incremental rebuilds). Instead of one huge source, consider dividing your project into multiple derivations or store paths that correspond to logical components. For instance, if your repo has multiple packages (genie, dotdot, mono, utils in your case), you could create a separate filtered source for each, and then have your top-level build depend on those. With Nix’s file set library, this is straightforward: you can create one fileset per subdirectory. The benefit is that if you change code in one sub-package, Nix can rebuild just that part’s source path and downstream derivations, but other pieces remain intact. As John’s blog demonstrates, a “naive” approach of one monolithic source means “ANY change to the files will cause all derivations to be rebuilt” oai_citation:22‡johns.codes. By contrast, breaking the build into parts using filesets (or separate src attributes) lets Nix see that, say, the utils package’s source hasn’t changed, so it can reuse its previous build results. This dramatically reduces unnecessary work.

• Avoid needless recopying by linking store paths. If you do split sources, you might then need to recombine them to build the final artifact. Copying them together (as you did with cp -R ... $out) can reintroduce cost. A more efficient trick is to use symlinks. Nix outputs can perfectly well be directories of symlinks pointing to other store content. For example, you can use pkgs.linkFarm to create a union of multiple store paths without copying files oai_citation:23‡discourse.nixos.org. This function takes a set of paths and makes a directory tree that symlinks to each file from those inputs. In your case, you could build a derivation that symlinks packages/@overeng/genie -> /nix/store/<hash>-genie-source, and so on for each component. The resulting output is a single directory with all needed files, but creation is much faster than copying (especially for lots of files), and it doesn’t double-use disk space. The Nix store ensures referential integrity – as long as those source store paths exist, the symlinks are valid. This approach maintains purity (everything is still in the Nix store) while minimizing I/O. The downside is a slightly less “normal” file structure (lots of symlinks), but most build tools (like compilers) handle that transparently. In summary, use the store’s content-addressed nature to your advantage: materialize large, slow-changing parts of the source once, and then just reference them on subsequent builds instead of copying again.

• Leverage fixed-output derivations or pre-materialization for heavy dependencies. If your build needs large blobs (say a huge vendored binary or a node modules tarball) that don’t change often, you can treat them as fixed-output inputs. For example, you could have a derivation that downloads a specific version of node_modules (or produce it with a known hash) and use that in your build. Since the hash is fixed, Nix will not re-fetch or rebuild it unless the content changes. In a local workflow, you might even manually add such content to the store with nix store add and refer to it via builtins.storePath. This is an advanced trick: essentially, you do the heavy copy once, note the store path, and then hardcode that path in your flake for reuse. This materialization of a store path breaks the need to copy those files each dev build (at the cost of a manual step to update when they do change). Haskell.nix calls this materialization – caching portions of the evaluation/build so you don’t redo them unnecessarily oai_citation:24‡input-output-hk.github.io. It’s a bit out of the ordinary for general Nix development, but it shows that whenever you have large data that is costly to regenerate, you can push it out-of-line and only update it when needed.

• Use content-addressed Nix (if available): Nix is gradually introducing content-addressed store paths, which can deduplicate identical files across builds oai_citation:25‡nix.dev. In theory, with content-addressing, if you build two different derivations that include the same file, Nix could store that file once and just reference it for both outputs. This feature is still experimental, but it’s worth keeping an eye on. Down the line, it could mean that your “dirty workspace” builds won’t need to duplicate tens of thousands of identical files for each rebuild – Nix would recognize unchanged content by its hash. Until then, the strategies above (filtering, splitting, and linking) are your best bet to avoid big copy storms while staying pure.

In short, to minimize copying: only include necessary files, split the build into modular pieces, and reuse/store outputs cleverly. This aligns with how Nix likes to build things – small, immutable pieces that compose. It fights the “giant workspace rebuild” scenario by not treating your entire repo as one black box blob.

Mitigating Repeated Flake Resolution Slowness

Your benchmarks show a stark contrast: ~0.16s for rsync vs ~174s for the Nix approach after prefetching. A large chunk of that Nix time is likely overhead from flake input resolution and Nixpkgs evaluation, especially on macOS. Here are strategies to tackle that:

• Cache Nixpkgs and other inputs aggressively. Even with a lock file, Nix may refetch or at least re-read inputs if not cached. Make sure you run nix flake prefetch (or nix build once) to get all inputs in the Nix store ahead of time. In your experiment, prefetching nixpkgs/release-25.11 took 74s oai_citation:26‡file-gwbnqbxbzxxxtfueqpeye5 – doing that upfront means subsequent builds don’t hit the network. You can take caching further: as noted, use nix flake archive + Cachix to prepopulate binary caches with those sources. Then a CI or a fresh machine can download the whole flake input closure in one go from cache, rather than doing a slower fetchGit or multiple HTTP requests. Essentially, treat the flake inputs as fixed outputs that you build/cache separately.

• Reuse the same store paths for inputs to avoid re-copying. One quirk observed is that Nix can end up “copying” a store path that it actually already has, due to the way flake inputs are realized. For example, on non-NixOS systems, using flake:nixpkgs often causes Nix to copy the Nixpkgs source into a new store path on each invocation, even if it hasn’t changed (this was reported as a bug) oai_citation:27‡discourse.nixos.org. On NixOS, this doesn’t happen because the system flake registry pins nixpkgs to a fixed store path oai_citation:28‡discourse.nixos.org. To mimic that, you can add an entry in your nix registry on macOS: e.g. nix registry add nixpkgs /nix/store/<hash>-source. The <hash>-source would be the store path of your pinned nixpkgs (find it via nix flake info or archive). By doing so, when your flake asks for github:NixOS/nixpkgs/your-commit, Nix will see it’s already available at that path and skip any copying. This is an advanced optimization, but it can save hundreds of MB of I/O each run. A simpler variant: use --override-input nixpkgs nixpkgs (assuming you have a channel or registry “nixpkgs” set up) which forces it to use the preexisting Nixpkgs in your profile/registry oai_citation:29‡siraben.dev oai_citation:30‡siraben.dev. The idea is to avoid re-downloading or even re-hashing Nixpkgs when it hasn’t changed.

• Amortize evaluation in a long-lived shell. Flakes have to evaluate flake.nix (and import nixpkgs, etc.) for each nix build invocation. Nixpkgs is huge – evaluating it can easily take a couple of seconds or more, especially on Darwin. If you are doing rapid iterative builds, one approach is to enter a development shell (nix develop) that sets up the environment and then run your build steps inside it (perhaps via Bun directly). The dev shell will still incur the flake eval and setup cost once, but then you can rebuild the project as needed with bun or a Makefile script without Nix repeatedly re-evaluating and checking inputs. Essentially, you trade a fully Nix-driven rebuild for a manual rebuild inside a Nix-provided environment. This might be acceptable since TypeScript/Bun can handle incremental builds on its own. You mentioned using devenv/direnv – that’s great for keeping an environment alive so that subsequent operations don’t continually resolve nixpkgs or reinstall tools. While this isn’t speeding up Nix itself, it avoids invoking Nix for repetitive tasks.

• Profile and upgrade Nix if necessary. As noted, some Nix versions have performance issues. The jump from Nix 2.18 to 2.19, for example, introduced a huge slowdown on macOS when copying flake inputs oai_citation:31‡github.com. If you haven’t already, ensure you’re using the latest Nix (2.20+ if available) or even consider using the Flakes-enabled Nix from the nixUnstable package, which might have performance fixes. Also, increasing macOS’s file descriptor limit (ulimit -n) was necessary in your case to avoid errors during flake fetching oai_citation:32‡file-gwbnqbxbzxxxtfueqpeye5 – keeping that high (e.g. 8192 or more) prevents runs from choking on too many open files. These tweaks won’t eliminate the inherent costs, but they prevent pathological slowdowns.

• Use binary caches for development builds. If you push local build results to a cache (even just a local nix-store --export / --import), you can avoid rebuilding or re-copying identical results. For instance, after you generate the cli-workspace output once, you could store it. Then, if you revert a change, Nix might realize that the output for a given input hash already exists and just reuse it. In practice, for a truly dirty (changing) workspace, this is tricky – every code change makes a new hash. But caching helps when toggling between a few known states or branches.

Finally, let’s analyze the root causes of the 174s vs 0.16s discrepancy and how the above mitigations address them:

• Flake input resolution and nixpkgs fetching: A large portion of that 174s was spent just getting nixpkgs and other inputs ready (your log shows ~74s for fetching nixpkgs alone) oai_citation:33‡file-gwbnqbxbzxxxtfueqpeye5. An imperative rsync doesn’t need to evaluate or fetch anything – it just copies files. By caching nixpkgs (prefetching, registry pinning, etc.), you eliminate that cost in subsequent runs. Ideally, you pay that cost once (like how a long-lived dev shell or a one-time prefetch works) rather than on every rebuild.

• Nix evaluation time: Importing nixpkgs and evaluating the flake can easily be 5-10s, and more if your flake does a lot. Rsync doesn’t have this step at all. Using a simpler evaluation (by moving logic out of Nix or into smaller fixed derivations) can help. The file set approach, for example, might let you simplify the filter logic (your current cleanSourceWith filter is fairly complex string logic in Nix; the fileset DSL might be more straightforward and possibly faster). Still, pure Nix will never beat rsync’s “do nothing but copy a few files” in raw speed – it’s doing more work to ensure purity. The hermeticity overhead (hashing, safety checks, evaluations) is the price paid for reproducibility oai_citation:34‡siraben.dev. Keeping Nix expressions lean and using profiles/shells to reuse evaluations helps mitigate this overhead.

• File copying and hashing on macOS: Even after inputs were resolved, your experiment spent ~100s on the nix build that created the filtered workspace oai_citation:35‡file-gwbnqbxbzxxxtfueqpeye5 oai_citation:36‡file-gwbnqbxbzxxxtfueqpeye5. This likely comes down to copying thousands of files (even after filtering). Rsync, by contrast, was syncing a smaller set of changes (often far fewer files). To improve Nix here, the best approach is to reduce the number of files and bytes it needs to process: exclude unneeded files (we did), and break the job so it doesn’t always recalc everything (split sources, linkFarm as discussed). Also, ensure you aren’t accidentally copying heavy files – e.g. if a dependency like a large .o file snuck into the source, filter it out. On macOS APFS, creating thousands of files (like a full copy) can be slower than on Linux ext4. Symlinking (as suggested) is much lighter weight. Users have noted it’s “quite expensive to copy 400 MiB after every rebuild” in Nix oai_citation:37‡discourse.nixos.org – so the strategy is to avoid that copy entirely by reusing the store path or linking it. Once you set up such patterns, the difference can be dramatic: copying 10,000 files vs making 10,000 symlinks or 0 file operations at all if nothing changed.

In conclusion, there are Nix-native ways to approach a “dirty workspace”:

• Use flake inputs or builtins to include your working directory without commits (purity is maintained except for the source itself).
• Filter aggressively (via lib.fileset, cleanSourceWith, etc.) to limit the build input to what’s actually needed, keeping hashes stable when irrelevant files change.
• Structure your Nix build to avoid monolithic copies – reuse unchanged parts through the store, and consider symlink-based assembly of the final workspace.
• Pre-cache and pin heavy dependencies like nixpkgs to avoid paying their cost on each run.
• Accept some overhead: Nix will never be as instant as rsync for incremental file syncing, but you can get closer. By implementing the above, many developers find the gap narrows to something acceptable given the benefits of Nix (pure, repeatable builds).

Ultimately it’s a trade-off. The rsync method is fast because it’s doing the minimal, impure thing – but it’s ad-hoc. The Nix methods strive for purity and transparency at the cost of extra work. With careful tuning, you can achieve a happy medium where your edit–build–test cycle is still snappy (perhaps a few seconds instead of 0.1s, but much better than 174s!) while retaining Nix’s guarantees for your CLI builds.

Sources:

• Tweag Blog – Source filtering with file sets oai_citation:38‡discourse.nixos.org oai_citation:39‡discourse.nixos.org
• NixOS Discourse – Easy source filtering with file sets (Infinisil) oai_citation:40‡discourse.nixos.org oai_citation:41‡discourse.nixos.org
• Nixpkgs Manual – lib.sources and lib.fileset usage oai_citation:42‡ryantm.github.io oai_citation:43‡ryantm.github.io
• User Benchmark Excerpt – Rsync vs Nix-derived workspace timing oai_citation:44‡file-gwbnqbxbzxxxtfueqpeye5 oai_citation:45‡file-gwbnqbxbzxxxtfueqpeye5
• NixOS Discourse – “Nix copying a store-path into the store” (input copying issue) oai_citation:46‡discourse.nixos.org oai_citation:47‡discourse.nixos.org
• GitHub Issues – Flakes dirty worktree & fetchGit, Nix performance on macOS oai_citation:48‡github.com oai_citation:49‡github.com
• Siraben’s Musings – “Dirty flake” hacks and Nix hermeticity costs oai_citation:50‡siraben.dev oai_citation:51‡siraben.dev
• John’s Code Blog – Efficient Nix derivations with file sets (incremental builds) oai_citation:52‡johns.codes.