Models as packages

model-packager-design.org

PRD + Tech Spec — .NET NuGet “Model Fetch + Cache” System (HuggingFace default, configurable source, optional MSBuild prefetch)

1. Problem Statement

We ship a .NET NuGet package that uses an AI model. The model can be large (100MB–multiple GB). Putting the model inside the NuGet package is a bad fit (slow restore, duplicated storage per version, plan quotas, painful CI).

We need a robust, boring, predictable way to:

• Default to downloading from HuggingFace (HF).
• Allow consumers to override the model source (e.g., internal mirror, GitHub release asset, S3/Azure blob).
• Cache the model locally and reuse it.
• Verify integrity.
• Provide an optional MSBuild-driven prefetch that is explicit opt-in (never surprise network calls on restore).

The end result should feel like: “install NuGet, call one method, it just works.”

2. Goals / Non-Goals

2.1 Goals

1. Tiny NuGet package: package ships code + metadata, not multi-GB blobs.
2. Default model source = HuggingFace (public or token-auth).
3. Configurable source override using:
   • runtime options (explicit parameter)
   • environment variables
   • MSBuild properties (propagated to runtime)
4. Local caching with deterministic paths and safe atomic writes.
5. Integrity verification: SHA256 required; file size checks strongly recommended.
6. Optional prefetch workflow that can be triggered via MSBuild or CLI.
7. Good developer experience: clear logs, actionable errors, minimal setup steps.
8. Enterprise-friendly: supports internal mirrors; can run offline after first download.

2.2 Non-Goals

• Not implementing a full artifact hosting service.
• Not implementing a P2P distribution system.
• Not solving model licensing or export restrictions beyond documented warnings.
• Not building a full UI; console logs are enough.

3. Target Users / Use Cases

3.1 Users

• Library consumers building apps that depend on AI inference at runtime.
• CI maintainers wanting deterministic builds and optional prefetch.
• Enterprise teams wanting internal mirror control.

3.2 Use Cases

1. Developer installs NuGet, runs the app: model auto-downloads once, caches, and runs.
2. Developer sets an environment variable to download from a custom mirror.
3. CI pipeline prefetches model during Docker build to avoid runtime download.
4. Consumer uses a private HF repo with HF token.

4. Product Requirements

4.1 Functional Requirements (FR)

FR-1: Provide a public .NET API to “ensure model present” and return a local path. FR-2: Provide default model metadata (repo id, revision, file name, sha256, optional size). FR-3: Download model from default source (HF resolve URL). FR-4: Support private sources with bearer token (HF_TOKEN). FR-5: Support configurable model source override via:

• runtime argument/options
• env vars
• MSBuild property (embedded into assembly metadata or config file)

FR-6: Cache model in a stable OS-appropriate cache directory by default. FR-7: Ensure atomic download (temp file + rename). FR-8: Verify SHA256 after download; if mismatch, delete + error. FR-9: Avoid repeated downloads; if file exists and verifies, reuse. FR-10: Optional prefetch via CLI: `dotnet mylib-model prefetch`. FR-11: Optional MSBuild target to run CLI prefetch at build time when explicitly enabled. FR-12: Logs must be human-readable and show how to fix failures.

4.2 Non-Functional Requirements (NFR)

NFR-1: No network calls during restore by default. NFR-2: Compatible with Windows/Linux/macOS. NFR-3: Support .NET 8+ (or .NET Standard 2.0 for library + separate tool; decide in Implementation). NFR-4: Safe concurrent usage: multiple processes may attempt download simultaneously. NFR-5: Resumable download is a “nice-to-have” (phase 2) but not required for MVP. NFR-6: Security: do not log tokens; do not store tokens on disk by default.

5. Developer Experience (DX) Requirements

5.1 Consumer “Hello Model” Experience

• Install NuGet: `dotnet add package MyLib.Model`
• Add one line of code:

var modelPath = await ModelManager.EnsureModelAsync();

This should:

• download if missing
• verify
• return absolute local path

5.2 Override Source Experience

• Environment variable override:

export MYLIB_MODEL_SOURCE="https://internal.example.com/models/model.onnx"

• MSBuild override (consumer project):

<PropertyGroup>
  <MyLibModelSource>https://internal.example.com/models/model.onnx</MyLibModelSource>
</PropertyGroup>

5.3 Prefetch Experience

• CLI:

dotnet mylib-model prefetch
dotnet mylib-model prefetch --source "https://..."

• MSBuild (opt-in):

<PropertyGroup>
  <MyLibPrefetchModel>true</MyLibPrefetchModel>
</PropertyGroup>

6. High-Level Design

We ship two deliverables:

1. NuGet library package: `MyLib.Model`
2. Optional .NET tool (recommended): `mylib-model` to prefetch and diagnose

Core components:

• ModelMetadata (default manifest)
• ModelSourceResolver (override precedence)
• ModelCache (paths, locking, validation)
• ModelDownloader (HTTP streaming, auth)
• IntegrityVerifier (sha256, size)
• CLI (prefetch/verify/info)
• MSBuild targets (opt-in prefetch + pass source to runtime)

7. Data Model / Configuration

7.1 Default Manifest (embedded JSON)

Ship a file inside the library assembly resources, e.g. `model-manifest.json`:

{
  "modelId": "my-org/my-model",
  "revision": "main",
  "file": "model.onnx",
  "sha256": "REPLACE_WITH_REAL_SHA256",
  "expectedBytes": 1234567890,
  "defaultSource": "hf"
}

7.2 Source Concepts

Source types:

• HF: derive URL from (modelId, revision, file)
• Direct URL: consumer-provided URL to file
• Mirror base URL: consumer-provided base; append file name

Represent as:

public enum ModelSourceKind { HuggingFace, DirectUrl, MirrorBaseUrl }

7.3 Override Precedence (MUST match docs)

Highest → lowest:

1. Explicit runtime `ModelOptions.SourceOverride`
2. Environment variable `MYLIB_MODEL_SOURCE`
3. Assembly metadata `MyLibModelSource` (from MSBuild)
4. Manifest default (HF)

If multiple are set, highest wins. Log which one was used.

8. Public API Spec

8.1 Library API

Namespace: `MyLib.Modeling`

public static class ModelManager
{
    public static Task<string> EnsureModelAsync(
        ModelOptions? options = null,
        CancellationToken cancellationToken = default);

    public static Task<ModelInfo> GetModelInfoAsync(
        ModelOptions? options = null,
        CancellationToken cancellationToken = default);

    public static Task VerifyModelAsync(
        ModelOptions? options = null,
        CancellationToken cancellationToken = default);
}

public sealed record ModelOptions
{
    // Overrides
    public string? SourceOverride { get; init; } // direct url or mirror base url
    public string? CacheDirOverride { get; init; } // absolute path
    public string? HuggingFaceToken { get; init; } // optional; if null use env HF_TOKEN
    public bool ForceRedownload { get; init; } = false;

    // Advanced (optional)
    public Action<string>? Logger { get; init; } // for host logging
}

public sealed record ModelInfo(
    string ModelId,
    string Revision,
    string FileName,
    string Sha256,
    long? ExpectedBytes,
    string ResolvedSource,
    string LocalPath);

Behavior:

• EnsureModelAsync returns local path; downloads if needed.
• VerifyModelAsync checks presence + sha; no download unless ForceRedownload true.
• GetModelInfoAsync returns resolved source + expected local path.

8.2 CLI Tool API (mylib-model)

Commands:

• `prefetch` (downloads if missing; verifies)
• `verify` (verifies; fails non-zero if invalid)
• `info` (prints resolved source, cache path, manifest metadata)
• `clear-cache` (optional; remove model files)

Example:

dotnet mylib-model info
dotnet mylib-model prefetch --source "https://internal/..." --cache-dir "/tmp/models"
dotnet mylib-model verify

Exit codes:

• 0 success
• 1 usage error
• 2 download failed
• 3 verification failed
• 4 permission/path error
• 5 unexpected error

9. MSBuild Integration Spec

Principles:

• Never download during restore by default.
• Opt-in prefetch by setting `MyLibPrefetchModel=true`.

9.1 Packaged Files Layout (NuGet)

The NuGet library package must include:

• /lib/net8.0/MyLib.Model.dll
• /contentFiles/any/any/model-manifest.json (optional; or embedded resource)
• /build/MyLib.Model.props
• /build/MyLib.Model.targets

9.2 Props: define defaults

File: `build/MyLib.Model.props`

<Project>
  <PropertyGroup>
    <!-- default source: HF; consumer can override -->
    <MyLibModelSource Condition="'$(MyLibModelSource)'==''"></MyLibModelSource>

    <!-- opt-in -->
    <MyLibPrefetchModel Condition="'$(MyLibPrefetchModel)'==''">false</MyLibPrefetchModel>

    <!-- optional: override cache dir -->
    <MyLibModelCacheDir Condition="'$(MyLibModelCacheDir)'==''"></MyLibModelCacheDir>
  </PropertyGroup>
</Project>

9.3 Targets: embed build-time metadata + optional prefetch

File: `build/MyLib.Model.targets`

<Project>
  <ItemGroup>
    <!-- Pass MSBuild properties to runtime via assembly metadata -->
    <AssemblyAttribute Include="System.Reflection.AssemblyMetadataAttribute">
      <_Parameter1>MyLibModelSource</_Parameter1>
      <_Parameter2>$(MyLibModelSource)</_Parameter2>
    </AssemblyAttribute>

    <AssemblyAttribute Include="System.Reflection.AssemblyMetadataAttribute">
      <_Parameter1>MyLibModelCacheDir</_Parameter1>
      <_Parameter2>$(MyLibModelCacheDir)</_Parameter2>
    </AssemblyAttribute>
  </ItemGroup>

  <Target Name="MyLibPrefetchModel"
          AfterTargets="Build"
          Condition="'$(MyLibPrefetchModel)'=='true'">

    <Message Text="[MyLib] Prefetch enabled. Source=$(MyLibModelSource) CacheDir=$(MyLibModelCacheDir)"
             Importance="high" />

    <!-- Call CLI tool (preferred) OR fallback to dotnet exec on bundled helper -->
    <Exec Command="dotnet mylib-model prefetch --source &quot;$(MyLibModelSource)&quot; --cache-dir &quot;$(MyLibModelCacheDir)&quot;" />
  </Target>
</Project>

Notes:

• The MSBuild properties may be empty; runtime resolver falls back to env/default HF.
• If consumer never installs the CLI tool, Exec will fail; therefore:
  • Either ship the CLI as a transitive tool dependency (hard),
  • Or ship a small helper executable/dll inside the package and run it with `dotnet exec`.

MVP approach recommendation:

• Ship helper as `MyLib.Model.Prefetcher.dll` in the package and Exec `dotnet “$(MSBuildThisFileDirectory)..\tools\prefetcher\MyLib.Model.Prefetcher.dll” …`

(Implementation details in section 12.)

10. Download + Cache Behavior

10.1 Default Cache Path

Implement OS-specific cache location:

• Windows: `%LOCALAPPDATA%\MyLib\ModelCache\{modelId}\{revision}\{file}`
• Linux/macOS: `~/.cache/mylib/{modelId}/{revision}/{file}`

Allow override via:

• ModelOptions.CacheDirOverride
• env `MYLIB_MODEL_CACHE_DIR`
• assembly metadata `MyLibModelCacheDir`

10.2 Atomic Write

Algorithm:

1. Ensure cache directory exists
2. Download to `file.partial.{guid}`
3. Verify SHA256 and (optional) bytes
4. Rename/move to final file name (atomic if same filesystem)
5. Cleanup partial files on success/failure

10.3 Concurrency / Locking

Minimum viable locking:

• Use a lock file `model.onnx.lock` in the target directory.
• Acquire exclusive lock:
  • On Windows: FileStream with FileShare.None
  • On Unix: FileStream lock works sufficiently in most cases
• If lock cannot be acquired, wait with backoff until:
  • final file exists and validates, or
  • lock is released and then attempt download

Timeout:

• default 15 minutes (configurable)

10.4 Retry Policy

• 3 retries with exponential backoff for transient HTTP failures (429/5xx, connection reset).
• Do not retry on 401/403 without token fix.

10.5 Integrity Verification

Required: SHA256 Optional: expectedBytes (if known)

If mismatch:

• Delete final file
• Delete partial file
• Throw exception with “Expected sha=…, got sha=…”

11. HuggingFace Integration Details

11.1 Public model URL format

Resolved download URL: `https://huggingface.co/{modelId}/resolve/{revision}/{file}`

Example: `https://huggingface.co/my-org/my-model/resolve/main/model.onnx`

11.2 Private model authentication

• Token from:
  • ModelOptions.HuggingFaceToken
  • else env `HF_TOKEN`
• Use HTTP header:

`Authorization: Bearer <token>`

• Never log token; redact if present.

11.3 Redirects

HF often returns redirects to blob storage. HttpClient must allow redirects (default true) or handle 302/307.

12. Packaging / Build Implementation Plan

12.1 Repo Layout (suggested)

• src/MyLib.Model (library)
• src/MyLib.Model.Prefetcher (small console app or tool)
• pack (NuGet packing scripts if needed)

12.2 Library project

• Embeds manifest as resource OR includes in contentFiles.
• Reads assembly metadata for MSBuild-provided values.
• Implements downloader/cacher/verifier.

12.3 Prefetcher helper

Two viable approaches:

A) Helper Console App shipped inside the NuGet package (recommended for MVP)

• Pack output into `tools/prefetcher/`
• MSBuild `Exec` runs `dotnet <path-to-dll> prefetch …`

Pros: no separate tool install needed. Cons: slightly more packaging complexity.

B) Separate dotnet tool package `mylib-model` Pros: clean UX for advanced users. Cons: requires installation; MSBuild Exec may fail if tool not installed.

Recommendation:

• MVP: Approach A (bundled helper)
• Phase 2: Publish tool package for broader UX

12.4 NuGet packing

Use SDK-style packing:

• Include `.props` and `.targets` via `build/` folder.
• Include helper dll in `tools/` folder.
• Validate installed layout in a sample consumer repo.

Example csproj pack includes:

<ItemGroup>
  <None Include="build\MyLib.Model.props" Pack="true" PackagePath="build\" />
  <None Include="build\MyLib.Model.targets" Pack="true" PackagePath="build\" />

  <None Include="..\MyLib.Model.Prefetcher\bin\Release\net8.0\MyLib.Model.Prefetcher.dll"
        Pack="true" PackagePath="tools\prefetcher\" />
</ItemGroup>

13. Error Messages / Logging (must be actionable)

All thrown exceptions must include:

• What failed (download / verify / permissions)
• Which source was used
• Where it tried to write
• How to fix

Examples:

• “Model download failed with 403. Set HF_TOKEN for private HF repos or override MYLIB_MODEL_SOURCE.”
• “SHA mismatch. Delete cache at … or set ForceRedownload=true.”

Logging guidelines:

• Log selected source and cache path at “info”.
• Log progress optionally (bytes downloaded) if feasible.
• Never log secrets. Redact tokens and query strings if needed.

14. Security Considerations

• Never embed credentials in package.
• Do not persist HF token.
• Verify SHA256 to prevent tampering.
• Prefer HTTPS sources.
• Optional: allow “allowed host list” for enterprise (phase 2).

15. Testing Plan

15.1 Unit Tests

• Resolver precedence (runtime/env/assembly/default)
• Cache path generation across OS
• SHA256 verification
• Atomic write behavior (simulate failure mid-download)
• Locking (two concurrent EnsureModelAsync calls)

15.2 Integration Tests

• Download from a small public HF file (mock HTTP server preferred)
• Private model flow (mock auth header requirement)
• Mirror URL override (mock server)
• MSBuild prefetch end-to-end:
  • consumer project sets MyLibPrefetchModel=true
  • build triggers helper
  • model appears in cache and verifies

15.3 CI

• Run tests on Windows + Ubuntu
• Use mocked HTTP server for deterministic tests (avoid live HF flakiness)

16. Performance Considerations

• Stream downloads; avoid loading into memory.
• Use HttpCompletionOption.ResponseHeadersRead.
• SHA computation should stream file.
• Avoid repeated hash if file is known-good:
  • store `model.onnx.sha256` sidecar file (optional optimization) but must validate safely.

17. Rollout Plan / Milestones

MVP (1)

• Library API + manifest
• HF download + token support
• Cache + atomic + sha verify
• Env var overrides
• Assembly metadata override reading
• Bundled prefetcher helper
• MSBuild opt-in prefetch

Phase 2

• Separate dotnet tool package
• Resume downloads via Range
• Sidecar hash caching
• Enterprise policy hooks (allowed hosts)
• Multi-model support (multiple files / shards)

18. Acceptance Criteria (Definition of Done)

• A new consumer can:
  1. Add NuGet package
  2. Call EnsureModelAsync
  3. First run downloads model from HF and caches it
  4. Subsequent runs do not download again
  5. Setting MYLIB_MODEL_SOURCE overrides HF
  6. Setting <MyLibModelSource> in csproj overrides runtime default (verified by reading assembly metadata)
  7. Setting <MyLibPrefetchModel>true</MyLibPrefetchModel> triggers prefetch on build (no restore side effects)
  8. SHA mismatch causes clear error and no corrupted cache remains
• All tests pass on Windows + Linux.

19. Implementation Notes (AI Agent Checklist)

1. Create library project with manifest + metadata reader.
2. Implement resolver with precedence and logging.
3. Implement cache path logic + override logic.
4. Implement lock file mechanism and backoff.
5. Implement downloader with redirects + auth header + retries.
6. Implement sha verification streaming.
7. Implement bundled prefetcher that calls same library internals (no duplicated logic).
8. Implement pack layout with props/targets/tools.
9. Add sample consumer repo for manual verification.
10. Add tests and CI pipeline.

20. Example: Minimal Consumer Usage (docs snippet)

using MyLib.Modeling;

var path = await ModelManager.EnsureModelAsync(new ModelOptions
{
    Logger = Console.WriteLine
});

Console.WriteLine($"Model ready at: {path}");

21. Example: Consumer Overrides (docs snippet)

Environment variable

export MYLIB_MODEL_SOURCE="https://mirror.company.net/models/model.onnx"
export MYLIB_MODEL_CACHE_DIR="/var/cache/mylib"
export HF_TOKEN="hf_xxx"  # only if using private HF model

MSBuild

<PropertyGroup>
  <MyLibModelSource>https://mirror.company.net/models/model.onnx</MyLibModelSource>
  <MyLibModelCacheDir>$(UserProfile)\.cache\mylib</MyLibModelCacheDir>
  <MyLibPrefetchModel>true</MyLibPrefetchModel>
</PropertyGroup>