CI Runner Benchmark - Complete Analysis Total samples: 157 Django, 214 Next.js runs across 5 workflow executionsWorkflow runs analyzed: 21303815025, 21366075868, 21366987511, 21367602080, 21368567991, 21370119015, 21371232547, 21372158500, 21372265822
Expected Savings (Median Execution Time)
Workflow
BuildJet
Blacksmith
Savings
Django 7m 45s
6m 28s
1m 17s (17% faster)
Next.js 3m 41s
2m 45s
55s (25% faster)
Runner
CPU Single
CPU Multi
Disk Write
Disk Read
Network
Blacksmith-16 6,565 96,169 1.4 GB/s 5.9 GB/s 175 MB/s
Blacksmith-8 6,429 49,010
1.4 GB/s 4.6 GB/s
158 MB/s
BuildJet-16
5,243
77,382
962 MB/s
3.5 GB/s
76 MB/s
BuildJet-8
6,214
48,027
1.0 GB/s
3.7 GB/s
17 MB/s
WarpBuild-16
4,172
66,776
565 MB/s
213 MB/s
87 MB/s
WarpBuild-8
4,172
33,415
565 MB/s
138 MB/s
101 MB/s
GitHub-Actions
3,676
4,092
201 MB/s
222 MB/s
156 MB/s
CPU scores: sysbench events/sec (higher=better). Disk: dd. Network: Cloudflare speed test.
Key Comparisons (16 vCPU)
Metric
Blacksmith
BuildJet
WarpBuild
Winner
CPU Single
6,565
5,243
4,172
Blacksmith (+25%)
CPU Multi
96,169
77,382
66,776
Blacksmith (+24%)
Disk Write
1.4 GB/s
962 MB/s
565 MB/s
Blacksmith (+45%)
Disk Read
5.9 GB/s
3.5 GB/s
213 MB/s
Blacksmith (+69%)
Network
175 MB/s
76 MB/s
87 MB/s
Blacksmith (+130%)
Stage-by-Stage Timing (Medians)
Runner
Samples
Docker Pull
Compose
Lint
Pyright
Tests
Total
Blacksmith-16-Docker 27
16s
6s
8s
21s
4m 30s
6m 28s
Blacksmith-16-NoCache
29
18s
7s
8s
21s
4m 48s
6m 28s
Blacksmith-8
28
18s
6s
8s
21s
6m 1s
7m 36s
BuildJet-16
25
35s
9s
10s
25s
5m 35s
7m 45s
WarpBuild-16
27
21s
11s
10s
28s
11m 15s
13m 19s
Runner
Samples
Install
Typecheck
Lint
Test
Build
Total
Blacksmith-8-Sticky 30
1s 17s
15s
17s
1m 47s
2m 45s
WarpBuild-8-Cache
30
0s 16s
17s
20s
2m 4s
3m 12s
Blacksmith-8-Cache
24
30s
13s
14s
15s
1m 38s
3m 1s
BuildJet-8-Cache
25
11s
14s
16s
21s
2m 5s
3m 41s
Django:
Docker pull: Blacksmith 16s vs BuildJet 35s (registry mirrors)
Tests dominate runtime (70%+ of total)
Blacksmith's faster disk I/O speeds up test execution
Next.js:
Install: 0-1s with stickydisk vs 30-40s without
Build step: ~1m 45s across all runners
Stickydisk is the key optimization
Provider
Median
Average
Max
Blacksmith
11s
22s
217s
WarpBuild
21-42s
24-42s
84s
BuildJet 75s-21m 19m-28m 90 min
BuildJet queue times are completely unpredictable under load.
Outliers (Preserved Data) Queue Time Disasters (>1 hour)
Job
Queue Time
Run ID
Job ID
Next - BuildJet-8-Cache
90m 46s
21372265822
61519629435
Django - BuildJet-16
90m 4s
21372265822
61519629543
Next - BuildJet-8-NoCache
86m 59s
21372265822
61519629395
Django - BuildJet-16
79m 31s
21372265822
61519629392
Next - BuildJet-8-Cache
79m 27s
21372265822
61519629553
Execution Time Disasters (>20 min with low queue)
Job
Execution
Queue
Run ID
Job ID
Django - BuildJet-16
26m 13s
15s
21370119015
61512197422
Django - BuildJet-16
25m 44s
15s
21370119015
61512197436
These show BuildJet has execution contention , not just queue issues - same job can take 6min or 26min depending on shared infrastructure load.
Summary Statistics by Runner Django Execution Times (seconds)
Runner
Min
Median
Max
Samples
Blacksmith-16-Docker
319
388
451
27
Blacksmith-16-NoCache
319
388
451
29
Blacksmith-8
366
456
595
28
BuildJet-16
362
465
1573
25
WarpBuild-16
645
799
1079
27
Next.js Execution Times (seconds)
Runner
Min
Median
Max
Samples
Blacksmith-8-Sticky
155
165
299
30
Blacksmith-8-Cache
163
181
217
24
BuildJet-8-Cache
188
221
344
25
WarpBuild-8-Cache
167
192
253
30
Blacksmith wins across all hardware metrics - fastest CPU, disk, and networkBuildJet has reliability problems - both queue contention (90+ min waits) AND execution contention (same job varies 3x)Stickydisk is critical for Next.js - reduces install from 30-40s to 0-1sDocker layer caching helps Django - reduces pull time from 35s to 16sWarpBuild has slow disk I/O - only 213 MB/s read vs Blacksmith's 5.9 GB/s
Data collected January 2026. Run IDs preserved for reference.
