Goal: Cache widget API responses (reviews, badges) with Varnish on NVMe SSD to reduce latency and backend load.
Architecture:
┌───────────────────────────────────────────────┐
│ widget-cache node (m6id.2xlarge) │
│ 8 vCPU / 32 GiB / 474 GiB NVMe │
│ │
/api/widget │ ┌──────────┐ ┌───────────┐ ┌────────┐ │
──────────────────▶│ │ varnish │MISS│ widget- │ │ app- │ │
nginx ingress │ │ :6081 ├───▶│ proxy ├───▶│backend │ │
(consistent hash │ │ │ │ :8700 │ │ -api │ │
on $request_uri) │ │ HIT ─▶ ⚡│◀───┤ (Node.js) │ │ (.NET) │ │
│ └────┬─────┘ └───────────┘ └────────┘ │
│ │ │
│ ┌────▼─────┐ │
│ │ NVMe │ 474 GiB local SSD │
│ │ 50G │ Local PV (provisioner) │
│ └──────────┘ │
└───────────────────────────────────────────────┘
How it works:
- Consistent hashing at ingress: same URL always hits same Varnish
- Cache HIT: response from NVMe in microseconds, no backend call
- Cache MISS: Varnish forwards to widget-proxy → app-backend-api
- Request coalescing: 100 concurrent requests for same URL = 1 backend hit
- TTL: 5 min default, 1 hour grace (serves stale while revalidating)
What was built:
- Terraform: widget-cache node group (m6id.2xlarge, NVMe auto-formatted)
- Infrastructure: local-nvme StorageClass + volume provisioner DaemonSet
- k8s: Varnish StatefulSet, VCL config, ingress with consistent hashing
- Routing: /api/widget paths moved from widget-proxy to varnish ingress
Current state:
- Dev: Varnish running, cache HIT/MISS verified ✓
- Prod: manifests ready, node running, deploy when ready
- Cost: ~$342/month per node (1 node, max 2)