Orchestration Middleware for Imagine Slow Ramp.md

categories	project	people	resume	date
[[Plans]]	[[Design Anything]]		claude --resume b9db3a88-ec96-4277-b3b1-f9e7339e4fdc	2026-03-04

Orchestration Middleware for Imagine Slow Ramp

Proposed: Per-Model AI SDK Middleware via Transform

Instead of orchestrating at the fallback loop level, wrap each model with an ImageModelV3Middleware that handles availability + reservation lifecycle. Use a transform to apply it from context — no factories or HOCs needed.

Key Insight: AI SDK Middleware Layering

The AI SDK interleaves transformParams/wrapGenerate per layer (not all-transforms-then-all-wraps):

outermost.doGenerate(params):
  outermost.transformParams(params)   ← availability check HERE (throws if unavailable)
  outermost.wrapGenerate(() =>        ← reserve/release HERE
    inner.transformParams(...)        ← file downloads here
    inner.wrapGenerate(() =>          ← ai.image.generate span here
      model.doGenerate(...)
    )
  )

An outermost orchestration middleware can throw in transformParams before file downloads happen, and reserve/release in wrapGenerate before the trace span opens.

New: createOrchestrationMiddleware

// ai-images/framework/middleware/orchestration.middleware.ts
const createOrchestrationMiddleware = (config: {
  service: OrchestrationService
}): ImageModelV3Middleware => ({
  specificationVersion: 'v3',

  // Cheap availability check — throws before file downloads/normalization
  async transformParams({ params, model }) {
    const available = await config.service.isAvailable(model)
    if (!available) throw new ModelUnavailableError(model.modelId)
    return params
  },

  // Reserve → generate → release
  async wrapGenerate({ doGenerate, model }) {
    const reservation = await config.service.reserve(model)
    try {
      const result = await doGenerate()
      config.service.reportCompletion(reservation, 'success').catch(() => {})
      return result
    } catch (err) {
      config.service.reportCompletion(reservation, 'error').catch(() => {})
      throw err
    }
  },
})

New: withOrchestration transform

// ai-images/fallback/filters.ts
export const withOrchestration = (): ModelTransform<
  unknown,
  { orchestrationService: OrchestrationService }
> => {
  return (models, _req, ctx) =>
    models.map((m) =>
      wrapGammaImageModel(m, createOrchestrationMiddleware({
        service: ctx.orchestrationService,
      }))
    )
}

Fallback usage

Fallback stays a static export. Just add the transform:

// design-generate.fallback.ts
export type DesignGenerateContext = {
  featureFlagService: FeatureFlagService
  user: User
  orchestrationService: OrchestrationService
}

export const designGenerateFallback = createImageModelFallback<
  DesignGenerateRequest,
  DesignGenerateContext
>({
  models: DESIGN_GENERATE_MODELS,
  transforms: [
    (models, req) => models.filter(...),
    featureFlagFilter(),
    preferModel(),
    withOrchestration(),
  ],
})

Service wiring — just pass it in context:

// design-image-generate.service.ts
const context: DesignGenerateContext = {
  featureFlagService: this.featureFlagService,
  user,
  orchestrationService: this.orchestrationService,
}
const { result, meta } = await designGenerateFallback({ request, params, context })

Adding orchestration to design-edit.fallback.ts later = add withOrchestration() to transforms + pass service in context.