ReqLLM v2 analysis

reqllm_v2_analysis.md

ReqLLM v2 Analysis (Design Alternatives)

This gist summarizes the comparison between ReqLlmNext (metadata‑driven spike) and the current ReqLLM v1, plus a recommended v2 simplification path.

---

1) What ReqLlmNext Simplifies

• Central pipeline clarity: Resolver → Validation → Constraints → Adapter → Wire → Provider.
• Metadata-driven model expansion: adding models via LLMDB updates is genuinely faster.
• Provider boilerplate shrinks: base_url + auth only.
• Scenario + fixture testing matches the metadata-first world.

---

2) Where Complexity Reappears

• Conditional quirks aren’t static: Anthropic thinking rules, OpenAI reasoning defaults, etc. already require adapters.
• Transport complexity is real: OAuth, SigV4, Azure URL pathing, Bedrock, etc. cannot be metadata-only.
• Wire protocols are code, not config: new API families will still require new wire modules.
• Response assembly is thinner than v1: no equivalent to ResponseBuilder parity and provider meta richness.
• Metadata as runtime truth is brittle: incorrect capabilities or limits become correctness bugs.

---

3) Conclusion (Net Assessment)

• It can simplify if metadata stays declarative and quirks are handled in code.
• It becomes brittle if you push conditional behaviors into config DSLs.
• The practical outcome is a hybrid: metadata for discovery/limits, code for behavior.

---

Recommended ReqLLM v2 Design

Core Idea

Split the system into Protocol, Transport, Policy, and Metadata.

• Protocol: encode/decode wire formats (OpenAI Chat, Responses, Anthropic Messages, Gemini, etc.)
• Transport: auth + base URL + headers + signing (Bearer, x‑api‑key, OAuth, SigV4, Azure)
• Policy: small conditional transforms (reasoning defaults, max_tokens rules, temperature removal)
• Metadata: discovery, capabilities, limits, pricing only

This keeps metadata declarative and moves conditional behavior into explicit, testable code.

---

Proposed v2 Architecture

1. Model Registry (LLMDB + overrides)
2. Protocol Layer (wire codecs)
3. Transport Layer (auth + URL + HTTP)
4. Policy Layer (quirk transforms)
5. ResponseBuilder (shared for streaming + non‑streaming)

---

API Changes (Minimal but Future‑Proof)

Keep convenience functions, but add a single explicit request API:

• ReqLLM.request/2 or /3 → returns Response (streaming or buffered)
• ReqLLM.stream/2 → returns StreamResponse
• Keep:
  • generate_text/3
  • generate_object/4
  • embed/3
  • generate_image/3

Normalize options around:

• operation: :chat | :embed | :image | :moderate
• output: :text | :json | :tool
• tools, tool_choice
• stream: true | false
• protocol_options and provider_options

---

Why This is Simpler Than v1 (and Safer Than Config‑Only)

• Provider modules become thin transports.
• Protocol logic is centralized and reusable.
• Conditional quirks stay explicit and testable.
• Streaming/non‑streaming share a single builder.
• Metadata stays static (no “config DSL”).

---

Bottom Line

Use metadata for discovery/limits, protocols for encoding, transports for auth, and policies for quirks. This keeps ReqLLM v2 simpler, predictable, and far less brittle than a config‑heavy design.