Pattern Decision Tree

## Full reference: a constraint-first decision tree

### 1. Frame the problem in one sentence

Write this down before picking anything:

> *"Given <inputs>, produce <required outputs> within <budget> at <accuracy bar>."*

If you can't fill all four blanks, stop and go define them. Pattern choice without a constraint is architecture-astronauting.

### 2. The constraint axes

Every agent design lives in a four-dimensional constraint space. Rank them in order; only the top-1 or top-2 matter.

| Axis | Typical symptom | Dominant pattern |
|---|---|---|
| **Latency** | User-facing UI, streaming chat, p95 budget <2s | Routing, single-shot, caching |
| **Cost** | High volume (>10k req/day), margin-sensitive | Routing (cheap-first), model stepping |
| **Accuracy** | Must not hallucinate; ≥95% bar | Evaluator-optimizer, rerank, human-in-loop |
| **Complexity** | Task spans tools, pages, time | Orchestrator-workers, prompt-chaining |

### 3. The tree

```
START
  │
  ├─ Is the task a single well-defined call?
  │    └─ YES → Do NOT build an agent. Call the tool directly. Stop.
  │
  ├─ Is top constraint LATENCY (<2s p95)?
  │    ├─ Multiple task types, some cheap? → ROUTING  (cheap fast, expensive slow)
  │    └─ One task, just needs to be fast? → SINGLE-SHOT with strong prompt + cache
  │
  ├─ Is top constraint COST (per-request <$0.01)?
  │    ├─ Mixed difficulty?             → ROUTING  (Haiku first, escalate)
  │    └─ Multi-step but each is small? → PROMPT-CHAINING with Haiku legs
  │
  ├─ Is top constraint ACCURACY (>95% required)?
  │    ├─ Have a golden set & rubric?    → EVALUATOR-OPTIMIZER  (candidate + judge loop)
  │    └─ No golden set yet              → STOP. Build one first. Then re-enter.
  │
  ├─ Is top constraint COMPLEXITY?
  │    ├─ Fixed N steps, known in advance?   → PROMPT-CHAINING
  │    ├─ N independent sub-queries?         → PARALLELIZATION (map-reduce)
  │    └─ Dynamic subtasks, depends on data? → ORCHESTRATOR-WORKERS
  │
  └─ Multiple constraints tie?
       └─ Pick the one the user will notice first. Usually:
            user-facing  → latency
            analytics    → cost
            agentic      → accuracy
```

### 4. Canonical patterns — 1-line definitions

Sourced from Anthropic's *Building effective agents* (Dec 2024) and aligned across the Tongyi NLA and Stanford meta-harness work.

- **Prompt-chaining** — fixed sequence of LLM calls, each step's output feeds the next. Use when you know the sub-steps.
- **Routing** — classifier picks which specialised prompt/model to hand off to. Use to save cost/latency on mixed task types.
- **Parallelization** — run N sub-tasks concurrently, then combine. Includes map-reduce and ensemble voting.
- **Orchestrator-workers** — a planner LLM spawns subtask LLMs dynamically. Use when the decomposition depends on the input.
- **Evaluator-optimizer** — candidate produces an output; evaluator judges; retry until it passes. Use when you can cheaply score outputs and can't tolerate misses.
- **Hybrid** — any composition of the above (e.g. router → orchestrator-workers → evaluator-optimizer).

### 5. Cost & latency envelopes (rough, order-of-magnitude)

| Pattern | Tokens per request | Latency p95 | Best when |
|---|---|---|---|
| Single-shot | 1× | ~1s | Simple, uniform tasks |
| Prompt-chaining (3 steps) | 3× | ~3s | Fixed pipeline |
| Routing | 1.1× (classifier + one leg) | 0.5–2s | Mixed difficulty |
| Parallelization (N=5) | 5× | ~1.5s (max leg) | Independent sub-tasks |
| Orchestrator-workers | 5–20× | 5–30s | Dynamic plans |
| Evaluator-optimizer (avg 1.5 retries) | 3–5× | 2–8s | Accuracy-critical |

### 6. Anti-patterns

- **Orchestrator-workers on fixed 3-step pipelines** → over-engineered; use prompt-chaining.
- **Evaluator-optimizer without a golden set** → the judge is unvalidated; you're measuring noise.
- **Routing with a too-expensive classifier** → classifier cost exceeds the savings. Classifier must be ≤1/4 of the cheap leg's cost.
- **Parallelization where sub-tasks share state** → you need orchestrator-workers instead; parallel assumes independence.
- **Single-shot when constraint is accuracy** → you can't hit >95% without a loop or a rerank.

### 7. Working example: "summarise this PDF with citations"

- Constraint rank: accuracy > cost > latency.
- Task decomposition: (1) chunk PDF, (2) retrieve relevant chunks, (3) generate summary with inline citations.
- Sub-task (2) is independent per chunk → **parallelization**.
- Sub-task (3) needs citation correctness → **evaluator-optimizer** wrapping the generator.
- Overall: **hybrid** — parallel retrieve → evaluator-optimizer generate. Pack recommendation: `rag-hybrid-bm25-vector` for retrieval + `evaluator-optimizer-gan` for the generator loop + `golden-eval-harness` for the CI gate.

### 8. When the tree says "don't build it"

If you land at "call the tool directly; stop" — that's a correct answer. Most "agent" features are better served by a direct API call and a good error message. Build an agent when the task is genuinely open-ended or branching; don't dress up procedural code as an agent to seem modern.

### 9. How to cite your choice in a PR

Include a block like:

> **Pattern:** routing → evaluator-optimizer.
> **Top constraint:** accuracy (>97% required).
> **Rejected:** single-shot (can't hit bar), orchestrator-workers (overkill; task is fixed-shape).
> **Budget:** 6k tokens, 3s p95, $0.012/request.
> **Pack:** `evaluator-optimizer-gan` for the loop; `routing-by-intent` for the upstream classifier.

This one paragraph has saved more re-architecture than any diagram.