claude-opus-5. One evidence package per trace, and one digest that says which models produced it.What a trace actually costs to attest.
The usual argument for splitting models across stages is unit economics, so it is worth putting the real number down before making it. A representative Warrant trace ingests ~12K input tokens and emits ~3K output tokens across the four stages. claude-opus-5 lists at $5 per million input tokens and $25 per million output tokens on the Anthropic price sheet as at 2026-06-24. Run the arithmetic:
# input cost per trace 12,000 input tokens * $5 / 1,000,000 = $0.060 # output cost per trace 3,000 output tokens * $25 / 1,000,000 = $0.075 # total per trace, all four stages on claude-opus-5 $0.060 + $0.075 = $0.135 ≈ $0.14 per trace at the per-stage budget shown in section 05
That is what runs today, at the per-stage token budget the table in section 05 publishes. Routing the two structure stages to claude-sonnet-5, which lists at $3 / $15 per million, would land the same budget at ~$0.10 — a 1.3× reduction, not the order of magnitude the framing usually implies. Both figures reconcile to the same input and output token totals; the model choice changes the per-token rate, not the budget.
At 1,000 traces a day that is ~$135 a day, ~$4,050 a month, on a single workload. The cheaper routing would take it to ~$100 a day, ~$3,000 a month. Those are the real stakes of the decision, and they are smaller than the argument they are usually used to win: a five-figure annual line item, against the cost of a wrong citation in a signed evidence package. That asymmetry, not the spreadsheet, is why the cheaper arm has not shipped.
Judgement vs structure.
Stages split cleanly into two camps once you stop treating them as one workload. Judgement versus structure.
Classify and Assess.
"Is this an advisory action or an informational one." "Does this action require a regulator-recognised authorisation that is not present in the trace." Calls that hinge on world knowledge of regulator language, on understanding what counts as a recommendation under the relevant regime.
Extract and Map.
"Pull the structured action object out of this trace." "For each action, list the obligation IDs it triggers from the menu." Calls that hinge on producing a deterministic JSON object that fits a schema, not on judgement.
That split is the reason the cheaper-model question is worth asking at all: if the two structure stages consume schema discipline rather than world knowledge of regulator language, they are the stages where a cheaper model would cost the least. Sonnet 5 lists at 60% of Opus 5's rate on both input and output — 1.67× cheaper, which is the actual size of the prize. What the split does not tell you is whether the cheaper model holds on those stages in practice. That is measurement, and section 06 is honest about not having it.
What the judgement stages are actually deciding.
The "advisory in disguise" case is the cleanest example of what these two stages carry. A retail brokerage trace contains a passage where the agent says "based on your recent portfolio mix, you might consider rebalancing toward duration-shorter exposure." Read one way that is informational, on the basis that the verb "consider" softens it. Read the other way it is advisory, because of the implicit personal recommendation embedded in "based on your portfolio," and it has to be flagged for the Assess stage.
Across the retail-advisory regimes Warrant maps, that reading is the line between a triggered obligation and a clean trace. A pipeline that misses the recommendation produces a clean evidence package an examiner will reject the moment the regulator reads the action as advisory. Assess has the same shape: the distinction between a platform consent click and a regulator-recognised authorisation under the relevant regime is world knowledge of regulator language, not schema discipline.
The right metric here is a model-versus-human agreement score on a labelled set, and Warrant does not have one. An earlier version of this section published such scores; they were never measured and they are withdrawn. What is on record instead is a directional observation on sample traces — cheaper models mis-read hedged recommendation language and accept platform terms as authorisation more often — with no counted rate attached. That is a weaker basis than a number and it is described here as one.
A measured score, with the failure classes named, is the work in progress. Until it exists, the conservative choice is the one that ships: the stronger model on every stage.
Why the structure stages are the open question.
Extract and Map do not consume world knowledge of regulator language. They consume schema discipline. Neither stage uses tool-use. Both are asked for a single strict JSON object in the system prompt and both returns are parsed as text — markdown fences stripped first, because the model sometimes fences them.
That makes them the two candidates for a cheaper model, at 60% of the input cost and 60% of the output cost. Two things stop it being a free win. First, we have not measured the schema-conformance rate for any model on these stages and do not publish one. The two stages are also not equally defended: Map's return passes through a typed boundary that rejects an unknown field or an obligation id absent from the corpus, while Extract has no such boundary — a reply that is not JSON raises, but a well-formed and wrong action list does not. Second, changing family and generation in one step makes a re-baseline impossible to attribute, so if it moves it moves one variable at a time.
The two shapes, as they are in the repo. Extract returns an envelope the pipeline reads by key; Map returns an envelope validated field by field before any downstream consumer sees it:
# stage 2 returns one JSON object as text. no tool call, no tool_choice. { "actions": [ {"action_id": "act_001", "actor": "...", "action": "issue_loan_decision", "subject": "...", "timestamp": "...", "inputs": {}, "outputs": {}, "step_number": 1} ], "extraction_confidence": 0.0, "truncation_warning": false, "refusal_reason": null } # the pipeline strips fences, parses, and reads the envelope by key: parsed = parse_json_response(strip_json_fences(response_text(response)), stage="stage2_extract_actions")
class ObligationRowModel(BaseModel): model_config = ConfigDict(extra="forbid", frozen=True) id: SubClauseId # an enum built at import time from the corpus compliance: ComplianceStatus confidence: float = Field(ge=0.0, le=1.0) evidence: str = Field(min_length=1, max_length=2000) unvalidated: bool = False class Stage4Output(BaseModel): model_config = ConfigDict(extra="forbid", frozen=True) obligations_by_action: dict[str, list[ObligationRowModel]] coverage_by_regime: dict[str, CoverageStatusLiteral] refusal_reason: str | None = None
The ids are sub-clause ids, not regime ids, and every one must appear verbatim in the filtered corpus block the prompt was handed — eu_ai_act_art_12.1, sebi_algo.2025_feb_circular.kill_switch. SubClauseId is built from the corpus at import time, so an id the corpus does not contain is rejected before it can be rendered. That check is at Map only. Extract has no equivalent.
The cost table at production prompt sizes.
Per-stage prompt budgets, the model each stage runs, and the resulting cost per trace. claude-opus-5 at $5 / $25 per million tokens; claude-sonnet-5, the candidate for stages 2 and 4, at $3 / $15 (introductory $2 / $10 through 2026-08-31). Every Sonnet figure in this post is computed at the $3 / $15 list rate, not the introductory one, so the saving available until 2026-08-31 is larger than the ratios given here. The arithmetic is the table:
| Stage | Input tokens | Output tokens | Model | Cost | Rationale |
|---|---|---|---|---|---|
| Classify | 1.2K | 280 | claude-opus-5 | $0.013 | judgement |
| Extract | 4.8K | 1.4K | claude-opus-5 | $0.059 | structure |
| Assess | 3.6K | 720 | claude-opus-5 | $0.036 | judgement |
| Map | 2.4K | 600 | claude-opus-5 | $0.027 | structure |
| total | $0.135 | ~$0.13–0.14 | |||
Traces drift around that figure depending on input size and output verbosity. Routing Extract and Map to claude-sonnet-5 at the same per-stage budget would land at ~$0.10, a ratio of ~1.3×. The quality side of that trade is not measured, and no delta is published here.
There is no router file, no orchestration graph and no abstraction layer. There is one mapping, in api/spec/pipeline_config.py, and the stages import from it rather than holding a literal:
from typing import Literal Stage = Literal[ "stage1_classify", "stage2_extract_actions", "stage3_assess_authorization", "stage4_map_obligations", ] # ONE definition. All four run Opus 5, ruled 2026-07-31. These strings are # hashed into pipeline_config_sha256, and that digest sits inside the signed # bytes: the model IS part of the attestation, not an implementation detail. STAGE_MODELS: dict[Stage, str] = { "stage1_classify": "claude-opus-5", "stage2_extract_actions": "claude-opus-5", "stage3_assess_authorization": "claude-opus-5", "stage4_map_obligations": "claude-opus-5", } def model_for(stage: Stage) -> str: try: return STAGE_MODELS[stage] except KeyError: raise ValueError(f"No model configured for stage {stage!r}.") from None
The four call sites read model_for(stage) and pass the string to the Anthropic SDK. What makes this more than tidiness is the second half: the same mapping is hashed, alongside the exact bytes of each stage prompt, the corpus digest and the confidence floors, into pipeline_config_sha256. That digest is inside the signed bytes, and the mapping is also copied onto the receipt in the clear as pipeline_models, keyed by stage id.
The consequence is the point of this post. Changing a stage model changes the digest on every package issued afterwards. Two packages over the same trace that disagree now have a visible, dated reason to disagree, and "same pipeline_config_sha256 plus same trace should produce the same verdict" becomes a claim a third party can test rather than a promise. Before this module existed, all four stages held their own hardcoded literal, three sources disagreed about what was running, and the receipt recorded none of them.
The eval that would justify this.
The model choice is a reasoned decision, not a measured one, and the difference matters enough to state plainly. What would settle it is a labelled set scored across configurations: production-shaped traces over the lending, advisory and KYC workloads, golden outputs at every stage labelled by a regulated-services compliance reviewer, and each candidate configuration scored against it. No compliance-reviewer-labelled set exists, and no agreement score between the pipeline and a human reviewer is published.
What does exist is a pinned eval harness at tests/fast-flow/eval-harness/ — 20 stage-3 cases and 20 stage-4 cases with expected outputs, plus a separate adversarial set — whose pin file records all four stages on claude-opus-5. That pin matters for a reason worth naming: before 2026-07-31 the harness pinned a different model for stage 4 than the pipeline actually ran, so it was scoring a configuration production did not use. Fixing the pin invalidated the pass rates measured under the old one, and that baseline is marked stale in the pin file itself. That harness baseline has not been re-run, so no pass rate from it is quoted here.
What the decision rests on, then, is cost arithmetic — checkable against a published price list — and the failure shapes in the next section, which are observations from working with sample traces rather than counted rates. Below is the shape of the comparison, cost column only.
| Configuration | Cost / trace | Basis for the cost figure | Quality delta vs production |
|---|---|---|---|
| All Opus 5 (production) | ~$0.135 | List price sheet as at 2026-06-24 × the per-stage token budget | Baseline |
| Sonnet 5 on stages 2 and 4 | ~$0.10 | List price sheet as at 2026-06-24 × the per-stage token budget | Not measured |
| All Sonnet 5 | ~$0.081 | List price sheet as at 2026-06-24 × the per-stage token budget | Not measured |
Only the cost column is populated, and only because it is arithmetic on a published price list. The quality column is the whole point of the eval and it is empty. The claim being made here is narrow: at constant prompt size, moving the two structure stages to Sonnet 5 would save about 1.3×, and the qualitative failure shapes below are why that has not been done. That is a hypothesis about quality, not a measurement of it.
The failure modes behind the decision.
Three failure modes surfaced while working the sample traces. None is theoretical — each came off a real trace — but none is a counted rate either, so read them as cases rather than as statistics. Between them they are why the cheaper arm is a question and not a shipped change.
Cheaper models under-classify personal recommendations as informational
The "advisory in disguise" case. Hedged recommendation language reads as informational, and the personal recommendation that triggers the obligation is missed. It showed up repeatedly on the advisory sample traces at the cheaper tier. How much more often is not measured. This is the observation behind keeping Classify on the stronger model.
Cheaper models invent sub-clause suffixes on long obligation menus
When the Map stage receives a regulator menu of more than 30 obligation ids, a cheaper model occasionally returns an id with a sub-clause suffix that does not exist in the corpus. This is the one failure the schema cannot catch — the return is well-formed and the citation is wrong — and it lands on the stage that produces the pinpoint an auditor reads. Observed on long synthetic menus; no rate is published. This is the strongest single reason Map has not moved.
The stronger model over-fills optional fields on Extract
On Extract the stronger model produces verbose JSON, filling in optional fields even where the schema marks them optional and the trace does not warrant them. The output is correct but bloated, and Extract is already the largest single line in the cost table at $0.059. This is the one failure mode that argues for the cheaper model rather than against it, and it is why stages 2 and 4 remain the live question rather than a closed one.
Two of the three argue for the stronger model and one against it. No eval has confirmed any of them, which is the honest limit of the argument — and the reason the shipped configuration is the conservative one rather than the cheap one.
What happens when the next model ships.
The point of a single mapping is the cost of the next swap. When the next model ships, the work is the eval, not a migration — run the set, compare, change one dict. What stops that being casual is the digest: the new string produces a new pipeline_config_sha256, so packages issued before and after the swap are visibly different artefacts, and the eval pin has to be rotated deliberately in the same change. That presumes a labelled set worth running, which is exactly the piece not built yet. Until it is, a model swap here is reviewed by reading output diffs, which is a weaker control and is named as one.
There is no graph to refactor, no chain template to rewrite, no abstraction to migrate. The pipeline is four explicit Anthropic API calls, each one parameterised by a string that comes from one place.
The same logic runs in the other direction. If an eval shows a cheaper model holding on the structure stages, one dict entry moves, the digest changes, the pin rotates, and cost per trace drops — with the change recorded on every package issued afterwards rather than absorbed silently. What the auditor sees either way is one evidence record per trace mapped to specific obligations, not a diagram of which model did what.
That record is the property the model choice has to preserve. On the regulatory side, EU AI Act Article 12(1) provides that high-risk AI systems "shall technically allow for the automatic recording of events ('logs') over the lifetime of the system" — a capability duty, not a duty to record everything. As of Regulation (EU) 2026/1744, in force 27 July 2026, which replaced Article 113(3)(c) of Regulation (EU) 2024/1689 and left Article 12 itself unamended, Article 12 applies from 2 December 2027 for Annex III high-risk systems and 2 August 2028 (subject to Article 2(13)) for the Annex I limb. Quoted from the text carried in Warrant's own corpus at digest 6871ee8b…; the enacted instrument is Regulation (EU) 2024/1689.
Questions an architect asks first.
What a Classify prompt actually looks like.
The Classify call is two blocks and nothing else: the stage prompt as a cached system block, and a user block carrying the trace inside <untrusted_trace> tags so the prompt's untrusted-input clause has a stable boundary. There are no few-shot exemplars and no tool definition. The output contract is stated in the prompt and the reply is parsed as text:
response = await create_message( client, stage="stage1_classify", stream=False, model=model_for("stage1_classify"), max_tokens=STAGE1_MAX_TOKENS, system=_SYSTEM_BLOCKS, # prompt text + ephemeral cache_control messages=[{"role": "user", "content": f"<untrusted_trace>\n{safe_trace}\n</untrusted_trace>"}], ) # the shape the prompt requires back — strict JSON, no prose, no markdown { "domain": "lending", # closed vocabulary, 10 values "jurisdictions": ["EU", "IN"], # ISO 3166-1 alpha-2, plus "EU" "applicable_regimes": ["EU AI Act Article 12", "SEBI Retail Algo Framework"], "risk_tier": "high-risk", # prohibited | high-risk | limited-risk | minimal "risk_tier_justification": "...", "confidence": 0.0, "risk_tier_confidence": 0.0, "refusal_reason": null }
Two details in that shape carry weight. applicable_regimes is prose, not corpus ids, and it does not decide which regimes the run is measured against — that filter is derived from the corpus's own applicability declarations, so the model cannot narrow the set it will be graded on. And the returned jurisdiction list is unioned with the ones the trace evidences deterministically, so the answer does not depend on the asking. There is no temperature parameter, and not by preference: on claude-opus-5 the sampling parameters are removed, and sending one returns HTTP 400. Determinism is not available as a knob, which is why the digest matters — you cannot promise identical outputs, so you record the conditions instead. pipeline_config_sha256 is taken over the bytes of each stage's prompt file alongside the model names, so editing a prompt in the repo is itself a provenance event.