Delivery step 04
Add AI only where it improves the flow
A task receives AI only when a rule cannot decide it and the output can be checked. Each task starts at the lowest model tier that can handle it.
What happens
- Test each task against rules first. If the rules decide it, no model runs.
- Where the rules cannot decide, try a classic or neural model, then a small language model, then a large language model.
- Give each task a measure. A task that does not beat its baseline is taken out of service.
How the platform carries it out
- The five model tiers and their confidence thresholds are described in The AI Technology Map.
- Guardrails apply to every task: purpose check, access check, citation check, schema check, and approval gate.
- Language models write summaries and drafts only. They do not write to the case record or move money.
AI role
This step is the AI routing itself. It chooses the lowest tier that can handle a task and records which tier made the decision.
Approval
The task owner approves each task before it goes live, and the approval is recorded.
Systems involved
- Azure AI Foundry
- Azure Document Intelligence
- Azure AI Search
- Rules service
Reference components
- claims document intake
- insurance policy Q&A with grounded citations
- claims payment guardrails
Pre-sizing
Business case, effort, time, and cost for this step.
These are planning estimates for scoping. They are based on the work listed here and must be checked with the client before a commitment is made.
Business justification
AI is only worth its cost where a rule cannot decide the work. Routing each task to the lowest capable tier keeps cost per task low, and the measured baselines show whether each task pays back.
Business value
- Work that rules can decide does not reach a model, which keeps cost down.
- Each AI task has a measure against its baseline, so keeping it is an evidence decision.
- Reviewers see cited output, which makes checking faster than typing from scratch.
Documentation to produce
- Task catalog with tier decision per task
- Evaluation set per task
- Guardrail policy
- Cost and latency measures per task
Work breakdown and effort
| Work item | Person-days |
|---|---|
| Build the rules catalog and the decision tests for each task | 4 |
| Build a classic model baseline for the scoring tasks | 6 |
| Integrate document extraction with field-level confidence | 6 |
| Fine-tune a small language model on reviewed examples | 8 |
| Build cited summaries with the citation check and guardrails | 8 |
| Build the evaluation harness and the measures | 5 |
| Total effort | 37 |
Delivery time
6 weeks for the full step.
Resources required
- AI engineer: 2 FTE
- Data engineer: 1 FTE
- Solution architect: 0.5 FTE
- Domain reviewer: 0.25 FTE
POC: 3 weeks
Scope
- One summary task with citations, measured against a reviewed set of cases
Exit criteria
- The citation check passes on the reviewed set
- The summary is rated at least as useful as the current manual draft by the reviewers
MVP: 4 weeks
Scope
- Extraction and cited summary for the first two case stages
- Rules-first routing and the guardrail set
Exit criteria
- Each task has a measure and a baseline
- Rejection rate and citation failures are within the agreed limits
Assumptions
- A reviewed set of past cases is available for evaluation
- Model hosting is available on Azure AI Foundry
Risks
- Model cost per task is higher than estimated
- Reviewer agreement on the reviewed set is low
Cost calculation
Enter your own day rate and weekly platform cost. Until they are entered, the calculator shows effort and delivery time only.
- Effort
- 37 person-days
- Delivery time
- 6 weeks
- Labor cost
- 0
- Platform cost
- 0
- Total estimate
- 0