# From measured model results to workflow scenarios

**What would this result mean for someone's work?** These calculators combine recorded public-data experiments with editable assumptions about workload and effort. They make the commercial question inspectable. They do not measure a client deployment, establish savings or recover confidential delivery records.

## What is measured

The inputs below come from the pinned artifacts linked from each study, with source revisions, checksums, saved predictions and reproducible scripts.

| Study | Recorded experiment | Relevant limitation |
| --- | --- | --- |
| [Support Intent](support-intent-lab.md) | BANKING77: 2,711 correct out of 3,080. The fixed review policy suggests 2,468 routes, including 111 wrong suggestions; 612 messages need manual fallback. | These are English banking categories. Scores are uncalibrated; another team's categories and messages may behave differently. |
| [Evidence Search](evidence-search-lab.md) | SciFact: BM25 misses every judged relevant document in 59/300 top-ten lists; hybrid misses in 51/300. | Hybrid was added after seeing the initial test results. Relevance judgments are incomplete and retrieval does not establish answer truth. |
| Hydraulic ML | UCI hydraulic recordings: 231 detected leakage cycles, 306 correctly unflagged cycles, zero false flags and 15 missed leakage cycles in 552 held-out cycles. | Random split from one test rig. Features use each complete 60-second cycle; this is not demonstrated early warning. |

Sources: [PolyAI BANKING77](https://github.com/PolyAI-LDN/task-specific-datasets), [BEIR](https://github.com/beir-cellar/beir), [UCI hydraulic systems](https://archive.ics.uci.edu/dataset/447/condition+monitoring+of+hydraulic+systems). These publishers are data sources, not clients or endorsements.

## What the visitor supplies

Work items per period, minutes per task, labour rate and operating overhead are **scenario assumptions**. They are not industry averages, price quotations or past client results. Changing an assumption changes the estimate; it does not change the underlying experiment. Multiplying benchmark proportions by a new workload is a conditional projection, not a prediction validated in that setting.

Any rupee figure represents the assumed value of staff time. Recovered capacity does not automatically reduce payroll, create revenue or establish willingness to pay. Setup, integration, maintenance, infrastructure and error consequences also require measurement. A negative result is a useful outcome and remains visible.

## Include the work AI leaves behind

**Support routing:** compare manually processing every message with reviewing every suggested route, manually processing every fallback, and doing additional correction work for wrong suggestions. Add operating overhead. No suggested category bypasses review. The scenario assumes reviewers identify and correct every wrong suggestion; actual reviewer errors need a separate pilot measurement. Do not describe 80.13% suggestion coverage as 80.13% time saved.

**Evidence retrieval:** compare BM25 with dense or hybrid search. Both include inspection for every query, full manual fallback for each method's all-judged-relevance misses, and any additional correction time. Include incremental operating cost over BM25. Dense misses 62/300 queries and may worsen the scenario. These rates do not establish real-world success, knowledge-base completeness or generated-answer truth. The hybrid result needs a fresh labeled query set.

**Hydraulic review:** show flagged, unflagged and missed completed cycles using the recorded confusion proportions. Review every alert and manually check every non-alert; add correction effort for model mistakes and overhead before comparing with a manual baseline. Catching missed leakage through review is hypothetical, not measured. Neither zero observed false flags nor 97.28% accuracy establishes safe automation. There is no demonstrated advance warning, avoided breakdown, downtime reduction or repair saving.

## How to explain this honestly

> “I evaluated the model on attributed public data, then built an editable workflow calculator including review, fallback and correction work. Its outputs are scenarios. I would validate workload, error rates and effort with the intended team before claiming a business result.”

Confidentiality can explain why a client's name is withheld. It does not corroborate a deployment or permit substituting invented results for missing records. Keep independently verified client contributions separate from these new studies.

A useful pilot needs permissioned representative inputs, a frozen evaluation plan, a comparable manual baseline, logged reviewer corrections, measured task times, all operating costs and a dated acceptance record. Only that evidence can support claims about actual workflow benefit.