Before any live pilot, Eli replays a school's historical cases with no live effect, compares its disposition to what the office did, and has to pass every product-quality gate. Those gates are evaluated separately from procurement, security, contract, identity, and data-access approval. This illustration runs over 182 synthetic cases in scope (the modeled pilot volume is 400).
Bring one workflow and one de-identified case. No data leaves your office for this meeting.
We walk the workflow the way your office actually runs it, seven questions in 45 minutes. The meeting has one goal: prove Eli understands the work, reproduces your known result, and produces the defense file. That proof is what earns data access for a pilot.
What exact event, file, or report tells you there is work? Who notices first? What can be missed?
How do you find every affected record? Which date starts which deadline? What report proves completeness?
The real SIS screens, portals, spreadsheets, and federal sites. Which one is authoritative at each stage?
One real de-identified case, first event to final close: every click, handoff, wait, calculation, and message.
The three cases that consume disproportionate time. What missing or conflicting fact stops the work? Who decides?
What is posted or transmitted? How do you know it was accepted? Which financial and system records must tie?
A different staff member reconstructs the case. What evidence is hard to find? What relies on memory?
Modeled on the cohort's per-case economics, no live effect
Security review: the model card, data-handling controls, RLS, retention schedule, and the mode-boundary memo. The artifacts a procurement and IT review need.
A de-identified historical cohort (300-500 cases) loads into a scoped tenant. Direct identifiers are masked; the distribution that makes the queue realistic is preserved.
Eli replays the cohort with no live effect. Its disposition is compared to what the office actually did, and the gates below decide whether to advance.
Only if the gates pass: Eli prepares real cases, your staff approve every call. Straight-through clearing stays off until the office is ready.
Optional: Eli's team works the queue on the same software while your office keeps the final sign-off, so the queue scales without new hires.
Seven dimensions, agreed with your office before any data is exchanged and measured at exit
| Dimension | Agreed up front | Evidence at exit |
|---|---|---|
| Population | Inclusion and exclusion rules, with control totals | Your report counts and dollars match Eli's import |
| Correctness | Golden cases and the acceptable tolerance | Every intermediate value and final result agree |
| Exceptions | Named unsupported and ambiguous scenarios | Each routes to the right owner, with no false confidence |
| Workflow | Required human gates and downstream states | Approval, exported action, response, and reconciliation on record |
| Efficiency | Baseline touches and cycle time, with a target | Measured before and after on the same cohort |
| Defense | Required audit-packet contents | A blind reviewer reproduces the decision |
| Data boundary | De-identification, retention, access, and deletion | A signed handling protocol and a purge report |
Eli's disposition vs the office's historical decision (182 cases replayed)
| Case | Office did | Eli would | Agreement | Audit packet |
|---|---|---|---|---|
| Bryce Whitaker | Route to human | Request docs | documented diff | complete |
| Fatou Park | Route to human | Route to human | ✓ match | complete |
| Liang Kowalski | Route to human | Route to human | ✓ match | complete |
| Mohammed Hernández | Route to human | Route to human | ✓ match | complete |
| Mateo Rossi | Request docs | Request docs | ✓ match | complete |
| Olivia Lim | Correct | Request docs | documented diff | complete |
| Sofía Romano | Request docs | Request docs | ✓ match | complete |
| Rohan Romano | Route to human | Route to human | ✓ match | complete |
| Dmitri Hernández | Request docs | Request docs | ✓ match | complete |
| Hana Ali | Request docs | Request docs | ✓ match | complete |
| Liang Romano | Request docs | Request docs | ✓ match | complete |
| Rowan Tran | Clear | Clear | ✓ match | complete |