AI Usage Disclosure & Governance Memo
Audience: financial-aid leadership, IT/security review, and institutional counsel evaluating Eli for a verification pilot. Last reviewed: 2026-07-13. Status: repository/design disclosure, not a deployment-specific model inventory, provider attestation, or counsel-reviewed contract commitment.
1. The one thing to take away
The AI never decides the outcome of a case. Every dollar figure, tolerance rule, and verification determination is produced by deterministic code that an institution can read, test, and audit. The AI reads documents, sequences the workflow, and explains its reasoning. The determination comes from the engine.
This is enforced in code, not just asserted in policy. When the agent finalizes a case, any determination the model proposes is ignored and the engine's result is used instead (lib/agent/finalize.ts, selectFinalDecision()):
"The only authority for the decision is the deterministic engine's most recent
reconcile_and_scoreoutput ... the model can sequence and narrate the workflow, but it can never change the dollar math or the determination."
The agent's tools each accept only a rationale string as input. The model physically cannot pass a dollar amount or a determination into the pipeline (lib/agent/orchestrator.ts).
2. Where AI is used, and what model
Eli uses Anthropic Claude via the Vercel AI SDK (lib/config.ts):
claude-sonnet-4-6— "deep" tier: extraction, the agent orchestrator, conflict detection, the staff copilot.claude-haiku-4-5— "fast" tier: document-type classification and the corrections self-check (small, well-bounded structured outputs).
Routed through the Vercel AI Gateway or a direct Anthropic API key. The model id is recorded on every AI step in the audit trail.
| AI function | What it does | What it does NOT do | File |
|---|---|---|---|
| Document extraction | Reads PDFs/images (W-2, IRS transcript, CSS Profile) and returns structured fields with a per-field confidence score | Choose the value of record (the "Per Computer" precedence rule is code) | lib/extraction/extract.ts |
| Document classification | Identifies an unlabeled document's type and reads the printed name + SSN-last-4 for routing | Make any eligibility decision | lib/extraction/classify.ts |
| Conflict detection | Flags material contradictions between a document and the application | Resolve the conflict (every conflict routes to a human) | lib/analysis/conflicts.ts |
| Agent orchestrator | Chooses the order of steps and narrates why | Decide the determination, do arithmetic | lib/agent/orchestrator.ts |
| Corrections self-check | Sanity-checks proposed corrections for sign flips / order-of-magnitude errors before any federal write | Clear a case; a concern only ever adds caution and routes to a human | lib/analysis/selfcheck.ts |
| Staff copilot | Answers questions, drafts outreach, surfaces stats; read-only and scoped to the user's institution | Mutate any record | app/api/copilot/route.ts |
3. The deterministic spine (the part that decides)
These are pure TypeScript, unit-tested, and contain no LLM call:
- Reconciliation engine (
lib/reconciliation/engine.ts) — compares each field against the application under the federal rules: the $25 difference correction trigger, the all-or-nothing rule (if any field triggers, all differing fields are submitted), and the IRS "Per Computer" precedence (the IRS-computed value is chosen in code, never by the model). - Decision logic (
lib/reconciliation/decision.ts) — computes the composite confidence (avg(extraction) × doc completeness × (1 − rule ambiguity)), applies the hard overrides, and returns one of four decisions:auto_clear,auto_correct,request_docs,route_human. The confidence threshold (default 0.85) is explicit and per-institution configurable. - Auto-approval qualifier (
lib/auto-approve.ts) — the conservative gate that lets an opted-in institution auto-resolve only its cleanest cases.
Because the decision is code, the same inputs always produce the same determination, and a Title IV program reviewer can re-run and inspect it. "The AI recommended it" is never the basis of a federal action; the engine's rule is.
4. Governance controls
- Human-in-the-loop, code-enforced. Identity (V4/V5), PJ, SAP, R2T4, non-filers, conflicts, household-size mismatches, and any low-confidence case always route to a person. See
decisions.md§3 andhuman-signoff-matrix.md. - Self-check before any automated federal write. A separate fast-model pass looks for implausible corrections; it can only route to a human, never clear.
- Confidence is a visible surface, not a vibe. The composite score and its breakdown are shown to reviewers and logged.
- Tamper-evident audit trail. Every AI step (model id, token count, latency, summary) and every human action is SHA-256 hash-chained; any later edit is detectable (
lib/audit-hash.ts). - Graceful degradation. With no AI key, AI steps no-op and the deterministic engine still adjudicates. If the agent stream hangs or errors, the case falls back to the deterministic pipeline (
lib/agent/orchestrator.ts). - Model portability. Swapping models is a single config change in
getModel(); the decision logic does not move. - No-training target and draft contract commitment. Eli does not operate a product training or fine-tuning pipeline for Institution Data, and the draft DPA prohibits model training, evaluation, benchmarking, or secondary use absent an explicit written opt-in. Before an official assurance, inventory the exact active gateway/model route and retain provider terms plus prompt/output/log retention settings that match that commitment. A draft clause or general provider posture is not proof of the assessed deployment's behavior.
5. Known limits (stated honestly)
- The accuracy corpus is synthetic documents, not scanned IRS originals; a scanned-document benchmark is in progress. Measured field accuracy is published on the in-app
/accuracypage. - Extraction confidence is the model's self-reported estimate; the engine treats it as one input to the composite score, not as ground truth, and routes anything below threshold to a human.
- SAI/Pell figures are directional estimates finalized against federal tables.
6. Mapping to HECVAT 4
HECVAT 4 includes AI-governance questions. This memo is preparation material for those answers: repository model configuration (§2), intended human oversight (§4), the no-training target/draft commitment (§4), explainability and audit design (§3–4), and known limitations (§5). The official response must add the deployment-specific inventory, active-provider terms, runtime settings, and operating evidence.