The DIE Progress Unit: Measuring AI Compliance in Steps, Not Perfection
Last updated: February 16, 2026
Last updated: April 2026
The DIE Progress Unit is a three-stage framework (Document, Implement, Evaluate) that measures AI compliance maturity for each governance requirement, replacing the binary “compliant or not” question with a quantifiable, defensible progression metric. Every AI compliance conversation I have with a general counsel eventually hits the same wall. They ask: “How do I know if we’re compliant?” I ask what they mean by compliant. They say: “Are we good, or are we not good?” That binary framing is the problem. Compliance with AI regulations isn’t a switch. There’s no moment where you go from non-compliant to compliant. There’s a spectrum, and you need a way to measure where you are on it.
Before you can measure compliance, you need to know what counts as AI in the first place. Once that’s settled, I developed a framework called the DIE Progress Unit. DIE stands for Document, Implement, Evaluate. Each AI governance requirement moves through these three stages. Your compliance maturity for any given requirement is measured by which stage you’ve reached. Your overall compliance maturity is the aggregate across all requirements.
Stage 1: Document
The Document stage means you’ve written down what you’re supposed to do. You have a policy. You have a procedure. You have a standard. It exists on paper (or in a shared drive, or in Notion, or wherever your organization keeps governance documents).
This is where most companies stop. They hire a law firm to write an AI policy. The policy says all the right things. Risk assessments will be conducted. Bias testing will be performed. Employees will be trained. The policy gets approved by the board. It goes into a folder. Nobody looks at it again.
Documentation without implementation is theater. (Ask the lawyers who got sanctioned for filing AI-generated briefs they never verified how well “we had a policy” held up.) But documentation without documentation is worse, because you can’t implement what you haven’t defined. So the Document stage has value. It’s the foundation. It’s just not the building.
For each requirement, the Document stage produces a specific artifact. For employee AI training, the artifact is a training curriculum and schedule. For bias testing, it’s a testing protocol with defined metrics and thresholds. For incident response, it’s a playbook with roles, escalation paths, and communication templates.
Stage 2: Implement
The Implement stage means you’re actually doing what the document says. The training curriculum isn’t just written; employees are completing the training. The bias testing protocol isn’t just defined; tests are running and results are being recorded. The incident response playbook isn’t just drafted; it’s been exercised and people know their roles.
Implementation is where resource constraints, organizational resistance, and technical complexity collide with policy aspirations. The policy says “conduct annual bias audits on all high-risk AI systems.” Implementation means finding a qualified auditor, negotiating the scope and methodology, getting access to system outputs, running the analysis, and remediating findings. That takes time, money, and organizational will.
The gap between Document and Implement is where most compliance programs live. They have policies that say things they’re not doing. This isn’t necessarily bad faith. It’s usually resource constraints and competing priorities. But it’s the gap that regulators and plaintiffs will target.
To track implementation, I use completion metrics. What percentage of employees completed AI training? What percentage of high-risk AI systems have undergone bias testing? What percentage of AI vendors have completed due diligence? These are measurable, reportable numbers that tell you (and your board, and your regulator) whether your policies are operational.
Stage 3: Evaluate
The Evaluate stage means you’re checking whether what you implemented is actually working. Training was completed, but did employee behavior change? Bias testing was performed, but were the results acted on? Incident response was exercised, but did it work when a real incident occurred?
Evaluation is the stage that separates compliance programs from governance programs. Compliance says “we did the thing.” Governance says “the thing worked.”
Evaluation metrics include: outcome measurements (did bias testing identify issues? Were they remediated? Did the remediation reduce bias in subsequent tests?), effectiveness measurements (after training, do employees use AI differently? Can they identify prohibited use cases?), and maturity measurements (are processes improving over time? Are cycle times decreasing? Are fewer incidents occurring?).
Ready to measure where you stand? Take the ACRA — it maps your current position on the DIE framework.
Most companies never reach the Evaluate stage for any requirement. The ones that do have a defensible compliance position even if individual requirements have gaps. A regulator asking “are you compliant?” will be far more satisfied with “here are our policies, here’s evidence of implementation, and here’s our evaluation showing what’s working and what we’re improving” than with “we have a policy.”
How to Use the Framework
Create a matrix. Rows are your AI governance requirements (drawn from applicable regulations, industry standards, and internal policy). Columns are the three DIE stages. Each cell is either empty (not started), in progress, or complete.
Example for a company subject to the Colorado AI Act:
| Requirement | Document | Implement | Evaluate |
|---|---|---|---|
| Risk management policy | Complete | In Progress | Not Started |
| Impact assessments for high-risk AI | Complete | Not Started | Not Started |
| Consumer notice of AI use | In Progress | Not Started | Not Started |
| Algorithmic discrimination reporting | Not Started | Not Started | Not Started |
| Employee AI training | Complete | Complete | In Progress |
This matrix tells you where your gaps are. It tells your board how mature your program is. It tells a regulator that you’re taking compliance seriously even if you’re not done. And it gives you a prioritization tool: focus implementation efforts on the requirements that are documented but not yet operational.
The “Good Faith” Defense
The DIE framework also builds a good faith defense. Multiple AI regulations (including Colorado’s) include provisions that consider whether a company made reasonable efforts to comply. A company that can show documented policies, evidence of implementation, and ongoing evaluation efforts has a much stronger “good faith” argument than a company that has nothing or has a policy it never followed.
This matters because early AI enforcement will likely focus on egregious violators, not companies making genuine compliance efforts. The FTC’s case-by-case approach confirms this: they’re picking off the worst actors first, but the circle is tightening. Regulators with limited resources will prioritize companies that have no AI governance at all over companies that have an imperfect but operational program. The DIE framework gives you a structured way to demonstrate “we’re working on it” with evidence rather than promises.
What to Do Now
Map your requirements. Pull every AI-related obligation from your applicable regulations (the regulatory patchwork is growing fast), industry standards (NIST AI RMF, ISO 42001), and internal policies. These are your matrix rows.
Assess each requirement honestly. For each requirement, mark whether you’ve documented it, implemented it, and evaluated it. No optimistic rounding. If the policy exists but nobody follows it, that’s Document-only. Mark it honestly.
Prioritize high-risk gaps. Any Tier 1 (high-risk) AI system requirement that’s below the Implement stage should be your first priority. These are the areas where regulatory exposure and liability risk are highest.
Report progress in DIE units. Tell your board: “We have 24 AI governance requirements. 20 are at Document stage. 14 are at Implement stage. 6 are at Evaluate stage.” That’s a governance maturity snapshot. Update it quarterly. Progress should be visible.
Stop asking “are we compliant?” and start asking “what stage are we at?” The first question has no useful answer. The second question has a specific, measurable, actionable answer for every requirement in your program.
For a practical implementation structure, the 5-Layer AI Compliance Stack maps directly onto the DIE framework, giving you a concrete architecture for each stage.
Compliance isn’t a destination. It’s a process with three stages. Know where you are. Know where you need to go. Measure the distance between them. That’s the DIE Progress Unit.
Stop guessing where you stand. Kaizen AI Lab maps your AI governance program against the DIE framework and builds the implementation plan to close the gaps. Talk to us.