Scope and Inventory
DEFINING ASSESSMENT BOUNDARY
Assessment begins with profiling the AI application: use case, model type, autonomy level, affected users, decisions influenced, jurisdictions, deployment pattern, and data used. Context drives risk — not model quality alone. This aligns with NIST AI RMF focus on governance, mapping context, measuring risk, and managing controls with continuous monitoring.
Three-Layer Scoring Model
INHERENT → CONTROLS → RESIDUAL
- Layer 1 — Inherent Risk: impact and likelihood before safeguards.
- Layer 2 — Control Effectiveness: maturity and coverage of safeguards.
- Layer 3 — Residual Risk & Compliance: what remains after mitigation.
Compliance Score = Σ(Requirement Weight × Control Pass Rate)
Domains and Weights
RULE FAMILIES CHECKED FOR EVERY AI APPLICATION
- Use‑case criticality (15%) — decision impact, autonomy, user harm.
- Data governance (15%) — consent, lineage, sensitivity, retention, data quality.
- Model performance (15%) — accuracy, drift, explainability, reproducibility.
- Fairness & ethics (15%) — bias, disparity, harmful content, contestability, transparency.
- Security (20%) — injection, poisoning, tampering, access control, dependencies, logging.
- Compliance governance (10%) — regulatory mapping, policies, approvals, audit trail.
- Monitoring & operations (10%) — incidents, fallback, alerts, retraining governance.
Metric Scale
CONSISTENT 1–5 SCORING
- 1 = Low risk / strong control / fully compliant
- 3 = Medium risk / partially controlled
- 5 = High risk / weak or missing control
Likelihood and impact drive inherent risk. Control maturity (coverage, quality, evidence) drives control effectiveness. Weighted aggregation produces the final residual risk and a compliance score for audit readiness.
Decision Thresholds
APPROVE, CONDITIONAL, REMEDIATE, OR REJECT
- 0–25: Low residual risk — approve.
- 26–50: Moderate risk — approve with documented controls.
- 51–75: High risk — remediation required before deployment.
- 76–100: Critical risk — reject or redesign.
Continuous Review
PROGRAM, NOT ONE‑TIME GATE
Reassess on model updates, data shifts, incidents, regulation changes, and periodic cycles. Store evidence in a compliance pack including policies, model cards, data sheets, test reports, approvals, and monitoring records for regulator‑ready audits.
Quick Summary
WHAT IT DOES
- Profiles the AI system and maps context (use case, autonomy, users, data, jurisdiction).
- Evaluates eight rule families using measurable metrics on a 1–5 scale.
- Aggregates scores with domain weights to compute inherent and residual risk.
- Rates control effectiveness and produces a compliance score with evidence pointers.
- Outputs a verdict and remediation roadmap, then supports continuous monitoring.