Model Research and Design by Kelly Emrick, DHSc, PhD, MBA
AI Governance Dashboard & Task List
A comprehensive framework for healthcare organizations to implement, monitor, and mature their AI governance programs — aligned with JC/CHAI, NIST AI RMF, HAIRA, and emerging state/federal regulations.
AI Governance Landscape
Healthcare AI governance sits at the intersection of accreditation standards, federal frameworks, state legislation, and organizational readiness. This dashboard integrates the leading frameworks into a single actionable tool.
Integrated Framework Sources
Navigate the Dashboard
AI Governance Committee Builder
Structure your AI governance using the People, Process, Technology, and Operations (PPTO) framework. An effective governance committee requires multidisciplinary expertise with clearly defined roles and decision-making authority.
- Executive Sponsor / Chief AI Officer designatedSenior leader with AI oversight aligned to enterprise strategy
- Clinical Operations representative appointedEnsures AI aligns with care delivery and serves as patient advocate
- IT/Technical Infrastructure lead assignedManages AI systems architecture, integration, and security
- Data Science / Clinical Informatics lead assignedBridges data analysis, model validation, and clinical practice
- Regulatory/Compliance officer includedNavigates FDA, state laws, HIPAA, and accreditation requirements
- Legal counsel engagedContract review, liability, data use agreements, vendor terms
- Ethics representative includedEvaluates AI projects for alignment with organizational values
- Cybersecurity officer includedAI-specific threat assessment, SBOM review, incident response
- Quality Improvement / Patient Safety officerIntegrates AI monitoring into existing QI infrastructure
- Patient/Community representative includedEnsures tools align with real-world patient priorities and builds trust
- Centralized AI intake process establishedSingle pathway for all AI proposals to enter governance review
- Risk classification methodology definedHigh/Medium/Low based on patient proximity and decision impact
- Lifecycle decision checkpoints documentedStage-gate approvals from intake through decommissioning
- Standard operating procedures writtenSOPs for evaluation, approval, deployment, monitoring
- Decision-making model formalizedVoting vs. consensus; quorum requirements; escalation paths
- Meeting cadence and reporting structure setRegular schedule with minutes, board reporting, and KPIs
- Vendor evaluation criteria standardizedValidation data, bias testing, demographic performance requirements
- Incident management protocols definedAI-specific reporting, response strategies, corrective actions
- Decommissioning criteria establishedTriggers for removing or replacing AI tools from production
- AI solution registry/inventory system deployedTrack all AI tools across the organization with metadata
- Document management for model artifactsVersioned storage for validation reports, change logs, audits
- Secure computing environment for validationIsolated environments for testing AI models on local data
- Data extraction and cohort tools availableInfrastructure for building representative validation datasets
- AI performance monitoring dashboards builtReal-time tracking of accuracy, drift, bias, and usage metrics
- Integration pathways mapped (EHR, PACS, etc.)APIs, FHIR endpoints, HL7 interfaces for clinical integration
- AI governance budget allocatedDedicated funding for committee operations, tools, and training
- FTE support for governance operations identifiedFull-time staff to manage day-to-day governance activities
- Committee member compensation structure definedProtected time, stipends, or role integration for participants
- Board reporting template createdStandardized format for regular updates to fiduciary board
- Annual governance review process scheduledYearly reassessment of policies, committee effectiveness, maturity
- Cross-departmental champion network builtAI advocates in each department to facilitate adoption and feedback
JC/CHAI Responsible Use of AI in Healthcare (RUAIH)
The Joint Commission and Coalition for Health AI released this landmark guidance in September 2025, establishing seven elements for responsible AI use across U.S. health systems. A voluntary AI certification program based on forthcoming playbooks is expected in 2026.
- Formal AI-usage policies established and documented
- Cross-functional governance committee formed with defined charter
- Committee includes compliance, IT, clinical, operations, privacy, cybersecurity
- Policies set expectations for AI use including permitted and prohibited uses
- AI policies aligned with other internal policies and external regulatory/ethical frameworks
- Regular AI usage reporting to the board of directors / fiduciary governing body
- Policies regularly reviewed and updated as regulations shift
- Data access and use policies specific to AI tools documented
- Mechanism to disclose AI’s role in patient care to patients and families
- Patient education materials on how AI may benefit their care
- Transparency extends to staff on how AI tools function and handle data
- Compliance with applicable state disclosure requirements (CA, TX, CO)
- All AI data uses comply with HIPAA Privacy and Security Rules
- Data encrypted in transit and at rest for all AI systems
- Role-based access controls enforced for AI tools
- Regular security risk assessments conducted for AI systems
- AI-specific incident response plan in place
- Data use agreements executed with all AI vendors
- DUAs define permitted uses, data minimization, re-identification prohibitions
- Third-party security obligations documented with audit rights
- Pre-deployment validation completed for each AI tool
- Post-deployment risk-based monitoring program established
- Validation evidence requested from vendors during procurement
- Bias evaluations reviewed and documented per vendor/tool
- AI performance dashboard deployed (accuracy, drift, usage)
- Post-implementation reviews at 30, 90, and 180 days
- Process for managing vendor updates and version changes
- Responsible parties for monitoring formally assigned
- Internal incident system updated to capture AI-related events
- AI near misses and harms tracked (unsafe recommendations, bias, degradation)
- De-identified AI safety events shared through existing channels (e.g., PSOs)
- Pattern recognition process for AI-related safety trends
- Feedback loop established with AI vendors on safety events
- Process for categorizing and documenting AI risk levels established
- Bias assessment process formalized for all AI tools
- Verification that AI tools are tuned to the population served
- Training data representativeness reviewed per AI tool
- Subgroup performance analysis required (age, sex, race/ethnicity)
- Ongoing bias monitoring integrated into quality monitoring process
- AI education program developed for all staff levels
- Role-specific AI training modules created (clinical, admin, IT)
- Training covers AI limitations, risks, and appropriate reliance
- AI tool access limited to need-to-use basis by role
- All staff know where to find AI policies and procedures
- Training updated when new AI tools are deployed or policies change
Regulatory & Legal Landscape Tracker
Healthcare AI governance operates within a complex, rapidly evolving regulatory environment. Federal frameworks provide voluntary guidance while state laws impose binding requirements. Organizations should adopt a "highest common denominator" compliance strategy.
Federal Frameworks & Guidance
Accreditation Standards
State Laws — Healthcare-Specific Requirements
- Audit all patient-facing AI systems for disclosure compliance
- Implement written patient disclosure protocols (TX, CA, CO)
- Review AI tools for implied licensure or misleading design elements
- Ensure practitioner review of all AI-generated clinical content
- Update vendor contracts with AI-specific data use provisions
- Map organizational AI use against Colorado high-risk categories
- Prepare impact assessments for high-risk AI systems (CO requirement)
- Monitor federal preemption developments and adjust strategy
AI Lifecycle Governance Task List
A comprehensive task list spanning all six phases of the AI lifecycle, integrating requirements from NIST AI RMF (Govern → Map → Measure → Manage), the HAIRA framework, and JC/CHAI guidance. Complete tasks are tracked across phases with live progress.
- Vendor due diligence checklist completed
- Validation data and testing results requested from vendor
- Bias testing results and demographic performance data reviewed
- FDA clearance/authorization status verified (if applicable)
- Training data representativeness evaluatedAge, sex, race/ethnicity, geography alignment with service population
- Model architecture and decision logic reviewed for transparency
- Data use agreement (DUA) negotiated and executed
- DUA includes data minimization, re-identification prohibitions, audit rights
- Security assessment completed (encryption, access controls, SBOM)
- Vendor willingness to tune/validate on local representative sample confirmed
- Vendor update/version change notification process agreed upon
- Monitoring responsibilities allocated between vendor and organization
- Contract includes performance guarantees and exit clauses
- Ethics review completed for the AI solution
- Local validation dataset assembled from representative population
- Performance testing completed (sensitivity, specificity, AUC, PPV/NPV)
- Subgroup analysis completed across demographic groupsNIST MEASURE 2.6 — Evaluate for bias across subpopulations
- Edge case and failure mode analysis conducted
- Clinical workflow integration tested (user acceptance testing)
- Interoperability with EHR/PACS/existing systems verified
- Performance meets or exceeds the established baseline threshold
- Safety testing for clinical decision-making scenarios completed
- Cybersecurity penetration testing / vulnerability assessment completed
- Validation results documented in standardized format
- Results reviewed and approved by governance committee
- Risk mitigation strategies documented for identified weaknesses
- Go/No-Go decision formally recorded
- Change management plan developed and communicated
- Staff training completed (role-specific modules)
- Patient disclosure protocols activatedState-specific requirements for CA, TX, CO, IL as applicable
- EHR/clinical workflow integration deployed
- Go-live support structure in place (helpdesk, escalation, clinical backup)
- Human-in-the-loop safeguards activated where required
- AI tool registered in organizational AI inventory/registry
- Monitoring infrastructure activated (dashboards, alerts, feedback loops)
- Incident reporting pathway communicated to all users
- Access controls and role-based permissions configured
- Communication to patients/community about new AI capability
- Equity communication to end-users and disadvantaged subgroupsHEAAL Decision Point 6 — Raise awareness about biases and consequences
- Deployment formally documented and governance committee notified
- 30-day post-implementation review completed
- 90-day post-implementation review completed
- 180-day post-implementation review completed
- Ongoing performance monitoring active (accuracy, drift detection)
- Data drift detection protocols in placeNIST MANAGE 4.1 — Monitor for changes in data distribution
- Concept drift monitoring active
- Bias monitoring ongoing with subgroup performance trackingHEAAL Decision Points 7–8 — Monitor equity impact continuously
- AI-related incidents captured and tracked in reporting system
- De-identified safety events shared with PSOs or industry channels
- Vendor updates/version changes reviewed and validated before deployment
- Clinician feedback mechanisms active and reviewed regularly
- Patient outcome tracking linked to AI tool usage
- Annual comprehensive reassessment completed
- Third-party vendor controls reassessed periodically
- AI policies and governance reviewed against evolving regulations
- Board reporting on AI performance and incidents maintained
- Decommissioning criteria defined and documented
- Triggers for AI tool replacement identified (performance, safety, equity)
- Process for evaluating major model updates/retraining established
- Re-validation requirements defined for significant changes
- PCCP compliance maintained for FDA-regulated devices
- Decommissioning workflow includes data retention/disposal plan
- Clinical workflow contingency plan ready (manual fallback)
- Staff communication plan for AI tool changes prepared
- Patient notification plan if AI tool is removed from care pathway
- Lessons learned documented and shared with governance committee
- AI inventory/registry updated to reflect decommissioned tools
- Post-decommission review conducted at 30 days
HAIRA Maturity Self-Assessment
Assess your organization across the seven critical domains of healthcare AI governance using the HAIRA maturity model. Your overall maturity level is determined by the "weakest-link" rule — capped at the lowest-scoring domain. Rate each domain from Level 1 (Ad Hoc) to Level 5 (Leading).
Health Equity & Bias Assessment
The HEAAL framework evaluates how AI implementation may affect health equity across five assessment domains and eight lifecycle decision points. This tab provides structured checklists for each domain to ensure AI tools do not worsen health disparities.
- Executive sponsor accountable for equity outcomes identified
- Equity objectives documented for each AI use case
- Governance committee includes equity/diversity expertise
- Patient/community voice incorporated in AI governance decisions
- Accountability for equity assigned at each lifecycle decision point
- Corrective action process established when equity objectives are not met
- Training data reviewed for demographic representation
- Outcome label definitions examined for proxy discriminationE.g., healthcare cost as proxy for illness severity can disadvantage minorities
- Performance metrics stratified by race, ethnicity, sex, age, socioeconomic status
- Fairness metrics applied (demographic parity, equalized odds, predictive parity)
- Bias audit toolkit deployed (IBM AI Fairness 360, Google Fairness Indicators, Aequitas)
- Disparate impact analysis conducted before deployment
- Ongoing fairness monitoring post-deployment with trigger thresholds
- AI tool validated on a population similar to the deployment context
- Clinical workflow context assessed for fit with diverse patient populations
- Language and literacy considerations evaluated for patient-facing tools
- Digital access barriers assessed (internet, devices, digital literacy)
- Pediatric vs. adult population considerations addressed where applicable
- External validation conducted on local population data
- Performance consistency across clinical sites verified
- Temporal stability assessed (performance over time)
- Data quality and completeness evaluated for equity impact
- Drift monitoring includes equity dimension (performance by subgroup over time)
- AI tool documentation includes known limitations and populations underrepresented
- End-users informed about AI limitations affecting disadvantaged groups
- Patients from disadvantaged subgroups specifically educated about AI’s role
- Equity monitoring results shared transparently with governance committee
- TRIPOD+AI or equivalent reporting standard used for model documentation
- Community engagement conducted on AI equity concerns
Auto-Generated Action Plan
This action plan is dynamically generated based on your self-assessment results and checklist completion across all tabs. Items are prioritized by urgency and impact. Complete checklists and adjust maturity sliders across the dashboard to see this plan update in real time.