Skip to content

assessment.md

Latest commit

 

History

History
312 lines (236 loc) · 7.37 KB

assessment.md

File metadata and controls

312 lines (236 loc) · 7.37 KB

AI Portfolio Assessment

Know where you are before deciding where to go.

Quick Assessment (60 Seconds)

Answer these three questions:

# Question Yes No
1 Can you list all your AI initiatives right now? ✅ ❌
2 Do you know the ROI of each? ✅ ❌
3 Do you review performance monthly? ✅ ❌

Scoring

3 Yes: You're in the top 5% — focus on optimization 2 Yes: You're ahead but have gaps — strengthen visibility 1 Yes: You're at risk — prioritize tracking 0 Yes: You're flying blind — start immediately

Full Assessment (15 Minutes)

Section 1: Portfolio Visibility

Question Score
How many AI initiatives does your organization have?
□ Don't know 0
□ Rough estimate 1
□ Documented list 2
□ Tracked in system 3
Question Score
What stage is each initiative in?
□ Don't know 0
□ Know some 1
□ Know most 2
□ All documented 3
Question Score
Who owns each initiative?
□ Unclear 0
□ Some have owners 1
□ Most have owners 2
□ All have clear owners 3

Section 1 Total: ___ / 9

Section 2: Layer 1 (Infrastructure)

Question Score
Do you know your total AI spend?
□ No visibility 0
□ Rough estimate 1
□ Monthly tracking 2
□ Real-time dashboard 3
Question Score
Are AI systems monitored?
□ No monitoring 0
□ Basic uptime 1
□ Performance metrics 2
□ Full observability 3
Question Score
Is access controlled?
□ Open access 0
□ Basic auth 1
□ Role-based access 2
□ Zero trust 3
Question Score
Are API calls logged?
□ No logging 0
□ Error logs only 1
□ Request logging 2
□ Full audit trail 3

Section 2 Total: ___ / 12

Section 3: Layer 2 (Intelligence)

Question Score
How is training data managed?
□ Ad hoc 0
□ Some documentation 1
□ Version controlled 2
□ Full data lineage 3
Question Score
Are models version controlled?
□ No versioning 0
□ Manual tracking 1
□ Git/similar 2
□ MLOps platform 3
Question Score
Can you roll back models?
□ No rollback capability 0
□ Manual process 1
□ Scripted process 2
□ Automated rollback 3
Question Score
Do you have learning loops?
□ No feedback 0
□ Manual review 1
□ Automated collection 2
□ Continuous learning 3

Section 3 Total: ___ / 12

Section 4: Layer 3 (Experience)

Question Score
How is user adoption tracked?
□ Not tracked 0
□ Anecdotal 1
□ Usage metrics 2
□ Full analytics 3
Question Score
How do users report issues?
□ No process 0
□ Email/ad hoc 1
□ Ticketing system 2
□ In-app feedback 3
Question Score
Is there human override capability?
□ No override 0
□ Technical only 1
□ User escalation 2
□ Seamless override 3

Section 4 Total: ___ / 9

Section 5: Governance

Question Score
Are initiatives risk-classified?
□ No classification 0
□ Some classified 1
□ Most classified 2
□ All classified 3
Question Score
Is documentation up to date?
□ No documentation 0
□ Outdated 1
□ Mostly current 2
□ Continuously updated 3
Question Score
How often do you review the portfolio?
□ Never 0
□ Annually 1
□ Quarterly 2
□ Monthly 3
Question Score
Do you have bias testing?
□ None 0
□ Ad hoc 1
□ Before deployment 2
□ Continuous monitoring 3

Section 5 Total: ___ / 12

Calculate Your Score

Section Your Score Max
Portfolio Visibility ___ 9
Layer 1 (Infrastructure) ___ 12
Layer 2 (Intelligence) ___ 12
Layer 3 (Experience) ___ 9
Governance ___ 12
TOTAL ___ 54

Interpret Your Results

Overall Score

Score Level Meaning
45-54 Optimizing Top 5%. Focus on refinement.
35-44 Established Good foundation. Close specific gaps.
25-34 Developing Making progress. Prioritize Layer 1.
15-24 Beginning Early stage. Start with basics.
0-14 Ad Hoc High risk. Immediate action needed.

Section Analysis

Lowest section = highest priority

If lowest is... Focus on...
Portfolio Visibility Deploy dashboard, create inventory
Layer 1 Cost monitoring, security, logging
Layer 2 Data pipelines, model management
Layer 3 User feedback, adoption tracking
Governance Risk classification, documentation

Recommended Actions by Score

Score 0-14: Immediate Actions

  1. This week: List all AI initiatives in a spreadsheet
  2. Next week: Identify owners for each
  3. Month 1: Implement basic cost tracking
  4. Month 2: Add security controls
  5. Month 3: Establish monthly review

Score 15-24: Foundation Building

  1. Month 1: Complete initiative inventory
  2. Month 2: Implement Layer 1 monitoring
  3. Month 3: Add governance basics
  4. Month 4: Begin Layer 2 standardization

Score 25-34: Gap Closing

  1. Identify: Which section scored lowest?
  2. Prioritize: Top 3 gaps in that section
  3. Implement: One improvement per month
  4. Measure: Track score improvement quarterly

Score 35-44: Optimization

  1. Analyze: Where are the remaining gaps?
  2. Benchmark: Compare to industry standards
  3. Optimize: Focus on efficiency gains
  4. Scale: Expand successful patterns

Score 45-54: Excellence

  1. Maintain: Keep processes running
  2. Share: Document and share learnings
  3. Innovate: Explore advanced capabilities
  4. Lead: Help others in your organization

Next Steps

Based on your assessment:

Your Score Start Here
0-24 Layer 1: Infrastructure
25-34 Your lowest-scoring section guide
35-44 Governance for refinement
45-54 Portfolio Intelligence for optimization

Tools to Help

Reassess Regularly

Schedule assessment reviews:

  • Quarterly: Full assessment (this document)
  • Monthly: Quick assessment (3 questions)
  • Weekly: Dashboard review

Track your score over time to measure improvement.

Assessment is the first step. Implementation is the journey.