AGENT SKILL

analyzing-ai-in-financial-services

Evaluates AI/ML applications in financial services with model governance, bias assessment, and regulatory considerations. Use when analyzing AI in finance, evaluating ML models, or assessing AI governance.

License 许可
Apache-2.0
最近更新
2026-11-09
Stars
★ 0
Forks
0

01 前言 Front Matter

name
analyzing-ai-in-financial-services
tags
[ "analysis", "financial-technology", "regulatory" ]
source
https://github.com/CaseMark/skills
license
Apache-2.0
language
en
metadata
{ "author": "casemark", "skill_modes": [ "Analysis" ], "document_types": [ "Analysis Report" ], "practice_areas": [ "Fintech", "Payments", "Digital Banking" ] }
description
Evaluates AI/ML applications in financial services with model governance, bias assessment, and regulatory considerations. Use when analyzing AI in finance, evaluating ML models, or assessing AI governance.
imported_at
2026-09-20
source_path
skills/finance/analyzing-ai-in-financial-services/SKILL.md
source_repository
CaseMark/skills

03 SKILL.md 正文

Analyzing AI In Financial Services

When To Use

  • Evaluating an AI/ML model deployed or proposed for credit decisioning, fraud detection, AML/KYC, trading, or customer-facing automation
  • Assessing model risk management (MRM) posture against SR 11-7, OCC 2011-12, or equivalent guidance [VERIFY jurisdiction-specific MRM guidance]
  • Reviewing algorithmic bias and fair-lending implications for underwriting or pricing models
  • Analyzing vendor-supplied AI tools (e.g., BaaS scoring engines, conversational AI for digital banking)
  • Preparing for regulatory examination or audit of AI/ML systems in a financial institution

Inputs To Gather

  • Model documentation: model cards, technical specifications, training data descriptions, feature lists
  • Performance metrics: accuracy, precision/recall, AUC-ROC, stability metrics (PSI), back-testing results
  • Governance artifacts: model inventory entry, validation reports, change-management logs, approval records
  • Bias/fairness data: disparate impact ratios across protected classes, HMDA or ECOA-relevant statistics [VERIFY applicable fair-lending statutes by jurisdiction]
  • Regulatory context: institution charter type (bank, credit union, non-bank lender), applicable supervisory framework, any open MRAs or consent orders
  • Use-case scope: specific business function the AI serves (e.g., real-time fraud scoring, deposit pricing, chatbot servicing)

Workflow

  1. Define scope and materiality

    • Classify the model by risk tier (critical, high, medium, low) based on financial exposure, customer impact, and regulatory sensitivity
    • Confirm whether the model is first-party built, vendor-supplied, or open-source derived — governance expectations differ for each
  2. Assess model governance framework

    • Map the institution's MRM program against SR 11-7 three-lines-of-defense structure [VERIFY if institution falls under OCC, Fed, or state regulator]
    • Check for documented model owner, independent validation function, and escalation procedures
    • Verify model inventory completeness — flag any shadow models or undocumented deployments
  3. Evaluate technical soundness

    • Review feature selection for proxies of protected characteristics (e.g., zip code as race proxy in credit models)
    • Examine training data vintage, representativeness, and label quality
    • Assess explainability: can adverse-action reasons be generated per ECOA/Reg B requirements? [VERIFY if adverse-action notice obligations apply]
    • Check for concept drift monitoring and automated retraining triggers
  4. Conduct bias and fairness analysis

    • Calculate disparate impact ratios for lending/pricing models across race, sex, age, and other protected classes
    • Apply appropriate fairness metrics (demographic parity, equalized odds, predictive parity) based on use case
    • Identify whether pre-processing, in-processing, or post-processing debiasing techniques are employed
    • Flag any model where disparate impact exceeds the four-fifths threshold without documented business justification
  5. Map regulatory and compliance exposure

    • Cross-reference against CFPB guidance on automated decisioning and adverse-action notices
    • For payment/digital-banking AI, evaluate PCI-DSS data handling, BSA/AML transaction-monitoring adequacy [VERIFY PCI scope for specific deployment]
    • Assess third-party risk management (TPRM) obligations if model is vendor-supplied (OCC 2013-29 / updated interagency guidance)
    • Note any state-specific AI laws (e.g., Colorado AI Act, NYC Local Law 144 for automated employment decisions) [VERIFY state/local AI legislation applicability]
  6. Synthesize findings and risk-rate

    • Assign overall risk rating with supporting rationale
    • Prioritize findings by severity: critical (immediate remediation), high (next exam cycle), moderate (roadmap), low (enhancement)
    • Identify compensating controls that mitigate identified gaps

Output

Produce an AI/ML Model Analysis Report containing:

  • Executive summary: model name, use case, risk tier, overall assessment (acceptable / acceptable with conditions / unacceptable)
  • Governance assessment: MRM program maturity, validation independence, inventory accuracy
  • Technical evaluation: model performance, explainability, drift monitoring, data quality
  • Bias and fairness findings: disparate impact results, fairness metrics, debiasing controls
  • Regulatory exposure map: applicable regulations, compliance gaps, pending enforcement trends
  • Remediation roadmap: prioritized findings with owners, timelines, and success criteria
  • Appendices: data tables, metric calculations, regulatory cross-reference matrix

Quality Checks

  • Every finding cites a specific regulatory provision, guidance document, or industry standard (SR 11-7, ECOA, CFPB circulars)
  • Disparate impact calculations include methodology notes and confidence intervals
  • Vendor-supplied models are assessed under both MRM and TPRM frameworks — do not evaluate under only one
  • All jurisdiction-dependent conclusions are tagged with [VERIFY] for reviewer confirmation
  • Adverse-action explainability is tested with sample outputs, not just documented as "supported"
  • Report distinguishes between model risk (technical) and compliance risk (regulatory) — do not conflate the two

04 同仓库更多 Skills