openevidence-prod-checklist
Production readiness checklist for OpenEvidence clinical AI deployments. Use when preparing for production launch, conducting pre-deployment reviews, or auditing OpenEvidence integration for compliance. Trigger with phrases like "openevidence production", "openevidence go-live", "openevidence checklist", "deploy openevidence", "openevidence readiness".
- License 许可
- MIT
- 最近更新
- 2026-05-26
- Stars
- ★ 0
- Forks
- 0
01 前言 Front Matter
- name
- openevidence-prod-checklist
- author
- Jeremy Longshore <jeremy@intentsolutions.io>
- license
- MIT
- version
- 1.0.0
- description
- Production readiness checklist for OpenEvidence clinical AI deployments. Use when preparing for production launch, conducting pre-deployment reviews, or auditing OpenEvidence integration for compliance. Trigger with phrases like "openevidence production", "openevidence go-live", "openevidence checklist", "deploy openevidence", "openevidence readiness".
- allowed-tools
- Read, Grep
- compatible-with
- claude-code, codex, openclaw
03 SKILL.md 正文
OpenEvidence Production Checklist
Overview
Comprehensive checklist for deploying OpenEvidence clinical AI integrations to production healthcare environments.
Prerequisites
- Completed development and testing phases
- Signed BAA with OpenEvidence
- Security review completed
- Compliance team sign-off
Production Readiness Checklist
1. Legal & Compliance
-
[ ] Business Associate Agreement (BAA) signed
- Contact: compliance@openevidence.com
- Required for any PHI handling
-
[ ] API Terms of Service accepted
-
[ ] Clinical use disclaimers in place
- All clinical outputs must include appropriate disclaimers
- "For informational purposes only. Verify with current guidelines."
-
[ ] Liability and malpractice considerations reviewed
- Legal review of clinical decision support usage
-
[ ] Data processing agreement documented
- What data is sent, stored, processed
2. Security
-
[ ] Production API keys generated
- Separate keys from development/staging
- Stored in secret manager (not environment variables)
-
[ ] Key rotation schedule established
- Recommend: 90-day rotation cycle
-
[ ] IP allowlist configured (if available)
- Restrict API access to known IP ranges
-
[ ] TLS 1.2+ enforced
- No fallback to older protocols
-
[ ] PHI sanitization implemented
- Review
openevidence-security-basics
- Review
-
[ ] Webhook signature verification active
- All webhook endpoints validate signatures
-
[ ] Audit logging enabled
- HIPAA-compliant access logging
- Minimum 6-year retention
3. Infrastructure
-
[ ] High availability configuration
// Multiple API endpoints for failover const config = { primaryEndpoint: 'https://api.openevidence.com', fallbackEndpoint: 'https://api-fallback.openevidence.com', timeout: 30000, retries: 3, }; -
[ ] Circuit breaker implemented
import CircuitBreaker from 'opossum'; const breaker = new CircuitBreaker(clinicalQuery, { timeout: 30000, errorThresholdPercentage: 50, resetTimeout: 30000, }); -
[ ] Rate limit handling configured
- Review
openevidence-rate-limits - Request queuing for traffic spikes
- Review
-
[ ] Timeout configuration appropriate
- Clinical queries: 15-30s
- DeepConsult: 5-10 minutes
-
[ ] Health check endpoint implemented
app.get('/health/openevidence', async (req, res) => { try { await client.health.check(); res.json({ status: 'healthy' }); } catch (error) { res.status(503).json({ status: 'unhealthy', error: error.message }); } });
4. Monitoring & Alerting
-
[ ] Metrics collection active
- Request count, latency, error rate
- Review
openevidence-observability
-
[ ] Alert rules configured
Alert Threshold Severity Error rate > 5% 5 min window Warning Error rate > 20% 5 min window Critical P95 latency > 15s 5 min window Warning Health check failed 2 consecutive Critical -
[ ] Dashboard created
- Real-time visibility into OpenEvidence integration health
-
[ ] Log aggregation configured
- Centralized logging for debugging
- PHI excluded from logs
5. Error Handling & Resilience
-
[ ] Graceful degradation implemented
async function clinicalQueryWithFallback(question: string) { try { return await openEvidenceQuery(question); } catch (error) { // Log error, alert, return cached or default response await alertService.send('OpenEvidence unavailable'); return { answer: 'Clinical evidence service temporarily unavailable. Please consult current guidelines directly.', fallback: true, }; } } -
[ ] Retry logic tested
- Exponential backoff with jitter
- Respects Retry-After headers
-
[ ] Error classification implemented
- Review
openevidence-common-errors
- Review
6. Performance
-
[ ] Response caching strategy
- Cache frequently queried clinical information
- Cache TTL appropriate for clinical data freshness
-
[ ] Connection pooling configured
- Reuse HTTP connections
-
[ ] Query optimization reviewed
- Clear, specific clinical questions
- Appropriate context provided
-
[ ] Load testing completed
- Test at 2x expected peak traffic
- Verify rate limits not exceeded
7. Documentation & Training
-
[ ] Runbook created
- Incident response procedures
- Review
openevidence-incident-runbook
-
[ ] On-call training completed
- Team knows how to triage OpenEvidence issues
-
[ ] User documentation updated
- How to use clinical decision support
- Limitations and appropriate use
-
[ ] Clinical staff training
- Understanding AI-assisted decision support
- When to rely on vs. question AI recommendations
8. Testing & Validation
-
[ ] Integration tests passing
- Against sandbox environment
-
[ ] End-to-end tests passing
- Full workflow validation
-
[ ] Clinical validation completed
- Sample queries reviewed by clinical team
- Responses appropriate and accurate
-
[ ] Performance benchmarks met
- P50 < 5s, P95 < 15s for clinical queries
- P50 < 3min, P95 < 5min for DeepConsult
9. Deployment
-
[ ] Blue-green or canary deployment planned
- Gradual rollout with monitoring
-
[ ] Rollback procedure documented
- Quick revert if issues detected
-
[ ] Feature flags configured
- Ability to disable OpenEvidence quickly
-
[ ] Database migrations completed
- Audit log tables, cache tables ready
10. Post-Launch
-
[ ] Monitoring dashboard reviewed daily (first week)
-
[ ] Error reports triaged immediately
-
[ ] User feedback collection active
-
[ ] Performance baseline established
Production Configuration Example
// config/openevidence.production.ts
export const productionConfig = {
// Credentials from secret manager
credentials: {
source: 'secret-manager',
apiKeyPath: 'projects/prod/secrets/openevidence-api-key',
orgIdPath: 'projects/prod/secrets/openevidence-org-id',
},
// Endpoints
api: {
baseUrl: 'https://api.openevidence.com',
timeout: 30000, # 30000: 30 seconds in ms
retries: 3,
},
// Rate limiting
rateLimits: {
enabled: true,
maxConcurrent: 10,
requestsPerMinute: 100,
},
// Circuit breaker
circuitBreaker: {
enabled: true,
errorThreshold: 50,
resetTimeout: 30000, # 30 seconds in ms
},
// Caching
cache: {
enabled: true,
ttlSeconds: 3600, # 3600: timeout: 1 hour
maxEntries: 10000, # 10000: 10 seconds in ms
},
// Monitoring
monitoring: {
metricsEnabled: true,
tracingEnabled: true,
logLevel: 'info',
},
};
Output
- Completed production readiness checklist
- Configuration validated
- All stakeholders signed off
- Go-live approved
Resources
Next Steps
For version upgrades, see openevidence-upgrade-migration.
Instructions
- Assess the current state of the deployment configuration
- Identify the specific requirements and constraints
- Apply the recommended patterns from this skill
- Validate the changes against expected behavior
- Document the configuration for team reference
Error Handling
| Error | Cause | Resolution |
|---|---|---|
| Authentication failure | Invalid or expired credentials | Refresh tokens or re-authenticate with deployment |
| Configuration conflict | Incompatible settings detected | Review and resolve conflicting parameters |
| Resource not found | Referenced resource missing | Verify resource exists and permissions are correct |
Examples
Basic usage: Apply openevidence prod checklist to a standard project setup with default configuration options.
Advanced scenario: Customize openevidence prod checklist for production environments with multiple constraints and team-specific requirements.
04 同仓库更多 Skills
zustand-store-ts
Create Zustand stores with TypeScript, subscribeWithSelector middleware, and proper state/action separation. Use when building React state management, creating global stores, or implementing reactive state patterns with Zustand.
zustand-store-creator
Zustand Store Creator - Auto-activating skill for Frontend Development. Triggers on: zustand store creator, zustand store creator Part of the Frontend Development skill category.
zero-trust-config-helper
Zero Trust Config Helper - Auto-activating skill for Security Advanced. Triggers on: zero trust config helper, zero trust config helper Part of the Security Advanced skill category.
zapier-integration-helper
Zapier Integration Helper - Auto-activating skill for Business Automation. Triggers on: zapier integration helper, zapier integration helper Part of the Business Automation skill category.
zai-cli
Execute z.AI CLI providing vision, search, reader, and GitHub exploration via CLI and MCP. Use when user needs image/video analysis, OCR, UI-to-code conversion, error diagnosis, real-time web search, web page to markdown extraction, or GitHub code exploration. Trigger with phrases like "analyze this image", "search the web for", "read this page", "explore this repo", or "use zai". Requires Z_AI_API_KEY.
yt-research
Research competitor YouTube channels, niches, and trending topics for your content strategy. Use this skill whenever the user says "research channels", "analyze competitors", "find trending topics", "niche analysis", "competitive research", "what are other creators doing", "scrape YouTube channels", or wants to understand the competitive landscape for a specific tool or topic area. Use when working with yt research. Trigger with 'yt', 'research'.