scope-api
Query and control the Scope platform API for analyzing benchmark data, managing runs/requests, skills, agents, reports, and insights. Use when the user asks about SCOPE data, wants to trigger runs, check status, generate reports, or interact with the SCOPE platform programmatically.
- License 许可
- MIT
- 所属仓库
- microsoft/scope
- 最近更新
- 2026-11-12
- Stars
- ★ 84
- Forks
- 32
01 前言 Front Matter
- name
- scope-api
- description
- Query and control the Scope platform API for analyzing benchmark data, managing runs/requests, skills, agents, reports, and insights. Use when the user asks about SCOPE data, wants to trigger runs, check status, generate reports, or interact with the SCOPE platform programmatically.
03 SKILL.md 正文
Scope API Skill
The Scope platform benchmarks AI coding agents. This skill explains how to query and control it.
Base URL
http://localhost:${API_PORT}
The API_PORT variable is defined in the project .env file (default: 3116). Always read it from the environment or .env before making calls.
Authentication
For authenticated calls, pass the existing IdP access token unchanged:
-H "Authorization: Bearer $SCOPE_TOKEN"
Use the caller-provided SCOPE_TOKEN when available; never print/log the token or
put it in a URL. There is no Scope session JWT, /auth/login, or token exchange.
The API verifies signature/claims before consulting its user-access cache.
Enrollment is explicit. Already-enrolled bearer callers remain compatible. For a new identity, intentionally call this once before other authenticated calls:
curl --fail-with-body -sS \
-X POST \
-H "Authorization: Bearer $SCOPE_TOKEN" \
-H "Cache-Control: no-store" \
"http://localhost:$API_PORT/api/v1/users/me"
Only POST /api/v1/users/me may JIT-create the user or update profile,
lastLoginAt, and eligible bootstrap-admin promotion. This POST has side effects:
never prefetch or poll it, and do not silently invoke it as an ordinary lookup
retry. lastLoginAt records this explicit upsert, not proof of an interactive login.
For normal identity checks use GET /api/v1/users/me; Invalid/repeated/structured
login values return 400; HEAD never enrolls.
Plain /me and other authenticated routes resolve an existing active Scope UUID/role
from Redis; miss/unavailability reads Mongo by exact (idp, tid, oid) and warms the
cache, without JIT. Missing users return 403 user_not_enrolled, disabled users
403 user_disabled, reserved/invalid principals 401; required Mongo/JWKS outages
return 503, unexpected errors 500. Do not interpret a cache miss as a denial or
turn a verified-identity denial into anonymous access.
The fixed/non-sliding cache TTL (AUTH_USER_CACHE_TTL_SECONDS, default 300 seconds)
means DB-only role/disable changes may not be visible until expiry. Public endpoints
and existing no-token/auth-disabled anonymous rollout remain supported; this is not
full route RBAC. See the auth contract.
Before Making API Calls
Always fetch the OpenAPI spec first to understand available endpoints, required parameters, and response schemas:
curl -s http://localhost:$API_PORT/openapi.json
Parse it with jq to find the relevant endpoint and its schema before constructing your API call. For example:
# Find endpoints matching a keyword
curl -s http://localhost:$API_PORT/openapi.json | jq '.paths | keys[] | select(contains("requests"))'
# Get schema for a specific endpoint
curl -s http://localhost:$API_PORT/openapi.json | jq '.paths["/api/v1/requests"]'
Key Endpoint Groups
- System:
/health,/ready,/about,/api/v1/version - Identity: GET
/api/v1/users/me(read-only); POST/api/v1/users/me(explicit enrollment/login refresh) - Requests & Runs:
/api/v1/requests/*(create, cancel, retry, pause, resume, bulk ops, logs, HAR, video, snapshots, tool-calls) - Skills:
/api/v1/skills/*(discover, search, external, resolve, revisions) - Agents:
/api/v1/agents/*(CRUD, versions) - Reports & Insights:
/api/v1/reports/*,/api/v1/insights/*(trigger, bulk, upvote/downvote) - Criteria & Prompt Features:
/api/v1/criteria/*,/api/v1/prompt-features/*(MDP, graph, seed, generate, extract) - Task Prompts:
/api/v1/task-prompts/*(generate, extract features) - Models:
/api/v1/models/*(list, sync) - Feature Flags:
/api/v1/feature-flags
Usage Pattern
When the user asks about SCOPE data or wants to control SCOPE:
- Fetch the OpenAPI spec from
http://localhost:$API_PORT/openapi.json - Find the relevant endpoint and inspect its parameters/schemas
- Make the API call with
curland parse results withjq - For complex queries, combine multiple API calls
Example Workflow
# 1. Fetch spec and find the endpoint you need
curl -s http://localhost:$API_PORT/openapi.json | jq '.paths["/api/v1/requests"].get'
# 2. Make the call
curl -s http://localhost:$API_PORT/api/v1/requests | jq '.[:5]'
# 3. Drill into details
curl -s http://localhost:$API_PORT/api/v1/requests/{id} | jq .
04 同仓库更多 Skills
github-actions
Use when creating, editing, reviewing, or troubleshooting GitHub Actions workflows in Scope.
ax-practitioner
This skill should be used when the user asks about AX (agent experience) or the AX Practitioner Playbook, for example "how do I evaluate my technology's agent experience", "write AX eval criteria", "design an eval scenario", "propensity vs efficacy", "lift and drag", "which agent profiles should I test", "how many eval runs", "how do I read an agent trajectory", "why does the agent ignore my MCP server or skill", "agent uses an outdated SDK version", "too many extensions", "how to read an AX readout", "how to get the owning team to fix the docs", "review my eval against the checklist", or the SPFx worked example. It answers interactively and strictly from the playbook, citing chapters. When the playbook has no answer, it says so and clearly labels any general-knowledge or web content.
typescript
TypeScript best practices for this monorepo. Use this skill when: (1) writing new TypeScript code, (2) reviewing TypeScript code, (3) working with API responses, fetch calls, or JSON parsing, (4) adding or modifying types and interfaces.
storybook
Maintain the Storybook component catalog for the portal app. Use this skill when: (1) adding new stories for components, (2) updating existing stories after component changes, (3) running a full Storybook refresh/sync with AI assistance.
scope-cli
Interact with the Scope benchmarking platform using the Scope CLI — submit runs, list results, manage criteria, generate reports, and administer agents, skills, and MCP servers.
rust-best-practices
Guide for writing idiomatic Rust code based on Apollo GraphQL's best practices handbook. Use this skill when: (1) writing new Rust code or functions, (2) reviewing or refactoring existing Rust code, (3) deciding between borrowing vs cloning or ownership patterns, (4) implementing error handling with Result types, (5) optimizing Rust code for performance, (6) writing tests or documentation for Rust projects.