autoskill
Observe the user's screen via screenpipe, detect repeated research workflows, match them against existing scientific-agent-skills, and draft new skills (or composition recipes that chain existing ones) for the patterns not yet covered. Use when the user asks to analyze their recent work and propose skills based on what they actually do. Requires the screenpipe daemon (https://github.com/screenpipe/screenpipe) running locally on port 3030 — the skill has no other data source and will refuse to run if screenpipe is unreachable. All detection runs locally; only redacted cluster summaries reach the LLM.
- License 许可
- MIT
- 最近更新
- 2026-06-05
- Stars
- ★ 2
- Forks
- 0
01 前言 Front Matter
- name
- autoskill
- license
- MIT license
- metadata
- { "requires": "screenpipe", "skill-author": "Lord1Egypt" }
- description
- Observe the user's screen via screenpipe, detect repeated research workflows, match them against existing scientific-agent-skills, and draft new skills (or composition recipes that chain existing ones) for the patterns not yet covered. Use when the user asks to analyze their recent work and propose skills based on what they actually do. Requires the screenpipe daemon (https://github.com/screenpipe/screenpipe) running locally on port 3030 — the skill has no other data source and will refuse to run if screenpipe is unreachable. All detection runs locally; only redacted cluster summaries reach the LLM.
- allowed-tools
- Read Write Edit Bash
02 资源文件 27
- 其他.gitignore34 B
- 参考文档config.yaml2.0 KB
- 参考文档references/https-proxy.md1.5 KB
- 参考文档references/screenpipe-config.yaml1.5 KB
- 脚本scripts/autoskill.py1.1 KB
- 脚本scripts/backends.py2.3 KB
- 脚本scripts/cluster.py1.7 KB
- 脚本scripts/doctor.py3.4 KB
- 脚本scripts/fetch_window.py1.1 KB
- 脚本scripts/match_skills.py1.3 KB
- 脚本scripts/promote.py1.5 KB
- 脚本scripts/redact.py1.6 KB
- 脚本scripts/run.py7.2 KB
- 脚本scripts/synthesize.py2.3 KB
- 脚本tests/conftest.py113 B
- 脚本tests/smoke_lmstudio.py2.0 KB
- 脚本tests/test_backends.py4.2 KB
- 脚本tests/test_cli.py1.5 KB
- 脚本tests/test_cluster.py2.7 KB
- 脚本tests/test_doctor.py3.0 KB
- 脚本tests/test_e2e.py11.9 KB
- 脚本tests/test_fetch_window.py3.7 KB
- 脚本tests/test_match_skills.py2.5 KB
- 脚本tests/test_promote.py3.5 KB
- 脚本tests/test_redact.py4.5 KB
- 脚本tests/test_run.py8.0 KB
- 脚本tests/test_synthesize.py3.1 KB
03 SKILL.md 正文
autoskill
Requires a running screenpipe daemon. This skill has no alternate data source — it reads exclusively from the local screenpipe HTTP API (default
http://localhost:3030). If the daemon isn't running,run()raisesScreenpipeUnreachablewith install instructions.
Network access & environment variables. This skill makes authenticated HTTP requests to (a) the user's local screenpipe daemon on loopback, and (b) the user-configured LLM backend — one of
http://localhost:1234/v1(LM Studio, default),https://api.anthropic.com(opt-in Claude), or a user-supplied BYOK Foundry gateway. The skill reads three environment variables —SCREENPIPE_TOKEN,ANTHROPIC_API_KEY,FOUNDRY_API_KEY— and uses each only to authenticate to the single endpoint its name implies. No other network destinations, no telemetry, no data egress to any third party.
Overview
Turn the user's own workflow history — captured passively by the local screenpipe daemon — into new skills. This skill is on-demand: the user invokes it with a time window, it queries screenpipe's local HTTP API, clusters repeated workflow patterns, compares each pattern against the existing skills in this repo, and produces a staged folder of proposals the user can review, edit, and promote.
When to Use This Skill
Invoke this skill when the user asks to:
- "Analyze my last 4 hours / day / week and propose new skills."
- "Look at what I've been doing and tell me what's not covered yet."
- "Draft a skill from my recent workflow."
- "Find composition recipes for workflows I repeat."
Do not invoke it for one-off questions about screenpipe itself, for real-time screen queries, or without an explicit user request — the skill analyzes sensitive local content and must stay explicitly user-triggered.
Privacy Posture
- Screenpipe handles app/window filtering at capture time. Install a starter deny-list by copying
references/screenpipe-config.yamlinto the user's screenpipe config. Sensitive apps (password managers, messaging, banking) are never OCR'd in the first place. - Raw OCR never leaves the machine.
scripts/fetch_window.pypulls data over localhost HTTP.scripts/cluster.pyreduces the timeline to app/duration/title summaries.scripts/redact.pystrips emails, API keys, bearer tokens, and phone numbers as defense-in-depth before any cluster summary reaches the LLM. - LLM backend defaults to
local. The recommended setup is LM Studio runningGemma-4-31B-it— strong reasoning at a size that fits on most workstation GPUs, and no data ever leaves your machine. Cloud backends (claude,foundry) are opt-in and documented inconfig.yamlfor users who explicitly want them. Detection and embeddings always run locally regardless of backend choice. - Dry-run mode (
--plan) prints the exact timeline that will be analyzed before any LLM call. - TLS for localhost (optional, for corporate policy): see
references/https-proxy.mdfor the Caddy pattern.
Prerequisites
1. Screenpipe daemon
Either install the official release or build from source. Either way the daemon binds HTTP on localhost:3030 by default.
From source (recommended if you want the CLI daemon without the desktop GUI):
git clone --depth 1 https://github.com/mediar-ai/screenpipe.git
cd screenpipe
cargo build -p screenpipe-engine --release
# System deps (macOS): cmake + full Xcode.app (not just Command Line Tools).
# brew install cmake
# # if xcodebuild plug-ins error: sudo xcodebuild -runFirstLaunch
./target/release/screenpipe doctor # confirm permissions + ffmpeg
./target/release/screenpipe record --disable-audio --use-pii-removal
First run will prompt for macOS Screen Recording permission. Grant it and relaunch.
2. Screenpipe API token
The local API now requires bearer auth. Retrieve your token and export it:
export SCREENPIPE_TOKEN=$(screenpipe auth token)
(Or set screenpipe.token directly in config.yaml — env var is preferred since it keeps secrets out of version control.)
3. Python environment
Via pipenv from the repo root:
pipenv install httpx pyyaml sentence-transformers
The embedding model (sentence-transformers/all-MiniLM-L6-v2, ~80 MB) downloads on first run.
4. Local LLM (default path) — LM Studio
- Install LM Studio.
- Download
Gemma-4-31B-it(or another strong reasoning model; adjustlocal.modelinconfig.yaml). - Load it via the CLI for headless use (no GUI required):
lms load gemma-4-31b-it --context-length 131072 --gpu max -y
lms status # confirm server running on :1234
5. Cloud LLM backends (optional, opt-in)
Only if you explicitly opt out of local:
claude: setANTHROPIC_API_KEY, flipbackend: claudeinconfig.yaml.foundry: setFOUNDRY_API_KEY, flipbackend: foundry, setfoundry.endpointto your corporate gateway URL.
Architecture
screenpipe daemon (user-installed)
│ HTTP on localhost:3030
▼
scripts/fetch_window.py → normalized timeline events
scripts/redact.py → regex scrub (defense-in-depth)
scripts/cluster.py → sessions + clusters (local only)
scripts/match_skills.py → top-k vs existing 135 skills (local embeddings)
scripts/synthesize.py → LLM judge: reuse / compose / novel
│
▼
~/.autoskill/proposed/<timestamp>/ (default; override with --out)
├── report.md
├── composition-recipes/<name>/SKILL.md
└── new-skills/<name>/SKILL.md
scripts/promote.py → user-approved proposal → scientific-skills/<name>/
Workflow
The skill ships a unified CLI at scripts/autoskill.py with three subcommands:
python scripts/autoskill.py doctor --config config.yaml --skills-dir ../
python scripts/autoskill.py run --start ... --end ... --config config.yaml
python scripts/autoskill.py promote --proposed ~/.autoskill/proposed/<ts> --skills-dir ../ --name <skill>
0. Preflight with doctor
Before a full run, verify every dependency in one shot:
python scripts/autoskill.py doctor \
--config scientific-skills/autoskill/config.yaml \
--skills-dir scientific-skills
The report covers config (backend choice valid), skills_dir (exists), screenpipe (reachable + authed), and llm (LM Studio serving or API key present). Non-zero exit on any failure, with the offending line marked error.
1. Run the pipeline
export SCREENPIPE_TOKEN=$(screenpipe auth token)
python scripts/autoskill.py run \
--start "2026-04-17T00:00:00Z" \
--end "2026-04-17T23:59:59Z" \
--config scientific-skills/autoskill/config.yaml \
--skills-dir scientific-skills
Proposals land in ~/.autoskill/proposed/<timestamp>/ by default, keeping experimental output out of the skills repo. Pass --out PATH to override.
Internally:
- Fetch —
fetch_windowpaginates screenpipe's/searchendpoint, normalizes events to{ts, app, window_title, text, content_type}. - Redact —
redactscrubs emails, API keys, bearer tokens, phones from OCR text and window titles as defense-in-depth over screenpipe's own PII removal. - Cluster —
segment_sessionssplits on idle gaps (default 10 min) and drops short sessions;cluster_sessionsgroups sessions by app-signature and keeps clusters of sizemin_cluster_size(default 2). - Match —
load_skill_descriptionsreads frontmatter from everySKILL.mdinscientific-skills/;top_k_matchesranks each cluster against all skills using localsentence-transformersembeddings (cosine similarity). - Synthesize —
synthesizeprompts the configured LLM backend to classify each cluster asreuse,compose, ornoveland emit a SKILL.md body where appropriate. - Report — writes
<out_dir>/<ts>/report.md, plusnew-skills/<name>/SKILL.mdorcomposition-recipes/<name>/SKILL.mdfor each proposal.
Add --dry-run to stop after clustering; this skips the LLM (and the sentence-transformers load), writing only plan.md for inspection.
2. Review and promote
Open ~/.autoskill/proposed/<ts>/report.md, edit drafts in place, delete anything you don't want. Then:
python scripts/autoskill.py promote \
--proposed ~/.autoskill/proposed/2026-04-17T14-30-00 \
--skills-dir scientific-skills \
--name zotero-pubmed-helper
promote moves the directory into scientific-skills/<name>/, refusing to overwrite an existing skill. Exits non-zero with a friendly error if the proposal isn't found or the target already exists.
Configuration
See config.yaml for the full shape. Default values (local-first):
backend: local
local:
endpoint: http://localhost:1234/v1 # LM Studio's Developer server
model: Gemma-4-31B-it
screenpipe:
url: http://localhost:3030 # or https://screenpipe.local via Caddy
cluster:
min_session_minutes: 5
idle_gap_minutes: 10
min_cluster_size: 2
To opt into a cloud backend:
backend: claude # or foundry
claude:
model: claude-opus-4-7
Composition recipes vs new skills
- compose: the LLM judged that chaining existing skills covers the workflow. The emitted SKILL.md is intentionally thin — frontmatter + a "Workflow" section that invokes existing skills in order. The same agent runtime that discovered the skill can then invoke it end-to-end.
- novel: no combination of existing skills covers it. A fuller SKILL.md is drafted, still following repo conventions (frontmatter, Overview, When to Use, Workflow). The user should always review new-skill drafts before promoting.
Testing
The skill is covered by a small pytest suite at tests/. Each script is unit-tested in isolation with dependency injection (mock HTTP transport, stub backend, stub embedder):
cd scientific-skills/autoskill
python -m pytest tests/ -v
Composition with other skills in this repo
The autoskill's embedding index covers all 135 sibling skills. Workflows that look like scientific writing will match scientific-writing / literature-review / citation-management; figure work will match scientific-schematics / generate-image / infographics; slide prep matches scientific-slides / pptx; etc. When a cluster scores high against two or three sibling skills the emitted composition recipe names them explicitly, so the user's future agent invocations use the optimized paths already documented in this repo.
04 同仓库更多 Skills
zarr-python
Chunked N-D arrays for cloud storage. Compressed arrays, parallel I/O, S3/GCS integration, NumPy/Dask/Xarray compatible, for large-scale scientific computing pipelines.
what-if-oracle
Run structured What-If scenario analysis with multi-branch possibility exploration. Use this skill when the user asks speculative questions like "what if...", "what would happen if...", "what are the possibilities", "explore scenarios", "scenario analysis", "possibility space", "what could go wrong", "best case / worst case", "risk analysis", "contingency planning", "strategic options", or any question about uncertain futures. Also trigger when the user faces a fork-in-the-road decision, wants to stress-test an idea, or needs to think through consequences before committing.
venue-templates
Access comprehensive LaTeX templates, formatting requirements, and submission guidelines for major scientific publication venues (Nature, Science, PLOS, IEEE, ACM), academic conferences (NeurIPS, ICML, CVPR, CHI), research posters, and grant proposals (NSF, NIH, DOE, DARPA). This skill should be used when preparing manuscripts for journal submission, conference papers, research posters, or grant proposals and need venue-specific formatting requirements and templates.
vaex
Use this skill for processing and analyzing large tabular datasets (billions of rows) that exceed available RAM. Vaex excels at out-of-core DataFrame operations, lazy evaluation, fast aggregations, efficient visualization of big data, and machine learning on large datasets. Apply when users need to work with large CSV/HDF5/Arrow/Parquet files, perform fast statistics on massive datasets, create visualizations of big data, or build ML pipelines that do not fit in memory.
usfiscaldata
Query the U.S. Treasury Fiscal Data API for federal financial data including national debt, government spending, revenue, interest rates, exchange rates, and savings bonds. Access 54 datasets and 182 data tables with no API key required. Use when working with U.S. federal fiscal data, national debt tracking (Debt to the Penny), Daily Treasury Statements, Monthly Treasury Statements, Treasury securities auctions, interest rates on Treasury securities, foreign exchange rates, savings bonds, or any U.S. government financial statistics.
treatment-plans
Generate concise (3-4 page), focused medical treatment plans in LaTeX/PDF format for all clinical specialties. Supports general medical treatment, rehabilitation therapy, mental health care, chronic disease management, perioperative care, and pain management. Includes SMART goal frameworks, evidence-based interventions with minimal text citations, regulatory compliance (HIPAA), and professional formatting. Prioritizes brevity and clinical actionability.