[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M71941B9FPMJ0GRCMMBMXEP7":3,"skill-same:01M71941B9FPMJ0GRCMMBMXEP7":79},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":31,"publishedAt":51,"linkOnly":52,"markdownContent":53,"markdownBody":54,"frontMatter":55,"refName":56,"commitSha":57,"skillDirectory":58,"forksCount":59,"repositoryHtmlUrl":60,"repositoryHomepageUrl":61,"ownerHtmlUrl":62,"ownerBlogUrl":61,"ownerDescription":63,"updatedAt":64,"install":65,"resources":78},"01M71941B9FPMJ0GRCMMBMXEP7","capture",null,"Capture web content and save as LLM-friendly markdown. Handles documentation sites (multi-page with search index), articles, blog posts, research papers, and PDFs. Supports custom target folders like ideas\u002F. Triggers on \"capture\", \"scrape\", \"save this article\", \"download docs\", \"grab this post\".","orionfold\u002Frelay","orionfold","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F276500412?v=4","01M7193JQD59YPB87TSHPD82EW","Orionfold","01M7193JR3MMP4G1R2867Y27PD",11,"Apache-2.0","https:\u002F\u002Fgithub.com\u002Forionfold\u002Frelay\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fcapture\u002FSKILL.md",[18,23,26],{"categoryNo":19,"categoryName":20,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000002","文档处理","SCENARIO","使用场景",{"categoryNo":24,"categoryName":25,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000018","写作与文案",{"categoryNo":27,"categoryName":28,"taxonomyType":29,"taxonomyTypeDesc":30},"01JBSEEDCAT000000000000036","内容创作者","PROFESSION","适用职业",[5,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50],"as","multi","with","sites","custom","triggers","multi-page","page","ideas","target","save","markdown","article","documentation","scrape","web content","search","papers","posts","2026-11-10 10:12:30",false,"---\nname: capture\ndescription: Capture web content and save as LLM-friendly markdown. Handles documentation sites (multi-page with search index), articles, blog posts, research papers, and PDFs. Supports custom target folders like ideas\u002F. Triggers on \"capture\", \"scrape\", \"save this article\", \"download docs\", \"grab this post\".\n---\n\n# Content Capture Skill\n\nCapture web content from a URL and save it as clean, LLM-friendly markdown. Supports documentation sites (multi-page), single articles, blog posts, research papers, and PDFs. Optionally saves to a custom target folder.\n\n## Input\n\nThe user provides a URL and an optional target folder:\n\n- `\u002Fcapture \u003Curl>` — auto-detect content type, use default target\n- `\u002Fcapture \u003Curl> ideas\u002F` — save to explicit target folder (positional)\n- `\u002Fcapture \u003Curl> --to ideas\u002F` — save to explicit target folder (flag-style)\n\nParse the URL to extract:\n- **domain**: e.g., `docs.example.com`\n- **path**: e.g., `\u002Fen\u002Fdocs`\n- **site-name**: derive from domain + path, e.g., `docs-example-com-en-docs` (replace `\u002F` and `.` with `-`, strip leading\u002Ftrailing `-`)\n\nDefault target by content type:\n- **Doc sites** → `.Codex\u002Freference\u002F\u003Csite-name>\u002F` (multi-file with search index)\n- **Single content** → `.Codex\u002Freference\u002F\u003Csite-name>\u002F` unless an explicit target folder is specified\n- **Explicit target** (e.g., `ideas\u002F`) → always single-file, flat in that folder\n\n---\n\n## Phase 0: Content Type Detection\n\nBefore choosing a workflow, detect the content type:\n\n| Signal | Content Type |\n|--------|-------------|\n| URL ends in `.pdf` | `pdf` |\n| Known single-content domain (medium.com, substack.com, arxiv.org, x.com, twitter.com, dev.to, hackernoon.com) | `article` |\n| User specified an explicit target folder (e.g., `ideas\u002F`) | `article` (single-file) |\n| URL path contains `\u002Fdocs\u002F`, `\u002Freference\u002F`, or domain starts with `docs.` | `doc-site` |\n| Default: fetch the page, check for 5+ same-domain navigation links | `doc-site` if yes, `article` if no |\n\nOnce detected, route to the appropriate path below.\n\n---\n\n## Doc-Site Path (content type = doc-site)\n\n### Phase 1: LLM-Friendly Discovery\n\nMany documentation sites publish LLM-optimized content. Check for these first — they are cleaner and more complete than scraping.\n\n1. **Check for `llms-full.txt`**: Use WebFetch to fetch `\u003Corigin>\u002Fllms-full.txt`\n   - If found (HTTP 200, non-empty, content looks like documentation not an error page): save it directly to `.Codex\u002Freference\u002F\u003Csite-name>\u002Fllms-full.txt` and create a minimal `index.md` pointing to it. **Done — skip all remaining phases.**\n\n2. **Check for `llms.txt`**: Use WebFetch to fetch `\u003Corigin>\u002Fllms.txt`\n   - If found: read it, parse out any linked resources (URLs ending in `.md`, `.txt`, or other documentation links)\n   - Save `llms.txt` to the reference directory\n   - Fetch each linked resource and save it with its original filename\n   - Create `index.md` listing all files. **Done — skip remaining phases.**\n\n3. **Fallback for WebFetch failures**: If WebFetch returns errors for both, retry using `curl -sL` via the Bash tool with a browser-like User-Agent:\n   ```\n   curl -sL -H \"User-Agent: Mozilla\u002F5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit\u002F537.36\" \"\u003Curl>\"\n   ```\n\nIf neither LLM-friendly file exists, proceed to Phase 2.\n\n---\n\n### Phase 2: HTML Scraping — Navigation Discovery\n\nFetch the main documentation page and extract its structure.\n\n1. **Fetch the page**: Use WebFetch on the provided URL\n   - If WebFetch fails, use curl via Bash\n   - If curl fails (e.g., JS-rendered content), use browser automation as last resort:\n     - Use `mcp__claude-in-chrome__tabs_create_mcp` to open a tab\n     - Use `mcp__claude-in-chrome__navigate` to load the URL\n     - Use `mcp__claude-in-chrome__get_page_text` or `mcp__claude-in-chrome__read_page` to get content\n\n2. **Extract navigation links**: From the fetched HTML\u002Fcontent, identify:\n   - Main navigation menu items (sidebar, top nav, or table of contents)\n   - Go **one level deep** — follow section links to find sub-page links\n   - Collect all unique documentation page URLs (absolute URLs, same domain only)\n   - Filter out: anchors to same page (`#`), external links, asset links (`.css`, `.js`, `.png`, etc.), API\u002Fauth endpoints\n\n3. **Deduplicate and organize**: Remove duplicate URLs. Group by section if the site has clear sections.\n\n4. **Cap at 50 pages maximum** to avoid excessive scraping. If more than 50 links found, prioritize:\n   - Getting started \u002F quickstart guides\n   - Core concepts \u002F fundamentals\n   - API reference\n   - Configuration \u002F setup\n   - Drop changelog, blog posts, release notes\n\n---\n\n### Phase 3: Parallel Content Capture\n\nUse subagents to fetch and convert pages in parallel.\n\n1. **Batch the links**: Split the collected URLs into batches of up to 10 links each.\n\n2. **Spawn up to 5 subagents in parallel** using the Agent tool. Each subagent receives:\n   - A batch of URLs to process\n   - The output directory path\n   - Instructions to follow the per-page process below\n\n3. **Per-page process** (each subagent does this for each URL in its batch):\n\n   a. **Fetch the page** using the fallback chain:\n      - Try WebFetch first\n      - If that fails, try `curl -sL` with browser User-Agent via Bash\n      - If that fails, try browser automation (`mcp__claude-in-chrome__navigate` + `get_page_text`) — note: this is slower and sequential, so batch browser-dependent pages together\n      - If all three fail, skip the page and note it in the output\n\n   b. **Convert HTML to clean markdown**:\n      - Extract the main content area (look for `\u003Cmain>`, `\u003Carticle>`, `[role=\"main\"]`, `.content`, `.docs-content`, or similar)\n      - Remove: navigation bars, headers, footers, sidebars, ads, cookie banners, script\u002Fstyle tags\n      - Convert HTML elements to markdown:\n        - `\u003Ch1>`-`\u003Ch6>` → `#`-`######`\n        - `\u003Cp>` → paragraph with blank lines\n        - `\u003Ca href=\"...\">text\u003C\u002Fa>` → `[text](href)` (make URLs absolute)\n        - `\u003Ccode>`, `\u003Cpre>` → backticks \u002F fenced code blocks (preserve language hints from class names like `language-python`)\n        - `\u003Cul>\u002F\u003Col>` → markdown lists\n        - `\u003Ctable>` → markdown tables\n        - `\u003Cstrong>\u002F\u003Cb>` → `**bold**`\n        - `\u003Cem>\u002F\u003Ci>` → `*italic*`\n        - `\u003Cimg>` → `![alt](src)` (only if alt text exists, skip decorative images)\n        - `\u003Cblockquote>` → `>`\n      - Strip all remaining HTML tags\n      - Collapse multiple blank lines into two\n      - Trim trailing whitespace\n\n   c. **Derive a filename** from the URL path:\n      - Use the last meaningful path segment(s), e.g., `\u002Fdocs\u002Fgetting-started` → `getting-started.md`\n      - If the path is `\u002F` or empty, use `index.md`\n      - Replace special characters with `-`\n      - Ensure `.md` extension\n\n   d. **Save the file** to `.Codex\u002Freference\u002F\u003Csite-name>\u002F\u003Cfilename>`\n\n   e. **Return a summary** to the parent: list of `{filename, title, url, success}` for each page processed\n\n4. **Collect results** from all subagents.\n\n---\n\n### Phase 4: Search Index Generation\n\nGenerate a rich `search-index.md` that enables precise, minimal-context lookups without loading all files.\n\n#### Step 4.1: Analyze each captured file\n\nFor each `.md` file (excluding index files), extract:\n- **H1 title** (first `#` heading)\n- **All H2\u002FH3 headings** (preserve hierarchy)\n- **Line count** (`wc -l`)\n- **First substantive paragraph** (first non-empty, non-heading line after H1)\n- **API names**: scan H3 headings for backtick-wrapped names (e.g., `` ### `functionName()` ``) — these are API elements\n\n#### Step 4.2: Auto-categorize each file\n\nAssign one category based on filename + headings:\n- **Getting Started**: overview, quickstart, getting-started, introduction\n- **Core Concepts**: agent-loop, architecture, concepts, how-it-works\n- **API Reference**: python, typescript, api, reference (or files with >10 backtick-wrapped headings)\n- **Features**: hooks, tools, mcp, streaming, subagents, skills, plugins, sessions, etc.\n- **Configuration**: permissions, config, settings, system-prompts\n- **Operations**: hosting, deployment, security, migration, cost, billing\n\n#### Step 4.3: Build the search-index.md\n\nWrite `.Codex\u002Freference\u002F\u003Csite-name>\u002Fsearch-index.md` with this structure:\n\n```markdown\n# Search Index: \u003CSite Name>\nSource: \u003Curl> | Captured: \u003Cdate> | Files: N | ~XKB total\n\n## Quick Reference\n| File | Category | Summary |\n|------|----------|---------|\n| overview.md | Getting Started | SDK overview, capabilities, architecture |\n| hooks.md | Features | Pre\u002Fpost tool hooks, validation, security |\n...\n\n## Heading Outlines\n### overview.md (575 lines)\n- ## Capabilities\n- ### Codex features\n- ## Get started\n...\n\n### python.md (2396 lines)\n- ## Functions\n  - ### `query()`\n  - ### `tool()`\n- ## Classes\n  - ### `ClaudeSDKClient`\n...\n\n## API Elements\nFunctions: `query()` → python.md, typescript.md | `tool()` → python.md\nClasses: `ClaudeSDKClient` → python.md\nTypes: `HookMatcher` → python.md, hooks.md | `PermissionMode` → permissions.md\n...\n\n## Task Map\n- Install\u002Fsetup → quickstart.md, overview.md\n- Create custom tools → custom-tools.md\n- Intercept tool calls → hooks.md\n...\n```\n\n**Section details:**\n- **Quick Reference** (~30 lines): One row per file. Summary from first paragraph, compressed to \u003C80 chars.\n- **Heading Outlines** (~120 lines): All H2\u002FH3 headings per file, indented to show hierarchy. Include line count.\n- **API Elements**: Group by kind (Functions, Classes, Types, Messages, Errors, Hooks). Map each name to file(s) containing it.\n- **Task Map**: Map common intents\u002Ftasks to 1-2 files. Derive from document titles + heading keywords.\n\n#### Step 4.4: Write backward-compatible index.md\n\nReplace `index.md` with a minimal pointer:\n\n```markdown\n# \u003CSite Name> Documentation Reference\n\nCaptured from: \u003Curl>\nDate: \u003Cdate>\nFiles: N | ~XKB total\n\n**For lookups, use the search index:** [search-index.md](search-index.md)\n\nThe search index provides Quick Reference, Heading Outlines, API Elements cross-reference, and Task Map for efficient topic-based lookups without loading all files into context.\n```\n\n---\n\n## Single-Content Path (content type = article, pdf, or social)\n\n### Article Workflow\n\n1. **Fetch the URL** using the fallback chain:\n   - Try WebFetch first\n   - If that fails, try `curl -sL` with browser User-Agent via Bash\n   - If that fails, use browser automation (mcp__claude-in-chrome__*)\n\n2. **Extract main content** → convert HTML to clean markdown using the same conversion rules as Phase 3 step 3b above.\n\n3. **Extract metadata**: title (`\u003Ch1>`, `og:title`, `\u003Ctitle>`), author (`\u003Cmeta name=\"author\">`, byline), date (`\u003Ctime>`, `article:published_time`), description (`og:description`, `\u003Cmeta name=\"description\">`).\n\n4. **Generate filename**: kebab-case from the title (e.g., `building-effective-agents.md`). Strip articles (\"a\", \"an\", \"the\") from the start. Max 60 chars before `.md`.\n\n5. **Write single markdown file** with YAML frontmatter:\n\n```yaml\n---\ntitle: \"Article Title\"\nsource: https:\u002F\u002Fexample.com\u002Farticle\nauthor: Author Name\ndate_published: 2024-01-15\ndate_captured: 2026-03-08\ncontent_type: article\n---\n```\n\n6. **Target folder determines output structure**:\n   - If target is `.Codex\u002Freference\u002F` (default) → also create `index.md` + `search-index.md` for `\u002Frefer` compatibility\n   - If target is `ideas\u002F` or any custom folder → just the single file, no index files\n\n### PDF Workflow\n\n1. **Download the PDF** via curl to a temp path:\n   ```\n   curl -sL -o \u002Ftmp\u002Fcaptured-doc.pdf \"\u003Curl>\"\n   ```\n\n2. **Read the PDF** using the Read tool (which supports PDF files). For large PDFs, read in page ranges (max 20 pages per request).\n\n3. **Convert to clean markdown**: Structure the extracted text with proper headings, paragraphs, and lists where identifiable.\n\n4. **Write the output file** with YAML frontmatter using `content_type: pdf`:\n\n```yaml\n---\ntitle: \"Paper Title\"\nsource: https:\u002F\u002Fexample.com\u002Fpaper.pdf\nauthor: Author Name\ndate_captured: 2026-03-08\ncontent_type: pdf\n---\n```\n\n5. Same target folder logic as the article workflow.\n\n### Social\u002FShort-Form Workflow\n\n1. **Use browser automation** (JS-rendered content is typical for social platforms):\n   - Use `mcp__claude-in-chrome__tabs_create_mcp` to open a tab\n   - Use `mcp__claude-in-chrome__navigate` to load the URL\n   - Use `mcp__claude-in-chrome__get_page_text` or `mcp__claude-in-chrome__read_page` to get content\n\n2. **Extract**: post content, author\u002Fhandle, date, thread context (if applicable).\n\n3. **Write single markdown file** with same YAML frontmatter format, using `content_type: social`.\n\n---\n\n## Fallback Chain Reference\n\nAll fetch operations use the same three-tier fallback. Choose the right starting tier based on known site patterns:\n\n| Failure Pattern | Symptom | Required Tier |\n|----------------|---------|---------------|\n| Standard HTML sites | WebFetch succeeds | Tier 1: WebFetch |\n| Bot protection (light) | WebFetch returns 403; curl with User-Agent works | Tier 2: curl |\n| Cloudflare challenge | Both WebFetch and curl return challenge HTML (e.g., \"Just a moment...\") | Tier 3: Browser |\n| JS-rendered SPA (Wix, React apps) | WebFetch\u002Fcurl return empty shell or JS bundles, no article content | Tier 3: Browser |\n| Paywall \u002F login wall | All tiers return partial content or login page | Skip + note |\n\n**Known sites requiring browser automation:**\n- `medium.com` \u002F `*.medium.com` — Cloudflare-protected\n- Wix-hosted sites (e.g., `leanware.co`) — JS-rendered SPAs\n- `x.com` \u002F `twitter.com` — JS-rendered social platform\n\n**Performance note:** WebFetch and curl calls can run in parallel. Browser automation is sequential (one tab at a time). When multiple URLs need browser fallback, batch them and process sequentially after all WebFetch\u002Fcurl attempts complete.\n\n---\n\n## Quality Guidelines\n\n- **No HTML artifacts**: The output should be clean markdown with zero HTML tags remaining\n- **Preserve code blocks**: Code examples are high-value content — preserve them exactly, with language annotations\n- **Keep internal links relative**: Convert links between captured pages to relative links (e.g., `.\u002Fother-page.md`)\n- **Preserve YAML frontmatter**: Single-content captures must include complete YAML frontmatter with metadata\n- **Descriptive filenames**: For `ideas\u002F` and custom targets, use descriptive kebab-case filenames derived from content title\n- **PDFs are a handled content type**: Download and extract text from PDFs rather than skipping them as binary\n- **Skip other binary content**: Don't try to download images or other binary assets — just reference them by URL\n- **Respect robots.txt**: If a site clearly blocks scraping, inform the user rather than forcing access\n\n## Error Handling\n\n- If the entire site is unreachable, inform the user with the specific error\n- If some pages fail but others succeed, capture what you can and list failures in `index.md`\n- If content looks like an error page (403, 404, login wall), skip it and note the issue\n- Always produce at least an `index.md` even if most pages fail, documenting what happened\n- If a PDF can't be parsed, save the raw extracted text and note the extraction quality issue in the file\n- If content type detection is ambiguous, default to `article` mode (single-file)\n\n## Example Output Structure\n\n**Doc-site capture** (existing behavior):\n```\n.Codex\u002Freference\u002Fdocs-example-com\u002F\n├── search-index.md          ← Rich search index (Quick Reference, Headings, API Elements, Task Map)\n├── index.md                 ← Minimal pointer to search-index.md\n├── getting-started.md\n├── configuration.md\n├── api-reference.md\n├── authentication.md\n└── deployment.md\n```\n\n**Single article to custom folder**:\n```\nideas\u002F\n└── building-effective-agents.md    ← Single file with YAML frontmatter\n```\n\n**Single article to default target**:\n```\n.Codex\u002Freference\u002Fanthropic-building-effective-agents\u002F\n├── search-index.md                 ← Search index for \u002Frefer compatibility\n├── index.md                        ← Minimal pointer to search-index.md\n└── building-effective-agents.md    ← Article with YAML frontmatter\n```\n\n**PDF capture**:\n```\nideas\u002F\n└── attention-is-all-you-need.md    ← Extracted PDF text with YAML frontmatter\n```\n","# Content Capture Skill\n\nCapture web content from a URL and save it as clean, LLM-friendly markdown. Supports documentation sites (multi-page), single articles, blog posts, research papers, and PDFs. Optionally saves to a custom target folder.\n\n## Input\n\nThe user provides a URL and an optional target folder:\n\n- `\u002Fcapture \u003Curl>` — auto-detect content type, use default target\n- `\u002Fcapture \u003Curl> ideas\u002F` — save to explicit target folder (positional)\n- `\u002Fcapture \u003Curl> --to ideas\u002F` — save to explicit target folder (flag-style)\n\nParse the URL to extract:\n- **domain**: e.g., `docs.example.com`\n- **path**: e.g., `\u002Fen\u002Fdocs`\n- **site-name**: derive from domain + path, e.g., `docs-example-com-en-docs` (replace `\u002F` and `.` with `-`, strip leading\u002Ftrailing `-`)\n\nDefault target by content type:\n- **Doc sites** → `.Codex\u002Freference\u002F\u003Csite-name>\u002F` (multi-file with search index)\n- **Single content** → `.Codex\u002Freference\u002F\u003Csite-name>\u002F` unless an explicit target folder is specified\n- **Explicit target** (e.g., `ideas\u002F`) → always single-file, flat in that folder\n\n---\n\n## Phase 0: Content Type Detection\n\nBefore choosing a workflow, detect the content type:\n\n| Signal | Content Type |\n|--------|-------------|\n| URL ends in `.pdf` | `pdf` |\n| Known single-content domain (medium.com, substack.com, arxiv.org, x.com, twitter.com, dev.to, hackernoon.com) | `article` |\n| User specified an explicit target folder (e.g., `ideas\u002F`) | `article` (single-file) |\n| URL path contains `\u002Fdocs\u002F`, `\u002Freference\u002F`, or domain starts with `docs.` | `doc-site` |\n| Default: fetch the page, check for 5+ same-domain navigation links | `doc-site` if yes, `article` if no |\n\nOnce detected, route to the appropriate path below.\n\n---\n\n## Doc-Site Path (content type = doc-site)\n\n### Phase 1: LLM-Friendly Discovery\n\nMany documentation sites publish LLM-optimized content. Check for these first — they are cleaner and more complete than scraping.\n\n1. **Check for `llms-full.txt`**: Use WebFetch to fetch `\u003Corigin>\u002Fllms-full.txt`\n   - If found (HTTP 200, non-empty, content looks like documentation not an error page): save it directly to `.Codex\u002Freference\u002F\u003Csite-name>\u002Fllms-full.txt` and create a minimal `index.md` pointing to it. **Done — skip all remaining phases.**\n\n2. **Check for `llms.txt`**: Use WebFetch to fetch `\u003Corigin>\u002Fllms.txt`\n   - If found: read it, parse out any linked resources (URLs ending in `.md`, `.txt`, or other documentation links)\n   - Save `llms.txt` to the reference directory\n   - Fetch each linked resource and save it with its original filename\n   - Create `index.md` listing all files. **Done — skip remaining phases.**\n\n3. **Fallback for WebFetch failures**: If WebFetch returns errors for both, retry using `curl -sL` via the Bash tool with a browser-like User-Agent:\n   ```\n   curl -sL -H \"User-Agent: Mozilla\u002F5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit\u002F537.36\" \"\u003Curl>\"\n   ```\n\nIf neither LLM-friendly file exists, proceed to Phase 2.\n\n---\n\n### Phase 2: HTML Scraping — Navigation Discovery\n\nFetch the main documentation page and extract its structure.\n\n1. **Fetch the page**: Use WebFetch on the provided URL\n   - If WebFetch fails, use curl via Bash\n   - If curl fails (e.g., JS-rendered content), use browser automation as last resort:\n     - Use `mcp__claude-in-chrome__tabs_create_mcp` to open a tab\n     - Use `mcp__claude-in-chrome__navigate` to load the URL\n     - Use `mcp__claude-in-chrome__get_page_text` or `mcp__claude-in-chrome__read_page` to get content\n\n2. **Extract navigation links**: From the fetched HTML\u002Fcontent, identify:\n   - Main navigation menu items (sidebar, top nav, or table of contents)\n   - Go **one level deep** — follow section links to find sub-page links\n   - Collect all unique documentation page URLs (absolute URLs, same domain only)\n   - Filter out: anchors to same page (`#`), external links, asset links (`.css`, `.js`, `.png`, etc.), API\u002Fauth endpoints\n\n3. **Deduplicate and organize**: Remove duplicate URLs. Group by section if the site has clear sections.\n\n4. **Cap at 50 pages maximum** to avoid excessive scraping. If more than 50 links found, prioritize:\n   - Getting started \u002F quickstart guides\n   - Core concepts \u002F fundamentals\n   - API reference\n   - Configuration \u002F setup\n   - Drop changelog, blog posts, release notes\n\n---\n\n### Phase 3: Parallel Content Capture\n\nUse subagents to fetch and convert pages in parallel.\n\n1. **Batch the links**: Split the collected URLs into batches of up to 10 links each.\n\n2. **Spawn up to 5 subagents in parallel** using the Agent tool. Each subagent receives:\n   - A batch of URLs to process\n   - The output directory path\n   - Instructions to follow the per-page process below\n\n3. **Per-page process** (each subagent does this for each URL in its batch):\n\n   a. **Fetch the page** using the fallback chain:\n      - Try WebFetch first\n      - If that fails, try `curl -sL` with browser User-Agent via Bash\n      - If that fails, try browser automation (`mcp__claude-in-chrome__navigate` + `get_page_text`) — note: this is slower and sequential, so batch browser-dependent pages together\n      - If all three fail, skip the page and note it in the output\n\n   b. **Convert HTML to clean markdown**:\n      - Extract the main content area (look for `\u003Cmain>`, `\u003Carticle>`, `[role=\"main\"]`, `.content`, `.docs-content`, or similar)\n      - Remove: navigation bars, headers, footers, sidebars, ads, cookie banners, script\u002Fstyle tags\n      - Convert HTML elements to markdown:\n        - `\u003Ch1>`-`\u003Ch6>` → `#`-`######`\n        - `\u003Cp>` → paragraph with blank lines\n        - `\u003Ca href=\"...\">text\u003C\u002Fa>` → `[text](href)` (make URLs absolute)\n        - `\u003Ccode>`, `\u003Cpre>` → backticks \u002F fenced code blocks (preserve language hints from class names like `language-python`)\n        - `\u003Cul>\u002F\u003Col>` → markdown lists\n        - `\u003Ctable>` → markdown tables\n        - `\u003Cstrong>\u002F\u003Cb>` → `**bold**`\n        - `\u003Cem>\u002F\u003Ci>` → `*italic*`\n        - `\u003Cimg>` → `![alt](src)` (only if alt text exists, skip decorative images)\n        - `\u003Cblockquote>` → `>`\n      - Strip all remaining HTML tags\n      - Collapse multiple blank lines into two\n      - Trim trailing whitespace\n\n   c. **Derive a filename** from the URL path:\n      - Use the last meaningful path segment(s), e.g., `\u002Fdocs\u002Fgetting-started` → `getting-started.md`\n      - If the path is `\u002F` or empty, use `index.md`\n      - Replace special characters with `-`\n      - Ensure `.md` extension\n\n   d. **Save the file** to `.Codex\u002Freference\u002F\u003Csite-name>\u002F\u003Cfilename>`\n\n   e. **Return a summary** to the parent: list of `{filename, title, url, success}` for each page processed\n\n4. **Collect results** from all subagents.\n\n---\n\n### Phase 4: Search Index Generation\n\nGenerate a rich `search-index.md` that enables precise, minimal-context lookups without loading all files.\n\n#### Step 4.1: Analyze each captured file\n\nFor each `.md` file (excluding index files), extract:\n- **H1 title** (first `#` heading)\n- **All H2\u002FH3 headings** (preserve hierarchy)\n- **Line count** (`wc -l`)\n- **First substantive paragraph** (first non-empty, non-heading line after H1)\n- **API names**: scan H3 headings for backtick-wrapped names (e.g., `` ### `functionName()` ``) — these are API elements\n\n#### Step 4.2: Auto-categorize each file\n\nAssign one category based on filename + headings:\n- **Getting Started**: overview, quickstart, getting-started, introduction\n- **Core Concepts**: agent-loop, architecture, concepts, how-it-works\n- **API Reference**: python, typescript, api, reference (or files with >10 backtick-wrapped headings)\n- **Features**: hooks, tools, mcp, streaming, subagents, skills, plugins, sessions, etc.\n- **Configuration**: permissions, config, settings, system-prompts\n- **Operations**: hosting, deployment, security, migration, cost, billing\n\n#### Step 4.3: Build the search-index.md\n\nWrite `.Codex\u002Freference\u002F\u003Csite-name>\u002Fsearch-index.md` with this structure:\n\n```markdown\n# Search Index: \u003CSite Name>\nSource: \u003Curl> | Captured: \u003Cdate> | Files: N | ~XKB total\n\n## Quick Reference\n| File | Category | Summary |\n|------|----------|---------|\n| overview.md | Getting Started | SDK overview, capabilities, architecture |\n| hooks.md | Features | Pre\u002Fpost tool hooks, validation, security |\n...\n\n## Heading Outlines\n### overview.md (575 lines)\n- ## Capabilities\n- ### Codex features\n- ## Get started\n...\n\n### python.md (2396 lines)\n- ## Functions\n  - ### `query()`\n  - ### `tool()`\n- ## Classes\n  - ### `ClaudeSDKClient`\n...\n\n## API Elements\nFunctions: `query()` → python.md, typescript.md | `tool()` → python.md\nClasses: `ClaudeSDKClient` → python.md\nTypes: `HookMatcher` → python.md, hooks.md | `PermissionMode` → permissions.md\n...\n\n## Task Map\n- Install\u002Fsetup → quickstart.md, overview.md\n- Create custom tools → custom-tools.md\n- Intercept tool calls → hooks.md\n...\n```\n\n**Section details:**\n- **Quick Reference** (~30 lines): One row per file. Summary from first paragraph, compressed to \u003C80 chars.\n- **Heading Outlines** (~120 lines): All H2\u002FH3 headings per file, indented to show hierarchy. Include line count.\n- **API Elements**: Group by kind (Functions, Classes, Types, Messages, Errors, Hooks). Map each name to file(s) containing it.\n- **Task Map**: Map common intents\u002Ftasks to 1-2 files. Derive from document titles + heading keywords.\n\n#### Step 4.4: Write backward-compatible index.md\n\nReplace `index.md` with a minimal pointer:\n\n```markdown\n# \u003CSite Name> Documentation Reference\n\nCaptured from: \u003Curl>\nDate: \u003Cdate>\nFiles: N | ~XKB total\n\n**For lookups, use the search index:** [search-index.md](search-index.md)\n\nThe search index provides Quick Reference, Heading Outlines, API Elements cross-reference, and Task Map for efficient topic-based lookups without loading all files into context.\n```\n\n---\n\n## Single-Content Path (content type = article, pdf, or social)\n\n### Article Workflow\n\n1. **Fetch the URL** using the fallback chain:\n   - Try WebFetch first\n   - If that fails, try `curl -sL` with browser User-Agent via Bash\n   - If that fails, use browser automation (mcp__claude-in-chrome__*)\n\n2. **Extract main content** → convert HTML to clean markdown using the same conversion rules as Phase 3 step 3b above.\n\n3. **Extract metadata**: title (`\u003Ch1>`, `og:title`, `\u003Ctitle>`), author (`\u003Cmeta name=\"author\">`, byline), date (`\u003Ctime>`, `article:published_time`), description (`og:description`, `\u003Cmeta name=\"description\">`).\n\n4. **Generate filename**: kebab-case from the title (e.g., `building-effective-agents.md`). Strip articles (\"a\", \"an\", \"the\") from the start. Max 60 chars before `.md`.\n\n5. **Write single markdown file** with YAML frontmatter:\n\n```yaml\n---\ntitle: \"Article Title\"\nsource: https:\u002F\u002Fexample.com\u002Farticle\nauthor: Author Name\ndate_published: 2024-01-15\ndate_captured: 2026-03-08\ncontent_type: article\n---\n```\n\n6. **Target folder determines output structure**:\n   - If target is `.Codex\u002Freference\u002F` (default) → also create `index.md` + `search-index.md` for `\u002Frefer` compatibility\n   - If target is `ideas\u002F` or any custom folder → just the single file, no index files\n\n### PDF Workflow\n\n1. **Download the PDF** via curl to a temp path:\n   ```\n   curl -sL -o \u002Ftmp\u002Fcaptured-doc.pdf \"\u003Curl>\"\n   ```\n\n2. **Read the PDF** using the Read tool (which supports PDF files). For large PDFs, read in page ranges (max 20 pages per request).\n\n3. **Convert to clean markdown**: Structure the extracted text with proper headings, paragraphs, and lists where identifiable.\n\n4. **Write the output file** with YAML frontmatter using `content_type: pdf`:\n\n```yaml\n---\ntitle: \"Paper Title\"\nsource: https:\u002F\u002Fexample.com\u002Fpaper.pdf\nauthor: Author Name\ndate_captured: 2026-03-08\ncontent_type: pdf\n---\n```\n\n5. Same target folder logic as the article workflow.\n\n### Social\u002FShort-Form Workflow\n\n1. **Use browser automation** (JS-rendered content is typical for social platforms):\n   - Use `mcp__claude-in-chrome__tabs_create_mcp` to open a tab\n   - Use `mcp__claude-in-chrome__navigate` to load the URL\n   - Use `mcp__claude-in-chrome__get_page_text` or `mcp__claude-in-chrome__read_page` to get content\n\n2. **Extract**: post content, author\u002Fhandle, date, thread context (if applicable).\n\n3. **Write single markdown file** with same YAML frontmatter format, using `content_type: social`.\n\n---\n\n## Fallback Chain Reference\n\nAll fetch operations use the same three-tier fallback. Choose the right starting tier based on known site patterns:\n\n| Failure Pattern | Symptom | Required Tier |\n|----------------|---------|---------------|\n| Standard HTML sites | WebFetch succeeds | Tier 1: WebFetch |\n| Bot protection (light) | WebFetch returns 403; curl with User-Agent works | Tier 2: curl |\n| Cloudflare challenge | Both WebFetch and curl return challenge HTML (e.g., \"Just a moment...\") | Tier 3: Browser |\n| JS-rendered SPA (Wix, React apps) | WebFetch\u002Fcurl return empty shell or JS bundles, no article content | Tier 3: Browser |\n| Paywall \u002F login wall | All tiers return partial content or login page | Skip + note |\n\n**Known sites requiring browser automation:**\n- `medium.com` \u002F `*.medium.com` — Cloudflare-protected\n- Wix-hosted sites (e.g., `leanware.co`) — JS-rendered SPAs\n- `x.com` \u002F `twitter.com` — JS-rendered social platform\n\n**Performance note:** WebFetch and curl calls can run in parallel. Browser automation is sequential (one tab at a time). When multiple URLs need browser fallback, batch them and process sequentially after all WebFetch\u002Fcurl attempts complete.\n\n---\n\n## Quality Guidelines\n\n- **No HTML artifacts**: The output should be clean markdown with zero HTML tags remaining\n- **Preserve code blocks**: Code examples are high-value content — preserve them exactly, with language annotations\n- **Keep internal links relative**: Convert links between captured pages to relative links (e.g., `.\u002Fother-page.md`)\n- **Preserve YAML frontmatter**: Single-content captures must include complete YAML frontmatter with metadata\n- **Descriptive filenames**: For `ideas\u002F` and custom targets, use descriptive kebab-case filenames derived from content title\n- **PDFs are a handled content type**: Download and extract text from PDFs rather than skipping them as binary\n- **Skip other binary content**: Don't try to download images or other binary assets — just reference them by URL\n- **Respect robots.txt**: If a site clearly blocks scraping, inform the user rather than forcing access\n\n## Error Handling\n\n- If the entire site is unreachable, inform the user with the specific error\n- If some pages fail but others succeed, capture what you can and list failures in `index.md`\n- If content looks like an error page (403, 404, login wall), skip it and note the issue\n- Always produce at least an `index.md` even if most pages fail, documenting what happened\n- If a PDF can't be parsed, save the raw extracted text and note the extraction quality issue in the file\n- If content type detection is ambiguous, default to `article` mode (single-file)\n\n## Example Output Structure\n\n**Doc-site capture** (existing behavior):\n```\n.Codex\u002Freference\u002Fdocs-example-com\u002F\n├── search-index.md          ← Rich search index (Quick Reference, Headings, API Elements, Task Map)\n├── index.md                 ← Minimal pointer to search-index.md\n├── getting-started.md\n├── configuration.md\n├── api-reference.md\n├── authentication.md\n└── deployment.md\n```\n\n**Single article to custom folder**:\n```\nideas\u002F\n└── building-effective-agents.md    ← Single file with YAML frontmatter\n```\n\n**Single article to default target**:\n```\n.Codex\u002Freference\u002Fanthropic-building-effective-agents\u002F\n├── search-index.md                 ← Search index for \u002Frefer compatibility\n├── index.md                        ← Minimal pointer to search-index.md\n└── building-effective-agents.md    ← Article with YAML frontmatter\n```\n\n**PDF capture**:\n```\nideas\u002F\n└── attention-is-all-you-need.md    ← Extracted PDF text with YAML frontmatter\n```\n",{"name":5,"description":7},"main","b80e0240f42f84cd950fe8361a11a10699aa7984",".agents\u002Fskills\u002Fcapture",3,"https:\u002F\u002Fgithub.com\u002Forionfold\u002Frelay","https:\u002F\u002Forionfold.com","https:\u002F\u002Fgithub.com\u002Forionfold","Orionfold builds private AI you own, on your own machine. Get an AI team without hiring one.","2026-09-04 21:49:30",{"source":8,"skillName":5,"commands":66,"prompt":76,"zipAvailable":77},[67,70,73],{"tool":68,"command":69},"npx","npx skills add orionfold\u002Frelay --skill capture",{"tool":71,"command":72},"bunx","bunx skills add orionfold\u002Frelay --skill capture",{"tool":74,"command":75},"pnpm","pnpm dlx skills add orionfold\u002Frelay --skill capture","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Forionfold\u002Frelay\u002Fblob\u002Fmain\u002F.agents\u002Fskills\u002Fcapture\u002FSKILL.md",true,[],[80,99,114,133,149,171],{"skillNo":81,"name":82,"nameZh":6,"description":83,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":84,"categories":85,"keywords":86,"publishedAt":51},"01M719952CZE2BR6GGER42MG51","reader-coach","Helps notice what was read this week. Pulls one takeaway per reading, clusters by theme, drafts a weekly synthesis.","https:\u002F\u002Fgithub.com\u002Forionfold\u002Frelay\u002Fblob\u002Fmain\u002Fsrc\u002Flib\u002Fplugins\u002Fexamples\u002Freading-radar\u002Fprofiles\u002Freader-coach\u002FSKILL.md",[],[87,88,89,90,91,92,93,94,95,96,97,98],"reader","reader coach","coach","Synthesis","Clusters","weekly synthesis","reading","drafts","read","theme","notice","weekly",{"skillNo":100,"name":101,"nameZh":6,"description":102,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":103,"categories":104,"keywords":105,"publishedAt":51},"01M71993PCY50PQMQW3C3SMMN6","personal-cfo","A patient personal CFO who categorizes transactions, flags unusual spend, and prepares monthly summaries.","https:\u002F\u002Fgithub.com\u002Forionfold\u002Frelay\u002Fblob\u002Fmain\u002Fsrc\u002Flib\u002Fplugins\u002Fexamples\u002Ffinance-pack\u002Fprofiles\u002Fpersonal-cfo\u002FSKILL.md",[],[106,107,108,109,110,111,112,113],"personal cfo","personal","cfo","flags","transactions","monthly","summaries","patient",{"skillNo":115,"name":116,"nameZh":6,"description":117,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":118,"categories":119,"keywords":120,"publishedAt":51},"01M71991XD74YK2BY4A3A6J361","relay-web-publisher--site-editor","Drafts structured website section rows for the static-site publisher","https:\u002F\u002Fgithub.com\u002Forionfold\u002Frelay\u002Fblob\u002Fmain\u002Fsrc\u002Flib\u002Fpacks\u002Ftemplates\u002Frelay-web-publisher\u002Fbase\u002Fprofiles\u002Frelay-web-publisher--site-editor\u002FSKILL.md",[],[121,122,123,124,125,126,127,128,129,94,130,131,132],"web","editor","relay","relay web publisher site editor","site","publisher","for","static","section","static-site","structured","website",{"skillNo":134,"name":135,"nameZh":6,"description":136,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":137,"categories":138,"keywords":139,"publishedAt":51},"01M719906C2NYTTVJQBJJ1F011","relay-web-assets--asset-curator","Curates web-design assets into a clean gallery inventory","https:\u002F\u002Fgithub.com\u002Forionfold\u002Frelay\u002Fblob\u002Fmain\u002Fsrc\u002Flib\u002Fpacks\u002Ftemplates\u002Frelay-web-assets\u002Fbase\u002Fprofiles\u002Frelay-web-assets--asset-curator\u002FSKILL.md",[],[140,121,123,141,142,143,144,145,146,147,148],"relay web assets asset curator","assets","asset","design assets","web-design","inventory","gallery","clean","design",{"skillNo":150,"name":151,"nameZh":6,"description":152,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":153,"categories":154,"keywords":156,"publishedAt":51},"01M7198Y634C0NJ2CKTSSTVY9R","relay-social--content-studio","The content inventory and repurpose planner — tracks source assets, decides what to repurpose next, and writes the channel brief for each creative","https:\u002F\u002Fgithub.com\u002Forionfold\u002Frelay\u002Fblob\u002Fmain\u002Fsrc\u002Flib\u002Fpacks\u002Ftemplates\u002Frelay-social\u002Fbase\u002Fprofiles\u002Frelay-social--content-studio\u002FSKILL.md",[155],{"categoryNo":27,"categoryName":28,"taxonomyType":29,"taxonomyTypeDesc":30},[157,158,123,159,160,161,162,127,163,164,165,166,167,145,168,141,169,170],"studio","content","social","social content","relay social content studio","content studio","planner","brief","tracks","next","repurpose","creative","source","channel",{"skillNo":172,"name":173,"nameZh":6,"description":174,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":175,"categories":176,"keywords":183,"publishedAt":51},"01M7198VQ1YYN3PFQH9Y7ZZ6SK","relay-social--campaign-runner","The organic-first campaign engine — composes, schedules, and measures a campaign from plan to published to results, setting the UTM that leads attribute back to","https:\u002F\u002Fgithub.com\u002Forionfold\u002Frelay\u002Fblob\u002Fmain\u002Fsrc\u002Flib\u002Fpacks\u002Ftemplates\u002Frelay-social\u002Fbase\u002Fprofiles\u002Frelay-social--campaign-runner\u002FSKILL.md",[177,180],{"categoryNo":178,"categoryName":179,"taxonomyType":21,"taxonomyTypeDesc":22},"01JBSEEDCAT000000000000019","营销推广",{"categoryNo":181,"categoryName":182,"taxonomyType":29,"taxonomyTypeDesc":30},"01JBSEEDCAT000000000000035","市场营销",[184,185,186,159,187,123,188,189,190,191,192,193,194,195,196],"relay social campaign runner","runner","campaign","social campaign","back","engine","leads","plan","organic","first","results","schedules","utm"]