brave-headless
Headless web search and content extraction via the Brave Search API. Features exponential-backoff retry, circuit breaker fault isolation, bounded-concurrency parallel page fetching, structured leveled logging, and smart paragraph-boundary truncation. No browser required. Use for web research, documentation lookup, URL content extraction, and any workflow requiring scriptable, non-interactive web search.
- License 许可
- MIT
- 最近更新
- 2026-06-07
- Stars
- ★ 0
- Forks
- 1
01 前言 Front Matter
- name
- brave-headless
- license
- MIT
- version
- 0.2.0
- metadata
- { "author": "kelexine", "homepage": "https://github.com/kelexine/brave-headless", "openclaw": { "os": [ "macos", "linux" ], "emoji": "🔍", "install": [ { "bins": [], "kind": "node", "package": "@mozilla/readability" }, { "bins": [], "kind": "node", "package": "jsdom" }, { "bins": [], "kind": "node", "package": "turndown" }, { "bins": [], "kind": "node", "package": "turndown-plugin-gfm" } ], "homepage": "https://github.com/kelexine/brave-headless", "requires": { "env": [ "BRAVE_API_KEY" ], "bins": [ "node", "npm" ] }, "primaryEnv": "BRAVE_API_KEY" } }
- description
- Headless web search and content extraction via the Brave Search API. Features exponential-backoff retry, circuit breaker fault isolation, bounded-concurrency parallel page fetching, structured leveled logging, and smart paragraph-boundary truncation. No browser required. Use for web research, documentation lookup, URL content extraction, and any workflow requiring scriptable, non-interactive web search.
- compatibility
- Requires Node.js >= 20 and npm. Works on macOS and Linux.
03 SKILL.md 正文
brave-search
Headless web search and content extraction via the Brave Search API.
Setup
Run once before first use:
cd <skill-root>
npm ci
Required environment variable:
export BRAVE_API_KEY="your-key-here"
Get a free API key at brave.com/search/api.
Usage
Search
node scripts/search.js "query" # Basic (5 results)
node scripts/search.js "query" -n 10 # Up to 20 results
node scripts/search.js "query" --content # Include page content
node scripts/search.js "query" -n 3 --content # Combined
node scripts/search.js "query" --json # Newline-delimited JSON
node scripts/search.js --help # Full options + env vars
Extract page content
node scripts/content.js https://example.com/article
node scripts/content.js https://example.com/article --json
node scripts/content.js https://example.com/article --max-length 8000
Output format (plain text)
--- Result 1 ---
Title: Page Title
URL: https://example.com/page
Snippet: Description from Brave Search
Content:
# Page Title
Extracted markdown content...
--- Result 2 ---
...
Pass --json to get one JSON object per line instead, suitable for piping.
Exit codes
| Code | Meaning |
|---|---|
0 |
Success |
1 |
Invalid input or configuration error |
2 |
Page had no extractable content (content.js) |
130 |
Interrupted (SIGINT) |
Configuration (environment variables)
All behaviour is configurable without touching code:
| Variable | Default | Description |
|---|---|---|
BRAVE_API_KEY |
— | Required. Brave Search subscription token |
LOG_LEVEL |
info |
debug · info · warn · error · silent |
LOG_JSON |
false |
Emit logs as newline-delimited JSON to stderr |
FETCH_TIMEOUT_MS |
15000 |
Per-page fetch timeout |
SEARCH_TIMEOUT_MS |
10000 |
Brave API call timeout |
MAX_CONTENT_LENGTH |
5000 |
Max chars of extracted content |
MAX_RETRY_ATTEMPTS |
3 |
Retry attempts on transient errors |
RETRY_BASE_DELAY_MS |
500 |
Base delay for exponential backoff |
RETRY_MAX_DELAY_MS |
30000 |
Backoff delay cap |
CONCURRENCY_LIMIT |
3 |
Parallel page fetches when --content is set |
CB_FAILURE_THRESHOLD |
5 |
Consecutive failures before circuit opens |
CB_RESET_TIMEOUT_MS |
60000 |
Circuit breaker reset window |
All variables are validated at startup — misconfigured runs fail immediately with a descriptive list of every bad value rather than crashing mid-execution.
Architecture
See references/ARCHITECTURE.md for a full module breakdown.
scripts/
├── search.js ← Search CLI entry point
├── content.js ← Content extraction CLI entry point
├── content-fetcher.js ← HTTP fetch + Readability + DOM fallback
├── config.js ← Schema-validated env config
├── circuit-breaker.js ← Fault isolation (CLOSED → OPEN → HALF_OPEN)
├── retry.js ← Exponential backoff with full jitter
├── concurrency.js ← Bounded parallel execution pool
├── utils.js ← htmlToMarkdown, smartTruncate, parseURL
├── logger.js ← Structured leveled logger → stderr
└── errors.js ← Typed error hierarchy
04 同仓库更多 Skills
crowd-prompting
A marketplace where AI agents improve prompts, system instructions, tool descriptions, and other text-based content with domain expertise from real-world operations — and earn tokens for valuable contributions.
ceaser
Interact with the Ceaser privacy protocol on Base L2 using the ceaser-mcp MCP tools. This skill uses the ceaser-mcp npm package for ALL operations -- shield, unshield, note management, and protocol queries. All ceaser tool calls use CLI subcommands (npx -y ceaser-mcp <subcommand>). Ceaser is a privacy-preserving ETH wrapper using Noir/UltraHonk zero-knowledge proofs on Base L2 (chain ID 8453).
ceaser-send
Fully automated private ETH transfer via Ceaser Protocol on Base L2 using the ceaser-mcp MCP tools. This skill uses the ceaser-mcp npm package for ALL operations -- all ceaser tool calls use CLI subcommands (npx -y ceaser-mcp <subcommand>). WARNING -- creates observable on-chain links between user wallet and hot wallet (see Privacy Warning). Generates an ephemeral hot wallet with BIP-39 mnemonic recovery, user funds it, agent signs and broadcasts the Shield TX automatically. Exactly one manual step required (funding the hot wallet with ETH). Uses Noir/UltraHonk zero-knowledge proofs.
dilemm-ai
Compete in DilemmAI, the prisoner's dilemma AI arena at dilemm.ai. Use when an OpenClaw agent wants to create an account, design and submit strategy prompts for their bot, enter matchmaking, analyze historical matches and leaderboards, and iterate on bot strategy. Requires headless browser automation (no API available yet).
moldium
Post and manage content on the Moldium blog platform. Triggered by "post to Moldium", "write a blog post", "publish an article", etc.
arxiv-paper-reviews
Interact with arXiv Crawler API to fetch papers, read reviews, submit comments, search papers, and import papers. Use when working with arXiv papers, fetching paper lists by date/category/interest, viewing paper details with comments, submitting paper reviews, searching papers by title, or importing papers from arXiv URLs via API at http://weakaccept.top:8000/.