open-web
Resilient public-page reading and OSINT keyword search — web_search (allowlisted-provider OSINT) and web_fetch (curl_cffi TLS-impersonation grid + headless-browser fallback that gets past WAF/403/anti-bot). Use when a fetch is blocked, a page is JS-rendered, or you need open-web OSINT about a target/org.
- License 许可
- Apache-2.0
- 最近更新
- 2026-07-22
- Stars
- ★ 3
- Forks
- 0
01 前言 Front Matter
- name
- open-web
- metadata
- { "tags": "open-web, osint, web-fetch, web-search, waf-bypass", "subdomain": "reconnaissance", "when_to_use": "blocked page, 403/402, WAF, cloudflare, akamai, datadome, captcha, JS-rendered page, open-web OSINT, read public page, reddit, x/twitter, youtube, search the web, third-party intel", "upstream_ref": "adapted from fivetaku/insane-search (MIT) — SKILL.md R1–R7; decepticon/sandbox_web engine" }
- description
- Resilient public-page reading and OSINT keyword search — web_search (allowlisted-provider OSINT) and web_fetch (curl_cffi TLS-impersonation grid + headless-browser fallback that gets past WAF/403/anti-bot). Use when a fetch is blocked, a page is JS-rendered, or you need open-web OSINT about a target/org.
- allowed-tools
- Read
03 SKILL.md 正文
Open-Web Acquisition — web_search / web_fetch
Two tools front the sandbox open-web engine (decepticon.sandbox_web):
web_search(query, provider="duckduckgo")— keyword OSINT over an allowlisted search provider. Target-EXEMPT (it hits the provider, not the target), so no engagement scope is required. Use it to DISCOVER: the org's public footprint, exposed tech / version-specific advisories, leaked references, doc/changelog/status pages, third-party subdomains and assets named in the wild.web_fetch(url, selector="", device="auto")— read ONE page's content, escalating past WAF / anti-bot blocks. RoE target-gated (theurlmust be inplan/roe.jsonscope) and SSRF-safe. Prefer it overcurl-in-bash whenever a public page is blocked, returns a challenge, or is JS-rendered.
Typical flow: web_search to find a URL → web_fetch to read it.
How web_fetch escalates (you do NOT drive this — the engine does)
web_fetch runs an escalation ladder inside the sandbox automatically:
- Phase 0 — official public APIs. Reddit / X(Twitter) / YouTube URLs are
auto-routed to their no-auth endpoints (Reddit
.rss, X tweet-result/oEmbed/ syndication, YouTubeyt-dlp) BEFORE any WAF grid. Just pass the normal page URL; a Phase-0 win showsverdictwithprofile=phase0:<platform>. - curl_cffi TLS-impersonation grid. Real Safari/Chrome/Firefox JA3/JA4 fingerprints × URL transforms (mobile subdomain, …) × referer strategies, tried exhaustively. This clears most WAFs (Cloudflare TLS, F5, AWS WAF, …).
- Headless browser tier. For JS challenges the curl grid can't clear (Cloudflare Turnstile, Akamai Bot Manager, DataDome), a local headless Chromium with stealth runs. There is no Playwright-MCP in the sandbox — this local browser IS the final rung. If it still fails, that is a real wall.
You never pick a TLS profile, transform, or browser — the engine detects the WAF
and chooses. Your only knobs are selector and device.
R2 — HTTP 200 is NOT success
The engine validates every response (4-layer: challenge markers / body size /
cookie sensor / your selector). A 200 that is actually a challenge or empty
SPA shell is reported as a FAILURE, not content. Trust the tool's verdict /
[web_fetch OK|FAILED] line, never a raw status. Always pass a selector
(e.g. article, #content, [class*='product']) when you know the content
marker — it upgrades a "looks clean" guess into proven success.
R6 — a give-up is NOT exhaustion (the discipline that matters)
When web_fetch returns [web_fetch FAILED] it includes stop_reason= and
grid_exhausted=. Do not declare a page unreadable on the first FAILED.
- TERMINAL walls —
stop_reasonisauth_required(login/paywall) ornot_found(404). These are real: record it and move on; retrying cannot help. - Everything else (
challenge,blocked,rate_limited, or a non-exhausted grid) still has escalation left:- rate_limited (429) is NOT terminal — back off, then retry.
- Retry once with
device="mobile"(mobile TLS +m.subdomain often walk past a desktop block) and/or a preciseselector. - Only after a terminal
stop_reasonmay you honestly conclude the page is unreachable.
This is the whole point of the engine: the cheap path failing is the start of escalation, not permission to stop.
RoE & OSINT boundary
web_searchis OSINT → not scope-gated (allowlisted provider egress only).web_fetchis target-gated → theurlmust be inplan/roe.jsonscope; an out-of-scope fetch returnsROE_REFUSED. SSRF-blocked (internal/metadata/ private) hosts are refused regardless.- Reading public pages is OSINT, not exploitation. Use
bash/http_requestfor ACTIVE probing of in-scope target infrastructure; useweb_fetchto READ pages (the target's own blocked/JS pages, or in-scope public assets).
Platform fast-paths (handled by Phase 0 — just pass the URL)
| Need | Pass to web_fetch |
|---|---|
| Reddit thread/sub | the reddit.com/... URL (auto .rss) |
| A specific tweet / X profile | the x.com/.../status/... or profile URL |
| YouTube metadata / captions | the youtube.com/watch?v=... URL (yt-dlp) |
| GitHub repo (metadata, stars, language) | the github.com/<owner>/<repo> URL → repos API JSON |
| npm package (deps, versions, maintainers) | the npmjs.com/package/<pkg> URL → registry JSON |
| PyPI package (versions, deps, urls) | the pypi.org/project/<pkg> URL → JSON API |
| Any WAF-protected article/page | the page URL — the grid + browser tier handle it |
GitHub/npm/PyPI return structured JSON (great for supply-chain / source recon) instead of HTML — just pass the normal page URL and the engine routes it.
For keyword discovery on a platform (e.g. "what's said about X on Reddit"),
web_search("site:reddit.com <topic>") first, then web_fetch the result URLs.
04 同仓库更多 Skills
wps-pixie-dust
WPS Pixie-Dust offline nonce attack (reaver -K / pixiewps) and fallback online PIN brute (bully) to recover the AP's WPA PSK without capturing a handshake.
wpa3-sae
WPA3-SAE transition-mode downgrade (DragonShift), SSID Confusion CVE-2023-52424, Dragonblood side-channels, and SAE captive-portal credential recovery against WPA3-Personal networks.
wpa2-psk
WPA/WPA2-PSK handshake capture via targeted deauth + PMKID (no deauth required) + offline hashcat cracking. The most common consumer/SMB encryption mode in 2026.
wpa-enterprise-eap
WPA/WPA2/WPA3-Enterprise (802.1X/EAP) rogue-RADIUS evil-twin for MSCHAPv2 capture, GTC downgrade, and PEAP relay. MSCHAPv2 capture equals a NetNTLM hash — the primary wireless on-ramp to Active Directory.
krack-fragattacks
KRACK key-reinstallation (CVE-2017-13077..13082) and FragAttacks fragmentation/aggregation flaws (CVE-2020-24586..24588, CVE-2020-26139..26147) against legacy or embedded 802.11 supplicants with poor patch cadence.
evil-twin-karma
Evil-twin rogue AP with KARMA/Mana PNL-probe response, captive-portal credential capture, and post-association MITM for PSK/open networks. Distinct from wpa-enterprise-eap which targets 802.1X.