[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"skill:01M78JABSDNTRMA92ZM5S043SW":3,"skill-same:01M78JABSDNTRMA92ZM5S043SW":95},{"skillNo":4,"name":5,"nameZh":6,"description":7,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":16,"categories":17,"keywords":18,"publishedAt":19,"linkOnly":20,"markdownContent":21,"markdownBody":22,"frontMatter":23,"refName":24,"commitSha":25,"skillDirectory":26,"forksCount":14,"repositoryHtmlUrl":27,"repositoryHomepageUrl":6,"ownerHtmlUrl":28,"ownerBlogUrl":6,"ownerDescription":29,"updatedAt":30,"install":31,"resources":44},"01M78JABSDNTRMA92ZM5S043SW","universal-scraping-architect",null,"Use for web scraping, crawling, document extraction, API parsing, or building validation-heavy data pipelines using Firecrawl or local Python scripts.","aneeba-pixel\u002Fclaude-skills","aneeba-pixel","https:\u002F\u002Favatars.githubusercontent.com\u002Fu\u002F218186338?v=4","01M78GRXBTG70217SH410P543Y","Aneeba Saeed","01M78GRXCDZ6TA0F8H2CZMDGSS",0,"MIT","https:\u002F\u002Fgithub.com\u002Faneeba-pixel\u002Fclaude-skills\u002Fblob\u002Fclaude\u002Fopen-prs-review-Of9HK\u002Fengineering\u002Funiversal-scraping-architect\u002FSKILL.md",[],[],"2026-11-15 14:19:17",false,"---\nname: \"universal-scraping-architect\"\ndescription: \"Use for web scraping, crawling, document extraction, API parsing, or building validation-heavy data pipelines using Firecrawl or local Python scripts.\"\n---\n\n# Universal Scraping Architect\n\nYou are an expert web scraping and data extraction engineer. Your goal is to design complete, robust data pipelines with intelligent routing, validation, and token budget tracking—not brittle one-off scripts.\n\n**Dependency Notice:** This skill utilizes `firecrawl`, `pandas`, `requests`, and `beautifulsoup4`. It uses a BYOK (Bring Your Own Key) pattern for Firecrawl. API keys must only be loaded via environment variables.\n\n## Before Starting\n**Check for context first:**\nIf `project-context.md` exists, read it before asking questions. Determine the target data format, scale of extraction, and deployment environment before writing any code.\n\n## How This Skill Works\n\nThis skill supports 3 extraction modes based on intelligent routing:\n\n### Mode 1: API-Driven (Firecrawl)\nUse when the source is a public URL, heavily dynamic (JS\u002FSPA), requires search-first discovery, or involves bulk crawling across a domain.\n### Mode 2: Local Python (Traditional)\nUse when extracting from local files (PDF, Excel, CSV), the data is private\u002Fsensitive, or the target is a simple static HTML page where Firecrawl is overkill.\n### Mode 3: Hybrid Pipeline\nUse when Firecrawl handles URL discovery\u002Fweb extraction, but local Python (Pandas) is required to clean, normalize, and structure the output before saving.\n\n## The Extraction Pipeline\n\nWhen executing a scraping task, always follow this sequence:\n1. **Route the Approach:** Explicitly state whether Firecrawl or Local Python is being used and why.\n2. **Track Budgets:** Estimate Firecrawl API quotas or LLM token context limits before executing large jobs. \n3. **Extract Safely:** Implement checkpointing for multi-page jobs. Handle pagination and dynamic layouts gracefully.\n4. **Validate & Clean:** Enforce required fields, catch empty outputs, flag duplicates, and normalize field names.\n5. **Format:** Default to CSV for tabular data, JSON for nested structures, and Markdown for clean text.\n\n## Proactive Triggers\n\nSurface these issues WITHOUT being asked when you notice them in context:\n- **Hardcoded API Keys** → Flag immediately and rewrite to use `os.getenv('FIRECRAWL_API_KEY')`.\n- **Private Data Leakage** → If the user asks to send local, sensitive files to an external API, flag the privacy risk and suggest Mode 2 (Local Python).\n- **Missing Pagination** → If the target implies hundreds of records but no pagination logic is requested, flag it and add checkpointing.\n\n## Output Artifacts\n\n| When you ask for... | You get... |\n|---------------------|------------|\n| \"Scrape this site\" | A fully validated Python extraction script with routing logic and error handling. |\n| \"Get data from this table\" | A clean CSV\u002FJSON dataset with a summary log of row counts and empty values. |\n| \"Crawl these docs\" | A Markdown deliverable chunked for LLM token limits. |\n\n## Anti-Patterns\n- **Brittle Selectors:** Never use highly nested CSS selectors (e.g., `div > span > ul > li:nth-child(3)`). Use data attributes or robust structural anchors.\n- **Ignoring Etiquette:** Never scrape without checking `robots.txt` or implementing sensible rate limits.\n- **No Validation:** Never blindly write scraped data to a file without checking if the array is empty or missing critical keys.\n\n## Related Skills\n- **data-cleaning**: Use when the scraped data requires complex statistical normalization or deduplication.\n- **browser-automation**: Use for highly interactive scraping requiring user emulation (clicks, logins) where Firecrawl is insufficient.\n","# Universal Scraping Architect\n\nYou are an expert web scraping and data extraction engineer. Your goal is to design complete, robust data pipelines with intelligent routing, validation, and token budget tracking—not brittle one-off scripts.\n\n**Dependency Notice:** This skill utilizes `firecrawl`, `pandas`, `requests`, and `beautifulsoup4`. It uses a BYOK (Bring Your Own Key) pattern for Firecrawl. API keys must only be loaded via environment variables.\n\n## Before Starting\n**Check for context first:**\nIf `project-context.md` exists, read it before asking questions. Determine the target data format, scale of extraction, and deployment environment before writing any code.\n\n## How This Skill Works\n\nThis skill supports 3 extraction modes based on intelligent routing:\n\n### Mode 1: API-Driven (Firecrawl)\nUse when the source is a public URL, heavily dynamic (JS\u002FSPA), requires search-first discovery, or involves bulk crawling across a domain.\n### Mode 2: Local Python (Traditional)\nUse when extracting from local files (PDF, Excel, CSV), the data is private\u002Fsensitive, or the target is a simple static HTML page where Firecrawl is overkill.\n### Mode 3: Hybrid Pipeline\nUse when Firecrawl handles URL discovery\u002Fweb extraction, but local Python (Pandas) is required to clean, normalize, and structure the output before saving.\n\n## The Extraction Pipeline\n\nWhen executing a scraping task, always follow this sequence:\n1. **Route the Approach:** Explicitly state whether Firecrawl or Local Python is being used and why.\n2. **Track Budgets:** Estimate Firecrawl API quotas or LLM token context limits before executing large jobs. \n3. **Extract Safely:** Implement checkpointing for multi-page jobs. Handle pagination and dynamic layouts gracefully.\n4. **Validate & Clean:** Enforce required fields, catch empty outputs, flag duplicates, and normalize field names.\n5. **Format:** Default to CSV for tabular data, JSON for nested structures, and Markdown for clean text.\n\n## Proactive Triggers\n\nSurface these issues WITHOUT being asked when you notice them in context:\n- **Hardcoded API Keys** → Flag immediately and rewrite to use `os.getenv('FIRECRAWL_API_KEY')`.\n- **Private Data Leakage** → If the user asks to send local, sensitive files to an external API, flag the privacy risk and suggest Mode 2 (Local Python).\n- **Missing Pagination** → If the target implies hundreds of records but no pagination logic is requested, flag it and add checkpointing.\n\n## Output Artifacts\n\n| When you ask for... | You get... |\n|---------------------|------------|\n| \"Scrape this site\" | A fully validated Python extraction script with routing logic and error handling. |\n| \"Get data from this table\" | A clean CSV\u002FJSON dataset with a summary log of row counts and empty values. |\n| \"Crawl these docs\" | A Markdown deliverable chunked for LLM token limits. |\n\n## Anti-Patterns\n- **Brittle Selectors:** Never use highly nested CSS selectors (e.g., `div > span > ul > li:nth-child(3)`). Use data attributes or robust structural anchors.\n- **Ignoring Etiquette:** Never scrape without checking `robots.txt` or implementing sensible rate limits.\n- **No Validation:** Never blindly write scraped data to a file without checking if the array is empty or missing critical keys.\n\n## Related Skills\n- **data-cleaning**: Use when the scraped data requires complex statistical normalization or deduplication.\n- **browser-automation**: Use for highly interactive scraping requiring user emulation (clicks, logins) where Firecrawl is insufficient.\n",{"name":5,"description":7},"claude\u002Fopen-prs-review-Of9HK","a48d8438efe8e36550acc4c2006bdcddf62911db","engineering\u002Funiversal-scraping-architect","https:\u002F\u002Fgithub.com\u002Faneeba-pixel\u002Fclaude-skills","https:\u002F\u002Fgithub.com\u002Faneeba-pixel","Student at University of Gujrat","2026-07-14 21:41:17",{"source":8,"skillName":5,"commands":32,"prompt":42,"zipAvailable":43},[33,36,39],{"tool":34,"command":35},"npx","npx skills add aneeba-pixel\u002Fclaude-skills --skill universal-scraping-architect",{"tool":37,"command":38},"bunx","bunx skills add aneeba-pixel\u002Fclaude-skills --skill universal-scraping-architect",{"tool":40,"command":41},"pnpm","pnpm dlx skills add aneeba-pixel\u002Fclaude-skills --skill universal-scraping-architect","安装技能，技能地址：https:\u002F\u002Fgithub.com\u002Faneeba-pixel\u002Fclaude-skills\u002Fblob\u002Fclaude\u002Fopen-prs-review-Of9HK\u002Fengineering\u002Funiversal-scraping-architect\u002FSKILL.md",true,[45,51,57,61,65,69,73,77,81,87,91],{"path":46,"resourceType":47,"resourceTypeDesc":48,"sizeBytes":49,"sourceHtmlUrl":50},".claude-plugin\u002Fplugin.json","REFERENCE","参考文档",767,"https:\u002F\u002Fgithub.com\u002Faneeba-pixel\u002Fclaude-skills\u002Fblob\u002Fclaude\u002Fopen-prs-review-Of9HK\u002Fengineering\u002Funiversal-scraping-architect\u002F.claude-plugin\u002Fplugin.json",{"path":52,"resourceType":53,"resourceTypeDesc":54,"sizeBytes":55,"sourceHtmlUrl":56},".gitignore","OTHER","其他",124,"https:\u002F\u002Fgithub.com\u002Faneeba-pixel\u002Fclaude-skills\u002Fblob\u002Fclaude\u002Fopen-prs-review-Of9HK\u002Fengineering\u002Funiversal-scraping-architect\u002F.gitignore",{"path":58,"resourceType":47,"resourceTypeDesc":48,"sizeBytes":59,"sourceHtmlUrl":60},"agents\u002Fcs-scraping-architect.md",227,"https:\u002F\u002Fgithub.com\u002Faneeba-pixel\u002Fclaude-skills\u002Fblob\u002Fclaude\u002Fopen-prs-review-Of9HK\u002Fengineering\u002Funiversal-scraping-architect\u002Fagents\u002Fcs-scraping-architect.md",{"path":62,"resourceType":47,"resourceTypeDesc":48,"sizeBytes":63,"sourceHtmlUrl":64},"commands\u002Fcs-scrape.md",181,"https:\u002F\u002Fgithub.com\u002Faneeba-pixel\u002Fclaude-skills\u002Fblob\u002Fclaude\u002Fopen-prs-review-Of9HK\u002Fengineering\u002Funiversal-scraping-architect\u002Fcommands\u002Fcs-scrape.md",{"path":66,"resourceType":47,"resourceTypeDesc":48,"sizeBytes":67,"sourceHtmlUrl":68},"references\u002Ffirecrawl-technical-guide.md",2711,"https:\u002F\u002Fgithub.com\u002Faneeba-pixel\u002Fclaude-skills\u002Fblob\u002Fclaude\u002Fopen-prs-review-Of9HK\u002Fengineering\u002Funiversal-scraping-architect\u002Freferences\u002Ffirecrawl-technical-guide.md",{"path":70,"resourceType":47,"resourceTypeDesc":48,"sizeBytes":71,"sourceHtmlUrl":72},"references\u002Flocal-extraction-patterns.md",2571,"https:\u002F\u002Fgithub.com\u002Faneeba-pixel\u002Fclaude-skills\u002Fblob\u002Fclaude\u002Fopen-prs-review-Of9HK\u002Fengineering\u002Funiversal-scraping-architect\u002Freferences\u002Flocal-extraction-patterns.md",{"path":74,"resourceType":47,"resourceTypeDesc":48,"sizeBytes":75,"sourceHtmlUrl":76},"references\u002Fscraping-ethics-security.md",2573,"https:\u002F\u002Fgithub.com\u002Faneeba-pixel\u002Fclaude-skills\u002Fblob\u002Fclaude\u002Fopen-prs-review-Of9HK\u002Fengineering\u002Funiversal-scraping-architect\u002Freferences\u002Fscraping-ethics-security.md",{"path":78,"resourceType":47,"resourceTypeDesc":48,"sizeBytes":79,"sourceHtmlUrl":80},"requirements.txt",300,"https:\u002F\u002Fgithub.com\u002Faneeba-pixel\u002Fclaude-skills\u002Fblob\u002Fclaude\u002Fopen-prs-review-Of9HK\u002Fengineering\u002Funiversal-scraping-architect\u002Frequirements.txt",{"path":82,"resourceType":83,"resourceTypeDesc":84,"sizeBytes":85,"sourceHtmlUrl":86},"scripts\u002Ffirecrawl_example.py","SCRIPT","脚本",8032,"https:\u002F\u002Fgithub.com\u002Faneeba-pixel\u002Fclaude-skills\u002Fblob\u002Fclaude\u002Fopen-prs-review-Of9HK\u002Fengineering\u002Funiversal-scraping-architect\u002Fscripts\u002Ffirecrawl_example.py",{"path":88,"resourceType":83,"resourceTypeDesc":84,"sizeBytes":89,"sourceHtmlUrl":90},"scripts\u002Flocal_bs4_example.py",10072,"https:\u002F\u002Fgithub.com\u002Faneeba-pixel\u002Fclaude-skills\u002Fblob\u002Fclaude\u002Fopen-prs-review-Of9HK\u002Fengineering\u002Funiversal-scraping-architect\u002Fscripts\u002Flocal_bs4_example.py",{"path":92,"resourceType":83,"resourceTypeDesc":84,"sizeBytes":93,"sourceHtmlUrl":94},"scripts\u002Fvalidate_extraction.py",1339,"https:\u002F\u002Fgithub.com\u002Faneeba-pixel\u002Fclaude-skills\u002Fblob\u002Fclaude\u002Fopen-prs-review-Of9HK\u002Fengineering\u002Funiversal-scraping-architect\u002Fscripts\u002Fvalidate_extraction.py",[96,103,110,117,124,131],{"skillNo":97,"name":98,"nameZh":6,"description":99,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":100,"categories":101,"keywords":102,"publishedAt":19},"01M78K120A3JKH3NH8GC19YG4M","syllabus","Generates a curated supplementary reading list from any course syllabus using Consensus academic search. Grill-me intake (syllabus input format + course audience + year range) plus a grouping forcing-options checkpoint before any search runs — so the reading list matches the course's level and recency need. Parses the syllabus to extract topics and learning outcomes, searches Consensus for recent peer-reviewed papers per topic, and produces a professionally formatted .docx with clickable Consensus links, plain-language summaries calibrated to audience level, and Bloom-higher-order discussion questions tied to course learning goals. Triggers whenever a user uploads a syllabus, course outline, or curriculum document and wants supplementary readings. Also triggers on: 'syllabus reading list', 'find papers for my course', 'create a reading list from this syllabus', 'recent research for my class', 'supplementary readings', 'find journal articles for these topics', 'what recent papers cover this material', 'any new research on these course topics', 'update my syllabus with recent papers'. Even casual mentions when a syllabus is attached should trigger this skill.","https:\u002F\u002Fgithub.com\u002Faneeba-pixel\u002Fclaude-skills\u002Fblob\u002Fclaude\u002Fopen-prs-review-Of9HK\u002Fresearch\u002Fsyllabus\u002Fskills\u002Fsyllabus\u002FSKILL.md",[],[],{"skillNo":104,"name":105,"nameZh":6,"description":106,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":107,"categories":108,"keywords":109,"publishedAt":19},"01M78K0QS9AGDJ43CPQYEHM5S0","research","Default entry point for any research request — a hybrid router that classifies the question deterministically and either delegates to a specialist research skill (pulse for trends\u002Fsentiment, grants for NIH funding, litreview for academic literature, syllabus for course reading, patent for prior-art + IP landscape, dossier for entity research) or runs its own plan-decompose-multi-source-search-synthesize-cite fallback workflow when no specialist matches. Always surfaces the routing decision so users can override. Triggers — \"research [topic]\", \"look into [topic]\", \"what do we know about [topic]\", \"investigate [topic]\", \"find me information on [topic]\", \"do some research on [topic]\", \"I need to understand [topic]\", or any research request that doesn't obviously match a more-specific specialist skill. Output is a markdown briefing (default) or .docx document (on request) with full citations and an audit log.","https:\u002F\u002Fgithub.com\u002Faneeba-pixel\u002Fclaude-skills\u002Fblob\u002Fclaude\u002Fopen-prs-review-Of9HK\u002Fresearch\u002Fresearch\u002Fskills\u002Fresearch\u002FSKILL.md",[],[],{"skillNo":111,"name":112,"nameZh":6,"description":113,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":114,"categories":115,"keywords":116,"publishedAt":19},"01M78K0FV3HMJ94SK965CWMNNP","pulse","Multi-source recency research skill that takes the pulse of any topic across Reddit, Hacker News, the open web, and optionally X\u002FTwitter within a configurable recent window (default 30 days). Forcing intake clarifies topic specificity, angle (trend\u002Fsentiment\u002Fproblems\u002Fopportunities\u002Fcomparison), time window, and platform scope before searching. Returns a synthesized briefing with citations, engagement metrics, and cross-platform pattern analysis. Triggers: 'pulse on [topic]', 'what's happening with [topic]', 'what are people saying about [topic]', 'current conversation about [topic]', 'take the pulse of [topic]', 'trending: [topic]', 'find me info on [topic]', or any variation requesting multi-source recency intelligence on a topic. Also use for competitor research, trend discovery, tool comparisons, and audience sentiment analysis.","https:\u002F\u002Fgithub.com\u002Faneeba-pixel\u002Fclaude-skills\u002Fblob\u002Fclaude\u002Fopen-prs-review-Of9HK\u002Fresearch\u002Fpulse\u002Fskills\u002Fpulse\u002FSKILL.md",[],[],{"skillNo":118,"name":119,"nameZh":6,"description":120,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":121,"categories":122,"keywords":123,"publishedAt":19},"01M78K04E77B04KYBB6M01EV3J","patent","Patent prior-art and landscape intelligence skill — not generic patent help. Commits to one of five sub-use-cases via forcing intake (novelty search \u002F freedom-to-operate \u002F competitive landscape \u002F acquisition diligence \u002F litigation prior-art) before any search runs. Searches Google Patents, Espacenet, USPTO, and optionally Lens.org for citation-graph signals. Output is an editable Word document (.docx) with verdict, ranked closest art (claim-text extracted), CPC-class-aware landscape, family-resolved hits, geographic coverage, FTO flags where applicable, strategy recommendations, and full audit log. Triggers: 'prior art search for [invention]', 'patent search on [topic]', 'freedom to operate analysis', 'FTO for [product]', 'patent landscape for [field]', 'is [invention] novel', 'patents on [topic]', 'competitive patent analysis', 'prior art for litigation', 'patent diligence on [company]'. Produces search signal, not legal advice — always recommends consulting a patent attorney before filing or licensing decisions. Trademark, copyright, and trade-secret questions are out of scope.","https:\u002F\u002Fgithub.com\u002Faneeba-pixel\u002Fclaude-skills\u002Fblob\u002Fclaude\u002Fopen-prs-review-Of9HK\u002Fresearch\u002Fpatent\u002Fskills\u002Fpatent\u002FSKILL.md",[],[],{"skillNo":125,"name":126,"nameZh":6,"description":127,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":128,"categories":129,"keywords":130,"publishedAt":19},"01M78JZW8XEFZP6HSF53M8D4M1","notebooklm","Browser automation skill for controlling Google's NotebookLM. Handles reading and querying notebooks, adding sources (URLs, text, files, YouTube links, synthesized content), generating Studio outputs (Audio Overview, infographics, slide decks, study guides, briefing docs, mind maps, timelines, FAQs), and creating new notebooks. Triggers on any phrase involving NotebookLM — 'open NotebookLM', 'check my [name] notebook', 'pull info from NotebookLM', 'ask my notebook about X', 'add [source] to NotebookLM', 'create an infographic in NotebookLM', 'use NotebookLM Studio', 'generate a slide deck from my notebook', or any variation where the goal involves NotebookLM. Requires browser automation environment — fails gracefully when unavailable.","https:\u002F\u002Fgithub.com\u002Faneeba-pixel\u002Fclaude-skills\u002Fblob\u002Fclaude\u002Fopen-prs-review-Of9HK\u002Fresearch\u002Fnotebooklm\u002Fskills\u002Fnotebooklm\u002FSKILL.md",[],[],{"skillNo":132,"name":133,"nameZh":6,"description":134,"descriptionZh":6,"repositoryFullName":8,"ownerLogin":9,"ownerAvatarUrl":10,"ownerAccountNo":11,"ownerDisplayName":12,"repositoryNo":13,"starsCount":14,"licenseSpdxId":15,"licenseFileUrl":6,"sourceHtmlUrl":135,"categories":136,"keywords":137,"publishedAt":19},"01M78JZHX45S2Y0GPY9Z5V6TQ3","litreview","Academic literature orientation skill that searches papers via Consensus, builds a strategic search plan using PICO (default) or SPIDER \u002F Decomposition \u002F hybrid as fallbacks, and synthesizes findings into a professionally formatted Word document (.docx) research guide. Grill-me intake (research question specificity + framework hint + tentative depth) before the recon search; a second forcing checkpoint after Phase 2 confirms framework + sub-areas + depth before searches consume budget. Configurable depth (5\u002F10\u002F20 queries) controls coverage vs. speed. Output is a 'launching pad' — not a finished review, but an orientation guide that lets a researcher dive in confidently. Triggers: 'litreview on [topic]', 'literature review on [topic]', 'I'm starting a literature review on X', 'I'm writing a paper on X', 'help me research X', 'I'm doing research on X', 'can you help me research X'. Do NOT trigger for single one-off paper searches where the user just wants a quick list — that's a plain Consensus search.","https:\u002F\u002Fgithub.com\u002Faneeba-pixel\u002Fclaude-skills\u002Fblob\u002Fclaude\u002Fopen-prs-review-Of9HK\u002Fresearch\u002Flitreview\u002Fskills\u002Flitreview\u002FSKILL.md",[],[]]