data-feeds
Extract structured data from 40+ websites including Amazon, LinkedIn, Instagram, TikTok, Facebook, YouTube, and more. Uses Bright Data's Web Data APIs with automatic polling. Returns clean JSON with product details, profiles, reviews, posts, and comments.
- License 许可
- MIT
- 最近更新
- 2026-05-10
- Stars
- ★ 1
- Forks
- 1
01 前言 Front Matter
- name
- data-feeds
- description
- Extract structured data from 40+ websites including Amazon, LinkedIn, Instagram, TikTok, Facebook, YouTube, and more. Uses Bright Data's Web Data APIs with automatic polling. Returns clean JSON with product details, profiles, reviews, posts, and comments.
02 资源文件 3
- 参考文档.skill-source.json481 B
- 脚本scripts/datasets.sh8.4 KB
- 脚本scripts/fetch.sh1.9 KB
03 SKILL.md 正文
Bright Data - Structured Data Feeds
Extract structured data from major websites with automatic parsing. No scraping logic needed - just provide a URL and get clean JSON data.
Setup
Environment Variables (Required)
export BRIGHTDATA_API_KEY="your-api-key"
Optional
export BRIGHTDATA_POLLING_TIMEOUT=600 # Max seconds to wait (default: 600)
Get your API key from Bright Data Dashboard.
Usage
bash scripts/datasets.sh <dataset_type> <url> [additional_params...]
Available Datasets
E-Commerce
| Dataset | Command | Description |
|---|---|---|
| Amazon Product | datasets.sh amazon_product <url> |
Product details, pricing, ratings |
| Amazon Reviews | datasets.sh amazon_product_reviews <url> |
Customer reviews for a product |
| Amazon Search | datasets.sh amazon_product_search <keyword> <domain_url> |
Search results |
| Walmart Product | datasets.sh walmart_product <url> |
Product details from Walmart |
| Walmart Seller | datasets.sh walmart_seller <url> |
Seller information |
| eBay Product | datasets.sh ebay_product <url> |
eBay listing details |
| Home Depot | datasets.sh homedepot_products <url> |
Home Depot product data |
| Zara | datasets.sh zara_products <url> |
Zara product details |
| Etsy | datasets.sh etsy_products <url> |
Etsy listing data |
| Best Buy | datasets.sh bestbuy_products <url> |
Best Buy product info |
Professional Networks
| Dataset | Command | Description |
|---|---|---|
| LinkedIn Person | datasets.sh linkedin_person_profile <url> |
Profile data (experience, skills) |
| LinkedIn Company | datasets.sh linkedin_company_profile <url> |
Company page data |
| LinkedIn Jobs | datasets.sh linkedin_job_listings <url> |
Job posting details |
| LinkedIn Posts | datasets.sh linkedin_posts <url> |
Post content and engagement |
| LinkedIn Search | datasets.sh linkedin_people_search <url> <first> <last> |
Find people |
| Crunchbase | datasets.sh crunchbase_company <url> |
Company funding, employees |
| ZoomInfo | datasets.sh zoominfo_company_profile <url> |
Company profile data |
| Dataset | Command | Description |
|---|---|---|
| Profiles | datasets.sh instagram_profiles <url> |
Bio, followers, following |
| Posts | datasets.sh instagram_posts <url> |
Post details, likes, captions |
| Reels | datasets.sh instagram_reels <url> |
Reel data and metrics |
| Comments | datasets.sh instagram_comments <url> |
Post comments |
| Dataset | Command | Description |
|---|---|---|
| Posts | datasets.sh facebook_posts <url> |
Post content and reactions |
| Marketplace | datasets.sh facebook_marketplace_listings <url> |
Listing details |
| Reviews | datasets.sh facebook_company_reviews <url> [num] |
Company reviews |
| Events | datasets.sh facebook_events <url> |
Event details |
TikTok
| Dataset | Command | Description |
|---|---|---|
| Profiles | datasets.sh tiktok_profiles <url> |
Creator profile data |
| Posts | datasets.sh tiktok_posts <url> |
Video details and metrics |
| Shop | datasets.sh tiktok_shop <url> |
TikTok Shop product data |
| Comments | datasets.sh tiktok_comments <url> |
Video comments |
YouTube
| Dataset | Command | Description |
|---|---|---|
| Profiles | datasets.sh youtube_profiles <url> |
Channel data |
| Videos | datasets.sh youtube_videos <url> |
Video details and stats |
| Comments | datasets.sh youtube_comments <url> [num] |
Video comments (default: 10) |
Other Social
| Dataset | Command | Description |
|---|---|---|
| X (Twitter) | datasets.sh x_posts <url> |
Tweet data |
datasets.sh reddit_posts <url> |
Post and comment data |
Google Services
| Dataset | Command | Description |
|---|---|---|
| Maps Reviews | datasets.sh google_maps_reviews <url> [days] |
Business reviews (default: 3 days) |
| Shopping | datasets.sh google_shopping <url> |
Product comparison data |
| Play Store | datasets.sh google_play_store <url> |
App details and reviews |
Other
| Dataset | Command | Description |
|---|---|---|
| Apple App Store | datasets.sh apple_app_store <url> |
iOS app data |
| Reuters News | datasets.sh reuter_news <url> |
News article content |
| GitHub | datasets.sh github_repository_file <url> |
Repository file data |
| Yahoo Finance | datasets.sh yahoo_finance_business <url> |
Stock and company data |
| Zillow | datasets.sh zillow_properties_listing <url> |
Property listing details |
| Booking.com | datasets.sh booking_hotel_listings <url> |
Hotel listing data |
Examples
Get LinkedIn Profile
bash scripts/datasets.sh linkedin_person_profile "https://www.linkedin.com/in/satyanadella/"
Get Amazon Product
bash scripts/datasets.sh amazon_product "https://www.amazon.com/dp/B09V3KXJPB"
Get Instagram Profile
bash scripts/datasets.sh instagram_profiles "https://www.instagram.com/natgeo/"
Get YouTube Comments
bash scripts/datasets.sh youtube_comments "https://www.youtube.com/watch?v=dQw4w9WgXcQ" 20
Search Amazon
bash scripts/datasets.sh amazon_product_search "wireless headphones" "https://www.amazon.com"
Output Format
Returns structured JSON with website-specific fields. Example for LinkedIn profile:
{
"name": "Satya Nadella",
"headline": "Chairman and CEO at Microsoft",
"location": "Greater Seattle Area",
"connections": "500+",
"experience": [...],
"education": [...],
"skills": [...]
}
How It Works
- Trigger: Sends URL to Bright Data's Web Data API
- Poll: Waits for data collection to complete (checks every second)
- Return: Outputs structured JSON when ready
The polling mechanism handles rate limits and ensures data quality by waiting for full extraction.
Advanced: Direct Fetch
For custom dataset IDs or advanced use cases:
bash scripts/fetch.sh <dataset_id> '<json_input>'
Example:
bash scripts/fetch.sh gd_l1viktl72bvl7bjuj0 '{"url":"https://linkedin.com/in/someone"}'
04 同仓库更多 Skills
zustand-store-ts
Create Zustand stores following established patterns with proper TypeScript types and middleware.
zod-validation-expert
Expert in Zod — TypeScript-first schema validation. Covers parsing, custom errors, refinements, type inference, and integration with React Hook Form, Next.js, and tRPC.
zinc-database
Access ZINC (230M+ purchasable compounds). Search by ZINC ID/SMILES, similarity searches, 3D-ready structures for docking, analog discovery, for virtual screening and drug discovery.
zarr-python
Chunked N-D arrays for cloud storage. Compressed arrays, parallel I/O, S3/GCS integration, NumPy/Dask/Xarray compatible, for large-scale scientific computing pipelines.
zapier-workflows
Manage and trigger pre-built Zapier workflows and MCP tool orchestration. Use when user mentions workflows, Zaps, automations, daily digest, research, search, lead tracking, expenses, or asks to "run" any process. Also handles Perplexity-based research and Google Sheets data tracking.
zapier-make-patterns
No-code automation democratizes workflow building. Zapier and Make (formerly Integromat) let non-developers automate business processes without writing code. But no-code doesn't mean no-complexity - these platforms have their own patterns, pitfalls, and breaking points. This skill covers when to use which platform, how to build reliable automations, and when to graduate to code-based solutions. Key insight: Zapier optimizes for simplicity and integrations (7000+ apps), Make optimizes for power