conducting-external-reconnaissance-with-osint
Conduct external recon using OSINT techniques to map an organization's external attack surface without touching target systems, gathering DNS records, certificate transparency logs, search results, social media, code repositories, and breach databases into a target profile. Use for the passive info-gathering phase of a pentest, external footprinting, or collecting employee/email intel for a social engineering campaign.
- License 许可
- Apache-2.0
- 最近更新
- 2026-08-25
- Stars
- ★ 0
- Forks
- 0
01 前言 Front Matter
- name
- conducting-external-reconnaissance-with-osint
- tags
- [ "OSINT", "reconnaissance", "attack-surface", "footprinting", "passive-recon" ]
- author
- mahipal
- domain
- cybersecurity
- license
- Apache-2.0
- version
- 1.0.0
- nist_csf
- [ "ID.RA-01", "ID.RA-06", "GV.OV-02", "DE.AE-07" ]
- subdomain
- penetration-testing
- description
- Conduct external recon using OSINT techniques to map an organization's external attack surface without touching target systems, gathering DNS records, certificate transparency logs, search results, social media, code repositories, and breach databases into a target profile. Use for the passive info-gathering phase of a pentest, external footprinting, or collecting employee/email intel for a social engineering campaign.
- mitre_attack
- [ "T1595", "T1592", "T1589", "T1590" ]
02 资源文件 3
- 许可证LICENSE11.0 KB
- 参考文档references/api-reference.md1.8 KB
- 脚本scripts/agent.py9.1 KB
03 SKILL.md 正文
Conducting External Reconnaissance with OSINT
When to Use
- Performing the initial reconnaissance phase of a penetration test to gather intelligence before active scanning
- Mapping an organization's external attack surface to identify unknown or shadow IT assets
- Collecting employee information, email formats, and organizational structure for social engineering campaigns
- Identifying exposed credentials, leaked data, or sensitive documents published on the internet
- Scoping the breadth of an organization's digital footprint prior to a red team engagement
Do not use for stalking, harassment, or unauthorized surveillance of individuals. OSINT gathering must be conducted within the scope of an authorized engagement and comply with applicable privacy laws (GDPR, CCPA).
Prerequisites
- Written authorization to perform reconnaissance against the target organization
- Dedicated research workstation with a VPN or Tor for anonymized queries when required
- OSINT framework tools installed: Amass, theHarvester, Shodan CLI, Recon-ng, SpiderFoot
- API keys for Shodan, Censys, SecurityTrails, Hunter.io, VirusTotal, and GitHub for enhanced results
- Disposable email accounts for accessing services that require registration during research
Workflow
Step 1: Domain and DNS Enumeration
Enumerate all domains, subdomains, and DNS records associated with the target:
- Root domain identification: Start with the primary domain and identify all related domains through reverse WHOIS lookups on registrant name, email, and organization using
whoxy.comordomaintools.com - Subdomain enumeration: Run multiple tools for comprehensive coverage:
amass enum -passive -d target.com -o amass_subs.txtfor passive subdomain discovery from 40+ data sourcessubfinder -d target.com -all -o subfinder_subs.txtfor fast passive enumerationcrt.shcertificate transparency log queries:curl -s "https://crt.sh/?q=%25.target.com&output=json" | jq -r '.[].name_value' | sort -u
- DNS record analysis: Query for all record types:
dig target.com ANY, check for SPF, DKIM, DMARC records that reveal email infrastructure, and enumerate MX records to identify email providers - Zone transfer attempt:
dig axfr @ns1.target.com target.comto check for misconfigured DNS servers - Consolidate results: Merge, deduplicate, and resolve all discovered subdomains to IP addresses. Map IP addresses to ASN and hosting providers.
Step 2: Infrastructure and Service Discovery
Identify internet-facing infrastructure without directly scanning target systems:
- Shodan:
shodan search "ssl.cert.subject.cn:target.com"to find all internet-facing services with TLS certificates for the target domain. Also search by organization name and IP ranges. - Censys: Search for target's IP ranges and TLS certificates to identify services, technologies, and potential vulnerabilities indexed from internet-wide scanning
- Cloud asset discovery: Check for S3 buckets (
target-com,target-backup,target-dev), Azure Blob storage (target.blob.core.windows.net), and GCP storage using tools likecloud_enum - WAF and CDN identification: Use
wafw00f target.comto identify web application firewalls and CDN providers that may mask the origin server IP - Historical data: Use Wayback Machine (
web.archive.org) to find removed pages, old application versions, and forgotten endpoints
Step 3: Email and Personnel Intelligence
Gather employee information and email addresses for social engineering preparation:
- Email harvesting:
theHarvester -d target.com -b all -f harvest_results.htmlto collect emails from search engines, LinkedIn, and data sources - Email format identification: Use
hunter.ioto determine the email format (first.last, flast, firstl) and verify deliverability - LinkedIn reconnaissance: Identify employees by department, particularly IT administrators, security team members, and executives. Note technologies mentioned in job postings and employee profiles.
- Organizational chart: Build an org chart from LinkedIn data to understand reporting structures, identify key personnel, and map departments
- Social media analysis: Review employee social media profiles for information about internal tools, technologies, office locations, badge photos, and security practices
- Job postings: Analyze current and historical job postings on the company career page and job boards for technology stack details, tools, and infrastructure information
Step 4: Credential and Data Leak Analysis
Search for exposed credentials and sensitive data:
- Breach databases: Check
haveibeenpwned.comAPI for breached email addresses associated with the target domain - Paste sites: Search Pastebin, GitHub Gists, and similar paste sites for leaked credentials, configuration files, or internal documents
- Code repositories: Search GitHub, GitLab, and Bitbucket for:
org:target "password",org:target "api_key",org:target "secret"- Use
trufflehogorgitleaksfor automated secret scanning across the target's public repositories
- Document metadata: Download publicly available documents (PDF, DOCX, XLSX) from the target website and extract metadata using
exiftoolto reveal internal usernames, software versions, printer names, and file paths - Google dorking: Use targeted search operators:
site:target.com filetype:pdffor public documentssite:target.com inurl:adminfor admin panelssite:target.com "index of /"for directory listingssite:pastebin.com "target.com"for paste site mentions
Step 5: Technology Stack Profiling
Identify the technologies, frameworks, and services used by the target:
- Web technology fingerprinting: Use
whatweb target.comor Wappalyzer browser extension to identify CMS, frameworks, JavaScript libraries, analytics, and server software - SSL/TLS analysis:
sslyze target.comortestssl.sh target.comto identify cipher suites, protocol versions, certificate details, and cryptographic weaknesses - JavaScript analysis: Download and review JavaScript files for framework identifiers, API endpoints, internal hostnames, and version strings
- DNS-based service identification: Review TXT records for service providers (e.g.,
v=spf1 include:_spf.google.comindicates Google Workspace,MS=msXXXXXXindicates Microsoft 365) - Mobile app analysis: Download the target's mobile applications from app stores and analyze with
apktool(Android) orfridafor hardcoded URLs, API endpoints, and embedded credentials
Key Concepts
| Term | Definition |
|---|---|
| OSINT | Open Source Intelligence; intelligence collected from publicly available sources including websites, social media, public records, and government data |
| Passive Reconnaissance | Information gathering without directly interacting with target systems, leaving no footprint in target logs |
| Active Reconnaissance | Information gathering that involves direct interaction with target systems (scanning, probing) and may be logged |
| Certificate Transparency | Public logs of TLS certificates issued by certificate authorities, queryable to discover subdomains and infrastructure |
| Attack Surface | The sum of all points where an unauthorized user can attempt to enter or extract data from an environment |
| Google Dorking | Using advanced Google search operators to find sensitive information indexed by search engines that was not intended to be public |
| Shadow IT | Technology systems and services deployed by employees or departments without the knowledge or approval of the IT department |
Tools & Systems
- Amass (OWASP): Comprehensive subdomain enumeration tool that combines passive sources, DNS brute-forcing, and certificate transparency log analysis
- Shodan: Internet-wide scanning database that indexes services, banners, and metadata for internet-connected devices, searchable by IP, domain, or organization
- theHarvester: OSINT tool for gathering emails, subdomains, hosts, employee names, and open ports from public sources
- SpiderFoot: Automated OSINT collection platform that queries 200+ data sources and correlates findings into a unified graph
- Recon-ng: Modular web reconnaissance framework with a database backend for organizing and cross-referencing discovered intelligence
Common Scenarios
Scenario: Pre-Engagement Reconnaissance for a Red Team Exercise
Context: A technology company has contracted a red team assessment. Before active testing begins, the team conducts passive OSINT to map the attack surface and identify potential entry points. The target is a SaaS company with 500 employees and a primary domain of techcorp.io.
Approach:
- Enumerate 147 subdomains via Amass and crt.sh, including staging.techcorp.io, jenkins.techcorp.io, and vpn.techcorp.io
- Shodan reveals a forgotten Elasticsearch instance on port 9200 with no authentication exposed to the internet
- theHarvester collects 89 employee email addresses, revealing the format first.last@techcorp.io
- GitHub search discovers a former developer's public repository containing a
.envfile with AWS access keys - LinkedIn analysis reveals the company uses Okta for SSO, Jira for project management, and AWS for hosting
- Google dorking finds a directory listing on docs.techcorp.io exposing internal architecture diagrams
- Compile all intelligence into a reconnaissance report that feeds directly into the threat modeling and attack planning phases
Pitfalls:
- Relying on a single subdomain enumeration tool and missing assets found by other tools using different data sources
- Failing to check cloud storage services (S3, Azure Blob, GCP) for publicly accessible buckets
- Not searching for credentials in public code repositories, which frequently yield immediate access
- Conducting active scanning (port scans, vulnerability scans) during what should be a passive-only phase
Output Format
## External Reconnaissance Report - TechCorp.io
### Attack Surface Summary
- **Domains discovered**: 3 (techcorp.io, techcorp.com, techcorpapp.com)
- **Subdomains enumerated**: 147 unique subdomains across all domains
- **Unique IP addresses**: 34 IPs mapped across AWS us-east-1 and us-west-2
- **Email addresses collected**: 89 valid corporate email addresses
- **Exposed services**: 12 internet-facing services identified via Shodan/Censys
### Critical Findings
**1. Unauthenticated Elasticsearch Instance**
- Host: 52.xx.xx.xx:9200 (elastic.techcorp.io)
- Indexed data: Application logs containing user session tokens and PII
- Source: Shodan search "ssl.cert.subject.cn:techcorp.io"
**2. AWS Credentials in Public GitHub Repository**
- Repository: github.com/former-dev/techcorp-scripts
- File: .env containing AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
- Status: Keys appear active (not tested - out of scope for passive recon)
**3. Directory Listing Exposing Internal Documents**
- URL: https://docs.techcorp.io/internal/
- Contents: Architecture diagrams, network topology, runbooks
- Source: Google dork "site:techcorp.io intitle:index.of"
### Recommendations
1. Immediately rotate the exposed AWS credentials and audit CloudTrail logs
2. Restrict Elasticsearch access to internal networks or add authentication
3. Disable directory listings on docs.techcorp.io and audit all web servers
4. Implement GitHub secret scanning across all organization repositories
04 同仓库更多 Skills
syllabus
Generates a curated supplementary reading list from any course syllabus using Consensus academic search. Grill-me intake (syllabus input format + course audience + year range) plus a grouping forcing-options checkpoint before any search runs — so the reading list matches the course's level and recency need. Parses the syllabus to extract topics and learning outcomes, searches Consensus for recent peer-reviewed papers per topic, and produces a professionally formatted .docx with clickable Consensus links, plain-language summaries calibrated to audience level, and Bloom-higher-order discussion questions tied to course learning goals. Use when the user uploads a syllabus, course outline, or curriculum document and wants supplementary readings (e.g., 'create a reading list from this syllabus', 'find recent papers for my course') — even casual mentions with a syllabus attached should trigger this skill.
pulse
Multi-source recency research skill that takes the pulse of any topic across Reddit, Hacker News, the open web, and optionally X/Twitter within a configurable recent window (default 30 days). Forcing intake clarifies topic specificity, angle (trend/sentiment/problems/opportunities/comparison), time window, and platform scope before searching. Returns a synthesized briefing with citations, engagement metrics, and cross-platform pattern analysis. Use when the user requests multi-source recency intelligence on a topic (e.g., 'pulse on [topic]', 'what's happening with [topic]', 'what are people saying about [topic]', 'current conversation about [topic]', 'take the pulse of [topic]', 'trending: [topic]', 'find me info on [topic]'), and for competitor research, trend discovery, tool comparisons, and audience sentiment analysis.
patent
Patent prior-art and landscape intelligence skill — not generic patent help. Commits to one of five sub-use-cases via forcing intake (novelty search / freedom-to-operate / competitive landscape / acquisition diligence / litigation prior-art) before any search runs. Searches Google Patents, Espacenet, USPTO, and optionally Lens.org for citation-graph signals. Output is an editable Word document (.docx) with verdict, ranked closest art (claim-text extracted), CPC-class-aware landscape, family-resolved hits, geographic coverage, FTO flags where applicable, strategy recommendations, and full audit log. Use when the user asks for patent searching or analysis (e.g., 'prior art search for [invention]', 'freedom to operate analysis for [product]'). Produces search signal, not legal advice — always recommends consulting a patent attorney before filing or licensing decisions. Trademark, copyright, and trade-secret questions are out of scope.
notebooklm
Browser automation skill for controlling Google's NotebookLM. Use when the user wants anything done in NotebookLM (e.g., 'open NotebookLM', 'check my [name] notebook', 'ask my notebook about X', 'add [source] to NotebookLM', 'generate a Video Overview from my notebook', 'use NotebookLM Studio'). Handles reading and querying notebooks, adding sources (URLs, text, files, YouTube links, synthesized content), generating Studio outputs (Audio/Video Overviews, Mind Maps, Reports incl. Briefing Doc/Study Guide/FAQ, Flashcards, Quiz, slide decks, infographics — discover the exact set from the live Studio panel; the UI evolves fast), and creating new notebooks. Requires browser automation environment — fails gracefully when unavailable.
litreview
Academic literature orientation skill that searches papers via free keyless APIs (PubMed E-utilities + OpenAlex) by default — with the Consensus MCP as an optional enhancement lane when connected — builds a strategic search plan using PICO (default) or SPIDER / Decomposition / hybrid as fallbacks, and synthesizes findings into a formatted Word (.docx) research guide. Grill-me intake (research question specificity + framework hint + tentative depth) before the recon search; a second forcing checkpoint after Phase 2 confirms framework + sub-areas + depth before searches consume budget. Configurable depth (5/10/20 queries) controls coverage vs. speed. Output is a 'launching pad' — an orientation guide that lets a researcher dive in confidently, not a finished review. Use when the user starts literature-oriented research (e.g., 'litreview on [topic]', 'literature review on [topic]', 'I'm starting a literature review on X', 'I'm writing a paper on X', 'help me research X', 'I'm doing research on X', 'can you help me research X'). Do NOT use for single one-off paper searches wanting a quick list — that's a plain PubMed/OpenAlex (or Consensus) query.
grants
NIH grant research skill for clinical researchers. Grill-me intake (research idea + career stage + preliminary data + environment + submission posture + known institute targets) locks down the funding strategy before any search runs. Runs a 5-facet Consensus positioning analysis (with draft Significance/Innovation language), maps the research to the right NIH institutes and study sections via RePORTER, finds NOSIs and funded overlap, and produces an editable Word document (.docx) with budget/scope-aware mechanism recommendations, submission timelines, and a mandatory program officer recommendation. Use when the user asks about research funding or makes any grant-related request (e.g., 'grants for [topic]', 'find grants for my research idea', 'what grants match my research', 'help me find NIH funding', 'grant opportunities for my research'). NIH-only scope — non-NIH funders (PCORI, DOD CDMRP, VA, foundations) are out of scope and flagged at intake.