AGENT SKILL

sf-civic-digest

Track San Francisco city government activity — Board of Supervisors, Land Use committee, Public Safety committee, SFMTA Engineering Public Hearings, SF Planning Commission, Historic Preservation Commission, Zoning Administrator hearings, Planning project notices, and SF.gov district events. Fetches agendas and recaps from SF Legistar, sfmta.com, sfplanning.org, the sf.gov Wagtail CMS API (api.sf.gov), and Socrata open data via curl-based scraping. Filtered by supervisorial district, neighborhood, streets, or topic keywords. Use when a user asks what's happening at City Hall, wants to track legislation or development in their neighborhood, wants a weekly or daily SF civic digest, asks about upcoming public hearings, planning variances, or wants to know what their supervisor is working on. SF-specific.

License 许可
MIT
最近更新
2026-06-10
Stars
★ 3
Forks
0

01 前言 Front Matter

name
sf-civic-digest
description
Track San Francisco city government activity — Board of Supervisors, Land Use committee, Public Safety committee, SFMTA Engineering Public Hearings, SF Planning Commission, Historic Preservation Commission, Zoning Administrator hearings, Planning project notices, and SF.gov district events. Fetches agendas and recaps from SF Legistar, sfmta.com, sfplanning.org, the sf.gov Wagtail CMS API (api.sf.gov), and Socrata open data via curl-based scraping. Filtered by supervisorial district, neighborhood, streets, or topic keywords. Use when a user asks what's happening at City Hall, wants to track legislation or development in their neighborhood, wants a weekly or daily SF civic digest, asks about upcoming public hearings, planning variances, or wants to know what their supervisor is working on. SF-specific.

03 SKILL.md 正文

SF Civic Digest — SKILL.md

Cold-start note: This file is designed to be self-sufficient. An agent reading only SKILL.md should be able to run reports, generate digests, and understand the full data stack. No other context required.


Quick Start

# Step 1: Find the absolute path to this skill's scripts/ directory
#         (it's always a sibling of this SKILL.md file)
SKILL_DIR="/absolute/path/to/sf-civic-digest"

# Step 2: Run scripts using that absolute path (--district N required)
python3 "$SKILL_DIR/scripts/sf_weekly_digest.py" --district 5 --json

# Step 3: Synthesize the JSON output into a narrative report (see STYLE.md)

Note: sf_weekly_digest.py makes ~22 network calls sequentially and can take 2-3 minutes. For a quick test, try a single script first: python3 "$SKILL_DIR/scripts/sf_journalism.py" --district 5 --days 7 --json

District: Pass --district N (1–11) to any script. User preferences and neighborhood context live in USER.md — read it before writing a report. There is no civic_config.json or civic_config.example.json; those files have been removed.

⚠️ Path rule — critical for any agent (any agent, IDE, or human):

  • All scripts must be run with the absolute path to the scripts/ directory
  • Find it: scripts/ is always a sibling of this SKILL.md file
  • Example: if SKILL.md is at /home/alice/.openclaw/workspace/skills/sf-civic-digest/SKILL.md, then scripts are at /home/alice/.openclaw/workspace/skills/sf-civic-digest/scripts/
  • Never use bare relative paths like scripts/foo.py — they break when the working directory is anything other than the skill root
  • When in doubt: realpath on any script path gives you the absolute path

Architecture

Scripts are data pipelines. The agent writes the narrative.

The scripts fetch, parse, dedup, and output JSON. The agent (you) reads that JSON and synthesizes it into a report following the style guide. There is no auto-generated report — the editorial judgment is yours.

sf_weekly_digest.py --json
    └── calls all subscripts → returns combined JSON
            │
            ▼
    Agent reads JSON + STYLE.md
            │
            ▼
    Agent writes narrative report

For report generation, see Report Generation section below.


All Scripts

Canonical script location: skills/sf-civic-digest/scripts/

Script What it does Method Notes
sf_weekly_digest.py Master aggregator — calls all subscripts — Use --json for machine-readable output; pass to agent for synthesis
sf_civic_digest.py Board of Supervisors + all committees via Legistar curl --daily for delta only; --days N; state in sf_civic_state.json
sfmta_hearings.py SFMTA Engineering Public Hearings curl + PDF Street changes, parking meters, signals. Every ~2 weeks.
sf_planning_notices.py SF Planning project notices (Section 311) curl Earliest signal layer — before commission hearings. Needs Accept: text/html header.
sf_planning_commission.py Planning Commission, HPC, ZA full agendas curl + PDF Server-rendered HTML (no browser). --body hpc for HPC, --body za for ZA.
sf_board_of_appeals.py Board of Appeals Wagtail API sf.gov CMS API (api.sf.gov). 1st & 3rd Wednesdays, 5pm. No browser needed.
sf_building_permits.py Building permits citywide Socrata REST Every permit filed. Classified HIGH/NOTABLE/noise. State-tracked.
sfgov_events.py Supervisor office hours, district events REST/JSON Very sparse — supervisor events only. Low signal.
sf_311.py 311 service requests by district Socrata REST Spike detection. Query two 7-day windows for trend comparison.
sf_ethics.py Lobbyist contacts + campaign contributions RSS + Socrata Contacts: 5f5n-tdbf. Contributions: e6py-fg8b. Column names have NO underscores.
sf_rent_board.py Rent Board commission meetings Wagtail API sf.gov CMS API (api.sf.gov). No browser needed. Rent increase: links to sf.gov (no hardcoded value).
sf_sfmta_board.py SFMTA Board of Directors curl 1st & 3rd Tuesdays 1pm. Service changes, fare decisions, bike/ped safety.
sf_rec_park.py Rec & Park Commission curl (Granicus) view_id=91. sfrecpark.org is unreachable from server — Granicus only.
sf_housing_pipeline.py AB 2011 / SB 35 / density bonus housing pipeline Socrata REST Cross-refs parcels dataset for district. Watchlist tracks 400 Divisadero. State-tracked.
sf_journalism.py News aggregator — 5 SF outlets RSS Mission Local, Streetsblog SF, SF YIMBY, 48 Hills, SF Standard. SF Examiner 404s. Archive: sf_journalism_archive.json.
sf_volunteer_cleanups.py Community cleanup events curl/JS bundle Refuse Refuse SF + DPW Love Our City NBDs. GoDaddy SPA — data is in JS bundle, not HTML.
sf_bart_board.py BART Board of Directors + advisory bodies Legistar API webapi.legistar.com/v1/bart/Events. Board = HIGH, advisory = MEDIUM. --next for next meeting.
sf_sfusd_board.py SFUSD Board of Education meetings Google Sheets + BoardDocs 2nd & 4th Tuesdays 5pm. Falls back to projected schedule if sources unavailable.
sf_evictions.py Eviction notices by district Socrata REST Dataset 5cei-gny5. Ellis Act + OMI = displacement signals. Trend comparison (two windows). --district N.
sf_sfcta.py SF County Transportation Authority curl (sfcta.org) Congestion pricing, Muni Forward, bike/ped funding, Prop L sales tax. Board + CAC + committees.
sf_civic_actions.py Protests, rallies, civic actions Mobilize.us API + Indybay No auth. Rallies/marches = HIGH, canvass/phone bank = MEDIUM. --type rally to filter.
sf_mission_local.py Mission Local news only RSS Subset of sf_journalism.py. Use sf_journalism.py for multi-outlet.
config_loader.py District config loader — All 11 districts pre-populated. Pass --district N to any script.

No browser required

All scripts now work without a browser. sf_board_of_appeals.py and sf_rent_board.py use the sf.gov Wagtail CMS API (api.sf.gov/api/v2/pages/). sf_planning_commission.py uses curl against server-rendered Drupal HTML (the sfplanning.org calendar pages are not JS-dependent despite initial assumptions).


District Configuration

All scripts accept --district N (1–11). That's all you need for a basic run.

User preferences and neighborhood context (streets, addresses, people to flag) live in USER.md — a human-readable profile file. There is no civic_config.json; that file has been removed. Read USER.md to understand what the user cares about before writing any report.

District→neighborhood mappings: references/sf-districts.md Build timeline benchmarks: references/sf-build-timelines.md — Read before writing about any housing project. Includes typical phase durations, financing tracking methods, and context for whether a project is on track or stalled.


Report Generation

Agent-driven (recommended for one-off reports)

# 1. Collect data (use absolute paths — see path rule above)
python3 /absolute/path/to/sf-civic-digest/scripts/sf_weekly_digest.py --json > /tmp/civic_data.json
python3 /absolute/path/to/sf-civic-digest/scripts/sf_housing_pipeline.py --district 5 --json >> /tmp/civic_data.json
python3 /absolute/path/to/sf-civic-digest/scripts/sf_volunteer_cleanups.py --district 5 --days 14 --json >> /tmp/civic_data.json

# 2. Agent synthesizes following STYLE.md
# Read STYLE.md, read the JSON, write narrative to reports/

Agent-driven reports (quarterly, annual, deep dives)

Give your agent the workspace path and let it orchestrate:

Read STYLE.md and USER.md. Run the scripts in scripts/ to collect data for District 5, then write a Q1 2026 civic digest. Save to reports/.

⚠️ Path gotcha: Agents default to wrong relative paths. Always resolve the absolute path to scripts/ from the location of this SKILL.md before running anything. Use full absolute paths — e.g. python3 /full/path/to/scripts/sf_housing_pipeline.py. Never just scripts/SCRIPTNAME.py.

Report style

Read STYLE.md before writing any report. Key rules:

  • TLDR (5-7 bullets) first — last bullet is always one action
  • Officials box: YOUR elected officials only (supervisor, mayor, state reps)
  • Short sections (3-5 sentences), scan-and-skip
  • Connect threads across sources (lobbying + permits + 311 + journalism = one story)
  • End each thread with a specific action (date, email address, what to say)
  • Rates over counts. "Up 15%" beats "95 reports."
  • Tone: pro-building, pro-tech, car-free lens. See STYLE.md for full framing guidance.

Reference: see STYLE.md for rules and examples.


Key Data Gotchas

These will burn you if you don't know them:

Socrata column names:

  • 311 (vw6y-z8j6): supervisor_district = '5.0' (float string), date field = requested_datetime
  • Building Permits (i98e-djp9): supervisor_district = '5'. Address split across street_number/street_name/street_suffix — no combined field. estimated_cost is a string, cast in Python.
  • Lobbyist Contacts (5f5n-tdbf): NO underscores — officialname, clientname, lobbyistname
  • Lobbyist Contributions (e6py-fg8b): same — lobbyistname, candidatename, sourceoffunds
  • Planning Records (qvu5-m3a2): pim_link returns {"url": "..."} dict, not string. No supervisor_district — cross-reference parcels (acdm-wktn) by block/lot.

Legistar: The API is broken for SF (returns "Agenda Draft Status" error). Use the RSS feed instead: https://sfgov.legistar.com/Feed.ashx?M=C&ID=17442&GUID=EEE85B7C-1A1C-4A56-873E-355A0A0DE5C3&Mode=All&Format=rss Returns ~100 meetings (~6 months). Parse meeting IDs from RSS, then fetch_meeting_detail() for agenda items.

Reddit: Direct curl blocked (403, datacenter IP). Use web_fetch tool — routes differently, works.

sfrecpark.org: Completely unreachable from this server. Granicus (view_id=91) is the only path.

311 trend pattern: Query two 7-day windows (current + prior week) and show week-over-week delta. "Encampments up 15%" is useful. "95 encampments" is not.

Journalism archive: sf_journalism.py persists articles to sf_journalism_archive.json (cap 2000). For quarterly history, use --archive --days 90.


Socrata Dataset Reference

Dataset ID Key fields
Building Permits i98e-djp9 permit_number, permit_type, address (split), supervisor_district ('5'), estimated_cost (string)
311 Cases vw6y-z8j6 service_name, requested_datetime, supervisor_district ('5.0')
Planning Dept Records qvu5-m3a2 record_id, description, record_status, project_address, block, lot, number_of_units_net, pim_link (dict)
SF Parcels acdm-wktn block_num, lot_num, supervisor_district, zoning_code
Lobbyist Contacts 5f5n-tdbf officialname, lobbyistname, clientname, outcomesought, subjectarea, filenumber
Lobbyist Contributions e6py-fg8b lobbyistname, candidatename, sourceoffunds, committeename
Assessor Parcels fk72-cxc3 property_location, parcel_number, assessed_land_value, zoning_code
Eviction Notices 5cei-gny5 eviction_id, address, neighborhood, file_date, supervisor_district, ellis_act_withdrawal, owner_move_in

Query pattern: https://data.sfgov.org/resource/{id}.json?$where=supervisor_district='5'&$order=date+DESC&$limit=50


Scraping Method Reference

Site curl works? Browser needed?
Legistar (sfgov.legistar.com) ✅ No
SFMTA (sfmta.com) ✅ No
SF Planning notices (sfplanning.org/notices) ✅ (needs Accept header) No
SF Planning calendars (sfplanning.org/hearings-*) ✅ (use -list views) No
SF.gov meetings (api.sf.gov) ✅ (Wagtail API) No
Socrata (data.sfgov.org) ✅ No
SF Ethics (sfethics.org) ✅ No
Mission Local / journalism RSS ✅ No
Granicus (Rec & Park) ✅ No
Refuse Refuse SF (refuserefusesf.org) ✅ (JS bundle) No
sfrb.org ✅ (Wagtail API via api.sf.gov) No

Scheduling

Weekly digest (Monday morning)

{
  "schedule": { "kind": "cron", "expr": "0 14 * * 1", "tz": "America/Los_Angeles" },
  "payload": { "kind": "systemEvent", "text": "Run SF civic weekly digest and send summary" }
}

Daily delta (weekday noon check-in)

{
  "schedule": { "kind": "cron", "expr": "0 12 * * 1-5", "tz": "America/Los_Angeles" },
  "payload": { "kind": "systemEvent", "text": "Run SF civic daily delta and report any new agenda items" }
}

File Layout

skills/sf-civic-digest/
├── SKILL.md              ← you are here
├── STYLE.md              ← report style guide (read before writing any report)
├── USER.md               ← user profile: district, neighborhoods, streets, interests (fill this in)
├── .gitignore            ← excludes runtime state from version control
├── scripts/              ← all data pipeline scripts
│   ├── sf_weekly_digest.py      ← master aggregator
│   ├── sf_civic_digest.py       ← Board of Supervisors (Legistar)
│   ├── sfmta_hearings.py        ← SFMTA Engineering Hearings
│   ├── sf_planning_notices.py   ← Planning project notices
│   ├── sf_planning_commission.py← Planning Commission / HPC / ZA
│   ├── sf_board_of_appeals.py   ← Board of Appeals
│   ├── sf_building_permits.py   ← Building permits (Socrata)
│   ├── sfgov_events.py          ← Supervisor events
│   ├── sf_311.py                ← 311 service requests (Socrata)
│   ├── sf_ethics.py             ← Lobbyist contacts + contributions
│   ├── sf_rent_board.py         ← Rent Board
│   ├── sf_sfmta_board.py        ← SFMTA Board of Directors
│   ├── sf_rec_park.py           ← Rec & Park Commission
│   ├── sf_housing_pipeline.py   ← Housing pipeline (AB2011/SB35/density bonus)
│   ├── sf_journalism.py         ← News aggregator (5 outlets)
│   ├── sf_volunteer_cleanups.py ← Community cleanups (Refuse Refuse + DPW)
│   ├── sf_evictions.py          ← Eviction notices (Socrata)
│   ├── sf_bart_board.py         ← BART Board of Directors (Legistar)
│   ├── sf_sfusd_board.py        ← SFUSD Board of Education
│   ├── sf_sfcta.py              ← SF County Transportation Authority
│   └── config_loader.py         ← District config loader
├── reports/              ← generated reports
│   └── archive/          ← older versions
└── references/
    ├── sf-districts.md       ← district map, supervisor names, neighborhood lists
    └── sf-build-timelines.md ← build phase benchmarks, financing signals, project context

Participation Guide

How to comment at SFMTA Engineering Hearings

  • Online: join at sfmta.com/EngHearing when your item is called, raise hand
  • Phone: number + conference ID in hearing notice
  • Written: email the contact listed per item with "Public Hearing" in subject
  • Decisions posted the following Friday at sfmta.com/EngineeringResults

How to comment at Planning Commission / HPC

  • Written comments accepted before each hearing via sfplanning.org
  • HPC meets 1st and 3rd Wednesdays (low attendance, high impact)
  • In-person: City Hall Room 400

How to reach your supervisor

  • Mahmood (D5): bilal.mahmood@sfgov.org
  • Constituent meetings: request via supervisor's website
  • Board public comment: first Tuesday of each month, City Hall Room 250

04 同仓库更多 Skills