shadow-frog-update

ShadowFrog Update

Updates .shadow/ from two sources: code changes (git diff) and conversational knowledge (what the user said during the session). Prerequisite: .shadow/ exists.

Triggers

1. Hook reminder: preToolUse injects a staleness warning when .shadow/_meta/state.json#last_commit differs from HEAD. The hook only reminds — it does NOT auto-run update.
2. Manual: user invokes /shadow-frog-update

Phase 1: Detect Changes

# Read last_commit defensively: it may be missing, or the literal "none"
# when init ran without git (e.g. inside a container). `git diff none HEAD`
# would abort with "fatal: bad revision 'none'", and a missing key would
# make $LAST_COMMIT empty so `git diff HEAD` silently reports the wrong set.
LAST_COMMIT=$(python3 -c "import json,sys; print(json.load(sys.stdin).get('last_commit','none'))" < .shadow/_meta/state.json 2>/dev/null || echo "none")
if git rev-parse --verify "$LAST_COMMIT" >/dev/null 2>&1; then
    git diff --name-only "$LAST_COMMIT" HEAD   # committed changes since last update
else
    echo "WARNING: state.json has no usable last_commit — falling back to a full re-scan."
fi
git diff --name-only HEAD                       # uncommitted changes
git diff --name-only --cached                   # staged changes

Categorize: modified, added, deleted, renamed.

Phase 2: Update Per-File Shadows (Symbol-Level)

Use the same extraction interface as init, rather than inventing a separate symbol scanner. From the project root:

python .github\skills\shadow-frog-init\shadow-init.py --symbols-only
# Claude Code: use .claude\skills\shadow-frog-init\shadow-init.py instead.

This prints JSON without changing .shadow/. It inherits the optional symbol_backend configuration from _meta/state.json (absent means regex). For Astred, run python install-astred.py (Linux: python3) from the ShadowFrog checkout once; setup defaults to latest stable, with optional --version <exact-version>. The executable is discovered from per-user setup configuration. SHADOWFROG_ASTRED_COMMAND or --astred-command override it; do not store its path in shared metadata. Unchanged Python files reuse _meta/symbols.json; changed Python files are batched through the pinned backend. C/C++ uses the recorded build-derived Astred project (or --astred-project). Native projects are always reparsed, even without source diffs, because headers can change declarations in unchanged translation units. Keep the configured compiler environment, includes, defines, and forced headers available; setup alone does not generate an OS build project. Other languages explicitly use regex. Stop on extraction failure; a failed inventory does not mean that all symbols disappeared. Use each record's qualified_name for file::symbol references and heading for Markdown headings.

When adopting Astred, upgrading its release/normalizer, or changing the native project/compile settings, first run --preview-migration --symbol-backend astred with the intended executable. Treat all affected shadow files as changed, even without a source diff. Also compare freshly extracted native symbols for unchanged source files after header changes; source hashes alone are insufficient. Review scope-change candidates; do not assume same-leaf names prove identity. Preserve unmatched/ambiguous sections and their discoveries as history until resolved. Apply only confirmed mappings, updating Also involves: and _cross/ **Refs**: consistently, without changing trust metadata or citation scores. Never use --reset to migrate. This is a reviewed agent workflow, not an automatic knowledge-rewriting command.

For each changed file, update its shadow at the symbol level:

Symbol change	Action
Added	Add the appropriate ##/### heading
Removed	Mark the section REMOVED; keep discoveries for history
Renamed	Move discoveries to the updated heading
Modified	Check whether discoveries still hold

Only mark source: user discoveries stale if the symbol is completely removed.

Lightweight update (auto/hook): re-extract symbols, update headings, flag stale. Deep update (manual/dream): read diffs, generate new discoveries, verify existing ones.

Phase 3: Capture Conversational Knowledge

When the user shares knowledge during the session, write it immediately. Do not batch for later.

Capture warnings, gotchas, design intent, history, deprecations, contracts, and conventions. Representative examples:

Signal	Example	Category
Warning	"Don't change the retry logic, it's subtle"	warning
Design intent	"We use this pattern because the API is unreliable"	intent
Convention	"Always use the helper in utils.py, not raw SQL"	convention

Write as:

- <user's words, as close to verbatim as possible>
  _(verified, source: user, citation_score: 0)_

For knowledge emerging from collaborative work (debugging, refactoring, test failures):

- <what was discovered and how>
  _(verified, source: interaction, citation_score: 0)_

Anchor to the specific file::symbol. source: user and source: interaction are always verified.

Auto-Placement

Users will not specify where to store their knowledge. You must find the correct location. Procedure:

1. Parse the user's statement for code references — file names, function names, class names, module names, variable names, error messages, CLI flags.
2. If the knowledge is a project-wide preference or convention with no code references (e.g., "always use snake_case", "no backward compatibility", "prefer small PRs") → write to _prefs.md.
3. If explicit references found → look up those file::symbol paths in _index.md and the corresponding shadow files.
4. If no explicit references → use context:
   • What file is the user currently viewing or editing?
   • What files were recently modified in this session?
   • Search shadow files: grep -rl "<keyword>" .shadow/ --include="*.md"
5. If multiple candidate locations → pick the most specific symbol that the knowledge applies to. Prefer a single file::symbol over file-level.
6. If the knowledge spans 3+ files → create a _cross/<slug>.md entry and add back-pointers to each involved file's ## Cross-References. For 2-file discoveries, use per-file entries with Also involves: instead.
7. If no matching location exists (e.g., the user mentions a concept not yet in the shadow) → place at the file-level ## File-Level section of the most relevant file, or create a new _cross/ entry for repo-wide knowledge.

Never ask the user "where should I put this?" — always resolve placement yourself.

Phase 4: Extract Session Insights

At session end or manual trigger, review the session for:

1. Files modified and why
2. Patterns revealed by the changes
3. Unrecorded conversational knowledge (user statements not yet shadowed)
4. Cross-cutting discoveries (create in _cross/<slug>.md if 3+ files involved)

Phase 5: Handle Structural Changes

Added files:

1. Check .shadow/.shadowignore — skip if the file matches an ignore pattern
2. Create .shadow/<path>/<file>.md with symbol-organized template
3. Add ## Cross-References section
4. Add to _index.md

Deleted files:

1. Add ORPHANED marker to shadow header
2. Keep shadow (discoveries explain history)
3. Mark [REMOVED] on any _cross/ refs pointing to this file
4. Update _index.md

Renamed files:

1. Move .shadow/<old>.md to .shadow/<new>.md
2. Update all _cross/ **Refs**: entries (old path → new path)
3. Update all Also involves: in other per-file shadows
4. Update _index.md
5. Preserve all discoveries

Phase 6: Verify and Dedup

Follow the dedup and writing rules in /shadow-frog — read before write, merge or update existing entries, fix bad format in place.

Verify exploration discoveries using the observe-based or do-based methods in /shadow-frog § Verification. source: user and source: interaction → always verified; only re-verify if the underlying code changes.

Phase 7: Verify Reference Integrity

Check the five core invariants (full 7-invariant set in /shadow-frog):

• Every _cross/<slug>.md ref has a back-pointer in per-file ## Cross-References
• Every ## Cross-References entry has a corresponding _cross/<slug>.md
• No duplicate cross-cutting filenames
• All Also involves: use file::symbol notation
• No duplicate discoveries (same behavioral claim at same symbol)

Repair any violations before proceeding.

Phase 8: Update Metadata

Preserve dream_cycles_completed from the existing state — only dream-reconcile.py increments it. Also preserve optional symbol_backend metadata. After a successful update, save the validated extraction inventory (without its optional migration section) to _meta/symbols.json; refresh fallback_languages from that report. For a reviewed backend/version migration, adopt the report's symbol_backend only after all affected headings and references are reconciled. Never publish new backend metadata or replace a good cache after a failed extraction or incomplete migration.

{
  "version": 1,
  "initialized_at": "<preserved>",
  "last_update_at": "<now ISO>",
  "last_commit": "<full 40-char HEAD SHA>",
  "last_update_type": "init|auto|manual|dream|meditate",
  "total_files": N,
  "total_symbols": N,
  "total_discoveries": N,
  "dream_cycles_completed": <preserved>
}

Refresh _index.md with current counts.

Discovery Writing Rules

See /shadow-frog § Discovery Format for the verbatim per-file, cross-cutting, and preference formats. Rules to keep in mind during update sessions:

• Start new claims/preferences with citation_score: 0; preserve the score when updating the same knowledge. Citation-only edits do not add discoveries.
• Record deliberately consulted existing entries once per task using the core shadow-cite.py helper, not a retrieval requirement or a second hidden store.
• Be behavioral: "silently returns None on expired tokens" not "handles token expiration"
• source: user and source: interaction → always verified, use user's own words
• source: exploration → mark uncertain unless verified by code reading or tests
• If 3+ files involved → create in _cross/<slug>.md instead, add back-pointers
• If project-wide preference with no file reference → write to _prefs.md
• Slug naming: kebab-case derived from title (e.g., "Token expiry config split" → token-expiry-config-split.md)