chore(agents): migrate to agents.md protocol — .agents/ wiki/plans/rules/skills

Root AGENTS.md is now a thin entry point + knowledge map; the batch-workflow
detail it duplicated already lived in .claude/skills/animation/SKILL.md.
Folded docs/ (dances registry + iclone-bridge stub), root plans/, and
devops-reports/ into .agents/wiki/ and .agents/plans/ per the per-repo .agents/
convention. Added .agents/SOUL.md, .agents/AGENTS.md, wiki/ARCHITECTURE.md,
wiki/architecture/, and wiki/master-plan.md (derived from the closed plans +
the dance registry's live ceremony-slot table). Fixed doc-path references in
exchange/GUIDE.md, the animation skill, and the iclone-bridge stub. All moves
via mv (git detected as renames) — no content deleted.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

.agents/plans/animation-gen-pipeline-2026-07-13.md

@@ -0,0 +1,114 @@
+# Animation Gen Pipeline — Video → Recreated Dance
+
+**Goal:** give an agent a video of people dancing; the agent breaks the choreography
+down and recreates it in-game as a dance JSON, using Kevin Iglesias (KI) clips, UAL
+clips, composites, and Mixamo retargets — then verifies its own recreation against the
+source and iterates.
+
+**Honest framing:** this produces a *cover version*, not motion capture. Continuous
+human motion gets quantized into a discrete vocabulary of clips + composites. For game
+choreography (kapa haka teams, dance floor content) that is the desired output.
+
+## Verified infrastructure (all exists, all tested 2026-07-13)
+
+| Piece | Where | Status |
+|---|---|---|
+| Dance runtime + hot-reload JSON | `src/Testing/Dance/`, `assets/dances/*.json` | SHIPPED — test bed plays dances, reloads in ~0.5s |
+| Composite baking (upper/lower masks) | `CompositeClipBaker.cs` | SHIPPED |
+| BPM beat clock + quantized transitions | `BeatClock.cs`, `bpm` in dance JSON | SHIPPED |
+| Viewport → mp4 recording | `VideoRecorder.cs` → `~/Downloads/dance_*.mp4` | SHIPPED |
+| Mixamo FBX → Quaternius GLB | `tools/mixamo_retarget.py` (Blender 5.1.2 headless) | PROVEN (`northern_soul.glb`) |
+| Clip vocabulary | 12 KI packs + UAL1/2 + mixamo dir, ~810 clips; KI social = Dance01–18, DancePoses, claps, cheers | SHIPPED |
+| Pose extraction from video | `pose-estimation` skill (user-level, MediaPipe Tasks) | BUILT + TESTED today |
+| Pose sequence comparison (DTW, 8 joint angles) | `compare_poses.py` in same skill | BUILT + TESTED (identity=100.0, cross-segment≈71) |
+| Animation creation recipes | `.claude/skills/animation-creation/SKILL.md` | BUILT today |
+
+**Key experimental finding (2026-07-13):** MediaPipe tracks the stylized Quaternius
+characters at **94% detection** when ONE character is crop-zoomed to fill the frame
+(`crop=140:180:350:420,scale=560:720:flags=lanczos` on a test bed recording). Wide
+formation shots detect **0%**. Every capture step below therefore frames a single dancer.
+
+## Pipeline stages
+
+### Stage 1 — Ingest the source video
+1. `ffprobe` duration/fps; extract audio (`ffmpeg -vn audio.wav`).
+2. **BPM + beat grid**: `librosa.beat.beat_track` (add librosa to the pose-estimation
+   venv when first needed). Beat times become the dance's `bpm` and the sampling grid.
+3. Extract pose: `extract_pose.py video.mp4 --times "<beat grid>" --overlay check.mp4`.
+   If multi-person/wide: crop-zoom the clearest dancer first. Eyeball the overlay.
+4. Extract frames on the beat grid for the vision pass (`ffmpeg -vf select=...`).
+
+### Stage 2 — Choreography breakdown
+- **Programmatic move segmentation**: the 8-angle signature (elbows/shoulders/hips/knees)
+  from the pose JSON gives a per-beat pose vector; a spike in angle-space distance
+  between consecutive beats = likely move boundary. Cheap, deterministic.
+- **Vision pass**: agent reads beat-grid frames and writes a per-segment description
+  ("beats 1–8: crouched stomp, arms slapping thighs..."). Combines with the programmatic
+  boundaries — vision names the moves, angles locate them.
+- Output: `breakdown.json` — segments with beat ranges, text description, pose signature.
+
+### Stage 3 — Clip card catalog (the one missing build item)
+One-time batch job; the enabler for matching. For each dance-relevant clip
+(KI Dance/DancePose/claps/cheers + UAL emotes + mixamo pack):
+1. Test bed plays the clip on ONE dancer, camera framed close (solo framing per the
+   94% finding). VideoRecorder captures ~2 loops.
+2. `extract_pose.py` on the capture → **pose signature** stored per clip.
+3. Delegated vision (pi `nova_describe` / cheap model) writes a **text card** per clip.
+4. Output: `assets/dances/catalog/clip_cards.json` — `{clipRef, text, bpm-ish tempo,
+   energy, pose_signature_file}`. Rebuild only when packs change.
+
+Build item: a `CatalogMode` in the dance test bed (iterate clips → frame solo dancer →
+record → next). Everything else is scripting.
+
+### Stage 4 — Matching (segment → clip or composite)
+1. **Numeric**: DTW each video segment's angle sequence against every catalog clip
+   signature (`compare_poses.py` logic). Rank by score.
+2. **Split-mask matching for composites**: score arms (4 arm angles) and legs (4 leg
+   angles) *independently*. If no whole clip scores well but clip A wins arms and clip
+   B wins legs → propose `{layers:[{mask:"upper",clip:A},{mask:"lower",clip:B}]}`.
+3. **Vision tiebreak**: agent compares top-3 candidates' text cards against the
+   segment description; picks; records rationale.
+
+### Stage 5 — Gap-fill via Mixamo
+Segment with no acceptable match → agent emits a **shopping list** (search terms for
+mixamo.com; downloads are behind Adobe login, so Jeremy grabs FBXs manually — ~5 min)
+→ `mixamo_retarget.py` → new pack in `assets/quaternius/mixamo/` → add to catalog
+(Stage 3 for just that pack) → re-match.
+
+### Stage 6 — Emit the dance
+Write `assets/dances/<name>.json`: `bpm` from Stage 1, one move per segment, `loops`
+from beat counts. Hot-reload shows it immediately if the test bed is running.
+
+### Stage 7 — Verify loop (closes the circle)
+1. Test bed plays the dance; camera solo-framed on one dancer; VideoRecorder captures.
+2. `extract_pose.py` on the capture → `compare_poses.py` vs source, per segment
+   (`--a-range/--b-range` from the beat grid).
+3. `score_0_100` per segment + `per_joint_error_deg` says WHERE it diverges (arms vs
+   legs → swap just that layer of a composite). Iterate moves until segments clear a
+   threshold (start ~65; calibrate).
+4. Final deliverable: side-by-side mp4 (`ffmpeg hstack`) + per-segment score table.
+
+## Deliverables ledger
+
+| Item | Status |
+|---|---|
+| `pose-estimation` skill (extract + compare, venv, models) | DONE 2026-07-13 |
+| `animation-creation` skill (project-level recipes) | DONE 2026-07-13 |
+| This pipeline doc | DONE 2026-07-13 |
+| Test bed `CatalogMode` (solo-frame + record each clip) | TODO — first build item |
+| Catalog batch script + `clip_cards.json` | TODO |
+| librosa beat grid helper | TODO (trivial, add when needed) |
+| Segment/match/emit agent workflow (`/dance-from-video`) | TODO — after catalog |
+
+## Risks / open questions
+- **Camera angle mismatch**: source video vs game capture viewpoint differ; world
+  landmarks + joint angles are view-invariant in principle, but MediaPipe's 3D lift is
+  weaker at oblique angles. Mitigation: match game camera height/angle roughly to source.
+- **Multi-dancer sources**: pick ONE dancer (ideally the leader) and track them; group
+  formation is authored separately (formations are already a test bed parameter).
+- **Catalog scale**: ~810 clips × record+describe is hours of batch time. Start with
+  the ~40 obviously dance-relevant clips; expand on demand.
+- **Score threshold semantics**: 71 = "different segments of the same dance", so
+  same-move recreations should land higher; calibrate on known pairs before trusting.
+- KI clips are polished loops; Mixamo retargets can have foot slide — acceptable for
+  dance (no locomotion), watch for it.