animation/animation-context-for-claude-desktop.md

# Animation Pipeline — Context Brief

> Compiled from claude-mem across the `ariki-game` and `animation` projects (July 2026 sessions). This is a portable context dump so a fresh Claude (e.g. Claude Desktop) understands how the ARIKI game's animation system works, what assets and tools exist, and the conventions/gotchas that matter.

---

## 1. The big picture

There are **two related repos** and **two distinct animation domains**.

**Repos**
- `~/Tinqs/local.repo/ariki-game` — the Godot 4 / C# game project. Consumes finished animation clips.
- `~/Tinqs/local.repo/animation` — a **Mac↔PC pipeline hub**. Bridges iClone/Character Creator output (produced on PC) with the Mac side for conversion and import into the game. Uses Git LFS for large binary assets (FBX, iProject files, texture maps, reference videos).

**Two animation domains inside the game (do not conflate them):**
1. **Character-clip / dance pipeline** — skeletal animation clips retargeted onto the shared Quaternius skeleton, plus hot-reloadable JSON dance choreography. Test bed lives in `src/Testing/Dance/`; runtime consumer is `src/Animation/`.
2. **FX / VFX pipeline** — effects animation via Tweens, `GpuParticles3D`, shaders, and per-feature "Beat" controllers. Lives in `src/Rendering/` and `src/Beats/`.

---

## 2. Runtime animation code (`src/Animation/`)

The runtime character-animation layer is **compact — 5 files** — and sits between the clip library and actual gameplay:

- `CharacterAnimDriver.cs` — low-level clip playback + state machine; shared base/adapter between the clip pipeline and game character nodes.
- `PlayerAnimationLogic.cs` — shared transition/blend-tree logic (was a late discovery; doesn't contain "dance"/"AnimationTree" keywords so early greps missed it).
- `PlayerAnimController.cs` — in-game player animation controller; Godot `AnimationTree` integration + IK hooks. (Distinct from `DebugAnimPlayer` in the test bed.)
- `PlayerIKRig.cs` — runtime inverse kinematics (foot placement, hand targets).
- `AnimalAnimationLogic.cs` — separate animation path for animal characters; canonical state keys via `StateKeyFor`.

Likely layering: `CharacterAnimDriver` → `PlayerAnimationLogic` → `PlayerAnimController` → `PlayerIKRig`.

**Not animation:** `src/Character/` is outfit/clothing systems (OutfitCatalog, OutfitSystem, ClothingItem, CharacterBuffSystem, CharacterPreview), not animation.

**QA surface is ~2x the runtime:** 9 test-bed files (ClipStation, UpperBodyBlendStation, AnimGalleryStation, AnimationShowcase, PlayerAnimTestBed, AnimActionTestBed, JumpAnimTestBed, LandingSlideTestBed, AiAnimationTestScene) vs 4–5 production files. `PlayerAnimTestBed.cs` (~410 lines) is the largest/most dance-relevant test bed and models the clip-loading pattern.

Player setup entry point: `src/Viewer/PlayerController.cs` (`SetupAnimations` around line 508).

---

## 3. Animation assets

### Character clips (shared Quaternius skeleton)
- **Kevin Iglesias packs** — `assets/quaternius/kevin/`, ~12 GLB packs, **~810 clips total** (combat, idles, misc, movement, social, work — male + female).
  - `kevin_male_social.glb`: 68 anims (military salutes w/ weapon variants, 7 dance poses as begin/loop/stop triplets, 18 dance clips, claps/waves/cheers, talk/emotion cycles).
  - `kevin_male_movement.glb`: 157 anims (rolling, sliding, 8-directional walking, crouch idle/turn/strafe/walk) — each in **root-motion (`_RM`) and in-place variants**.
  - Plus idles, misc (eat/drink/sleep/sit), work (chop/mine/hammer/fish/gather/farm/water/carry/skin).
- **UAL libraries** — `UAL1_Standard.glb` (anim-lib-1, ~19 player clips: idle/walk/run/sprint/jump chain/dance/death/swim/interact/punch/throw/crouch) and `UAL2_Standard.glb` (anim-lib-2, running-jump chain: NinjaJump Start/Idle/Land).
- **Combined** — `anim-combined/Run_Punch.glb`.
- **Pre-extracted single-clip FBX** — `assets/animations/kevin/` (13 gameplay-critical FBX: Archer combat/death/idle/run/damage, Throwing@BigAxe, Villager gathering/skinning) and `assets/animations/basic-motions/` (Idle01, Walk01, Run01, Sprint01). **Note:** social/misc/dance clips were *not* extracted to individual FBX — they live only inside the GLB packs.
- **Generated dances** — `assets/quaternius/dancegen/` (auto-discovered by `ClipCatalog`): fertility_dance_01(_pp), taming_dance_01(_pp), hakadance1_static.

**Load order (later overrides earlier for same clip name):** `PlayerController.SetupAnimations` loads UAL1 → UAL2 → Kevin packs. Example override: Kevin `work` chop overrides UAL1 Sword_Attack.

**Naming conventions (Quaternius):** `CharacterName@AnimationName.fbx`; suffix `_RM` = root motion; `_Loop` = looping; directionals `_Forward`/`_BackwardLeft`/etc.; dances use `_Begin` / `_Loop` / `_Stop` triplets.

### Dance choreography (hot-reload JSON)
`assets/dances/*.json` — hot-reloadable: clap_stomp, fertility_dance_01(_pp), haka_tutorial, hakadance1_static, hype_line, taming_dance_01(_pp), test_trio, haka_mocap.

**Dance type assignments (configured):** Haka = "war dance 1", Aloha = "fertility", Dance1 = "taming dance". **Requirement: every dance must ship a dedicated `_Loop` version** so it plays continuously without restarting.

### Animals
`assets/models/animals_catalog.json` — **96 species, 320 distinct clips** (avg 3.33/species, max 6 = Chick). Fields: `anims`, `category`, `state_clips`, `default_clip`, `glb`, `size_m`. Canonical states: idle, walk, run, fly, swim, attack, death, + ambient eat/sleep/drink/graze.

### Retarget bone maps
`assets/retarget/kevin_bonemap.tres`, `assets/retarget/bozo_bonemap.tres`. No standalone `.anim`/AnimationLibrary resources — animations are embedded in FBX/GLB.

---

## 4. Tooling (Python retarget + GLB utilities)

Located under `tools/`:

- `mocap_retarget.py` — **MediaPipe pose JSON → solved bone rotations → Quaternius-skeleton GLB** via headless Blender. Uses limb-aim deltas, torso-basis solve, char-frame conjugation, squat-height reconstruction from leg extent, shortest-arc rotation solving.
- `cc_retarget.py` — Reallusion Character Creator / iClone `CC_Base_*` FBX retargeting; rejects motionless `TempMotion` placeholder takes.
- `mixamo_retarget.py` — Mixamo FBX → Quaternius.
- `kevin_retarget.py` — Kevin FBX → Quaternius.
- `convert_all_fbx.py` — batch FBX conversion.
- `anim_qc.py` / `anim_qc.py` — loop-seam / velocity / stutter quality checks.
- GLB utilities: `glb_oracle.py`, `validate_glb_structure.py`, `batch_convert_glb.py`, `strip_glb_textures.py`, `normalize_animal_glbs.py`, `normalize_item_glbs.py`.

`ClipCatalog.cs` (`src/Testing/Dance/`) scans `quaternius/dancegen`, `kevin`, and `mixamo` paths for clips.

**Proof-of-concept that works:** MediaPipe→GLB ran end-to-end on a YouTube haka tutorial → **208 frames / 26 s @ 30fps** `HakaFull`, with **0.0° elbow-angle error** across all sampled frames. Output: `assets/quaternius/mixamo/haka_mocap.glb`. Detection caveat: **94% on solo-framed characters, 0% on wide shots.**

---

## 5. Claude skills & MCP setup

**ariki-game skills** (`.claude/skills/`):
- `animation-creation` — the master skill. Documents **6 creation paths, cheapest-first:** (1) reuse existing ~810 clips, (2) retarget Mixamo/Kevin FBX with Blender scripts, (3) author composite clips in JSON (upper/lower body masks), (4) author full dance choreography sequences, (5) live Blender (blender-mcp), (6) video mocap. Companion recipes: `blender-mcp-recipes.md`, `gltf-transform-recipes.md`.
- `iclone-video-mocap` — feeds `cc_retarget.py`. Backed by a 31KB `references/iclone8-core-animation.md` (Motion Clips + Layer Keys model, Edit Motion Layer / HumanIK, Timeline ops, Curve Editor F12 filters, Motion Correction, AccuLips face anim, FBX/USD export). iClone is Z-up, 1 unit = 1 cm, 60fps internal. **BVH export NOT available from iClone 8.** 3DXchange discontinued (merged into iClone 8 + CC4). Reallusion content needs an Export License for FBX/OBJ/USD.
- `retarget-animations` (agent-level, `.agents/skills/`) — **deprecated/retired May 2026** (was Kevin→BoZo).

**MCP:** `.mcp.json` registers **blender-mcp via `uvx blender-mcp`** (replaced the old Playwright entry + removed a hardcoded token). Five mature **Godot MCP** projects exist that can drive `AnimationPlayer`/scenes directly (godot-mcp-pro = 175 tools, Godot AI = 150+ w/ fade/slide/shake/pulse presets, tugcantopaloglu `game_play_animation`, mkdevkit, tomyud1). Closes the loop: **Blender MCP (rig/animate) → glTF-Transform (optimize GLB) → Godot MCP (engine integration).**

**General media/animation skills available (`~/.claude/skills/`):** `video-editing` (FFmpeg/Remotion/ElevenLabs/fal.ai), `videodb`, `manim-video`, `remotion-video-creation`, `fal-ai-media`.

---

## 6. The video→dance generation pipeline (7 stages)

Documented in `.agents/plans/animation-gen-pipeline-2026-07-13.md`. Honest framing: it produces a **cover version using a discrete clip vocabulary, not true motion capture.**

1. Ingest source video with a beat grid.
2. Programmatic + vision choreography breakdown.
3. Clip-card catalog of the ~810 existing clips (blocked on building `CatalogMode` in the test bed — the identified next build item).
4. DTW matching with split-mask composite fallback.
5. Mixamo gap-fill for unmatched segments.
6. Emit dance JSON.
7. Self-verify: play in-game, capture via VideoRecorder, compare poses back to the source (uses the `pose-estimation` skill for scoring).

**Risks flagged:** camera-angle mismatch, multi-dancer sources, catalog scale, score-threshold calibration.

---

## 7. Hard constraints & gotchas

- **Never use `*RM` root-motion variants** in contexts expecting in-place animation.
- **Skeleton path remap is mandatory** when retargeting.
- **Clip-name fallback logic:** exact → try `+"_Loop"` → try `-"_Loop"`.
- **Skip `RESET` clips.** `.uid` files auto-generate.
- **Every dance needs a `_Loop` version** (see §3).
- Retarget tools **reject motionless `TempMotion` placeholder takes** (CC export artifact).
- The `animation` repo is LFS-heavy — pulls can be 100+ MB (one iClone `.iProject` was 109 MB).
- Pose detection fails on wide shots (0%); frame subjects solo for mocap.

## 8. iClone correction workflow notes (from the animator)

- To fade a correction in/out over a range: **bracket the fix with a clean keyframe before and after** the blend zone.
- Use the **Animation Layer panel (F11)** — stackable named layers, each with a keyframeable weight slider — for non-destructive experiments; merge down when satisfied.
- Curve Editor (F12) Optimize/Smooth/Butterworth filters clean up mocap jitter after sampling clips to layer keys.
- Motion Correction auto-creates hand/foot prints + Reach keys to fix sliding; requires iterative Flatten→deactivate→correct cycling.

---

## 9. Future / planned

- **Crowd-animation platform** (`.agents/wiki/crowd-animation-platform.md`): baked bone-matrix **animation-texture GPU instancing** to render thousands of colonists, beyond the current per-skeleton `PlayerAnimController` path.
- A dedicated `animation` skill for recording animator preferences/lessons was requested (source of the §3 dance-type + §8 iClone notes).