animation/animation-context-for-claude-desktop.md

Animation Pipeline — Context Brief

Compiled from claude-mem across the ariki-game and animation projects (July 2026 sessions). This is a portable context dump so a fresh Claude (e.g. Claude Desktop) understands how the ARIKI game's animation system works, what assets and tools exist, and the conventions/gotchas that matter.

---

1. The big picture

There are two related repos and two distinct animation domains.

Repos

• ~/Tinqs/local.repo/ariki-game — the Godot 4 / C# game project. Consumes finished animation clips.
• ~/Tinqs/local.repo/animation — a Mac↔PC pipeline hub. Bridges iClone/Character Creator output (produced on PC) with the Mac side for conversion and import into the game. Uses Git LFS for large binary assets (FBX, iProject files, texture maps, reference videos).

Two animation domains inside the game (do not conflate them):

1. Character-clip / dance pipeline — skeletal animation clips retargeted onto the shared Quaternius skeleton, plus hot-reloadable JSON dance choreography. Test bed lives in src/Testing/Dance/; runtime consumer is src/Animation/.
2. FX / VFX pipeline — effects animation via Tweens, GpuParticles3D, shaders, and per-feature "Beat" controllers. Lives in src/Rendering/ and src/Beats/.

---

2. Runtime animation code (src/Animation/)

The runtime character-animation layer is compact — 5 files — and sits between the clip library and actual gameplay:

• CharacterAnimDriver.cs — low-level clip playback + state machine; shared base/adapter between the clip pipeline and game character nodes.
• PlayerAnimationLogic.cs — shared transition/blend-tree logic (was a late discovery; doesn't contain "dance"/"AnimationTree" keywords so early greps missed it).
• PlayerAnimController.cs — in-game player animation controller; Godot AnimationTree integration + IK hooks. (Distinct from DebugAnimPlayer in the test bed.)
• PlayerIKRig.cs — runtime inverse kinematics (foot placement, hand targets).
• AnimalAnimationLogic.cs — separate animation path for animal characters; canonical state keys via StateKeyFor.

Likely layering: CharacterAnimDriver → PlayerAnimationLogic → PlayerAnimController → PlayerIKRig.

Not animation: src/Character/ is outfit/clothing systems (OutfitCatalog, OutfitSystem, ClothingItem, CharacterBuffSystem, CharacterPreview), not animation.

QA surface is ~2x the runtime: 9 test-bed files (ClipStation, UpperBodyBlendStation, AnimGalleryStation, AnimationShowcase, PlayerAnimTestBed, AnimActionTestBed, JumpAnimTestBed, LandingSlideTestBed, AiAnimationTestScene) vs 4–5 production files. PlayerAnimTestBed.cs (~410 lines) is the largest/most dance-relevant test bed and models the clip-loading pattern.

Player setup entry point: src/Viewer/PlayerController.cs (SetupAnimations around line 508).

---

3. Animation assets

Character clips (shared Quaternius skeleton)

• Kevin Iglesias packs — assets/quaternius/kevin/, ~12 GLB packs, ~810 clips total (combat, idles, misc, movement, social, work — male + female).
  • kevin_male_social.glb: 68 anims (military salutes w/ weapon variants, 7 dance poses as begin/loop/stop triplets, 18 dance clips, claps/waves/cheers, talk/emotion cycles).
  • kevin_male_movement.glb: 157 anims (rolling, sliding, 8-directional walking, crouch idle/turn/strafe/walk) — each in root-motion (_RM) and in-place variants.
  • Plus idles, misc (eat/drink/sleep/sit), work (chop/mine/hammer/fish/gather/farm/water/carry/skin).
• UAL libraries — UAL1_Standard.glb (anim-lib-1, ~19 player clips: idle/walk/run/sprint/jump chain/dance/death/swim/interact/punch/throw/crouch) and UAL2_Standard.glb (anim-lib-2, running-jump chain: NinjaJump Start/Idle/Land).
• Combined — anim-combined/Run_Punch.glb.
• Pre-extracted single-clip FBX — assets/animations/kevin/ (13 gameplay-critical FBX: Archer combat/death/idle/run/damage, Throwing@BigAxe, Villager gathering/skinning) and assets/animations/basic-motions/ (Idle01, Walk01, Run01, Sprint01). Note: social/misc/dance clips were not extracted to individual FBX — they live only inside the GLB packs.
• Generated dances — assets/quaternius/dancegen/ (auto-discovered by ClipCatalog): fertility_dance_01(_pp), taming_dance_01(_pp), hakadance1_static.

Load order (later overrides earlier for same clip name): PlayerController.SetupAnimations loads UAL1 → UAL2 → Kevin packs. Example override: Kevin work chop overrides UAL1 Sword_Attack.

Naming conventions (Quaternius): CharacterName@AnimationName.fbx; suffix _RM = root motion; _Loop = looping; directionals _Forward/_BackwardLeft/etc.; dances use _Begin / _Loop / _Stop triplets.

Dance choreography (hot-reload JSON)

assets/dances/*.json — hot-reloadable: clap_stomp, fertility_dance_01(_pp), haka_tutorial, hakadance1_static, hype_line, taming_dance_01(_pp), test_trio, haka_mocap.

Dance type assignments (configured): Haka = "war dance 1", Aloha = "fertility", Dance1 = "taming dance". Requirement: every dance must ship a dedicated _Loop version so it plays continuously without restarting.

Animals

assets/models/animals_catalog.json — 96 species, 320 distinct clips (avg 3.33/species, max 6 = Chick). Fields: anims, category, state_clips, default_clip, glb, size_m. Canonical states: idle, walk, run, fly, swim, attack, death, + ambient eat/sleep/drink/graze.

Retarget bone maps

assets/retarget/kevin_bonemap.tres, assets/retarget/bozo_bonemap.tres. No standalone .anim/AnimationLibrary resources — animations are embedded in FBX/GLB.

---

4. Tooling (Python retarget + GLB utilities)

Located under tools/:

• mocap_retarget.py — MediaPipe pose JSON → solved bone rotations → Quaternius-skeleton GLB via headless Blender. Uses limb-aim deltas, torso-basis solve, char-frame conjugation, squat-height reconstruction from leg extent, shortest-arc rotation solving.
• cc_retarget.py — Reallusion Character Creator / iClone CC_Base_* FBX retargeting; rejects motionless TempMotion placeholder takes.
• mixamo_retarget.py — Mixamo FBX → Quaternius.
• kevin_retarget.py — Kevin FBX → Quaternius.
• convert_all_fbx.py — batch FBX conversion.
• anim_qc.py / anim_qc.py — loop-seam / velocity / stutter quality checks.
• GLB utilities: glb_oracle.py, validate_glb_structure.py, batch_convert_glb.py, strip_glb_textures.py, normalize_animal_glbs.py, normalize_item_glbs.py.

ClipCatalog.cs (src/Testing/Dance/) scans quaternius/dancegen, kevin, and mixamo paths for clips.

Proof-of-concept that works: MediaPipe→GLB ran end-to-end on a YouTube haka tutorial → 208 frames / 26 s @ 30fps HakaFull, with 0.0° elbow-angle error across all sampled frames. Output: assets/quaternius/mixamo/haka_mocap.glb. Detection caveat: 94% on solo-framed characters, 0% on wide shots.

---

5. Claude skills & MCP setup

ariki-game skills (.claude/skills/):

• animation-creation — the master skill. Documents 6 creation paths, cheapest-first: (1) reuse existing ~810 clips, (2) retarget Mixamo/Kevin FBX with Blender scripts, (3) author composite clips in JSON (upper/lower body masks), (4) author full dance choreography sequences, (5) live Blender (blender-mcp), (6) video mocap. Companion recipes: blender-mcp-recipes.md, gltf-transform-recipes.md.
• iclone-video-mocap — feeds cc_retarget.py. Backed by a 31KB references/iclone8-core-animation.md (Motion Clips + Layer Keys model, Edit Motion Layer / HumanIK, Timeline ops, Curve Editor F12 filters, Motion Correction, AccuLips face anim, FBX/USD export). iClone is Z-up, 1 unit = 1 cm, 60fps internal. BVH export NOT available from iClone 8. 3DXchange discontinued (merged into iClone 8 + CC4). Reallusion content needs an Export License for FBX/OBJ/USD.
• retarget-animations (agent-level, .agents/skills/) — deprecated/retired May 2026 (was Kevin→BoZo).

MCP: .mcp.json registers blender-mcp via uvx blender-mcp (replaced the old Playwright entry + removed a hardcoded token). Five mature Godot MCP projects exist that can drive AnimationPlayer/scenes directly (godot-mcp-pro = 175 tools, Godot AI = 150+ w/ fade/slide/shake/pulse presets, tugcantopaloglu game_play_animation, mkdevkit, tomyud1). Closes the loop: Blender MCP (rig/animate) → glTF-Transform (optimize GLB) → Godot MCP (engine integration).

General media/animation skills available (~/.claude/skills/): video-editing (FFmpeg/Remotion/ElevenLabs/fal.ai), videodb, manim-video, remotion-video-creation, fal-ai-media.

---

6. The video→dance generation pipeline (7 stages)

Documented in .agents/plans/animation-gen-pipeline-2026-07-13.md. Honest framing: it produces a cover version using a discrete clip vocabulary, not true motion capture.

1. Ingest source video with a beat grid.
2. Programmatic + vision choreography breakdown.
3. Clip-card catalog of the ~810 existing clips (blocked on building CatalogMode in the test bed — the identified next build item).
4. DTW matching with split-mask composite fallback.
5. Mixamo gap-fill for unmatched segments.
6. Emit dance JSON.
7. Self-verify: play in-game, capture via VideoRecorder, compare poses back to the source (uses the pose-estimation skill for scoring).

Risks flagged: camera-angle mismatch, multi-dancer sources, catalog scale, score-threshold calibration.

---

7. Hard constraints & gotchas

• Never use *RM root-motion variants in contexts expecting in-place animation.
• Skeleton path remap is mandatory when retargeting.
• Clip-name fallback logic: exact → try +"_Loop" → try -"_Loop".
• Skip RESET clips. .uid files auto-generate.
• Every dance needs a _Loop version (see §3).
• Retarget tools reject motionless TempMotion placeholder takes (CC export artifact).
• The animation repo is LFS-heavy — pulls can be 100+ MB (one iClone .iProject was 109 MB).
• Pose detection fails on wide shots (0%); frame subjects solo for mocap.

8. iClone correction workflow notes (from the animator)

• To fade a correction in/out over a range: bracket the fix with a clean keyframe before and after the blend zone.
• Use the Animation Layer panel (F11) — stackable named layers, each with a keyframeable weight slider — for non-destructive experiments; merge down when satisfied.
• Curve Editor (F12) Optimize/Smooth/Butterworth filters clean up mocap jitter after sampling clips to layer keys.
• Motion Correction auto-creates hand/foot prints + Reach keys to fix sliding; requires iterative Flatten→deactivate→correct cycling.

---

9. Future / planned

• Crowd-animation platform (.agents/wiki/crowd-animation-platform.md): baked bone-matrix animation-texture GPU instancing to render thousands of colonists, beyond the current per-skeleton PlayerAnimController path.
• A dedicated animation skill for recording animator preferences/lessons was requested (source of the §3 dance-type + §8 iClone notes).