Files
jeremy 31ba2911df init: animation pipeline hub — Mac↔PC bridge for converting and implementing animations
Skills (.claude/skills/): animation-creation, iclone-video-mocap,
pose-estimation (MediaPipe models via LFS), retarget-animations (deprecated).
Tools: cc/mixamo/kevin/mocap retargeters + composite baker.
Plans: animation-gen-pipeline + skills-adoption.
exchange/: incoming-fbx, converted-glb, reference-video (LFS for fbx/glb/mp4).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 10:29:48 -07:00

125 lines
6.6 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
name: animation-creation
description: Create new skeletal animations for ariki-game on the shared Quaternius skeleton — retarget Mixamo/Kevin-Iglesias FBX to game-ready GLB packs, bake composite clips (upper body of one clip over lower body of another), and author dance sequences as hot-reload JSON. Use when the user wants a new animation, dance, or move that doesn't exist in the shipped packs.
---
# Animation Creation (ariki-game)
Every animation in the game runs on the **one shared Quaternius skeleton** (65 bones,
UE names: `pelvis`, `spine_01..03`, `Head` capital-H, `thigh_l`, ...). Creating an
animation means getting motion onto that skeleton. Four proven paths, cheapest first.
## Path 1 — It probably already exists (~810 clips)
Check before creating. Packs: `assets/quaternius/kevin/*.glb` (12 packs, male+female ×
combat/idles/misc/movement/social/work — social has `Dance01``Dance18`,
`DancePose01_Loop``07` with `_Begin`/`_Stop`, claps, waves, cheers), UAL1/UAL2
standard libraries, `assets/quaternius/mixamo/*.glb`. List a pack's clips:
```bash
# clip names live in the GLB's JSON chunk
node -e "const b=require('fs').readFileSync('assets/quaternius/kevin/kevin_male_social.glb');const len=b.readUInt32LE(12);const j=JSON.parse(b.slice(20,20+len));console.log(j.animations.map(a=>a.name).join('\n'))"
```
## Path 2 — Retarget a Mixamo FBX (new motion from Mixamo's huge library)
Download FBX(s) from mixamo.com (any character, "Without Skin" is fine), then:
```bash
/Applications/Blender.app/Contents/MacOS/Blender --background \
--python tools/mixamo_retarget.py -- \
--src "<dir-or-fbx>[,<more>...]" --out assets/quaternius/mixamo/<pack_name>.glb
```
- World-rotation-delta method; handles `mixamorig:` prefix variants automatically.
- Proven end-to-end: `northern_soul.glb`, `run_to_dive.glb`.
- Output GLBs in `assets/quaternius/mixamo/` are auto-discovered by ClipCatalog /
AnimationShowcase — referenceable immediately as `<pack_name>/<ClipName>`.
- Kevin Iglesias FBX sources use the sibling `tools/kevin_retarget.py` (different rig
root handling — Kevin has a demo-scene offset bone).
## Path 3 — Composite clips (new move from two existing clips, no Blender)
The dance system bakes "upper body of clip A over lower body of clip B" at load time.
Author it as a move in `assets/dances/*.json` — hot-reloads in ~0.5s while the dance
test bed runs:
```json
{ "layers": [
{ "mask": "upper", "clip": "kevin_male_social/HandClap01" },
{ "mask": "lower", "clip": "kevin_male_movement/Crouch01_Walk_Forward" } ],
"duration": 4.0 }
```
- Masks: `lower` = `root|pelvis|thigh*|calf*|foot*|ball*` (owns hip position AND
rotation — always); `upper` = everything else. Authoritative predicate:
`tools/bake_run_punch.mjs`.
- For a composite needed **outside** the dance system, pre-bake a merged pack with a
`bake_run_punch.mjs`-style node script instead (same mask logic, writes a GLB).
## Path 4 — Dance sequences (choreography = ordered moves + BPM)
A full "animation" at the choreography level is a JSON file in `assets/dances/`:
`{ "name", "bpm", "moves": [ {"clip": "pack/Clip", "loops": 2}, {composite...} ] }`.
Move transitions quantize to the beat clock. View in the test bed:
`SCENE=dance_test_bed bash tools/game.sh spawn` (ASK Jeremy before launching the game).
See `.agents/plans/dance-test-bed-2026-07-13.md` for full format + class specs.
## Path 5 — Author in live Blender (blender-mcp)
For motion that exists nowhere and can't be composited from existing clips — hand-keyed
poses, IK-assisted tweaks, NLA layering of retargeted clips. Prerequisites: Blender 5.1.2
open with the BlenderMCP addon connected (sidebar → BlenderMCP → Connect) and the `blender`
MCP server registered in `.mcp.json`. Workflow: import an existing game GLB to pull in the
real Quaternius armature → author (keyframes, IK, NLA) → export GLB into
`assets/quaternius/mixamo/` so ClipCatalog auto-discovers it as `<pack>/<ClipName>`.
Recipes: `references/blender-mcp-recipes.md`.
## GLB surgery without Blender (gltf-transform)
Inspect clip names, transplant or merge clips between same-skeleton GLBs, and
resample/prune/dedup to shrink packs — no Blender required. Runs via
`npx @gltf-transform/cli`, or as a JS SDK from a node script like the existing
`tools/bake_run_punch.mjs`. Complements Path 3's node-script compositing when the job is a
clip move or pack merge, not new motion. Recipes: `references/gltf-transform-recipes.md`.
## Hard rules (violations = invisible or broken animation)
- **Never use `*RM` root-motion clip variants** in dances/composites — they translate the rig.
- **Skeleton path remap is mandatory** when playing raw GLB clips on a game rig: GLB
tracks target `Armature/Skeleton3D:bone`; derive the rig's real prefix from an
already-remapped clip (`ap.GetAnimation("idle").TrackGetPath(0)`), remap once, share.
- Clip name fallback: exact → `+"_Loop"``-"_Loop"` (Godot import sometimes strips it).
- Skip `RESET` clips; set `LoopMode.Linear` on extracted clips.
- `.uid` files auto-generate — never hand-author.
## Verify what you created
- Single clips / packs: `scenes/animation_showcase.tscn` pages through every shipped clip.
- Dances / composites: dance test bed; `VideoRecorder` (in-scene) records the viewport
to `~/Downloads/*.mp4` for review — feed that to the `pose-estimation` skill to score
a recreation against source footage (crop-zoom one dancer first; wide shots don't detect).
## Path 6 — Generate from video (pose-driven mocap, BUILT 2026-07-13)
`tools/mocap_retarget.py` turns `pose-estimation` skill output (extract_pose.py JSON,
world landmarks) into a GLB clip on the Quaternius skeleton:
```bash
/Applications/Blender.app/Contents/MacOS/Blender --background \
--python tools/mocap_retarget.py -- \
--pose <pose.json> --out assets/quaternius/mixamo/<pack>.glb --name <ClipName> \
--range 166:192 --smooth 5 --render-check /tmp/check # renders frames to eyeball
```
- Limb bones: shortest-arc aim from rest direction (preserves rest roll); pelvis/spine:
full-basis deltas conjugated through char-frame alignment (same convention as
mixamo_retarget.py — landmark space is subject-left=+X, up=+Z, facing=-Y, NOT identity).
- MediaPipe world landmarks are hip-centered → no root motion; pelvis height is
reconstructed from pelvis-above-ankle extent vs standing (carries squats).
- Known v1 limits: no twist control, no fingers, no horizontal root motion.
- Proven: `haka_mocap.glb` / `HakaFull` (26s, 208 keys) generated from a YouTube haka
tutorial via `~/.claude/skills/pose-estimation`.
Full pipeline context: `.agents/plans/animation-gen-pipeline-2026-07-13.md`.