init: animation pipeline hub — Mac↔PC bridge for converting and implementing animations
Skills (.claude/skills/): animation-creation, iclone-video-mocap, pose-estimation (MediaPipe models via LFS), retarget-animations (deprecated). Tools: cc/mixamo/kevin/mocap retargeters + composite baker. Plans: animation-gen-pipeline + skills-adoption. exchange/: incoming-fbx, converted-glb, reference-video (LFS for fbx/glb/mp4). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,124 @@
|
||||
---
|
||||
name: animation-creation
|
||||
description: Create new skeletal animations for ariki-game on the shared Quaternius skeleton — retarget Mixamo/Kevin-Iglesias FBX to game-ready GLB packs, bake composite clips (upper body of one clip over lower body of another), and author dance sequences as hot-reload JSON. Use when the user wants a new animation, dance, or move that doesn't exist in the shipped packs.
|
||||
---
|
||||
|
||||
# Animation Creation (ariki-game)
|
||||
|
||||
Every animation in the game runs on the **one shared Quaternius skeleton** (65 bones,
|
||||
UE names: `pelvis`, `spine_01..03`, `Head` capital-H, `thigh_l`, ...). Creating an
|
||||
animation means getting motion onto that skeleton. Four proven paths, cheapest first.
|
||||
|
||||
## Path 1 — It probably already exists (~810 clips)
|
||||
|
||||
Check before creating. Packs: `assets/quaternius/kevin/*.glb` (12 packs, male+female ×
|
||||
combat/idles/misc/movement/social/work — social has `Dance01`–`Dance18`,
|
||||
`DancePose01_Loop`–`07` with `_Begin`/`_Stop`, claps, waves, cheers), UAL1/UAL2
|
||||
standard libraries, `assets/quaternius/mixamo/*.glb`. List a pack's clips:
|
||||
|
||||
```bash
|
||||
# clip names live in the GLB's JSON chunk
|
||||
node -e "const b=require('fs').readFileSync('assets/quaternius/kevin/kevin_male_social.glb');const len=b.readUInt32LE(12);const j=JSON.parse(b.slice(20,20+len));console.log(j.animations.map(a=>a.name).join('\n'))"
|
||||
```
|
||||
|
||||
## Path 2 — Retarget a Mixamo FBX (new motion from Mixamo's huge library)
|
||||
|
||||
Download FBX(s) from mixamo.com (any character, "Without Skin" is fine), then:
|
||||
|
||||
```bash
|
||||
/Applications/Blender.app/Contents/MacOS/Blender --background \
|
||||
--python tools/mixamo_retarget.py -- \
|
||||
--src "<dir-or-fbx>[,<more>...]" --out assets/quaternius/mixamo/<pack_name>.glb
|
||||
```
|
||||
|
||||
- World-rotation-delta method; handles `mixamorig:` prefix variants automatically.
|
||||
- Proven end-to-end: `northern_soul.glb`, `run_to_dive.glb`.
|
||||
- Output GLBs in `assets/quaternius/mixamo/` are auto-discovered by ClipCatalog /
|
||||
AnimationShowcase — referenceable immediately as `<pack_name>/<ClipName>`.
|
||||
- Kevin Iglesias FBX sources use the sibling `tools/kevin_retarget.py` (different rig
|
||||
root handling — Kevin has a demo-scene offset bone).
|
||||
|
||||
## Path 3 — Composite clips (new move from two existing clips, no Blender)
|
||||
|
||||
The dance system bakes "upper body of clip A over lower body of clip B" at load time.
|
||||
Author it as a move in `assets/dances/*.json` — hot-reloads in ~0.5s while the dance
|
||||
test bed runs:
|
||||
|
||||
```json
|
||||
{ "layers": [
|
||||
{ "mask": "upper", "clip": "kevin_male_social/HandClap01" },
|
||||
{ "mask": "lower", "clip": "kevin_male_movement/Crouch01_Walk_Forward" } ],
|
||||
"duration": 4.0 }
|
||||
```
|
||||
|
||||
- Masks: `lower` = `root|pelvis|thigh*|calf*|foot*|ball*` (owns hip position AND
|
||||
rotation — always); `upper` = everything else. Authoritative predicate:
|
||||
`tools/bake_run_punch.mjs`.
|
||||
- For a composite needed **outside** the dance system, pre-bake a merged pack with a
|
||||
`bake_run_punch.mjs`-style node script instead (same mask logic, writes a GLB).
|
||||
|
||||
## Path 4 — Dance sequences (choreography = ordered moves + BPM)
|
||||
|
||||
A full "animation" at the choreography level is a JSON file in `assets/dances/`:
|
||||
`{ "name", "bpm", "moves": [ {"clip": "pack/Clip", "loops": 2}, {composite...} ] }`.
|
||||
Move transitions quantize to the beat clock. View in the test bed:
|
||||
`SCENE=dance_test_bed bash tools/game.sh spawn` (ASK Jeremy before launching the game).
|
||||
See `.agents/plans/dance-test-bed-2026-07-13.md` for full format + class specs.
|
||||
|
||||
## Path 5 — Author in live Blender (blender-mcp)
|
||||
|
||||
For motion that exists nowhere and can't be composited from existing clips — hand-keyed
|
||||
poses, IK-assisted tweaks, NLA layering of retargeted clips. Prerequisites: Blender 5.1.2
|
||||
open with the BlenderMCP addon connected (sidebar → BlenderMCP → Connect) and the `blender`
|
||||
MCP server registered in `.mcp.json`. Workflow: import an existing game GLB to pull in the
|
||||
real Quaternius armature → author (keyframes, IK, NLA) → export GLB into
|
||||
`assets/quaternius/mixamo/` so ClipCatalog auto-discovers it as `<pack>/<ClipName>`.
|
||||
Recipes: `references/blender-mcp-recipes.md`.
|
||||
|
||||
## GLB surgery without Blender (gltf-transform)
|
||||
|
||||
Inspect clip names, transplant or merge clips between same-skeleton GLBs, and
|
||||
resample/prune/dedup to shrink packs — no Blender required. Runs via
|
||||
`npx @gltf-transform/cli`, or as a JS SDK from a node script like the existing
|
||||
`tools/bake_run_punch.mjs`. Complements Path 3's node-script compositing when the job is a
|
||||
clip move or pack merge, not new motion. Recipes: `references/gltf-transform-recipes.md`.
|
||||
|
||||
## Hard rules (violations = invisible or broken animation)
|
||||
|
||||
- **Never use `*RM` root-motion clip variants** in dances/composites — they translate the rig.
|
||||
- **Skeleton path remap is mandatory** when playing raw GLB clips on a game rig: GLB
|
||||
tracks target `Armature/Skeleton3D:bone`; derive the rig's real prefix from an
|
||||
already-remapped clip (`ap.GetAnimation("idle").TrackGetPath(0)`), remap once, share.
|
||||
- Clip name fallback: exact → `+"_Loop"` → `-"_Loop"` (Godot import sometimes strips it).
|
||||
- Skip `RESET` clips; set `LoopMode.Linear` on extracted clips.
|
||||
- `.uid` files auto-generate — never hand-author.
|
||||
|
||||
## Verify what you created
|
||||
|
||||
- Single clips / packs: `scenes/animation_showcase.tscn` pages through every shipped clip.
|
||||
- Dances / composites: dance test bed; `VideoRecorder` (in-scene) records the viewport
|
||||
to `~/Downloads/*.mp4` for review — feed that to the `pose-estimation` skill to score
|
||||
a recreation against source footage (crop-zoom one dancer first; wide shots don't detect).
|
||||
|
||||
## Path 6 — Generate from video (pose-driven mocap, BUILT 2026-07-13)
|
||||
|
||||
`tools/mocap_retarget.py` turns `pose-estimation` skill output (extract_pose.py JSON,
|
||||
world landmarks) into a GLB clip on the Quaternius skeleton:
|
||||
|
||||
```bash
|
||||
/Applications/Blender.app/Contents/MacOS/Blender --background \
|
||||
--python tools/mocap_retarget.py -- \
|
||||
--pose <pose.json> --out assets/quaternius/mixamo/<pack>.glb --name <ClipName> \
|
||||
--range 166:192 --smooth 5 --render-check /tmp/check # renders frames to eyeball
|
||||
```
|
||||
|
||||
- Limb bones: shortest-arc aim from rest direction (preserves rest roll); pelvis/spine:
|
||||
full-basis deltas conjugated through char-frame alignment (same convention as
|
||||
mixamo_retarget.py — landmark space is subject-left=+X, up=+Z, facing=-Y, NOT identity).
|
||||
- MediaPipe world landmarks are hip-centered → no root motion; pelvis height is
|
||||
reconstructed from pelvis-above-ankle extent vs standing (carries squats).
|
||||
- Known v1 limits: no twist control, no fingers, no horizontal root motion.
|
||||
- Proven: `haka_mocap.glb` / `HakaFull` (26s, 208 keys) generated from a YouTube haka
|
||||
tutorial via `~/.claude/skills/pose-estimation`.
|
||||
|
||||
Full pipeline context: `.agents/plans/animation-gen-pipeline-2026-07-13.md`.
|
||||
@@ -0,0 +1,159 @@
|
||||
# blender-mcp recipes
|
||||
|
||||
Practical `bpy` snippets the agent pastes through blender-mcp's `execute-code` tool to
|
||||
author animations on the shared Quaternius skeleton and export them as game-ready GLB
|
||||
packs. These mirror the proven conventions in `tools/mixamo_retarget.py` — keep them in
|
||||
lockstep.
|
||||
|
||||
## Setup assumptions
|
||||
|
||||
- Blender **5.1.2** at `/Applications/Blender.app` (call headless as
|
||||
`/Applications/Blender.app/Contents/MacOS/Blender`).
|
||||
- The BlenderMCP addon is installed and **connected** (3D View → sidebar → BlenderMCP →
|
||||
Connect). If the `blender` MCP tool isn't available, the addon isn't connected.
|
||||
- **One game GLB imported** so the scene holds the real Quaternius armature. The game
|
||||
ships these as animations-only GLBs; import one to reuse its exact rig:
|
||||
|
||||
```python
|
||||
bpy.ops.import_scene.gltf(filepath="/Users/jeremykashkett/Tinqs/local.repo/ariki-game/assets/quaternius/kevin/kevin_male_social.glb")
|
||||
arm = next(o for o in bpy.data.objects if o.type == "ARMATURE")
|
||||
arm.name = "Armature" # match the convention mixamo_retarget.py enforces
|
||||
bpy.context.scene.render.fps = 30
|
||||
```
|
||||
|
||||
## Skeleton orientation
|
||||
|
||||
65 bones, UE-style names: `pelvis`, `spine_01`, `spine_02`, `spine_03`, `neck_01`,
|
||||
`Head` (**capital H**), `clavicle_l`, `upperarm_l`, `lowerarm_l`, `hand_l`, `thigh_l`,
|
||||
`calf_l`, `foot_l`, `ball_l`, and the `_r` twins, plus 3-segment fingers
|
||||
(`index_0{l,r}` … `pinky_03`, `thumb_0{l,r}` … `thumb_03`). The armature object is named
|
||||
`Armature`; work in pose mode via `bpy.context.object.pose.bones["<name>"]`.
|
||||
|
||||
## Keyframing recipe
|
||||
|
||||
Build an action from an explicit list of poses (name, frame, per-bone quaternions).
|
||||
Quaternius bones are authored in **quaternion** rotation mode, so set every pose bone's
|
||||
`rotation_mode = "QUATERNION"` first (this is what `mixamo_retarget.py` does):
|
||||
|
||||
```python
|
||||
import bpy
|
||||
from mathutils import Quaternion
|
||||
|
||||
arm = bpy.data.objects["Armature"]
|
||||
for pb in arm.pose.bones:
|
||||
pb.rotation_mode = "QUATERNION"
|
||||
|
||||
act = bpy.data.actions.new("Wave01")
|
||||
arm.animation_data_create()
|
||||
arm.animation_data.action = act
|
||||
bpy.context.scene.frame_start, bpy.context.scene.frame_end = 1, 30
|
||||
|
||||
# poses: (frame, {bone: (w,x,y,z)})
|
||||
poses = [
|
||||
(1, {"hand_l": (1,0,0,0), "lowerarm_l": (1,0,0,0)}),
|
||||
(15, {"hand_l": (0.7,0,0,0.7), "lowerarm_l": (0.9,0.3,0,0)}), # raise hand
|
||||
(30, {"hand_l": (1,0,0,0), "lowerarm_l": (1,0,0,0)}),
|
||||
]
|
||||
for frame, bones in poses:
|
||||
bpy.context.scene.frame_set(frame)
|
||||
for name, q in bones.items():
|
||||
pb = arm.pose.bones[name]
|
||||
pb.rotation_quaternion = Quaternion(q)
|
||||
pb.keyframe_insert(data_path="rotation_quaternion", frame=frame)
|
||||
```
|
||||
|
||||
## IK setup recipe
|
||||
|
||||
Drive `calf_l` (or `lowerarm_l`) toward a target empty with a 2-bone chain, then bake
|
||||
the solved motion to real keyframes and drop the constraint — the game never needs IK:
|
||||
|
||||
```python
|
||||
import bpy
|
||||
|
||||
arm = bpy.data.objects["Armature"]
|
||||
target = bpy.data.objects.new("IK_target_l", None) # empty
|
||||
bpy.context.collection.objects.link(target)
|
||||
target.location = arm.matrix_world @ arm.pose.bones["foot_l"].head + (0.0, 0.0, 0.2)
|
||||
|
||||
pb = arm.pose.bones["calf_l"]
|
||||
ik = pb.constraints.new("IK")
|
||||
ik.target = target
|
||||
ik.chain_count = 2 # thigh_l + calf_l
|
||||
ik.use_tail = True
|
||||
|
||||
# Bake the visual (constraint-solved) result to plain keyframes, then remove the constraint.
|
||||
bpy.ops.object.select_all(action="DESELECT")
|
||||
arm.select_set(True)
|
||||
bpy.context.view_layer.objects.active = arm
|
||||
bpy.ops.nla.bake(frame_start=1, frame_end=30, visual_keying=True,
|
||||
bake_types={"POSE"}, clean_curves=True)
|
||||
for c in list(pb.constraints):
|
||||
pb.constraints.remove(c)
|
||||
```
|
||||
|
||||
## NLA layering recipe
|
||||
|
||||
The glTF exporter emits each **NLA track** as a separate animation clip. To ship multiple
|
||||
clips in one GLB (or layer retargeted clips), push each action onto its own NLA track —
|
||||
this is the exact pattern `tools/mixamo_retarget.py` uses to export a pack:
|
||||
|
||||
```python
|
||||
import bpy
|
||||
arm = bpy.data.objects["Armature"]
|
||||
if arm.animation_data is None:
|
||||
arm.animation_data_create()
|
||||
for act in bpy.data.actions: # every action you want to ship
|
||||
tr = arm.animation_data.nla_tracks.new()
|
||||
tr.name = act.name # clip name = track name
|
||||
tr.strips.new(act.name, 1, act)
|
||||
arm.animation_data.action = None # avoid a stray duplicate clip
|
||||
```
|
||||
|
||||
To **blend** two clips on the stack (e.g. a base loop under a gesture), set the upper
|
||||
strip's `blend_type` (`"ADD"`, `"REPLACE"`) and `blend_in`/`blend_out` frames, then bake
|
||||
the combined result down to a single action (visual keying) and re-export that.
|
||||
|
||||
## Export recipe
|
||||
|
||||
The canonical export block lives in `tools/mixamo_retarget.py` and **must be mirrored
|
||||
exactly** — the game relies on these flags (`NLA_TRACKS` mode = one clip per track;
|
||||
`export_force_sampling` for clean per-bone channels; size optimization off to preserve
|
||||
retargeted curves). Select only the armature and drop meshes first so the GLB is
|
||||
animations-only (the game binds clips to its own body):
|
||||
|
||||
```python
|
||||
import bpy
|
||||
out = "/Users/jeremykashkett/Tinqs/local.repo/ariki-game/assets/quaternius/mixamo/wave_pack.glb"
|
||||
|
||||
# Drop meshes — animations-only GLB (mirrors mixamo_retarget.py).
|
||||
for o in [o for o in bpy.data.objects if o.type == "MESH"]:
|
||||
bpy.data.objects.remove(o, do_unlink=True)
|
||||
|
||||
bpy.ops.object.select_all(action="DESELECT")
|
||||
arm = bpy.data.objects["Armature"]
|
||||
arm.select_set(True)
|
||||
bpy.context.view_layer.objects.active = arm
|
||||
|
||||
bpy.ops.export_scene.gltf(
|
||||
filepath=out, use_selection=True,
|
||||
export_animations=True, export_animation_mode="NLA_TRACKS",
|
||||
export_force_sampling=True, export_optimize_animation_size=False)
|
||||
print("[blender-mcp] EXPORTED →", out)
|
||||
```
|
||||
|
||||
Output GLBs in `assets/quaternius/mixamo/` are auto-discovered by ClipCatalog /
|
||||
AnimationShowcase — referenceable immediately as `<pack_name>/<ClipName>` where
|
||||
`<ClipName>` is the NLA track (Blender action) name.
|
||||
|
||||
## Gotchas
|
||||
|
||||
- **Never export `*RM` root-motion variants** for dances/composites — they translate the
|
||||
rig and break compositing. Author clips in place.
|
||||
- **Skip the `RESET`/rest action** — don't push it to an NLA track or it ships as a clip.
|
||||
- **Clip name = Blender action/NLA-track name.** The game refers to clips as
|
||||
`<pack>/<ClipName>` (e.g. `wave_pack/Wave01`).
|
||||
- **One armature per exported GLB.** If you imported a GLB that brought its own body
|
||||
meshes, delete them before export (see the export block).
|
||||
- **Match `fps=30`** so frame counts line up with shipped packs and the dance beat clock.
|
||||
- **Quaternion mode** for all pose bones before keyframing (matches the retargeter).
|
||||
- `.uid` files auto-generate in Godot — never hand-author them.
|
||||
@@ -0,0 +1,101 @@
|
||||
# gltf-transform recipes
|
||||
|
||||
Headless GLB surgery with gltf-transform (donmccurdy, gltf-transform.dev) — inspect,
|
||||
merge, transplant, and shrink animation packs with no Blender required. Same Quaternius
|
||||
skeleton across all game GLBs, so node names line up 1:1 and clip moves are mechanical.
|
||||
|
||||
## Install / run
|
||||
|
||||
CLI (no global install — `npx` pulls it on demand):
|
||||
|
||||
```bash
|
||||
npx @gltf-transform/cli --help
|
||||
npx @gltf-transform/cli inspect assets/quaternius/kevin/kevin_male_social.glb
|
||||
```
|
||||
|
||||
Or as a JS SDK from a node script (mirror the layout of the existing
|
||||
`tools/bake_run_punch.mjs`):
|
||||
|
||||
```js
|
||||
import { NodeIO } from "@gltf-transform/core";
|
||||
import { prune, dedup, resample } from "@gltf-transform/functions";
|
||||
const io = new NodeIO();
|
||||
const doc = io.read("path/to/file.glb");
|
||||
```
|
||||
|
||||
Install once into the repo's node tooling: `npm i -D @gltf-transform/core @gltf-transform/functions`
|
||||
(the orchestrator installs — you author the script only).
|
||||
|
||||
## Inspect clips
|
||||
|
||||
```bash
|
||||
# Full report: meshes, animations, channels, duration.
|
||||
npx @gltf-transform/cli inspect assets/quaternius/kevin/kevin_male_social.glb
|
||||
```
|
||||
|
||||
For a quick clip-name list without the SDK, the pure-node JSON-chunk one-liner already in
|
||||
`SKILL.md` (Path 1) reads the GLB's JSON chunk and prints `animations[].name` — use it to
|
||||
confirm a clip exists before importing/transplanting.
|
||||
|
||||
## Clip transplant between same-skeleton GLBs
|
||||
|
||||
Both files share the Quaternius skeleton, so every channel's target node matches by name.
|
||||
Sketch: read source + target, copy the source `Animation` into the target document, then
|
||||
re-point each channel's `targetNode` to the target-doc node with the same name.
|
||||
|
||||
```js
|
||||
// verify against gltf-transform v4 API before first use
|
||||
import { NodeIO } from "@gltf-transform/core";
|
||||
import { copyToDocument } from "@gltf-transform/functions";
|
||||
|
||||
const io = new NodeIO();
|
||||
const target = io.read("assets/quaternius/mixamo/northern_soul.glb"); // pack to extend
|
||||
const source = io.read("downloads/clip_to_add.glb"); // same skeleton
|
||||
|
||||
const srcAnim = source.getRoot().listAnimations()
|
||||
.find(a => a.getName() === "NewClip");
|
||||
const targetNodes = new Map(target.getRoot().listNodes().map(n => [n.getName(), n]));
|
||||
|
||||
copyToDocument(target, source, [srcAnim]); // copy the Animation property
|
||||
const dstAnim = target.getRoot().listAnimations().at(-1);
|
||||
for (const ch of dstAnim.listChannels()) {
|
||||
const srcNode = ch.getTargetNode();
|
||||
if (srcNode) ch.setTargetNode(targetNodes.get(srcNode.getName())); // re-point by name
|
||||
}
|
||||
io.write("assets/quaternius/mixamo/northern_soul.glb", target); // overwrites — back up first
|
||||
```
|
||||
|
||||
Common variations: copy **all** source animations (iterate `listAnimations()`), rename on
|
||||
the way in (`dstAnim.setName("Wave01")`), or strip a clip (remove its channels then
|
||||
`prune`). If `copyToDocument` isn't the right entry point in your installed version, fall
|
||||
back to walking `srcAnim.listChannels()` and rebuilding each (sampler interpolation +
|
||||
input/output accessors + targetNode/path) on a fresh `target.createAnimation()`.
|
||||
|
||||
## Shrink / clean
|
||||
|
||||
Standard transforms, applied in this order for pack size:
|
||||
|
||||
```bash
|
||||
npx @gltf-transform/cli optimize <in.glb> <out.glb> # resample + prune + dedup combined
|
||||
# or individually:
|
||||
npx @gltf-transform/cli <in.glb> <out.glb> resample # drop redundant keyframes
|
||||
npx @gltf-transform/cli <in.glb> <out.glb> prune # remove unreferenced data
|
||||
npx @gltf-transform/cli <in.glb> <out.glb> dedup # merge identical accessors/meshes
|
||||
```
|
||||
|
||||
In the SDK: `await doc.transform(prune(), dedup(), resample())`. `resample` takes an
|
||||
optional `tolerance`/`ratio` to trade fidelity for size — retargeted clips have clean
|
||||
curves and tolerate aggressive resampling; mocap-sourced clips do not.
|
||||
|
||||
## When to prefer this over Blender
|
||||
|
||||
Prefer gltf-transform when there's **no new motion** to author:
|
||||
|
||||
- Same-skeleton clip moves (one clip from a downloaded GLB into a shipped pack).
|
||||
- Batch renames (fix a pack of `*_Loop` suffixes, normalize clip names).
|
||||
- Pack merges / splits (combine two `assets/quaternius/mixamo/` packs).
|
||||
- Headless size passes before shipping (resample/prune/dedup).
|
||||
|
||||
Reach for **Blender (Path 2 retarget / Path 5 authoring)** when the source uses a
|
||||
*different* skeleton (needs the world-rotation-delta retarget), or when you must author,
|
||||
IK-solve, or layer new motion that exists nowhere.
|
||||
Reference in New Issue
Block a user