jeremy
31ba2911df
Skills (.claude/skills/): animation-creation, iclone-video-mocap, pose-estimation (MediaPipe models via LFS), retarget-animations (deprecated). Tools: cc/mixamo/kevin/mocap retargeters + composite baker. Plans: animation-gen-pipeline + skills-adoption. exchange/: incoming-fbx, converted-glb, reference-video (LFS for fbx/glb/mp4). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
76 lines
8.5 KiB
Markdown
76 lines
8.5 KiB
Markdown
# ariki-game — Integrating iClone Animation Exports
|
||
|
||
How motion authored in iClone 8 (incl. Video Mocap output) lands on ariki-game's shared Quaternius skeleton. Mapped 2026-07-15 from the repo.
|
||
|
||
**Headline: the retargeter already exists — `tools/cc_retarget.py` (created 2026-07-14) handles `CC_Base_*` (Reallusion CC/iClone) FBX → Quaternius GLB.** The pipeline is: iClone FBX export → `cc_retarget.py` → mesh-stripped GLB in an auto-discovered pack dir → referenceable as `<pack>/<ClipName>` in dances/showcase.
|
||
|
||
## 1. The shared Quaternius skeleton (retarget target)
|
||
|
||
- **Canonical rig:** `assets/quaternius/base-characters/Universal Base Characters[Standard]/Base Characters/Godot - UE/Superhero_Male_FullBody.gltf` — all retargeters default to it (`--target` overrides). `kevin_retarget.py:26` is the exception (hardcoded absolute path, non-portable).
|
||
- **65 bones, UE-style names, T-pose rest.** Spine: `root, pelvis, spine_01..03, neck_01, Head` (capital H). Per side: `clavicle_, upperarm_, lowerarm_, hand_, thigh_, calf_, foot_, ball_, ball_leaf_` + fingers `thumb/index/middle/ring/pinky_01..03_` with `_04_leaf_` tips.
|
||
|
||
## 2. ClipCatalog discovery (`src/Testing/Dance/ClipCatalog.cs`)
|
||
|
||
- Hardcoded packs (lines 20-25): UAL1, UAL2, Run_Punch.
|
||
- Auto-discovered dirs (lines 37-45): `res://assets/quaternius/kevin`, `res://assets/quaternius/mixamo`, `res://assets/quaternius/dancegen` — drop a GLB in one and it's live; **pack name = GLB basename**.
|
||
- Clip ref format: `"<pack>/<ClipName>"` (e.g. `dancegen/HakaFull`).
|
||
- Lazy extraction (lines 80-118): first `AnimationPlayer` in GLB scene, skips `RESET`, forces `LoopMode.Linear`, caches. Name fallback: `exact → +"_Loop" → -"_Loop"`.
|
||
- Gotcha baked into the code (lines 63-64): `DirAccess.GetNext()` returns `""` at end, not null — null check pegs CPU.
|
||
|
||
## 3. `tools/cc_retarget.py` — the iClone/CC ingestion tool
|
||
|
||
Blender 5 headless, same world-rotation-delta method as `mixamo_retarget.py`:
|
||
|
||
```bash
|
||
/Applications/Blender.app/Contents/MacOS/Blender --background \
|
||
--python tools/cc_retarget.py -- \
|
||
--src "<dir-or-fbx>[,<more>...]" --out assets/quaternius/dancegen/<pack>.glb \
|
||
[--name <ClipName>] [--target <gltf>]
|
||
```
|
||
|
||
- **BONE_MAP (lines 25-48):** `CC_Base_Hip→pelvis`, `CC_Base_Waist→spine_01`, `CC_Base_Spine01→spine_02`, `CC_Base_Spine02→spine_03`, `CC_Base_NeckTwist01→neck_01`, `CC_Base_Head→Head`; limbs `CC_Base_L_Clavicle/Upperarm/Forearm/Hand/Thigh/Calf/Foot/ToeBase → clavicle_l/upperarm_l/lowerarm_l/hand_l/thigh_l/calf_l/foot_l/ball_l`. **Thumbs 3-seg, other fingers 1-seg** (`CC_Base_L_Index1→index_01_l` etc. — game-optimized CC rigs have one segment per finger). Twist helpers (`UpperarmTwist01…`) unmapped → ride parent rigidly.
|
||
- **Char-frame conjugation:** `q_delta = C @ (q_src·q_src_rest⁻¹) @ C⁻¹` with `C = char_frame(target) @ char_frame(source)⁻¹`; source frame from `CC_Base_Hip`/`CC_Base_Head`/`CC_Base_L_Upperarm` (lines 128-129).
|
||
- **Units/scale:** `unit = src_arm.scale.x` captured BEFORE `transform_apply` (importer's cm→m 0.01); pelvis = world-space delta `× tscale × unit` (lines 176-177). Handles iClone's cm natively.
|
||
- **fps 60** (only tool at 60 — "iClone exports at 60; glTF export resamples fine").
|
||
- **Motion-presence guard** (`pick_action`, lines 89-105): picks the longest action whose fcurves reference `CC_Base_` bones; **rejects `TempMotion` <5-frame placeholders** with error "FBX exported WITHOUT motion — Re-export with Include Motion".
|
||
- Export tail (shared with all retargeters): drop MESH objects, one NLA track per action (track name = clip name = glTF animation name), `export_scene.gltf(export_animation_mode="NLA_TRACKS", export_force_sampling=True, export_optimize_animation_size=False)`.
|
||
|
||
### ⚠ Open validation item: A-pose
|
||
The tool's comments claim "CC rigs rest in T-pose, verified on ActorBuild exports", but iClone/CC3+ characters bind in **A-pose**. The char-frame delta math measures off actual rest and should self-cancel — but the left-arm char-frame axis (`CC_Base_L_Upperarm` bone-Y) points down-and-out in A-pose, not straight +X. **QA the first A-pose export**: if arms bake rotated, fix the char-frame or add a per-arm rest-alignment quaternion. This is the one genuine divergence vs the proven Mixamo (T-pose) path. Alternatively force T-pose on the iClone side: FBX Advanced Settings > **Use T-Pose As Bind Pose**.
|
||
|
||
## 4. Recommended iClone export settings for this pipeline
|
||
|
||
- iClone: `File > Export > Export Fbx...`, Target Tool Preset **Blender** (or Unity 3D), Export Range **All**/**Range**, FPS `Project(60)` fine (tool handles it). Advanced: **Preserve Bone Names (CC Base)** ON (default — the bone map depends on it), consider **Use T-Pose As Bind Pose** (see §3 caveat), **Reset Bone Scale** ON.
|
||
- Run **Menu > Animation > Flatten All Motion with Constraint** first if the take uses Reach/Link/Look At/Path — those keys don't survive FBX.
|
||
- iClone 8 always exports mesh+animation (no motion-only mode); `cc_retarget.py` strips the mesh anyway, so don't fight it — but uncheck Embed Textures / set Max Texture Size low to keep the FBX small.
|
||
- Animation-only alternative: export via CC4 (`FBX Options: Motion`) — the retargeter still needs a rest pose, and Blender only gets bind pose from a weighted mesh, so prefer the iClone mesh+motion export.
|
||
- Store-content license wall: Reallusion marketplace motions/characters need a per-pack **Export License** to leave via FBX; your own Video Mocap recordings and characters are unrestricted.
|
||
|
||
## 5. Runtime consumers
|
||
|
||
- **AnimationShowcase** (`src/Testing/AnimationShowcase.cs`, `scenes/animation_showcase.tscn`): pages all clips. `SCENE=animation_showcase bash tools/game.sh spawn` (game.sh:926). ASK Jeremy before launching the game.
|
||
- **Dance test bed** (`scenes/dance_test_bed.tscn`, `src/Testing/Dance/DanceTestBed.cs:118-124`): hot-reload dances via `DanceLibrary.Poll`; in-scene `VideoRecorder` → `~/Downloads/*.mp4` at 30fps. `SCENE=dance_test_bed bash tools/game.sh spawn` (game.sh:927).
|
||
- **Dance JSON** (`assets/dances/*.json`): `{ "name", "bpm", "moves": [...] }`; move = `{ "clip": "pack/Clip", "loops": N }` or composite `{ "layers": [{"mask":"upper","clip":...},{"mask":"lower","clip":...}], "duration": s }`. Transitions quantize to the BPM beat clock. Examples: `haka_mocap.json` (uses `haka_mocap/HakaFull`), `clap_stomp.json` (110), `war_challenge.json` (120).
|
||
- **CompositeClipBaker.cs**: lower mask = `root|pelvis|thigh*|calf*|foot*|ball*` (always owns hip pos+rot — prevents hip-fight/foot-slide); upper = rest; first-writer-wins per track. Cache keyed by definition → JSON edits need no invalidation.
|
||
- **Hard rules:** skeleton path remap mandatory for raw GLB clips (GLB tracks target `Armature/Skeleton3D:bone`; derive real prefix from an already-remapped clip — `PlayerController.LoadUalLibrary` is canonical). **Never use `*RM`/`*RootMotion` clip variants** — everything is in-place.
|
||
|
||
## 6. Godot import conventions
|
||
|
||
- `.import` sidecars: `importer="scene"`, **`animation/fps=30`** (even 60fps cc output resamples to 30 at import), `animation/remove_immutable_tracks=true`, `animation/import_rest_as_RESET=false`, `meshes/generate_lods=false`. `.uid` files auto-generate — never hand-author.
|
||
- Commit GLB + let Godot import → immediately referenceable as `<pack>/<ClipName>`.
|
||
|
||
## 7. Precedents
|
||
|
||
- **Kevin Iglesias**: 199 Unity FBX (`B-*` rig) → 12 packs in `assets/quaternius/kevin/` via `kevin_retarget.py` batch — the batch-directory template.
|
||
- **Video-to-mocap (free, local)**: `extract_pose.py` (MediaPipe) → `mocap_retarget.py` (rotation-solving, no source armature) → `dancegen/haka_mocap.glb` / `HakaFull`. iClone Video Mocap is the paid, higher-quality alternative feeding the same `cc_retarget.py` path.
|
||
- Four bone-map dicts exist: `kevin_retarget.py:29`, `mixamo_retarget.py:26`, `cc_retarget.py:25`, `mocap_retarget.py` AIM_BONES.
|
||
|
||
## 8. End-to-end recipe (iClone Video Mocap → in-game dance)
|
||
|
||
1. Film per the Video Mocap guidelines (eye-level static camera, full body, single actor, textured clothing; stack takes with 2s gaps into one ≤60s region).
|
||
2. Generate in the plugin (Full Body [+ Includes Fingers]), apply to a CC character, clean up in iClone (Motion Correction for feet, Curve Editor smooth/Butterworth for jitter — see the other references).
|
||
3. Export FBX per §4.
|
||
4. `cc_retarget.py --src <fbx> --out assets/quaternius/dancegen/<pack>.glb --name <ClipName>`.
|
||
5. Verify: open in animation_showcase or reference `<pack>/<ClipName>` from a dance JSON in `assets/dances/` (hot-reloads in the dance test bed).
|
||
6. First A-pose export: eyeball arm orientation (§3 caveat) before batching.
|