150 lines
8.5 KiB
Markdown
150 lines
8.5 KiB
Markdown
|
# Plan: `tools/loop_fix.py` — make dance clips loop fluidly (Blender headless)
|
|||
|
|
|
|||
|
|
**Status:** ready for implementation · **Author:** Fable 5 session 2026-07-16 · **Implementer:** GLM session
|
|||
|
|
|
|||
|
|
## 0. Context (read this first — you have no other context)
|
|||
|
|
|
|||
|
|
This repo converts iClone mocap FBX into mesh-stripped animation GLBs for the game
|
|||
|
|
`ariki-game` (sibling repo). The game loops each clip with Godot `LoopMode.Linear`:
|
|||
|
|
after the last keyframe it wraps straight back to frame 0. Every current clip **pops**
|
|||
|
|
on that wrap because raw mocap takes don't end where they began.
|
|||
|
|
|
|||
|
|
We already have the **detector**: `tools/loop_qc.py` (read it before coding — your
|
|||
|
|
output must satisfy it). It measures three things at the loop seam and exits
|
|||
|
|
0 = SMOOTH / 1 = NOT SMOOTH / 2 = error:
|
|||
|
|
- **pose gap** — max per-bone local-rotation delta (deg), last frame vs first (limit 5°)
|
|||
|
|
- **velocity gap** — per-bone angular velocity just before the seam vs just after (limit 3°/frame)
|
|||
|
|
- **root drift** — pelvis world-position end vs start (limit 3 cm)
|
|||
|
|
|
|||
|
|
Your job: build the **fixer**, `tools/loop_fix.py`, that transforms a failing GLB into
|
|||
|
|
one that passes `loop_qc.py`, and run it on all seven clips.
|
|||
|
|
|
|||
|
|
## 1. Environment
|
|||
|
|
|
|||
|
|
- Machine: macOS, Apple Silicon. Blender: `/Applications/Blender.app/Contents/MacOS/Blender` (5.1.2).
|
|||
|
|
- Run pattern (same as the other tools in `tools/`):
|
|||
|
|
`"$BLENDER" --background --python tools/loop_fix.py -- --src <in.glb> --out <out.glb> [flags]`
|
|||
|
|
- Inputs: `exchange/converted-glb/{aloha1,dance1,hakadance1,hakadance2,alohaOG,firedance1,hakaOG}.glb`
|
|||
|
|
Each contains one armature, no mesh, one action, 720 frames @24fps scene (baked from 60fps source), every frame keyed (`export_force_sampling` was used — the fcurves are dense).
|
|||
|
|
- Outputs: `exchange/looped-glb/<same-basename>.glb` (create the dir; do NOT overwrite the originals).
|
|||
|
|
- **The animation name inside each GLB must not change** (e.g. `Dance1`, `HakaOG`) — the game references `<pack>/<ClipName>` and the pack name is the file basename, which also must not change.
|
|||
|
|
- Export exactly like `tools/cc_retarget.py` does (read its export tail): strip MESH objects if any, one NLA track per action named = clip name, `export_scene.gltf(export_animation_mode="NLA_TRACKS", export_force_sampling=True, export_optimize_animation_size=False)`.
|
|||
|
|
|
|||
|
|
## 2. Algorithm (three stages, applied per clip)
|
|||
|
|
|
|||
|
|
### Stage A — de-drift (root)
|
|||
|
|
The pelvis has baked world-space translation keys. Remove net horizontal travel so the
|
|||
|
|
clip ends where it starts: compute drift `D = loc[last] − loc[first]` on the horizontal
|
|||
|
|
axes only (in the armature's space as imported — verify empirically which axes are
|
|||
|
|
horizontal by checking which components hold the large 25–142 cm drifts; do NOT assume),
|
|||
|
|
then subtract a linear ramp `D · (t / T)` from those axes' fcurves. Vertical stays
|
|||
|
|
untouched (crouches are choreography). This alone must bring `root drift` under 3 cm.
|
|||
|
|
|
|||
|
|
### Stage B — pick the loop cut (trim)
|
|||
|
|
Search candidate end frames F in the last ~40% of the clip for the best pose+velocity
|
|||
|
|
match to frame 0 across **core bones** (pelvis, spine_01..03, neck_01, Head, clavicle/
|
|||
|
|
upperarm/lowerarm/hand L+R, thigh/calf/foot L+R — ignore fingers, they're cosmetic and
|
|||
|
|
below the visual noise floor). Score = maxPoseGap(F vs 0) + 2·maxVelGap. Keep at least
|
|||
|
|
60% of the clip. Trim the action to [0, F]. (Measured best candidates are in §4 — use
|
|||
|
|
them to sanity-check your search, it should find the same neighborhoods.)
|
|||
|
|
|
|||
|
|
### Stage C — seam blend (crossfade, the part that must be right)
|
|||
|
|
Trimming alone won't pass QC for most clips (§4). Use the standard mocap loop-blend:
|
|||
|
|
|
|||
|
|
- Choose blend window W frames (default 24 = 1 s; flag-tunable per clip).
|
|||
|
|
- Set loop start `a = W`, loop end `b = F` (from stage B). The output clip is frames
|
|||
|
|
`[a, b]` rebased to start at 0.
|
|||
|
|
- For `k in [0, W]`: replace frame `b−W+k` with
|
|||
|
|
`blend( original[b−W+k], original[a−W+k], w(k/W) )` where `w` is smoothstep
|
|||
|
|
(`3t²−2t³`), rotations via **quaternion slerp with hemisphere correction**
|
|||
|
|
(if `dot(q1,q2) < 0` negate one first — skipping this causes spins), locations via lerp.
|
|||
|
|
- Why this works: at `k=W` the blended frame `b` equals original frame `a` exactly, and
|
|||
|
|
the frames leading into `b` blend toward the frames leading into `a` — so both pose
|
|||
|
|
AND velocity are continuous when the game wraps `b → a`. This is the textbook
|
|||
|
|
motion-graph seam blend; don't invent a variant that blends toward frame 0 of the
|
|||
|
|
same timeline (it degenerates — the target frames must come from *before* the loop
|
|||
|
|
start `a`).
|
|||
|
|
- Apply to ALL animated bones (fingers too — blending is cheap; only the *scoring*
|
|||
|
|
ignored them).
|
|||
|
|
|
|||
|
|
Edge case: `hakadance2` already matches at its ends (0.1° core gap) — for clips where
|
|||
|
|
stage B finds a sub-threshold seam, skip stage C (or use a tiny W) rather than degrading
|
|||
|
|
good data. Make the tool decide per-clip: measure first, do the minimum.
|
|||
|
|
|
|||
|
|
## 3. CLI spec
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
--src <in.glb> required
|
|||
|
|
--out <out.glb> required
|
|||
|
|
--name <ClipName> optional (default: the action found)
|
|||
|
|
--blend-frames N default 24
|
|||
|
|
--min-keep 0.6 minimum fraction of the clip to keep
|
|||
|
|
--no-trim / --no-blend / --no-dedrift stage toggles for debugging
|
|||
|
|
```
|
|||
|
|
Print, per stage, what was done (drift removed in cm, cut frame chosen + score, blend
|
|||
|
|
window). Exit 0 on success, 2 on error. Follow the code style of `tools/cc_retarget.py`
|
|||
|
|
(plain procedural, `[loop_fix]`-prefixed prints).
|
|||
|
|
|
|||
|
|
## 4. Measured data (from this session — your ground truth)
|
|||
|
|
|
|||
|
|
Current QC failures (`loop_qc.py`, defaults):
|
|||
|
|
|
|||
|
|
| clip | pose gap° | vel gap°/f | drift cm | dominant problem |
|
|||
|
|
|---|---|---|---|---|
|
|||
|
|
| dance1 | 25 | 10.3 | 128.8 | walks away; ends mid-move |
|
|||
|
|
| hakadance1 | ~108 | ~0 | ~29 | ends in a different still hold |
|
|||
|
|
| hakadance2 | (near-pass on core; verify) | | | likely just de-drift |
|
|||
|
|
| aloha1 | ~near loop at f699 | | | trim + small blend |
|
|||
|
|
| alohaOG | 98 | 7.4 | 25 | held start, mid-motion end |
|
|||
|
|
| firedance1 | 96 | 13 | 142.5 | worst: travel + mid-motion end |
|
|||
|
|
| hakaOG | 108 | 0.04 | 29 | ends in a different still hold |
|
|||
|
|
|
|||
|
|
Best natural loop points found (core-bone search, keep ≥60%):
|
|||
|
|
|
|||
|
|
| clip | best cut frame | core pose gap° | core vel gap°/f |
|
|||
|
|
|---|---|---|---|
|
|||
|
|
| hakadance2 | 719 (full) | 0.1 | 0.11 |
|
|||
|
|
| aloha1 | ~699 | 13–19 | 4–8 |
|
|||
|
|
| dance1 | 719 (full) | 35 | 10.3 |
|
|||
|
|
| alohaOG | ~610 | 44+ | 8.7 |
|
|||
|
|
| firedance1 | ~469 | 61 | 2.9 |
|
|||
|
|
| hakaOG / hakadance1 | ~516 | ~98 | ~1.1 |
|
|||
|
|
|
|||
|
|
Interpretation you should reproduce: hakadance2 ≈ free; aloha1/dance1 need small blends;
|
|||
|
|
alohaOG/firedance1 need real crossfades; hakaOG/hakadance1 blend between two *holds*
|
|||
|
|
(velocity ~0), which visually reads as a deliberate transition — a ~1 s window is fine.
|
|||
|
|
|
|||
|
|
## 5. Acceptance criteria (all must hold)
|
|||
|
|
|
|||
|
|
1. `tools/loop_fix.py` exists, runs headless, no errors, matches the CLI spec.
|
|||
|
|
2. For **all seven clips**: `exchange/looped-glb/<clip>.glb` exists and
|
|||
|
|
`loop_qc.py --src exchange/looped-glb/<clip>.glb` **exits 0 with default thresholds**.
|
|||
|
|
Capture exit codes directly — do NOT pipe the Blender run through `grep`/`tail` when
|
|||
|
|
reading `$?` (the pipe's status wins).
|
|||
|
|
3. Animation name and file basename unchanged; output has no mesh; ≥60% of original
|
|||
|
|
duration kept.
|
|||
|
|
4. Do not modify `tools/loop_qc.py` or its thresholds — the gate defines success.
|
|||
|
|
5. Write a results report `plans/loop-fix-results-2026-07-16.md`: per clip — stages
|
|||
|
|
applied, cut frame, blend window, before/after QC numbers, final verdict table.
|
|||
|
|
|
|||
|
|
## 6. Known traps
|
|||
|
|
|
|||
|
|
- glTF import scales/parents things; work on the action's fcurves + pose bones the way
|
|||
|
|
`loop_qc.py` and `cc_retarget.py` do. Frame range comes from `action.frame_range`.
|
|||
|
|
- Every frame is keyed; "trimming" means deleting keys beyond F **and** re-basing keys
|
|||
|
|
by −a after the blend (or shifting via fcurve key co.x). Verify final
|
|||
|
|
`action.frame_range` starts at 0.
|
|||
|
|
- Quaternion fcurves are 4 channels (`rotation_quaternion` w,x,y,z per bone). Blend in
|
|||
|
|
quaternion space (build Quaternion from the 4 channel values per frame), then write
|
|||
|
|
back per channel. Hemisphere-correct BEFORE slerp.
|
|||
|
|
- Scene fps after import is 24 — keep it; don't resample.
|
|||
|
|
- `hakadance1.glb` and `hakaOG.glb` are near-identical takes; expect near-identical results.
|
|||
|
|
- Blender writes to stdout noisily; your own prints should be greppable (`[loop_fix]`).
|
|||
|
|
|
|||
|
|
## 7. Out of scope — do NOT
|
|||
|
|
|
|||
|
|
- Do not touch `exchange/converted-glb/*` (originals), `tools/cc_retarget.py`,
|
|||
|
|
`tools/loop_qc.py`, anything in `.claude/`, or the ariki-game repo.
|
|||
|
|
- Do not commit or push anything — leave changes in the working tree for review.
|
|||
|
|
- Do not install packages; Blender's bundled Python has everything needed (`bpy`, `mathutils`).
|