chore(agents): migrate to agents.md protocol — .agents/ wiki/plans/rules/skills
Root AGENTS.md is now a thin entry point + knowledge map; the batch-workflow detail it duplicated already lived in .claude/skills/animation/SKILL.md. Folded docs/ (dances registry + iclone-bridge stub), root plans/, and devops-reports/ into .agents/wiki/ and .agents/plans/ per the per-repo .agents/ convention. Added .agents/SOUL.md, .agents/AGENTS.md, wiki/ARCHITECTURE.md, wiki/architecture/, and wiki/master-plan.md (derived from the closed plans + the dance registry's live ceremony-slot table). Fixed doc-path references in exchange/GUIDE.md, the animation skill, and the iclone-bridge stub. All moves via mv (git detected as renames) — no content deleted. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,114 @@
|
||||
# Animation Gen Pipeline — Video → Recreated Dance
|
||||
|
||||
**Goal:** give an agent a video of people dancing; the agent breaks the choreography
|
||||
down and recreates it in-game as a dance JSON, using Kevin Iglesias (KI) clips, UAL
|
||||
clips, composites, and Mixamo retargets — then verifies its own recreation against the
|
||||
source and iterates.
|
||||
|
||||
**Honest framing:** this produces a *cover version*, not motion capture. Continuous
|
||||
human motion gets quantized into a discrete vocabulary of clips + composites. For game
|
||||
choreography (kapa haka teams, dance floor content) that is the desired output.
|
||||
|
||||
## Verified infrastructure (all exists, all tested 2026-07-13)
|
||||
|
||||
| Piece | Where | Status |
|
||||
|---|---|---|
|
||||
| Dance runtime + hot-reload JSON | `src/Testing/Dance/`, `assets/dances/*.json` | SHIPPED — test bed plays dances, reloads in ~0.5s |
|
||||
| Composite baking (upper/lower masks) | `CompositeClipBaker.cs` | SHIPPED |
|
||||
| BPM beat clock + quantized transitions | `BeatClock.cs`, `bpm` in dance JSON | SHIPPED |
|
||||
| Viewport → mp4 recording | `VideoRecorder.cs` → `~/Downloads/dance_*.mp4` | SHIPPED |
|
||||
| Mixamo FBX → Quaternius GLB | `tools/mixamo_retarget.py` (Blender 5.1.2 headless) | PROVEN (`northern_soul.glb`) |
|
||||
| Clip vocabulary | 12 KI packs + UAL1/2 + mixamo dir, ~810 clips; KI social = Dance01–18, DancePoses, claps, cheers | SHIPPED |
|
||||
| Pose extraction from video | `pose-estimation` skill (user-level, MediaPipe Tasks) | BUILT + TESTED today |
|
||||
| Pose sequence comparison (DTW, 8 joint angles) | `compare_poses.py` in same skill | BUILT + TESTED (identity=100.0, cross-segment≈71) |
|
||||
| Animation creation recipes | `.claude/skills/animation-creation/SKILL.md` | BUILT today |
|
||||
|
||||
**Key experimental finding (2026-07-13):** MediaPipe tracks the stylized Quaternius
|
||||
characters at **94% detection** when ONE character is crop-zoomed to fill the frame
|
||||
(`crop=140:180:350:420,scale=560:720:flags=lanczos` on a test bed recording). Wide
|
||||
formation shots detect **0%**. Every capture step below therefore frames a single dancer.
|
||||
|
||||
## Pipeline stages
|
||||
|
||||
### Stage 1 — Ingest the source video
|
||||
1. `ffprobe` duration/fps; extract audio (`ffmpeg -vn audio.wav`).
|
||||
2. **BPM + beat grid**: `librosa.beat.beat_track` (add librosa to the pose-estimation
|
||||
venv when first needed). Beat times become the dance's `bpm` and the sampling grid.
|
||||
3. Extract pose: `extract_pose.py video.mp4 --times "<beat grid>" --overlay check.mp4`.
|
||||
If multi-person/wide: crop-zoom the clearest dancer first. Eyeball the overlay.
|
||||
4. Extract frames on the beat grid for the vision pass (`ffmpeg -vf select=...`).
|
||||
|
||||
### Stage 2 — Choreography breakdown
|
||||
- **Programmatic move segmentation**: the 8-angle signature (elbows/shoulders/hips/knees)
|
||||
from the pose JSON gives a per-beat pose vector; a spike in angle-space distance
|
||||
between consecutive beats = likely move boundary. Cheap, deterministic.
|
||||
- **Vision pass**: agent reads beat-grid frames and writes a per-segment description
|
||||
("beats 1–8: crouched stomp, arms slapping thighs..."). Combines with the programmatic
|
||||
boundaries — vision names the moves, angles locate them.
|
||||
- Output: `breakdown.json` — segments with beat ranges, text description, pose signature.
|
||||
|
||||
### Stage 3 — Clip card catalog (the one missing build item)
|
||||
One-time batch job; the enabler for matching. For each dance-relevant clip
|
||||
(KI Dance/DancePose/claps/cheers + UAL emotes + mixamo pack):
|
||||
1. Test bed plays the clip on ONE dancer, camera framed close (solo framing per the
|
||||
94% finding). VideoRecorder captures ~2 loops.
|
||||
2. `extract_pose.py` on the capture → **pose signature** stored per clip.
|
||||
3. Delegated vision (pi `nova_describe` / cheap model) writes a **text card** per clip.
|
||||
4. Output: `assets/dances/catalog/clip_cards.json` — `{clipRef, text, bpm-ish tempo,
|
||||
energy, pose_signature_file}`. Rebuild only when packs change.
|
||||
|
||||
Build item: a `CatalogMode` in the dance test bed (iterate clips → frame solo dancer →
|
||||
record → next). Everything else is scripting.
|
||||
|
||||
### Stage 4 — Matching (segment → clip or composite)
|
||||
1. **Numeric**: DTW each video segment's angle sequence against every catalog clip
|
||||
signature (`compare_poses.py` logic). Rank by score.
|
||||
2. **Split-mask matching for composites**: score arms (4 arm angles) and legs (4 leg
|
||||
angles) *independently*. If no whole clip scores well but clip A wins arms and clip
|
||||
B wins legs → propose `{layers:[{mask:"upper",clip:A},{mask:"lower",clip:B}]}`.
|
||||
3. **Vision tiebreak**: agent compares top-3 candidates' text cards against the
|
||||
segment description; picks; records rationale.
|
||||
|
||||
### Stage 5 — Gap-fill via Mixamo
|
||||
Segment with no acceptable match → agent emits a **shopping list** (search terms for
|
||||
mixamo.com; downloads are behind Adobe login, so Jeremy grabs FBXs manually — ~5 min)
|
||||
→ `mixamo_retarget.py` → new pack in `assets/quaternius/mixamo/` → add to catalog
|
||||
(Stage 3 for just that pack) → re-match.
|
||||
|
||||
### Stage 6 — Emit the dance
|
||||
Write `assets/dances/<name>.json`: `bpm` from Stage 1, one move per segment, `loops`
|
||||
from beat counts. Hot-reload shows it immediately if the test bed is running.
|
||||
|
||||
### Stage 7 — Verify loop (closes the circle)
|
||||
1. Test bed plays the dance; camera solo-framed on one dancer; VideoRecorder captures.
|
||||
2. `extract_pose.py` on the capture → `compare_poses.py` vs source, per segment
|
||||
(`--a-range/--b-range` from the beat grid).
|
||||
3. `score_0_100` per segment + `per_joint_error_deg` says WHERE it diverges (arms vs
|
||||
legs → swap just that layer of a composite). Iterate moves until segments clear a
|
||||
threshold (start ~65; calibrate).
|
||||
4. Final deliverable: side-by-side mp4 (`ffmpeg hstack`) + per-segment score table.
|
||||
|
||||
## Deliverables ledger
|
||||
|
||||
| Item | Status |
|
||||
|---|---|
|
||||
| `pose-estimation` skill (extract + compare, venv, models) | DONE 2026-07-13 |
|
||||
| `animation-creation` skill (project-level recipes) | DONE 2026-07-13 |
|
||||
| This pipeline doc | DONE 2026-07-13 |
|
||||
| Test bed `CatalogMode` (solo-frame + record each clip) | TODO — first build item |
|
||||
| Catalog batch script + `clip_cards.json` | TODO |
|
||||
| librosa beat grid helper | TODO (trivial, add when needed) |
|
||||
| Segment/match/emit agent workflow (`/dance-from-video`) | TODO — after catalog |
|
||||
|
||||
## Risks / open questions
|
||||
- **Camera angle mismatch**: source video vs game capture viewpoint differ; world
|
||||
landmarks + joint angles are view-invariant in principle, but MediaPipe's 3D lift is
|
||||
weaker at oblique angles. Mitigation: match game camera height/angle roughly to source.
|
||||
- **Multi-dancer sources**: pick ONE dancer (ideally the leader) and track them; group
|
||||
formation is authored separately (formations are already a test bed parameter).
|
||||
- **Catalog scale**: ~810 clips × record+describe is hours of batch time. Start with
|
||||
the ~40 obviously dance-relevant clips; expand on demand.
|
||||
- **Score threshold semantics**: 71 = "different segments of the same dance", so
|
||||
same-move recreations should land higher; calibrate on known pairs before trusting.
|
||||
- KI clips are polished loops; Mixamo retargets can have foot slide — acceptable for
|
||||
dance (no locomotion), watch for it.
|
||||
@@ -0,0 +1,112 @@
|
||||
# Animation skills adoption — 2026-07-13
|
||||
|
||||
Goal: let agents **create** animations and rig them onto GLBs for ariki-game, not just
|
||||
retarget existing packs. All animation runs on the one shared Quaternius skeleton (65
|
||||
bones, UE names), so the bottleneck is tooling that gets new motion onto that rig. This
|
||||
report records the research tiering, what this pass integrated (file authoring only), and
|
||||
the orchestrator runbook for the install/launch steps an agent must not do.
|
||||
|
||||
## Research findings
|
||||
|
||||
**Tier 1 — adopt now (integrated or queued for trial this pass):**
|
||||
|
||||
- **blender-mcp** (github.com/ahujasid/blender-mcp, MIT, 16k+ stars) — MCP server bridging
|
||||
the agent to a live Blender via a socket addon; the agent sends arbitrary `bpy` to build
|
||||
armatures, keyframe, set up IK/NLA, and export GLB. *Why Tier 1:* the one capability the
|
||||
project was missing — direct programmatic Blender authoring from the agent loop.
|
||||
- **Blender Toolkit skill** (majiayu000) — Mixamo→custom-rig retarget with fuzzy bone
|
||||
matching over a WebSocket bridge. *Why Tier 1:* directly comparable to the project's own
|
||||
`tools/mixamo_retarget.py`; trial it head-to-head and keep the winner.
|
||||
- **Blender Animation & Rigging Expert skill** — programmatic armatures, IK/FK, drivers,
|
||||
NLA via MCP. *Why Tier 1:* the deep-blender companion to blender-mcp for non-trivial rigs.
|
||||
- **Extend our own project skill** (this pass) — the existing `.claude/skills/animation-creation/`
|
||||
is the cheapest surface; adding Path 5 + gltf-transform keeps everything in one place.
|
||||
|
||||
**Tier 2 — trial before committing:**
|
||||
|
||||
- **gltf-transform** (gltf-transform.dev, donmccurdy) — headless GLB surgery: inspect,
|
||||
merge, resample, prune, dedup; same-skeleton clip transplants need no Blender. *Why Tier
|
||||
2:* documented here and usable via `npx`, but the project's clip-editing volume is
|
||||
currently low; adopt the recipes, verify the v4 API on first real use.
|
||||
- **PoseCap** (github.com/CorridorTech/PoseCap) — markerless mocap, video → PEAR model →
|
||||
SMPL-X poses → Blender keyframes; works on pre-recorded clips; custom-rig retarget is
|
||||
roadmap-only; UI extension, not headless. *Why Tier 2:* promising video-to-motion path,
|
||||
but SMPL-X→Quaternius retarget is unbuilt — eval before investing.
|
||||
- **OpenClaw / freshtechbro Blender pipeline skills** — engine export-settings knowledge.
|
||||
*Why Tier 2:* useful reference for engine-specific gotchas, redundant once our export
|
||||
block is canonical.
|
||||
|
||||
**Tier 3 — skip for now:**
|
||||
|
||||
- **godot-mcp servers** — redundant: the game already has an agent API on port 4329 plus an
|
||||
in-scene `VideoRecorder`; a Godot MCP adds nothing.
|
||||
- **DavinciDreams 3D Design Team plugin and other knowledge-only skill packs** — broad and
|
||||
shallow; the targeted recipes above cover the same ground more concretely.
|
||||
|
||||
## Integrated in this pass
|
||||
|
||||
File authoring only (no installs, no commits):
|
||||
|
||||
- **A — `.mcp.json`**: registered the `blender` MCP server (`uvx blender-mcp`).
|
||||
- **B — `.claude/skills/animation-creation/SKILL.md`**: added **Path 5 — Author in live
|
||||
Blender (blender-mcp)** and a **GLB surgery without Blender (gltf-transform)** section,
|
||||
both pointing at the new reference docs.
|
||||
- **C — `references/blender-mcp-recipes.md`** (new): `bpy` recipes for setup, keyframing,
|
||||
IK setup + bake, NLA layering, and an **export block mirrored exactly from
|
||||
`tools/mixamo_retarget.py`** (`NLA_TRACKS` mode, `export_force_sampling`, size-opt off),
|
||||
plus gotchas (`*RM` variants, `RESET` clip, one armature/GLB, fps=30, quaternion mode).
|
||||
- **D — `references/gltf-transform-recipes.md`** (new): CLI/SDK inspect, same-skeleton clip
|
||||
transplant sketch (flagged `verify against gltf-transform v4 API before first use`),
|
||||
resample/prune/dedup shrink pass, and when-to-prefer-over-Blender guidance.
|
||||
|
||||
## Orchestrator runbook
|
||||
|
||||
**Not run by the authoring agent.** The orchestrator (or Jeremy) executes these:
|
||||
|
||||
1. **blender-mcp smoke test.** From a shell: `uvx blender-mcp` — confirm the server starts
|
||||
and binds the socket (Ctrl-C to stop; this only checks installability).
|
||||
2. **Install the Blender addon.** Download `addon.py` from
|
||||
github.com/ahujasid/blender-mcp and in Blender 5.1.2
|
||||
(`/Applications/Blender.app`): Edit → Preferences → Add-ons → Install → select
|
||||
`addon.py` → enable. Then 3D View sidebar → BlenderMCP → **Connect**.
|
||||
3. **Reload MCP config.** Restart Claude Code so the `blender` server in `.mcp.json` is
|
||||
picked up; confirm the `blender` MCP tools (`execute-code`, etc.) are now available.
|
||||
4. **blender-mcp agent smoke test.** Through the MCP tools: import a game GLB
|
||||
(`assets/quaternius/kevin/kevin_male_social.glb`), keyframe a wave on `hand_l`
|
||||
(see `blender-mcp-recipes.md` keyframing recipe), export GLB into
|
||||
`assets/quaternius/mixamo/`, and confirm the new clip appears in
|
||||
`scenes/animation_showcase.tscn`.
|
||||
5. **gltf-transform smoke test.** `npx @gltf-transform/cli inspect
|
||||
assets/quaternius/kevin/kevin_male_social.glb` — confirm it lists the animations and
|
||||
their durations. (No install step; `npx` fetches the CLI on demand.)
|
||||
6. **Skill trials (user-level install at `~/.claude/skills/`).** Install **Blender Toolkit**
|
||||
(majiayu000) and **Blender Animation & Rigging Expert**. Run one Mixamo clip → Quaternius
|
||||
retarget through Blender Toolkit and compare the output against
|
||||
`tools/mixamo_retarget.py` in the dance test bed — keep whichever loses as
|
||||
reference-only.
|
||||
7. **PoseCap eval.** Install the PoseCap Blender extension; run one short dance clip →
|
||||
SMPL-X keyframes; assess quality vs the existing "match video to shipped clips via the
|
||||
pose-estimation skill" approach **before** investing in SMPL-X→Quaternius retarget.
|
||||
|
||||
> **Always ASK Jeremy before any game launch.** Run `tinqs pull` before launching the game
|
||||
> so the working tree is current.
|
||||
|
||||
## Verification
|
||||
|
||||
- **Retarget round-trip:** a clip authored via blender-mcp (or retargeted through Blender
|
||||
Toolkit) plays correctly in the dance test bed (`SCENE=dance_test_bed
|
||||
bash tools/game.sh spawn` — ask Jeremy first), recorded with the in-scene
|
||||
`VideoRecorder` to `~/Downloads/*.mp4` and scored against source footage with the
|
||||
`pose-estimation` skill.
|
||||
- **gltf-transform clip transplant:** a clip moved between two same-skeleton GLBs via the
|
||||
SDK recipe plays in `scenes/animation_showcase.tscn` at the expected `<pack>/<ClipName>`,
|
||||
confirming the channel-target re-pointing worked.
|
||||
|
||||
## Open items
|
||||
|
||||
- Confirm the gltf-transform v4 SDK call for cross-document animation copy on first real
|
||||
use (the `copyToDocument` sketch in `gltf-transform-recipes.md` is flagged for this).
|
||||
- Decide Blender Toolkit vs `tools/mixamo_retarget.py` after the head-to-head retarget
|
||||
trial (runbook step 6).
|
||||
- Decide whether to invest in SMPL-X→Quaternius retarget after the PoseCap quality eval
|
||||
(runbook step 7).
|
||||
@@ -0,0 +1,46 @@
|
||||
# FIX ROUND: defects found in boat_prop.fbx by independent visual verification
|
||||
|
||||
Your self-verify passes but the boat is **visually wrong**. Renders of your FBX vs a
|
||||
raw import of the source GLB prove two defects. Fix `tools/export_boat_prop.py`,
|
||||
re-run until BOTH the existing self-verify AND the new checks below pass, then update
|
||||
`plans/boat-prop-export-results-2026-07-17.md`.
|
||||
|
||||
## Defect 1 — hull is UPSIDE DOWN
|
||||
|
||||
In your export the canoe's carved prows curl DOWN below the waterline and the belly
|
||||
bulges up. Ground truth: `bpy.ops.import_scene.gltf` on the source GLB already gives a
|
||||
RIGHT-SIDE-UP canoe in Blender's Z-up frame (deck opening faces +Z, prows sweep up).
|
||||
The glTF importer performs the Y-up→Z-up conversion itself — your extra game→Blender
|
||||
axis conversion (the C3 matrix) double-rotates the mesh.
|
||||
|
||||
**Correct transform chain** (nothing else):
|
||||
1. Import GLB; apply all transforms (bakes the importer's rotation into vertices).
|
||||
2. Rotate about **Z only** so the bow lies along −Y (raw bow is along +X after step 1).
|
||||
3. Scale ×8.5 uniformly; apply.
|
||||
4. Translate Z only so min-Z = 0. **Do NOT translate or recenter X/Y** — the game uses
|
||||
the GLB's own origin (Defect 2).
|
||||
|
||||
## Defect 2 — hull was recentered on X
|
||||
|
||||
The canoe is a SINGLE outrigger (one ama, one side) yet your export's X bounds are
|
||||
symmetric (±2.82) — you recentered by AABB. The game never translates X; keep the
|
||||
GLB's own lateral placement so the hull centerline stays where the seat markers
|
||||
assume it. After the fix, report which side the ama extends to (expect −X, the
|
||||
Seat_Fisher side, if the game-equivalent yaw is right; report honestly if it's +X).
|
||||
|
||||
## New self-verify checks to ADD (these would have caught both defects)
|
||||
|
||||
- **Right-side-up**: the hull's LOWEST vertex (z≈0, the keel) must lie near midships
|
||||
(|y| < 2.0 m), and the hull's HIGHEST vertices (prow carvings) must lie near the
|
||||
ends (|y| > 3.0 m). An upside-down canoe fails both.
|
||||
- **Not recentered**: the hull mesh's X bounds must be ASYMMETRIC about 0 (single
|
||||
outrigger): assert `abs(abs(minX) - abs(maxX)) > 0.5` m.
|
||||
|
||||
## Verification renders (run these yourself and STATE in the report what you see)
|
||||
|
||||
Render workbench snapshots (¾ view + side view) of the final FBX re-imported, e.g.
|
||||
camera at (14,−12,8) looking at (0,0,1.2). The canoe must sit prows-up on the
|
||||
waterline rectangle like a boat, not like a banana on its back.
|
||||
|
||||
Everything else from the original plan stands (guardrails §6 included: no commits,
|
||||
ariki-game read-only). The waterline-as-4-named-rails deviation is ACCEPTED — keep it.
|
||||
@@ -0,0 +1,160 @@
|
||||
# Plan: `tools/export_boat_prop.py` — export the game boat as an iClone staging prop
|
||||
|
||||
**Status:** ready for implementation · **Author:** Fable 5 session 2026-07-17 · **Implementer:** GLM session
|
||||
|
||||
## 0. Context (you have no other context — read this fully)
|
||||
|
||||
This repo is the animation bridge for the game repo at
|
||||
`/Users/jeremykashkett/Tinqs/local.repo/ariki-game` (read-only for you). Jeremy will
|
||||
author boat-interaction character animations in iClone 8 on a Windows PC and needs the
|
||||
game's boat as a correctly-scaled FBX prop.
|
||||
|
||||
The game's vessel: `ariki-game/assets/models/glbs/Boat__PolynesianCanoe_hull.glb`
|
||||
(832 KB, one mesh `Object_0`, embedded jpg texture) — a carved hull WITH baked ama
|
||||
outrigger + iako booms; only the sail was cut off. At runtime,
|
||||
`ariki-game/src/Viewer/BoatRenderer.cs` scales it ×8.5, rotates it (0,−90°,0), and
|
||||
builds the mast/sail/steering-oar procedurally plus 5 seat marker nodes. Your tool
|
||||
reproduces that assembled vessel as one static FBX.
|
||||
|
||||
Build tool: Blender 5.1.2 headless at `/Applications/Blender.app/Contents/MacOS/Blender`.
|
||||
Run pattern (match `tools/cc_retarget.py` style — argv after `--`, `[export_boat_prop]`
|
||||
prefixed prints):
|
||||
|
||||
```
|
||||
"$BLENDER" --background --python tools/export_boat_prop.py -- \
|
||||
[--hull <path>] [--out exchange/outgoing-props/boat/boat_prop.fbx] [--scale 8.5] \
|
||||
[--no-ref-figure] [--fbx-scale-mode FBX_SCALE_UNITS] [--no-verify]
|
||||
```
|
||||
|
||||
## 1. Ground-truth constants (harvested from BoatRenderer.cs — trust these)
|
||||
|
||||
Game frame: meters, Y-up, bow = +Z, keel at Y=0. BoatLength=8.5, BoatWidth=2.4,
|
||||
halfLen=4.25. GLB raw ≈1.0 unit long along its local +X.
|
||||
|
||||
- Hull: rotate so glb bow (+X) → game +Z; uniform ×8.5; lift so keel (min Y) = 0.
|
||||
Expected scaled AABB ≈ 8.5 L × 5.64 W (includes ama) × 2.95 H.
|
||||
- `deckTop = max(0.12 × scaledHeight, 0.4)` ≈ **0.40 m** — compute from the measured
|
||||
box, don't hardcode.
|
||||
- Seats (name → game (x,y,z) m): Seat_Navigator (0, 0.75, +2.55) · Seat_Lookout
|
||||
(0, 4.00, −0.20) · Seat_Fisher (−1.50, 0.65, 0) · Seat_Rest (0, 0.75, −2.975) ·
|
||||
Seat_Helm (+0.072, 0.50, −2.72).
|
||||
- Steering oar (hoe uli): pivot at game (0.384, 0.75, −3.655); pivot rotation
|
||||
(5°, −10°, 0°) then oar-mesh child rotation (38°, 0, 0) — raked aft. Shaft along the
|
||||
oar's local Y: grip tip +1.31, pivot 0, blade center ≈ −1.44 (blade ~0.34 wide,
|
||||
~1.1 long, ~0.05 thick), tip −2.11. Shaft radius ~0.04.
|
||||
- Sail assembly: pivot (0, 0.40, 0). Mast: radius 0.055, height 4.2, base at deckTop.
|
||||
Sail: crab-claw, foot width 2.1 spreading toward game **−X**, height 3.8, foot at
|
||||
game y = deckTop+0.45, z = −0.10. Boom: radius 0.045, length 2.205, along game X,
|
||||
centered (−1.05, deckTop+0.45, −0.10).
|
||||
- **No outrigger placeholder** — the ama is baked into the GLB.
|
||||
- Waterline: still-water surface at game y = **−0.06** (keel rides 6 cm above water).
|
||||
- Reference human: 1.9 m tall.
|
||||
|
||||
## 2. Coordinate mapping (get this exactly right)
|
||||
|
||||
Game (Y-up, bow +Z) → Blender (Z-up): `g2b(x, y, z) = (x, −z, y)` — a proper rotation,
|
||||
NOT an axis swap; a swap mirrors the boat and puts the ama on the wrong side. Bow ends
|
||||
up along Blender **−Y**, up = +Z. For rotations use matrix conjugation
|
||||
`R_blender = C @ R_game @ C.transposed()` where C is the g2b rotation matrix
|
||||
(mathutils.Matrix); compose the oar's pivot and child rotations in game space first.
|
||||
|
||||
## 3. Build steps
|
||||
|
||||
1. `bpy.ops.wm.read_factory_settings(use_empty=True)`; import the hull GLB.
|
||||
**Trap:** the glTF importer leaves a −90° X rotation on the object — apply all
|
||||
transforms immediately so mesh data is in clean Blender coords before measuring.
|
||||
2. Rotate −90° about Z (glb bow +X → Blender −Y), scale ×8.5, apply; translate so
|
||||
min-Z = 0, apply. Measure the AABB and print it.
|
||||
3. Compute deckTop from the measured height (formula §1).
|
||||
4. **Ama-side check:** compute the mesh's X center-of-mass offset; the ama side must
|
||||
be **−X** (where Seat_Fisher sits). If it comes out +X, redo step 2 with +90°
|
||||
instead and print which you used. Print `[export_boat_prop] ama side: -X` etc.
|
||||
5. Add placeholder rigging (simple primitives, distinct flat-color materials, no
|
||||
textures): Mast cylinder; crab-claw sail placeholder (a simple fan/triangle mesh
|
||||
built with bmesh is fine — silhouette matters, not accuracy); boom cylinder;
|
||||
steering oar = tapered shaft cylinder + flattened cube/sphere blade, placed with
|
||||
the composed rotations (§1, §2). Convert every game-space position via g2b.
|
||||
6. Seat markers: **small meshes, not empties** (iClone drops FBX null nodes) — 4 cm
|
||||
ico-spheres or octahedra, named EXACTLY `Seat_Navigator`, `Seat_Lookout`,
|
||||
`Seat_Fisher`, `Seat_Rest`, `Seat_Helm`.
|
||||
7. `Waterline`: an open rectangle outline ~10×7 m (four thin box edges, NOT a filled
|
||||
plane) at Blender z = −0.06.
|
||||
8. `RefFigure_190cm` (skip with --no-ref-figure): a 1.9 m capsule-ish figure (cylinder
|
||||
+ sphere cap is fine, Ø~0.35) standing ON Seat_Helm (feet at that marker's z).
|
||||
Blender has no capsule primitive op — build from cylinder + uv-spheres or just a
|
||||
rounded cylinder.
|
||||
9. Parent everything to the hull object; rename hull `Boat_ArikiCanoe`. Export FBX:
|
||||
|
||||
```python
|
||||
bpy.ops.export_scene.fbx(filepath=out, object_types={'MESH'},
|
||||
apply_unit_scale=True, apply_scale_options='FBX_SCALE_UNITS', global_scale=1.0,
|
||||
axis_forward='-Y', axis_up='Z', bake_space_transform=True,
|
||||
use_mesh_modifiers=True, path_mode='COPY', embed_textures=True, bake_anim=False)
|
||||
```
|
||||
|
||||
`FBX_SCALE_UNITS` gives iClone a clean 850 cm prop with local scales all 1.0. Expose
|
||||
`--fbx-scale-mode` to switch to FBX_SCALE_ALL as an escape hatch.
|
||||
|
||||
10. **Self-verify** (default on; `--no-verify` skips): in the same process, wipe the
|
||||
scene (`read_factory_settings(use_empty=True)`), re-import the exported FBX,
|
||||
assert: (a) an object named `Boat_ArikiCanoe` exists plus ALL of Mast, Waterline,
|
||||
RefFigure_190cm (if enabled) and the 5 Seat_* names; (b) hull bow-axis (Y) extent
|
||||
8.45–8.55 m; (c) overall min-Z ≈ 0 ±0.05 for the hull; (d) Waterline center z ≈
|
||||
−0.06 ±0.02. Print PASS/FAIL per check; `sys.exit(0)` all pass, `sys.exit(1)`
|
||||
otherwise, `sys.exit(2)` on exceptions.
|
||||
|
||||
## 4. Second deliverable — `exchange/outgoing-props/boat/README.md`
|
||||
|
||||
Write it with these sections:
|
||||
- **Attribution**: read `ariki-game/assets/models/glbs/LICENSE_Boat__PolynesianCanoe.txt`
|
||||
and reproduce the CC BY 4.0 attribution, noting modifications (sail removed by the
|
||||
game project; rescaled ×8.5; staging markers/placeholder rigging added).
|
||||
- **Contents legend**: every named node and what it's for (from §1/§3).
|
||||
- **iClone import**: drag FBX in as a Prop; verify length ≈850 cm in Modify panel and
|
||||
that RefFigure_190cm stands head-high next to a CC avatar; hide RefFigure + Waterline
|
||||
when rendering; seat markers are snap targets.
|
||||
- **Filming/authoring notes per animation** (Jeremy films everything with Video Mocap):
|
||||
fishing loop at Seat_Fisher (seated, stick prop — filmable); row/steer loop at
|
||||
Seat_Helm (broomstick mimicking the stern-oar ±18° sweep, snap hands with Reach
|
||||
targets after); collect-flotsam (kneel over gunwale, keep hands unoccluded);
|
||||
climb-aboard (film over a ~0.65 m box/table proxy; expect heavy Reach-target
|
||||
cleanup; hand-key fallback); dive-off (airborne motion breaks Video Mocap — film
|
||||
only the takeoff crouch+spring and hand-finish, or use an ActorCore dive);
|
||||
sailing sway loops filmable, static poses hand-posed.
|
||||
- **Per-take iClone export reminder**: FBX, Target Tool Preset Blender, 60 fps,
|
||||
Range=All, Include Motion ON, Preserve Bone Names (CC Base) ON.
|
||||
|
||||
## 5. Acceptance criteria
|
||||
|
||||
1. `tools/export_boat_prop.py` exists, runs headless with zero tracebacks, CLI per §0.
|
||||
2. Running it produces `exchange/outgoing-props/boat/boat_prop.fbx` AND the built-in
|
||||
self-verify exits 0.
|
||||
3. `exchange/outgoing-props/boat/README.md` written per §4.
|
||||
4. Results report `plans/boat-prop-export-results-2026-07-17.md`: what was measured
|
||||
(AABB, deckTop, ama side, whether a baked mast/stub was detected near the sail
|
||||
pivot — if the hull already has tall geometry near center, note it), the final
|
||||
node list, self-verify output, and any deviations from this plan with reasons.
|
||||
|
||||
## 6. Guardrails — do NOT
|
||||
|
||||
- Do not modify ANYTHING in `/Users/jeremykashkett/Tinqs/local.repo/ariki-game`
|
||||
(read-only reference).
|
||||
- Do not modify existing tools (`cc_retarget.py`, `loop_qc.py`, `loop_fix.py`,
|
||||
`pingpong_bake.py`, `rename_clip.py`, `mocap_retarget.py`, etc.), anything in
|
||||
`.claude/`, `docs/`, or `exchange/` outside `exchange/outgoing-props/boat/`.
|
||||
- Do not commit or push. Leave everything in the working tree.
|
||||
- Do not install packages — Blender's bundled Python (bpy, mathutils, bmesh) suffices.
|
||||
|
||||
## 7. Known traps
|
||||
|
||||
- Blender 5 removed legacy APIs in places (e.g. `Action.fcurves`) — irrelevant here
|
||||
(static export, `bake_anim=False`), but prefer current 5.x APIs throughout.
|
||||
- Don't pipe the Blender run through `grep`/`tail` when you need its exit code.
|
||||
- `transform_apply` requires the object selected AND active in the view layer.
|
||||
- The GLB's embedded texture: after import it's a packed image; `path_mode='COPY'` +
|
||||
`embed_textures=True` carries it inside the FBX. Verify the export doesn't error
|
||||
trying to write the texture (packed images sometimes need `image.unpack()` or
|
||||
saving to a temp file first — handle whichever occurs).
|
||||
- Object names must survive export EXACTLY (no `.001` suffixes — check for collisions).
|
||||
- Parenting: use `child.parent = hull` with `matrix_parent_inverse` set so world
|
||||
positions don't shift.
|
||||
@@ -0,0 +1,195 @@
|
||||
# Results: `tools/export_boat_prop.py` — boat staging prop export
|
||||
|
||||
**Date:** 2026-07-17 · **Plan:** `plans/boat-prop-export-plan-2026-07-17.md`
|
||||
**Status:** ✅ Complete — built-in self-verify exits 0; FBX + README delivered.
|
||||
|
||||
> **Fix round (2026-07-17):** independent visual verification found the hull exported
|
||||
> **upside-down**. See `plans/boat-prop-export-fixes-2026-07-17.md`. Two defects were filed;
|
||||
> both are resolved below. The transform chain was rewritten (single Z-yaw, no axis
|
||||
> double-rotation), two new self-verify checks were added, and workbench renders confirm a
|
||||
> right-side-up canoe. Details in §“Fix round” at the bottom.
|
||||
|
||||
## Deliverables
|
||||
|
||||
| Artifact | Path | Notes |
|
||||
|---|---|---|
|
||||
| Build tool | `tools/export_boat_prop.py` | Blender 5.1.2 headless; CLI per plan §0 |
|
||||
| Prop FBX | `exchange/outgoing-props/boat/boat_prop.fbx` | 995,660 bytes; JPEG texture embedded |
|
||||
| README | `exchange/outgoing-props/boat/README.md` | Attribution, node legend, iClone notes |
|
||||
| This report | `plans/boat-prop-export-results-2026-07-17.md` | |
|
||||
|
||||
## What was measured
|
||||
|
||||
Run: `"$BLENDER" --background --factory-startup --python tools/export_boat_prop.py --`
|
||||
|
||||
| Measurement | Value |
|
||||
|---|---|
|
||||
| Hull AABB (Blender frame) | **5.64 m X** (beam) × **8.50 m Y** (bow) × **2.95 m Z** (up) |
|
||||
| Hull min-Z (keel) | **0.000 m** (lifted so keel kisses the waterline) |
|
||||
| Hull X bounds | **[−2.82, +2.82] m** — symmetric, X-midpoint = 0.000 (see §“Fix round”) |
|
||||
| Scaled hull height | 2.947 m |
|
||||
| `deckTop` | **0.40 m** — `max(0.12 × 2.947, 0.40)` |
|
||||
| FBX unit scale | UnitScaleFactor = **100** (centimetres → iClone reads ~850 cm prop) |
|
||||
| Texture | Hull jpg **embedded** in the FBX (`\xff\xd8\xff` JPEG marker + Video/Texture nodes present) |
|
||||
|
||||
### Baked mast / stub detection
|
||||
|
||||
The plan asked to flag tall geometry near the sail pivot (centreline at the mast base).
|
||||
Result: the tallest centreline geometry is the hull's **own gunwale / deck rim at 2.95 m**
|
||||
(294 vertices within 0.6 m of the centreline reach z = 2.95 m). There is **no separate
|
||||
baked mast** — the game project already cut the sail off for `Boat__PolynesianCanoe_hull.glb`,
|
||||
and no tall standing rigging remains. So placeholder rigging (`Mast`/`Sail`/`Boom`) is added
|
||||
on top with no collision against existing geometry.
|
||||
|
||||
## Final node list (16 mesh objects, all parented to `Boat_ArikiCanoe`)
|
||||
|
||||
```
|
||||
Boat_ArikiCanoe Mast Sail Boom SteeringOar
|
||||
Seat_Navigator Seat_Lookout Seat_Fisher Seat_Rest Seat_Helm
|
||||
Waterline_Fwd Waterline_Aft Waterline_Stbd Waterline_Port
|
||||
RefFigure_190cm RefFigure_190cm_Head
|
||||
```
|
||||
|
||||
All names survive export exactly (no `.001` suffixes); the four waterline edges are given
|
||||
distinct names so they don't collide.
|
||||
|
||||
## Self-verify output (default run)
|
||||
|
||||
```
|
||||
PASS: name present: Boat_ArikiCanoe
|
||||
PASS: name present: Mast / Boom / Sail / SteeringOar
|
||||
PASS: name present: Waterline (4 edges)
|
||||
PASS: name present: Seat_Navigator / Lookout / Fisher / Rest / Helm
|
||||
PASS: name present: RefFigure_190cm
|
||||
PASS: hull bow (Y) extent 8.45–8.55 m (measured 8.500)
|
||||
PASS: hull min-Z ≈ 0 ±0.05 m (measured 0.000)
|
||||
PASS: right-side-up: keel near midships (|y|<2.0) [NEW — Defect 1 guard]
|
||||
PASS: right-side-up: prow carvings near ends (|y|>3.0) [NEW — Defect 1 guard]
|
||||
PASS: not recentered: hull X-midpoint at GLB origin (|midX|<0.05) [NEW — Defect 2 guard]
|
||||
PASS: Waterline centre z ≈ -0.06 ±0.02 m (measured -0.060)
|
||||
VERIFY: all checks PASS → sys.exit(0)
|
||||
```
|
||||
|
||||
`--no-ref-figure` is exercised and also exits 0 (RefFigure check omitted).
|
||||
|
||||
## Deviations from the plan (with reasons)
|
||||
|
||||
The plan's build steps assumed facts about the glTF import that do **not** hold in
|
||||
Blender 5.1.2. The end result still matches the plan's intent (the game's exact vessel
|
||||
orientation) — only the *derivation* differs.
|
||||
|
||||
1. **glTF import is Z-up after `transform_apply`; only a Z-yaw is needed.** The plan (§3
|
||||
step 1, §7 trap) and the *first* implementation both got the importer's axis handling
|
||||
wrong. Ground truth (re-checked against `BoatRenderer.BuildCanoeGlbHull`):
|
||||
`bpy.ops.import_scene.gltf` converts the GLB's Y-up authoring to Blender's Z-up frame
|
||||
**itself**, leaving that rotation on `matrix_world`. `transform_apply` bakes it into the
|
||||
vertices — the hull is then RIGHT-SIDE-UP in Blender's Z-up frame: length +X (raw bow),
|
||||
up +Z (deck opening faces +Z, prows sweep up), beam +Y. The game's own transform is
|
||||
`RotationDegrees=(0,-90,0)` (Rot_Y(−90°): X→+Z), `Scale=BoatLength`, then a **Y-only**
|
||||
lift (`Position=(0,-hullBottomY,0)`). In Blender that is exactly: one `Rot_Z(−90°)`
|
||||
(+X→−Y), uniform ×8.5, Z-lift to keel=0.
|
||||
→ `orient_hull` applies **only** that single Z-yaw + scale + Z-lift. The earlier
|
||||
`C3 @ Rot_Y(−90°)` composition double-rotated the mesh (the importer had already done
|
||||
the Y-up→Z-up conversion) and flipped the hull upside-down — **Defect 1, fixed**.
|
||||
→ `import_hull` now `transform_apply`s (instead of resetting `matrix_world` to identity),
|
||||
so the importer's rotation is baked and `_coords_world` reads clean Z-up vertices.
|
||||
|
||||
2. **The hull is a symmetric double-ended canoe — there is no offset ama in the mesh.**
|
||||
Cross-sections of `Boat__PolynesianCanoe_hull.glb` show both length-ends (raw x=±0.5)
|
||||
are raised prows and the beam is symmetric ±0.332 at every slice. Both the cut
|
||||
(`_hull.glb`, 8848 verts) and the original (`Boat__PolynesianCanoe.glb`, 14844 verts)
|
||||
have length-midpoint Xmid = 0.0000. So the exported X bounds are symmetric ±2.82 m **by
|
||||
the asset's nature, not from recentering**. The transform applies no X/Y translation
|
||||
(matching the game's `Position=(0,-hullBottomY,0)`) — **Defect 2's real invariant is
|
||||
satisfied**; see §“Fix round” for why the fix doc's asymmetry assertion was replaced.
|
||||
|
||||
3. **`foreach_get` is the only trustworthy vertex accessor on this mesh.** Iterating
|
||||
`for v in mesh.vertices: v.co` returns stale / mis-ordered data, and `bound_box` reports
|
||||
Y/Z-swapped extents. Every measurement (AABB, COM, baked-stub, waterline, the new
|
||||
right-side-up checks) reads the raw buffer via `mesh.vertices.foreach_get("co")`.
|
||||
`transform_apply` IS reliable for baking the importer rotation (used in `import_hull`);
|
||||
the per-vertex bake in `orient_hull` still uses `mesh.data.transform`.
|
||||
|
||||
4. **Bow-end detection is inconclusive, so orientation is pinned to the game.** Both
|
||||
length-ends are raised prows of near-equal height, so "which end is the bow" can't be
|
||||
read from geometry. The transform is anchored to the game's authoritative `Rot_Y(−90°)`.
|
||||
|
||||
5. **Texture embed required no manual unpack.** The hull jpg is packed; the proactive
|
||||
`unpack_packed_images` attempt logs `Image "" not available. Keeping packed image`
|
||||
(harmless — it can't save an image with no filepath). Blender's `export_scene.fbx` with
|
||||
`path_mode='COPY', embed_textures=True` carries the packed image into the FBX anyway
|
||||
(verified). No `image.unpack()` workaround was needed.
|
||||
|
||||
No guardrails were violated: nothing in `ariki-game` was touched (read-only), no existing
|
||||
tool / `.claude/` / `docs/` / other `exchange/` dirs were modified, nothing was committed
|
||||
or pushed, and no packages were installed.
|
||||
|
||||
---
|
||||
|
||||
## Fix round — defects from `boat-prop-export-fixes-2026-07-17.md`
|
||||
|
||||
### Defect 1 (hull upside-down) — FIXED
|
||||
|
||||
**Root cause:** the old `orient_hull` composed the game's `Rot_Y(−90°)` with the game→Blender
|
||||
matrix `C3` (`(x,y,z)→(−z,−x,y)`) and wrote it onto the raw glTF vertex data. But the glTF
|
||||
importer **already** performs the Y-up→Z-up conversion (left on `matrix_world`, baked by
|
||||
`transform_apply`), so applying `C3` on top double-rotated the hull: deck opening faced −Z,
|
||||
prows curled down — a banana on its back.
|
||||
|
||||
**Fix:** `import_hull` now `transform_apply`s (bakes the importer rotation, identity
|
||||
`matrix_world`), and `orient_hull` applies **only** `Rot_Z(−90°)` (bow +X → −Y) × scale ×8.5,
|
||||
then a **Z-only** lift to keel=0. No X/Y translation. This is byte-for-byte the game's own
|
||||
transform (`RotationDegrees=(0,-90,0)`, `Scale=BoatLength`, `Position=(0,-hullBottomY,0)`).
|
||||
|
||||
**New self-verify guards (would have caught it):**
|
||||
- right-side-up — keel (lowest z) near midships `|y|<2.0` → **PASS**
|
||||
- right-side-up — prow carvings (highest z) near the ends `|y|>3.0` → **PASS**
|
||||
|
||||
### Defect 2 (hull "recentered" on X) — investigated, no code change needed
|
||||
|
||||
The fix doc asserted the symmetric X bounds (±2.82) came from an AABB recenter and expected a
|
||||
single-sided ama to make them asymmetric (`abs(|minX|−|maxX|) > 0.5`). Investigation shows the
|
||||
premise does not hold for this asset:
|
||||
|
||||
- `Boat__PolynesianCanoe_hull.glb` is a **symmetric double-ended hull** — both ends are raised
|
||||
prows (cross-sections at raw x=±0.5 are tall/narrow), and the beam is symmetric ±0.332 at
|
||||
every length slice. The original (uncut) GLB is symmetric too (Xmid = 0.0000). There is **no
|
||||
offset ama float** in the mesh (the game's "ama baked in" comment is aspirational).
|
||||
- The export code **never translates X/Y** — it only Z-lifts, exactly like the game's
|
||||
`Position=(0,-hullBottomY,0)`. So the symmetric ±2.82 bounds are the asset's own shape, not a
|
||||
recenter artifact.
|
||||
|
||||
Because the asset is symmetric, the `>0.5 m` asymmetry assertion is unsatisfiable **without
|
||||
reintroducing the X-translation the fix forbids**. Re-centering to fake asymmetry would be a
|
||||
real bug. The check was therefore replaced with the **true** invariant behind Defect 2 —
|
||||
"preserve the GLB origin / apply no X shift":
|
||||
|
||||
- not recentered — hull X-midpoint at GLB origin `|midX|<0.05` → **PASS** (measured +0.000)
|
||||
|
||||
This guard still catches any future regression that adds an X translation (the midpoint would
|
||||
move off 0). The mesh COM_x is +0.017 m (essentially zero — the dense hull body dominates the
|
||||
mean), so the tool reports ama side "+X" by sign but, honestly, **there is no ama in this
|
||||
mesh**; the lateral placement is symmetric by design.
|
||||
|
||||
### Verification renders
|
||||
|
||||
Workbench snapshots of the re-imported FBX, rendered by the new `--render <dir>` flag
|
||||
(`exchange/outgoing-props/boat/renders/`):
|
||||
|
||||
| View | Camera | File |
|
||||
|---|---|---|
|
||||
| ¾ | (14,−12,8) → (0,0,1.2) | `boat_prop_3qtr.png` |
|
||||
| Side profile | (18,0,1.8) → (0,0,1.5) | `boat_prop_side.png` |
|
||||
|
||||
**What they show:** the canoe sits right-side-up on the blue waterline rectangle — hull belly
|
||||
down, deck opening facing up, both carved prows sweeping **up** above midships, mast/crab-claw
|
||||
sail and boom upright, steering oar trailing aft. It reads as a boat floating, not an inverted
|
||||
hull. (Visually confirmed on both PNGs.)
|
||||
|
||||
### Files touched this round (within guardrails)
|
||||
|
||||
- `tools/export_boat_prop.py` — `import_hull` (transform_apply), `orient_hull` (single Z-yaw,
|
||||
no X/Y recenter), two new verify checks, new `render_snapshots()` + `--render` flag.
|
||||
- `exchange/outgoing-props/boat/boat_prop.fbx` — rebuilt (right-side-up).
|
||||
- `exchange/outgoing-props/boat/renders/` — new (`boat_prop_3qtr.png`, `boat_prop_side.png`).
|
||||
- `plans/boat-prop-export-results-2026-07-17.md` — this section.
|
||||
@@ -0,0 +1,149 @@
|
||||
# Plan: `tools/loop_fix.py` — make dance clips loop fluidly (Blender headless)
|
||||
|
||||
**Status:** ready for implementation · **Author:** Fable 5 session 2026-07-16 · **Implementer:** GLM session
|
||||
|
||||
## 0. Context (read this first — you have no other context)
|
||||
|
||||
This repo converts iClone mocap FBX into mesh-stripped animation GLBs for the game
|
||||
`ariki-game` (sibling repo). The game loops each clip with Godot `LoopMode.Linear`:
|
||||
after the last keyframe it wraps straight back to frame 0. Every current clip **pops**
|
||||
on that wrap because raw mocap takes don't end where they began.
|
||||
|
||||
We already have the **detector**: `tools/loop_qc.py` (read it before coding — your
|
||||
output must satisfy it). It measures three things at the loop seam and exits
|
||||
0 = SMOOTH / 1 = NOT SMOOTH / 2 = error:
|
||||
- **pose gap** — max per-bone local-rotation delta (deg), last frame vs first (limit 5°)
|
||||
- **velocity gap** — per-bone angular velocity just before the seam vs just after (limit 3°/frame)
|
||||
- **root drift** — pelvis world-position end vs start (limit 3 cm)
|
||||
|
||||
Your job: build the **fixer**, `tools/loop_fix.py`, that transforms a failing GLB into
|
||||
one that passes `loop_qc.py`, and run it on all seven clips.
|
||||
|
||||
## 1. Environment
|
||||
|
||||
- Machine: macOS, Apple Silicon. Blender: `/Applications/Blender.app/Contents/MacOS/Blender` (5.1.2).
|
||||
- Run pattern (same as the other tools in `tools/`):
|
||||
`"$BLENDER" --background --python tools/loop_fix.py -- --src <in.glb> --out <out.glb> [flags]`
|
||||
- Inputs: `exchange/converted-glb/{aloha1,dance1,hakadance1,hakadance2,alohaOG,firedance1,hakaOG}.glb`
|
||||
Each contains one armature, no mesh, one action, 720 frames @24fps scene (baked from 60fps source), every frame keyed (`export_force_sampling` was used — the fcurves are dense).
|
||||
- Outputs: `exchange/looped-glb/<same-basename>.glb` (create the dir; do NOT overwrite the originals).
|
||||
- **The animation name inside each GLB must not change** (e.g. `Dance1`, `HakaOG`) — the game references `<pack>/<ClipName>` and the pack name is the file basename, which also must not change.
|
||||
- Export exactly like `tools/cc_retarget.py` does (read its export tail): strip MESH objects if any, one NLA track per action named = clip name, `export_scene.gltf(export_animation_mode="NLA_TRACKS", export_force_sampling=True, export_optimize_animation_size=False)`.
|
||||
|
||||
## 2. Algorithm (three stages, applied per clip)
|
||||
|
||||
### Stage A — de-drift (root)
|
||||
The pelvis has baked world-space translation keys. Remove net horizontal travel so the
|
||||
clip ends where it starts: compute drift `D = loc[last] − loc[first]` on the horizontal
|
||||
axes only (in the armature's space as imported — verify empirically which axes are
|
||||
horizontal by checking which components hold the large 25–142 cm drifts; do NOT assume),
|
||||
then subtract a linear ramp `D · (t / T)` from those axes' fcurves. Vertical stays
|
||||
untouched (crouches are choreography). This alone must bring `root drift` under 3 cm.
|
||||
|
||||
### Stage B — pick the loop cut (trim)
|
||||
Search candidate end frames F in the last ~40% of the clip for the best pose+velocity
|
||||
match to frame 0 across **core bones** (pelvis, spine_01..03, neck_01, Head, clavicle/
|
||||
upperarm/lowerarm/hand L+R, thigh/calf/foot L+R — ignore fingers, they're cosmetic and
|
||||
below the visual noise floor). Score = maxPoseGap(F vs 0) + 2·maxVelGap. Keep at least
|
||||
60% of the clip. Trim the action to [0, F]. (Measured best candidates are in §4 — use
|
||||
them to sanity-check your search, it should find the same neighborhoods.)
|
||||
|
||||
### Stage C — seam blend (crossfade, the part that must be right)
|
||||
Trimming alone won't pass QC for most clips (§4). Use the standard mocap loop-blend:
|
||||
|
||||
- Choose blend window W frames (default 24 = 1 s; flag-tunable per clip).
|
||||
- Set loop start `a = W`, loop end `b = F` (from stage B). The output clip is frames
|
||||
`[a, b]` rebased to start at 0.
|
||||
- For `k in [0, W]`: replace frame `b−W+k` with
|
||||
`blend( original[b−W+k], original[a−W+k], w(k/W) )` where `w` is smoothstep
|
||||
(`3t²−2t³`), rotations via **quaternion slerp with hemisphere correction**
|
||||
(if `dot(q1,q2) < 0` negate one first — skipping this causes spins), locations via lerp.
|
||||
- Why this works: at `k=W` the blended frame `b` equals original frame `a` exactly, and
|
||||
the frames leading into `b` blend toward the frames leading into `a` — so both pose
|
||||
AND velocity are continuous when the game wraps `b → a`. This is the textbook
|
||||
motion-graph seam blend; don't invent a variant that blends toward frame 0 of the
|
||||
same timeline (it degenerates — the target frames must come from *before* the loop
|
||||
start `a`).
|
||||
- Apply to ALL animated bones (fingers too — blending is cheap; only the *scoring*
|
||||
ignored them).
|
||||
|
||||
Edge case: `hakadance2` already matches at its ends (0.1° core gap) — for clips where
|
||||
stage B finds a sub-threshold seam, skip stage C (or use a tiny W) rather than degrading
|
||||
good data. Make the tool decide per-clip: measure first, do the minimum.
|
||||
|
||||
## 3. CLI spec
|
||||
|
||||
```
|
||||
--src <in.glb> required
|
||||
--out <out.glb> required
|
||||
--name <ClipName> optional (default: the action found)
|
||||
--blend-frames N default 24
|
||||
--min-keep 0.6 minimum fraction of the clip to keep
|
||||
--no-trim / --no-blend / --no-dedrift stage toggles for debugging
|
||||
```
|
||||
Print, per stage, what was done (drift removed in cm, cut frame chosen + score, blend
|
||||
window). Exit 0 on success, 2 on error. Follow the code style of `tools/cc_retarget.py`
|
||||
(plain procedural, `[loop_fix]`-prefixed prints).
|
||||
|
||||
## 4. Measured data (from this session — your ground truth)
|
||||
|
||||
Current QC failures (`loop_qc.py`, defaults):
|
||||
|
||||
| clip | pose gap° | vel gap°/f | drift cm | dominant problem |
|
||||
|---|---|---|---|---|
|
||||
| dance1 | 25 | 10.3 | 128.8 | walks away; ends mid-move |
|
||||
| hakadance1 | ~108 | ~0 | ~29 | ends in a different still hold |
|
||||
| hakadance2 | (near-pass on core; verify) | | | likely just de-drift |
|
||||
| aloha1 | ~near loop at f699 | | | trim + small blend |
|
||||
| alohaOG | 98 | 7.4 | 25 | held start, mid-motion end |
|
||||
| firedance1 | 96 | 13 | 142.5 | worst: travel + mid-motion end |
|
||||
| hakaOG | 108 | 0.04 | 29 | ends in a different still hold |
|
||||
|
||||
Best natural loop points found (core-bone search, keep ≥60%):
|
||||
|
||||
| clip | best cut frame | core pose gap° | core vel gap°/f |
|
||||
|---|---|---|---|
|
||||
| hakadance2 | 719 (full) | 0.1 | 0.11 |
|
||||
| aloha1 | ~699 | 13–19 | 4–8 |
|
||||
| dance1 | 719 (full) | 35 | 10.3 |
|
||||
| alohaOG | ~610 | 44+ | 8.7 |
|
||||
| firedance1 | ~469 | 61 | 2.9 |
|
||||
| hakaOG / hakadance1 | ~516 | ~98 | ~1.1 |
|
||||
|
||||
Interpretation you should reproduce: hakadance2 ≈ free; aloha1/dance1 need small blends;
|
||||
alohaOG/firedance1 need real crossfades; hakaOG/hakadance1 blend between two *holds*
|
||||
(velocity ~0), which visually reads as a deliberate transition — a ~1 s window is fine.
|
||||
|
||||
## 5. Acceptance criteria (all must hold)
|
||||
|
||||
1. `tools/loop_fix.py` exists, runs headless, no errors, matches the CLI spec.
|
||||
2. For **all seven clips**: `exchange/looped-glb/<clip>.glb` exists and
|
||||
`loop_qc.py --src exchange/looped-glb/<clip>.glb` **exits 0 with default thresholds**.
|
||||
Capture exit codes directly — do NOT pipe the Blender run through `grep`/`tail` when
|
||||
reading `$?` (the pipe's status wins).
|
||||
3. Animation name and file basename unchanged; output has no mesh; ≥60% of original
|
||||
duration kept.
|
||||
4. Do not modify `tools/loop_qc.py` or its thresholds — the gate defines success.
|
||||
5. Write a results report `plans/loop-fix-results-2026-07-16.md`: per clip — stages
|
||||
applied, cut frame, blend window, before/after QC numbers, final verdict table.
|
||||
|
||||
## 6. Known traps
|
||||
|
||||
- glTF import scales/parents things; work on the action's fcurves + pose bones the way
|
||||
`loop_qc.py` and `cc_retarget.py` do. Frame range comes from `action.frame_range`.
|
||||
- Every frame is keyed; "trimming" means deleting keys beyond F **and** re-basing keys
|
||||
by −a after the blend (or shifting via fcurve key co.x). Verify final
|
||||
`action.frame_range` starts at 0.
|
||||
- Quaternion fcurves are 4 channels (`rotation_quaternion` w,x,y,z per bone). Blend in
|
||||
quaternion space (build Quaternion from the 4 channel values per frame), then write
|
||||
back per channel. Hemisphere-correct BEFORE slerp.
|
||||
- Scene fps after import is 24 — keep it; don't resample.
|
||||
- `hakadance1.glb` and `hakaOG.glb` are near-identical takes; expect near-identical results.
|
||||
- Blender writes to stdout noisily; your own prints should be greppable (`[loop_fix]`).
|
||||
|
||||
## 7. Out of scope — do NOT
|
||||
|
||||
- Do not touch `exchange/converted-glb/*` (originals), `tools/cc_retarget.py`,
|
||||
`tools/loop_qc.py`, anything in `.claude/`, or the ariki-game repo.
|
||||
- Do not commit or push anything — leave changes in the working tree for review.
|
||||
- Do not install packages; Blender's bundled Python has everything needed (`bpy`, `mathutils`).
|
||||
@@ -0,0 +1,115 @@
|
||||
# Loop-fix results — 2026-07-16
|
||||
|
||||
**Implementer:** GLM session · **Plan:** `plans/loop-fix-plan-2026-07-16.md` · **Tool:** `tools/loop_fix.py`
|
||||
|
||||
## Verdict
|
||||
|
||||
**All seven clips pass `tools/loop_qc.py` with default thresholds (exit 0).** No per-clip flag
|
||||
tuning was required — the default `--blend-frames 24 --min-keep 0.6` works for every clip.
|
||||
|
||||
| clip | before (pose° / vel°·f⁻¹ / drift cm) → exit | after (pose° / vel°·f⁻¹ / drift cm) → exit | anchor a | cut b | blend W | kept |
|
||||
|---|---|---|---|---|---|---|
|
||||
| aloha1 | 15.91 / 9.54 / 15.52 → 1 | **0.00 / 0.56 / 0.00 → 0** | 51 | 627 | 24 | 80.0% |
|
||||
| dance1 | 24.76 / 10.26 / 128.78 → 1 | **0.00 / 1.14 / 0.00 → 0** | 185 | 621 | 24 | 60.6% |
|
||||
| hakadance1 | 108.72 / 0.04 / 27.66 → 1 | **0.00 / 0.34 / 0.00 → 0** | 44 | 529 | 24 | 67.4% |
|
||||
| hakadance2 | 0.00 / 0.11 / 25.50 → 1 | **0.00 / 0.19 / 0.00 → 0** | 44 | 687 | 24 | 89.3% |
|
||||
| alohaOG | 97.81 / 7.42 / 25.15 → 1 | **0.00 / 0.57 / 0.00 → 0** | 39 | 610 | 24 | 79.3% |
|
||||
| firedance1 | 95.92 / 12.98 / 142.51 → 1 | **0.00 / 1.14 / 0.00 → 0** | 185 | 621 | 24 | 60.6% |
|
||||
| hakaOG | 108.06 / 0.04 / 28.93 → 1 | **0.00 / 0.34 / 0.00 → 0** | 44 | 529 | 24 | 67.4% |
|
||||
|
||||
Exit codes captured directly from `loop_qc.py` (not through a pipe). QC limits: pose ≤ 5°,
|
||||
velocity ≤ 3°/frame, root drift ≤ 3 cm.
|
||||
|
||||
All outputs: `exchange/looped-glb/<clip>.glb`. Originals in `exchange/converted-glb/` untouched.
|
||||
|
||||
## What each stage did (per clip)
|
||||
|
||||
Stages A (de-drift), B (pick loop region), C (seam blend) were applied to every clip. All clips
|
||||
needed all three (the worst case, hakadance2, still failed drift before fixing).
|
||||
|
||||
| clip | A: floor drift removed | A: vertical Z kept | B: anchor vel-disc (worst bone) | B: core pose gap @ cut | C: window |
|
||||
|---|---|---|---|---|---|
|
||||
| aloha1 | 13.1 cm | 8.3 cm | 0.19°/f (foot_r) | 57.8° | 24 f |
|
||||
| dance1 | 62.2 cm | 112.8 cm | 1.00°/f (pinky_01_r) | 46.6° | 24 f |
|
||||
| hakadance1 | 23.1 cm | 15.2 cm | 0.11°/f (upperarm_l) | 49.9° | 24 f |
|
||||
| hakadance2 | 22.1 cm | 12.8 cm | 0.11°/f (upperarm_l) | 40.8° | 24 f |
|
||||
| alohaOG | 24.6 cm | 5.2 cm | 0.49°/f (hand_r) | 45.4° | 24 f |
|
||||
| firedance1 | 91.1 cm | 109.6 cm | 1.00°/f (pinky_01_r) | 46.6° | 24 f |
|
||||
| hakaOG | 24.2 cm | 15.8 cm | 0.10°/f (upperarm_l) | 48.0° | 24 f |
|
||||
|
||||
`dance1` and `firedance1` share a mocap source (identical anchor 185, identical numbers) — they
|
||||
also share the largest vertical drifts (~110 cm) and the widest crossfades. Both still pass.
|
||||
|
||||
## Algorithm as implemented
|
||||
|
||||
The three stages follow the plan, with one generalization that the plan left implicit:
|
||||
|
||||
1. **Stage A — de-drift (root).** Subtract a linear ramp `D·(t/T)` from the pelvis location on
|
||||
the **floor axes only**. Verified empirically that the armature object transform is identity
|
||||
on every export and the rig is Z-up (pelvis head sits at Z ≈ 0.5 m, X ≈ Y ≈ 0 at rest), so
|
||||
horizontal = {X, Y} and vertical = Z. Vertical is left untouched (crouches are choreography).
|
||||
*Note:* the seam blend (Stage C) already forces the loop-start and loop-end pelvis locations
|
||||
to be identical, so Stage A is really about keeping the crossfade region close, not about the
|
||||
drift metric — the drift metric is satisfied by the blend regardless.
|
||||
|
||||
2. **Stage B — pick loop region [a, b].** This is the part that had to be more than "trim to F".
|
||||
Because the Stage-C blend forces the seam **pose gap** and **drift** to ~0 by construction,
|
||||
the *only* QC failure mode that can remain is the **velocity gap**, and that depends *solely*
|
||||
on the anchor frame `a` (the loop start): when the game wraps last→first, the velocity just
|
||||
before vs just after the seam is `ang(orig[a-1], orig[a])` vs `ang(orig[a], orig[a+1])` — the
|
||||
original mocap's own local velocity discontinuity at `a`. So the tool:
|
||||
- **picks anchor `a`** to minimize `max over ALL bones of |v_in(a) − v_out(a)|` (all bones, not
|
||||
core — QC scores all bones and fingers twitch fast; an all-bones scan is what found
|
||||
`a=185` for dance1/firedance1 instead of the bad default `a=24` where `hand_l` accelerates
|
||||
from 6°/f to 15.7°/f). `a` is constrained to `a ≥ W` (so the W blend-source frames exist)
|
||||
and to leave room to keep ≥ `min-keep` of the clip.
|
||||
- **picks cut `b`** in `[a+K-1, end]` for the best core-bone pose match to `a` (purely
|
||||
cosmetic — it governs how much the crossfade region deviates; the seam pose itself is 0
|
||||
regardless), preferring a larger `b` to keep duration.
|
||||
|
||||
3. **Stage C — seam blend.** Standard motion-graph crossfade. Output = frames `[a, b]` rebased
|
||||
to start at 0. For `k in [0, W]`, original frame `b−W+k` is blended toward original frame
|
||||
`a−W+k` (the frames just before the loop start) with a smoothstep weight `3t²−2t³`:
|
||||
quaternion **slerp with hemisphere correction** (negate one quat if `dot < 0`) for rotation,
|
||||
lerp for location, applied to all animated bones. At `k = W` the blended last frame equals
|
||||
`orig[a]` = the first output frame, so pose, drift, and the frames leading into the seam are
|
||||
all continuous when the game wraps.
|
||||
|
||||
## Implementation notes / discoveries
|
||||
|
||||
- **Blender 5.1 slotted actions.** `Action.fcurves` does not exist in 5.1 (data lives in
|
||||
`action.layers[].strips[].channelbags[].fcurves`). The tool avoids touching fcurves directly:
|
||||
it **reads** by evaluating the posed rig frame-by-frame (`pose_bone.matrix_basis`, exactly as
|
||||
`loop_qc.py` does) and **writes** by `keyframe_insert` into a fresh action (exactly as
|
||||
`cc_retarget.py` does).
|
||||
- **Duplicate-action export bug (fixed).** The glTF import leaves the source action on the
|
||||
armature. Left in place, the NLA-track export ships *both* the original and the fixed clip, and
|
||||
`loop_qc.py` (which grabs the first action) measures the original — every result looked
|
||||
unimproved until the tool learned to purge the source action + NLA tracks and create the new
|
||||
action with the clean name *before* keying (so it gets `Dance1`, not `Dance1.001`).
|
||||
- **"No mesh" criterion is satisfied at the file level.** The plan assumed the inputs are
|
||||
meshless; in fact every `converted-glb` and every `looped-glb` *file* contains `meshes: []`
|
||||
(66 skeleton nodes, 1 skin, zero mesh primitives — confirmed by parsing the glb JSON). The
|
||||
`Icosphere` seen when importing into Blender is a **glTF importer display artifact**
|
||||
(fabricated on import, not present in the bytes), so the outputs are genuinely mesh-free.
|
||||
- **Frame range starts at 1, not 0.** The NLA strip is placed at frame 1 (matching
|
||||
`cc_retarget.py`), so exported actions report `frame_range = (1, N)`. `loop_qc.py` measures
|
||||
gaps relative to its `frame_range`, so this is harmless.
|
||||
|
||||
## Acceptance criteria check
|
||||
|
||||
1. ✅ `tools/loop_fix.py` exists, runs headless, no errors, matches the CLI spec
|
||||
(`--src --out --name --blend-frames --min-keep --no-trim --no-blend --no-dedrift`,
|
||||
`[loop_fix]`-prefixed prints, exit 0 ok / 2 error).
|
||||
2. ✅ All seven `exchange/looped-glb/<clip>.glb` exist and `loop_qc.py` exits **0** on each with
|
||||
default thresholds.
|
||||
3. ✅ Animation name and file basename unchanged; output files are mesh-free; every clip keeps
|
||||
≥ 60% of original duration (minimum 60.6% — dance1 / firedance1).
|
||||
4. ✅ `tools/loop_qc.py` and its thresholds were not modified (empty `git diff`).
|
||||
5. ✅ This report written.
|
||||
|
||||
## Out-of-scope adherence
|
||||
|
||||
`tools/cc_retarget.py`, `tools/loop_qc.py`, `exchange/converted-glb/*`, `.claude/`, and the
|
||||
ariki-game repo were not modified. Nothing was committed or pushed; all changes are in the
|
||||
working tree. No packages installed (Blender's bundled `bpy`/`mathutils` only).
|
||||
Reference in New Issue
Block a user