animation/plans/fbx-pipeline-plan-2026-07-21.md

FBX → Quaternius Pipeline — Formal Plan & Specification

Doc id: fbx-pipeline-plan-2026-07-21 · Status: APPROVED direction, pre-implementation Owner: Jeremy · Author: Claude (5-agent research audit, 2026-07-21) Supersedes: ad-hoc tools/* pipeline docs scattered across skills/READMEs.

---

1. Purpose & scope

After the FBX-vs-GLB study (2026-07-21) we adopted Recommendation 1: the game keeps GLB clips on the shared Quaternius skeleton; the conversion pipeline gets consolidated.

In scope

• S1. Relocate the FBX→Quaternius pipeline into a single folder pipeline/ at the animation-repo root, runnable end-to-end on this Windows machine and on the Mac.
• S2. Accept any registered FBX source ("regular fbx"), not only iClone exports.
• S3. House the iClone integration in the pipeline, driven by an export script over the TinqsBridge API (localhost 18800).
• S4. Define and enforce a standardized bone/rig contract for all incoming FBX.
• S5. A reviewed, regression-gated migration (baseline before/after parity).

Out of scope (non-goals)

• N1. No change to ariki-game runtime code, ClipCatalog, dance JSON, or shipped GLBs.
• N2. No skeleton change (CC-skeleton option was evaluated and rejected).
• N3. No FBX-in-Godot ingestion (Option 2 spike may be scheduled separately).
• N4. Video-mocap tools (mocap_retarget.py) and the reverse prop bridge (export_boat_prop.py) stay in tools/ unchanged.

Asset flow decision: all assets continue to run through exchange/ (§6). The pipeline reads and writes only exchange/ stage folders; no new asset roots.

---

2. Definitions

Term	Meaning
Canonical skeleton	The game's Quaternius 65-bone UE4-named rig (§4.1). Sole retarget target.
Rig family	A named source-skeleton convention with a registered bone map (§4.2).
Take	One FBX file containing one baked animation clip + skinned mesh.
Pack / Clip	GLB basename / animation name baked into the GLB; game ref is pack/Clip.
Intake contract	The validity rules every FBX must satisfy before retargeting (§4.3).
Stage folder	A subfolder of exchange/ representing one pipeline stage (§6).
Bridge	TinqsBridge iClone plugin + iclone_bridge.py client (verified 2026-07-21).

---

3. Requirements

Functional

• FR-1 One command converts a take: intake validation → retarget → ping-pong bake → loop QC (both variants) → per-take report. Partial failures stop at the failing gate.
• FR-2 Rig family is auto-detected; a take on an unregistered rig is rejected with the complete list of unmapped bones and a nonzero exit code.
• FR-3 Adding a rig family requires only a new bone-map file + registry entry (no retargeter code change).
• FR-4 iClone export is scriptable: export_take.py applies the standard export block (§4.4) and lands the FBX + metadata sidecar in exchange/incoming-fbx/.
• FR-5 All existing outputs remain reproducible: for each rig family, the unified retargeter's output matches the legacy tool's output within tolerance (§9).
• FR-6 The pipeline runs identically from Windows and macOS; Blender executable is resolved via $BLENDER → per-OS default paths, never hardcoded in commands.
• FR-7 The driver surfaces the next free nd_## id by parsing docs/dances/REGISTRY.md, and refuses output names that collide with registry rows.

Non-functional

• NFR-1 Fail loud: every gate exits nonzero on failure; no silent partial output. A failed stage must not leave a half-written GLB in a stage folder (write to temp, rename on success).
• NFR-2 Provenance: every produced GLB gets a sibling JSON recording source file hash, rig family, tool versions (pipeline git rev, Blender version), settings, and QC verdicts.
• NFR-3 Determinism: same input + same pipeline rev ⇒ byte-stable animation curves (float tolerance per §9).
• NFR-4 Compatibility window: legacy tools remain runnable (moved to tools/legacy/) until the Phase-2 regression matrix passes; deleted in a separate, single-purpose commit.
• NFR-5 3.8 constraint: any code executed inside iClone stays Python 3.8.8 compatible. Pipeline-side code targets the repo's standard Python 3.
• NFR-6 LFS hygiene: no new large-binary patterns; .fbx/.glb already ride LFS.

---

4. Standards (the contract)

4.1 Canonical skeleton

Quaternius 65-bone rig, UE4-style names: root, pelvis, spine_01..03, neck_01, Head (capital H); per side clavicle_, upperarm_, lowerarm_, hand_, thigh_, calf_, foot_, ball_, ball_leaf_; fingers thumb_01..03, index/middle/ring/pinky_01, *_04_leaf.

• Pinned snapshot committed at pipeline/retarget/rigs/target/quaternius_ue.gltf (mesh-stripped copy of ariki-game Superhero_Male_FullBody.gltf).
• verify_target.py compares pinned rig vs the local ariki-game checkout (bone names + rest transforms, tolerance 1e-5) and is run automatically by the driver; drift ⇒ hard fail with instructions to re-pin deliberately.

4.2 Rig-family registry — pipeline/retarget/rigs/*.json

One JSON per family:

{
  "family": "cc_base",
  "version": 1,
  "probe_bones": ["CC_Base_Hip", "CC_Base_L_Upperarm"],
  "char_frame": ["CC_Base_Hip", "CC_Base_Head", "CC_Base_L_Upperarm"],
  "bone_map": { "CC_Base_Hip": "pelvis", "...": "..." },
  "notes": "iClone 8 / CC4 exports; twist bones intentionally unmapped (ride parent)"
}

Registered at launch:

Family	Probe	Source	Seed map
cc_base	CC_Base_Hip + CC_Base_L_Upperarm	iClone/CC exports (primary)	cc_retarget.py BONE_MAP
bstyle	B-hips + B-upperArm.L	Kevin Iglesias packs (HumanM@*, BasicMotions@*)	kevin_retarget.py BONE_MAP
mixamo	mixamorig prefix (strip-variants handled)	Mixamo	mixamo_retarget.py BONE_MAP
quaternius_ue	pelvis + spine_01	Quaternius vendor FBX (UAL)	identity pass-through

Detection order: most-specific probe first; ambiguous or no match ⇒ reject (FR-2). Bone-map edits bump version; provenance sidecars record it (NFR-2).

4.3 Intake contract (validated by validate_fbx.py, pre-Blender)

#	Rule	Check	On violation
IC-1	Registered rig family	probe-bone scan of FBX node names	REJECT, list unmapped bones
IC-2	Baked motion present, >5 frames	anim-stack scan (TempMotion guard retained)	REJECT: "exported WITHOUT motion"
IC-3	T-pose bind preferred	cc_base: sidecar says t_pose: true, else WARN "A-pose — arm QA required"	WARN (blocks nothing; flags QA)
IC-4	In-place motion	not statically checkable; enforced downstream by loop-QC root-drift metric	n/a (QC gate)
IC-5	One take per file	anim-stack count == 1	REJECT
IC-6	Mesh + skin weights present	at least one skinned mesh node	REJECT (bind pose needs weighted mesh)
IC-7	Textures absent or ≤512px	embedded-texture scan	WARN (size hygiene)
IC-8	Metadata sidecar present for iClone takes	<take>.json next to FBX	WARN
IC-9	Output name valid + free	registry parse (FR-7)	REJECT on collision

4.4 iClone standard export block (enforced by export_take.py)

Target Tool Preset Blender · Export Range All · FPS 60 · Preserve Bone Names (CC Base) ON · Use T-Pose As Bind Pose ON · Reset Bone Scale ON · Embed Textures OFF/≤512 · precondition: Animation > Flatten All Motion with Constraint. The script writes the settings actually used into the metadata sidecar (IC-8), so QA can distinguish "contract says" from "export did".

---

5. Target layout

pipeline/
  README.md                  # operator doc; contract §4 + runbook. Source of truth.
  convert.py                 # FR-1 driver
  validate_fbx.py            # IC-1..IC-9 gate (no Blender; raw FBX scan; seconds)
  blender_env.py             # FR-6 Blender resolver ($BLENDER → per-OS defaults)
  retarget/
    retarget.py              # unified retargeter (replaces cc/kevin/mixamo_retarget.py)
    rigs/                    # §4.2 registry
      cc_base.json  bstyle.json  mixamo.json  quaternius_ue.json
      target/quaternius_ue.gltf   target/verify_target.py
  qc/loop_qc.py              # moved unchanged (then FBX-input tolerant later if needed)
  fix/loop_fix.py            # moved
  fix/pingpong_bake.py       # moved
  post/rename_clip.py        # moved
  post/dance_profile.py      # moved
  iclone/
    export_take.py           # FR-4 (bridge client side)
    bridge/                  # moved from tools/iclone_bridge/ (plugin src + installer)
tools/
  legacy/                    # NFR-4: cc_retarget.py, kevin_retarget.py,
                             #   mixamo_retarget.py until Phase-2 matrix passes
  export_boat_prop.py  mocap_retarget.py  bake_run_punch.mjs   # out of scope (N4)

6. Exchange data-flow contract (assets run through exchange/ — decided)

exchange/
  incoming-fbx/     STAGE 0  raw takes + .fbm textures + <take>.json sidecars
  converted-glb/    STAGE 1  retargeted GLB (+ provenance .json)      [convert.py]
  pingpong-glb/     STAGE 2  <pack>_pp.glb loop variants (+ provenance)[convert.py]
  looped-glb/       STAGE 2b optional loop_fix outputs (recreated on demand)
  reports/          STAGE 3  per-take markdown QC report               [convert.py]  ← NEW
  reference-video/  QA reference footage (unchanged)
  workingfiles-iclone/ iClone .iProject staging (unchanged)
  outgoing-props/   reverse direction, game→iClone (unchanged, out of scope)

Rules:

• E-1 Stages only move forward; a stage folder never contains hand-edited files.
• E-2 Ship step remains manual+reviewed: copy STAGE 1/2 GLBs into ariki-game assets/quaternius/dancegen/ with dance JSON, per existing skill workflow.
• E-3 Cleanup only after Jeremy confirms in-game (existing etiquette, unchanged).
• E-4 Everything in exchange/ is LFS-tracked already; sidecars/reports are plain text.

7. Component contracts

Component	CLI (canonical)	Exit codes
validate_fbx.py	python pipeline/validate_fbx.py <fbx> [--name <Clip>]	0 pass / 1 reject / 2 warn-only / 3 error
retarget.py	<blender> --background --python pipeline/retarget/retarget.py -- --src <fbx…> --out <glb> --name <Clip> [--family <f>] [--target <gltf>]	0 ok / 1 reject (unmapped/no-motion) / 2 error
loop_qc.py	unchanged: -- --src <glb>	0 SMOOTH / 1 NOT SMOOTH / 2 error (unchanged)
pingpong_bake.py, loop_fix.py, rename_clip.py	unchanged flags, new paths	unchanged
convert.py	python pipeline/convert.py --src exchange/incoming-fbx/<take>.fbx --id nd_02	0 all gates green / 1 QC not smooth / 2 rejected at intake / 3 error
export_take.py	python pipeline/iclone/export_take.py --take <name> [--project <.iProject>]	0 exported+valid / 1 bridge down / 2 export invalid vs §4.3

Driver behavior (FR-1): temp-write + atomic rename (NFR-1); provenance sidecars (NFR-2); report to exchange/reports/<id>-<date>.md containing IC results, QC metrics for base and _pp, worst-offending bones, and the ship checklist.

7.1 QC architecture (the QC element)

QC is layered, script-based, and mandatory — convert.py will not emit a shippable GLB past a red gate:

Gate	Tool	Runs	Catches
QC-0 Intake	validate_fbx.py (IC-1..9)	before Blender, seconds	wrong rig, missing motion, A-pose flag, name collisions
QC-1 Target integrity	verify_target.py	every convert.py run	pinned skeleton drifting from the game's rig
QC-2 Loop seam	qc/loop_qc.py on STAGE 1 GLB and the _pp variant	every take	pose gap / velocity gap / root drift at the loop seam, worst-offending bones
QC-3 Report + human eyeball	exchange/reports/<id>.md + test-bed viewing	every take	everything numbers can't catch (arm orientation, style, reversal reading as moonwalk)
QC-R Regression	§9 matrix	on retargeter/bone-map changes	math/output drift vs known-good clips

QC-3 stays human by policy: the boat-prop lesson (all asserts green, boat upside down) is why no automated verdict ships a clip without an eyeball in the test bed.

7.2 Model & token usage

The pipeline itself calls no AI models and consumes zero tokens. Every component in §7 is deterministic Python/Blender (required by NFR-3 — determinism and model calls are mutually exclusive here). There are no LLM API calls, no cloud services; the only network-adjacent piece is the TinqsBridge localhost socket to iClone.

Tokens are spent only when a Claude agent operates the pipeline, and only in the session doing the operating:

Consumer	Model	Typical cost	Notes
Scripts (convert.py, retarget, QC, bakes, export_take.py)	none	0 tokens	pure Python/Blender
loop-qc agent (.claude/agents/loop-qc.md)	inherits the session's model (no override pinned)	~5–15k tokens per clip; ~20–40k per small batch	convenience wrapper that runs loop_qc.py and interprets numbers
Claude running a full convert batch interactively	session model	~50–150k tokens per batch today	convert.py + reports are designed to shrink this: one command + one readable report instead of many tool round-trips

Cost-control decisions baked into this plan: QC verdicts live in exchange/reports/ markdown so future sessions read a summary instead of re-running tools; validate_fbx.py fails bad takes in seconds before any expensive step; and the loop-qc agent SHOULD be pinned to a small model (model: haiku) in its frontmatter during P1 — it only runs a script and relays numbers, which doesn't need a frontier model.

8. Phases, gates, deliverables

Phase	Work	Entry gate	Exit gate (must pass)	Est
P0 Baseline review	Run CURRENT tools on Windows (explicit Blender path) on hakadance1.fbx; curve-diff vs shipped war_dance_01.glb; write plans/fbx-pipeline-baseline-2026-07-21.md incl. per-tool audit table	plan approved	Windows/Mac parity verdict documented; blockers listed	0.5d
P1 Move & wire	git mv per §5; blender_env.py; fix kevin hardcoded path; sweep docs/skills (grep checklist: tools/, /Applications/Blender)	P0 pass	P0 run repeats green from new paths; grep sweep clean	0.5d
P2 Unified retargeter	retarget.py + rigs registry; port maps from legacy tools	P1 done	Regression matrix §9 green; legacy tools moved to tools/legacy/	1d
P3 iClone export script	Verify RLPy FBX-export API coverage of §4.4 FIRST; implement export_take.py; else checklist-fallback mode; T-pose QA on first export	P2 done; bridge pings	One take exported from iClone passes validate_fbx.py with t_pose: true; arm QA eyeballed in test bed	1d
P4 Driver + gates	convert.py, validate_fbx.py, reports, registry-id logic (FR-7)	P2 done (parallel w/ P3)	convert.py on a cc_base take: green end-to-end, report emitted	0.5–1d
P5 Docs + pilot	pipeline/README.md; skills point at it; pilot batch: 4× HumanM@Mining*.fbx (bstyle)	P2–P4 done	Pilot converted + QC'd + staged; registry updated	0.5d
P6 Retire legacy	Delete tools/legacy/ in a single-purpose commit	2 real batches through new pipeline w/o legacy fallback	commit merged	0.1d

Total ≈ 4 days. P3 and P4 can run in parallel.

9. Regression & test matrix (P2 exit gate)

Case	Family	Legacy tool	Comparison
aloha1.fbx	cc_base	cc_retarget.py	per-bone quaternion curves, max abs delta ≤ 1e-4; pelvis translation ≤ 1e-4 m; frame count equal
HumanM@Mining01 - Loop Ground.fbx	bstyle	kevin_retarget.py (target path fixed)	same
UAL1_Standard.fbx (one clip)	quaternius_ue	n/a (new capability)	output plays on pinned rig; bone set ⊆ canonical 65
mixamo	mixamo	mixamo_retarget.py	no source FBX on disk — gate on any newly downloaded Mixamo take; if none by P2, mark family "ported, unverified" in README
Negative: unknown rig	—	—	reject w/ unmapped-bone list (FR-2)
Negative: 2-frame TempMotion	cc_base	—	IC-2 reject message
verify_target.py	—	—	pinned rig == ariki-game rig

Float-tolerance rationale: Blender 5.1 (Win) vs 5.1.2 (Mac) may differ in low-order bits; 1e-4 is far below visible thresholds but catches real math changes.

10. Risk register

#	Risk	L×I	Mitigation / trigger
R1	RLPy export API lacks §4.4 settings	M×M	P3 verifies API surface before coding; fallback = scene-prep + human-clicked checklist; contract still enforced by validate_fbx.py
R2	T-pose-bind switch shifts cc_base output vs shipped clips	M×M	Deliberate: new takes only; shipped GLBs untouched; first T-pose take gets arm QA (P3 exit gate)
R3	Blender Win/Mac version drift	L×M	P0 curve-diff gate; pin "Blender 5.1.x" in README; upgrade both sides together
R4	Pinned target rig drifts from game skeleton	L×H	verify_target.py auto-run by driver (hard fail)
R5	Doc/skill path rot after move	H×L	P1 grep checklist is a named exit gate
R6	Unified retargeter regression on edge takes	M×H	§9 matrix + NFR-4 legacy window + P6 requires 2 clean real batches
R7	Registry parse (FR-7) breaks on table format change	M×L	Parser tolerant to column additions; on parse failure driver degrades to WARN, never blocks
R8	Mixamo family unverifiable at P2	M×L	Explicitly labeled "unverified" until a take arrives; not a launch blocker

11. Acceptance criteria (project-level)

1. python pipeline/convert.py --src exchange/incoming-fbx/<take>.fbx --id nd_## on this Windows machine produces STAGE 1+2 GLBs, provenance sidecars, and a QC report — zero Mac involvement, zero manual Blender paths.
2. All rig families in §4.2 convert; §9 matrix green (mixamo allowed "unverified").
3. Unknown-rig and no-motion FBX fail fast with actionable messages.
4. export_take.py (or its documented fallback) yields an intake-valid, T-pose iClone FBX with settings-recording sidecar.
5. Legacy tools deleted (P6) after two clean production batches.
6. Skills/docs reference only pipeline/ paths; pipeline/README.md is the runbook.

12. Open questions (tracked, non-blocking)

• Q1 RLPy FBX-export settings coverage (resolves in P3).
• Q2 Should loop_qc.py gain FBX input for pre-retarget QC? (defer; not needed for launch)
• Q3 Option-2 spike (Godot import-time retarget of FBX) — separate plan if/when wanted.
• Q4 hakadance1_static.glb orphan in dancegen — cleanup during next ariki-game touch.