animation/plans/fbx-pipeline-plan-2026-07-21.md

# FBX → Quaternius Pipeline — Formal Plan & Specification

**Doc id:** fbx-pipeline-plan-2026-07-21 · **Status:** APPROVED direction, pre-implementation
**Owner:** Jeremy · **Author:** Claude (5-agent research audit, 2026-07-21)
**Supersedes:** ad-hoc `tools/*` pipeline docs scattered across skills/READMEs.

---

## 1. Purpose & scope

After the FBX-vs-GLB study (2026-07-21) we adopted **Recommendation 1**: the game keeps
GLB clips on the shared Quaternius skeleton; the conversion pipeline gets consolidated.

**In scope**
- S1. Relocate the FBX→Quaternius pipeline into a single folder `pipeline/` at the
  animation-repo root, runnable end-to-end on this Windows machine and on the Mac.
- S2. Accept *any* registered FBX source ("regular fbx"), not only iClone exports.
- S3. House the iClone integration in the pipeline, driven by an export script over the
  TinqsBridge API (localhost 18800).
- S4. Define and enforce a standardized bone/rig contract for all incoming FBX.
- S5. A reviewed, regression-gated migration (baseline before/after parity).

**Out of scope (non-goals)**
- N1. No change to ariki-game runtime code, ClipCatalog, dance JSON, or shipped GLBs.
- N2. No skeleton change (CC-skeleton option was evaluated and rejected).
- N3. No FBX-in-Godot ingestion (Option 2 spike may be scheduled separately).
- N4. Video-mocap tools (`mocap_retarget.py`) and the reverse prop bridge
  (`export_boat_prop.py`) stay in `tools/` unchanged.

**Asset flow decision:** all assets continue to run through **`exchange/`** (§6). The
pipeline reads and writes only `exchange/` stage folders; no new asset roots.

---

## 2. Definitions

| Term | Meaning |
|---|---|
| **Canonical skeleton** | The game's Quaternius 65-bone UE4-named rig (§4.1). Sole retarget target. |
| **Rig family** | A named source-skeleton convention with a registered bone map (§4.2). |
| **Take** | One FBX file containing one baked animation clip + skinned mesh. |
| **Pack / Clip** | GLB basename / animation name baked into the GLB; game ref is `pack/Clip`. |
| **Intake contract** | The validity rules every FBX must satisfy before retargeting (§4.3). |
| **Stage folder** | A subfolder of `exchange/` representing one pipeline stage (§6). |
| **Bridge** | TinqsBridge iClone plugin + `iclone_bridge.py` client (verified 2026-07-21). |

---

## 3. Requirements

### Functional
- **FR-1** One command converts a take: intake validation → retarget → ping-pong bake →
  loop QC (both variants) → per-take report. Partial failures stop at the failing gate.
- **FR-2** Rig family is auto-detected; a take on an unregistered rig is rejected with
  the complete list of unmapped bones and a nonzero exit code.
- **FR-3** Adding a rig family requires only a new bone-map file + registry entry
  (no retargeter code change).
- **FR-4** iClone export is scriptable: `export_take.py` applies the standard export
  block (§4.4) and lands the FBX + metadata sidecar in `exchange/incoming-fbx/`.
- **FR-5** All existing outputs remain reproducible: for each rig family, the unified
  retargeter's output matches the legacy tool's output within tolerance (§9).
- **FR-6** The pipeline runs identically from Windows and macOS; Blender executable is
  resolved via `$BLENDER` → per-OS default paths, never hardcoded in commands.
- **FR-7** The driver surfaces the next free `nd_##` id by parsing
  `docs/dances/REGISTRY.md`, and refuses output names that collide with registry rows.

### Non-functional
- **NFR-1 Fail loud:** every gate exits nonzero on failure; no silent partial output.
  A failed stage must not leave a half-written GLB in a stage folder (write to temp,
  rename on success).
- **NFR-2 Provenance:** every produced GLB gets a sibling JSON recording source file
  hash, rig family, tool versions (pipeline git rev, Blender version), settings, and
  QC verdicts.
- **NFR-3 Determinism:** same input + same pipeline rev ⇒ byte-stable animation curves
  (float tolerance per §9).
- **NFR-4 Compatibility window:** legacy tools remain runnable (moved to
  `tools/legacy/`) until the Phase-2 regression matrix passes; deleted in a separate,
  single-purpose commit.
- **NFR-5 3.8 constraint:** any code executed *inside* iClone stays Python 3.8.8
  compatible. Pipeline-side code targets the repo's standard Python 3.
- **NFR-6 LFS hygiene:** no new large-binary patterns; `.fbx/.glb` already ride LFS.

---

## 4. Standards (the contract)

### 4.1 Canonical skeleton
Quaternius 65-bone rig, UE4-style names: `root, pelvis, spine_01..03, neck_01, Head`
(capital H); per side `clavicle_, upperarm_, lowerarm_, hand_, thigh_, calf_, foot_,
ball_, ball_leaf_`; fingers `thumb_01..03`, `index/middle/ring/pinky_01`, `*_04_leaf`.

- Pinned snapshot committed at `pipeline/retarget/rigs/target/quaternius_ue.gltf`
  (mesh-stripped copy of ariki-game `Superhero_Male_FullBody.gltf`).
- `verify_target.py` compares pinned rig vs the local ariki-game checkout (bone names +
  rest transforms, tolerance 1e-5) and is run automatically by the driver; drift ⇒ hard
  fail with instructions to re-pin deliberately.

### 4.2 Rig-family registry — `pipeline/retarget/rigs/*.json`
One JSON per family:
```json
{
  "family": "cc_base",
  "version": 1,
  "probe_bones": ["CC_Base_Hip", "CC_Base_L_Upperarm"],
  "char_frame": ["CC_Base_Hip", "CC_Base_Head", "CC_Base_L_Upperarm"],
  "bone_map": { "CC_Base_Hip": "pelvis", "...": "..." },
  "notes": "iClone 8 / CC4 exports; twist bones intentionally unmapped (ride parent)"
}
```
Registered at launch:

| Family | Probe | Source | Seed map |
|---|---|---|---|
| `cc_base` | `CC_Base_Hip` + `CC_Base_L_Upperarm` | iClone/CC exports (primary) | `cc_retarget.py` BONE_MAP |
| `bstyle` | `B-hips` + `B-upperArm.L` | Kevin Iglesias packs (`HumanM@*`, `BasicMotions@*`) | `kevin_retarget.py` BONE_MAP |
| `mixamo` | `mixamorig` prefix (strip-variants handled) | Mixamo | `mixamo_retarget.py` BONE_MAP |
| `quaternius_ue` | `pelvis` + `spine_01` | Quaternius vendor FBX (UAL) | identity pass-through |

Detection order: most-specific probe first; ambiguous or no match ⇒ reject (FR-2).
Bone-map edits bump `version`; provenance sidecars record it (NFR-2).

### 4.3 Intake contract (validated by `validate_fbx.py`, pre-Blender)
| # | Rule | Check | On violation |
|---|---|---|---|
| IC-1 | Registered rig family | probe-bone scan of FBX node names | REJECT, list unmapped bones |
| IC-2 | Baked motion present, >5 frames | anim-stack scan (`TempMotion` guard retained) | REJECT: "exported WITHOUT motion" |
| IC-3 | T-pose bind preferred | `cc_base`: sidecar says `t_pose: true`, else WARN "A-pose — arm QA required" | WARN (blocks nothing; flags QA) |
| IC-4 | In-place motion | not statically checkable; enforced downstream by loop-QC root-drift metric | n/a (QC gate) |
| IC-5 | One take per file | anim-stack count == 1 | REJECT |
| IC-6 | Mesh + skin weights present | at least one skinned mesh node | REJECT (bind pose needs weighted mesh) |
| IC-7 | Textures absent or ≤512px | embedded-texture scan | WARN (size hygiene) |
| IC-8 | Metadata sidecar present for iClone takes | `<take>.json` next to FBX | WARN |
| IC-9 | Output name valid + free | registry parse (FR-7) | REJECT on collision |

### 4.4 iClone standard export block (enforced by `export_take.py`)
Target Tool Preset **Blender** · Export Range **All** · FPS **60** ·
**Preserve Bone Names (CC Base)** ON · **Use T-Pose As Bind Pose** ON ·
**Reset Bone Scale** ON · Embed Textures OFF/≤512 ·
precondition: `Animation > Flatten All Motion with Constraint`.
The script writes the settings actually used into the metadata sidecar (IC-8), so QA
can distinguish "contract says" from "export did".

---

## 5. Target layout

```
pipeline/
  README.md                  # operator doc; contract §4 + runbook. Source of truth.
  convert.py                 # FR-1 driver
  validate_fbx.py            # IC-1..IC-9 gate (no Blender; raw FBX scan; seconds)
  blender_env.py             # FR-6 Blender resolver ($BLENDER → per-OS defaults)
  retarget/
    retarget.py              # unified retargeter (replaces cc/kevin/mixamo_retarget.py)
    rigs/                    # §4.2 registry
      cc_base.json  bstyle.json  mixamo.json  quaternius_ue.json
      target/quaternius_ue.gltf   target/verify_target.py
  qc/loop_qc.py              # moved unchanged (then FBX-input tolerant later if needed)
  fix/loop_fix.py            # moved
  fix/pingpong_bake.py       # moved
  post/rename_clip.py        # moved
  post/dance_profile.py      # moved
  iclone/
    export_take.py           # FR-4 (bridge client side)
    bridge/                  # moved from tools/iclone_bridge/ (plugin src + installer)
tools/
  legacy/                    # NFR-4: cc_retarget.py, kevin_retarget.py,
                             #   mixamo_retarget.py until Phase-2 matrix passes
  export_boat_prop.py  mocap_retarget.py  bake_run_punch.mjs   # out of scope (N4)
```

## 6. Exchange data-flow contract (assets run through `exchange/` — decided)

```
exchange/
  incoming-fbx/     STAGE 0  raw takes + .fbm textures + <take>.json sidecars
  converted-glb/    STAGE 1  retargeted GLB (+ provenance .json)      [convert.py]
  pingpong-glb/     STAGE 2  <pack>_pp.glb loop variants (+ provenance)[convert.py]
  looped-glb/       STAGE 2b optional loop_fix outputs (recreated on demand)
  reports/          STAGE 3  per-take markdown QC report               [convert.py]  ← NEW
  reference-video/  QA reference footage (unchanged)
  workingfiles-iclone/ iClone .iProject staging (unchanged)
  outgoing-props/   reverse direction, game→iClone (unchanged, out of scope)
```
Rules:
- E-1 Stages only move forward; a stage folder never contains hand-edited files.
- E-2 Ship step remains manual+reviewed: copy STAGE 1/2 GLBs into ariki-game
  `assets/quaternius/dancegen/` with dance JSON, per existing skill workflow.
- E-3 Cleanup only after Jeremy confirms in-game (existing etiquette, unchanged).
- E-4 Everything in `exchange/` is LFS-tracked already; sidecars/reports are plain text.

## 7. Component contracts

| Component | CLI (canonical) | Exit codes |
|---|---|---|
| `validate_fbx.py` | `python pipeline/validate_fbx.py <fbx> [--name <Clip>]` | 0 pass / 1 reject / 2 warn-only / 3 error |
| `retarget.py` | `<blender> --background --python pipeline/retarget/retarget.py -- --src <fbx…> --out <glb> --name <Clip> [--family <f>] [--target <gltf>]` | 0 ok / 1 reject (unmapped/no-motion) / 2 error |
| `loop_qc.py` | unchanged: `-- --src <glb>` | 0 SMOOTH / 1 NOT SMOOTH / 2 error (unchanged) |
| `pingpong_bake.py`, `loop_fix.py`, `rename_clip.py` | unchanged flags, new paths | unchanged |
| `convert.py` | `python pipeline/convert.py --src exchange/incoming-fbx/<take>.fbx --id nd_02` | 0 all gates green / 1 QC not smooth / 2 rejected at intake / 3 error |
| `export_take.py` | `python pipeline/iclone/export_take.py --take <name> [--project <.iProject>]` | 0 exported+valid / 1 bridge down / 2 export invalid vs §4.3 |

Driver behavior (FR-1): temp-write + atomic rename (NFR-1); provenance sidecars
(NFR-2); report to `exchange/reports/<id>-<date>.md` containing IC results, QC metrics
for base and `_pp`, worst-offending bones, and the ship checklist.

### 7.1 QC architecture (the QC element)

QC is layered, script-based, and mandatory — `convert.py` will not emit a shippable
GLB past a red gate:

| Gate | Tool | Runs | Catches |
|---|---|---|---|
| **QC-0 Intake** | `validate_fbx.py` (IC-1..9) | before Blender, seconds | wrong rig, missing motion, A-pose flag, name collisions |
| **QC-1 Target integrity** | `verify_target.py` | every `convert.py` run | pinned skeleton drifting from the game's rig |
| **QC-2 Loop seam** | `qc/loop_qc.py` on STAGE 1 GLB *and* the `_pp` variant | every take | pose gap / velocity gap / root drift at the loop seam, worst-offending bones |
| **QC-3 Report + human eyeball** | `exchange/reports/<id>.md` + test-bed viewing | every take | everything numbers can't catch (arm orientation, style, reversal reading as moonwalk) |
| **QC-R Regression** | §9 matrix | on retargeter/bone-map changes | math/output drift vs known-good clips |

QC-3 stays human by policy: the boat-prop lesson (all asserts green, boat upside down)
is why no automated verdict ships a clip without an eyeball in the test bed.

### 7.2 Model & token usage

**The pipeline itself calls no AI models and consumes zero tokens.** Every component in
§7 is deterministic Python/Blender (required by NFR-3 — determinism and model calls are
mutually exclusive here). There are no LLM API calls, no cloud services; the only
network-adjacent piece is the TinqsBridge localhost socket to iClone.

Tokens are spent only when a **Claude agent operates** the pipeline, and only in the
session doing the operating:

| Consumer | Model | Typical cost | Notes |
|---|---|---|---|
| Scripts (`convert.py`, retarget, QC, bakes, `export_take.py`) | none | 0 tokens | pure Python/Blender |
| `loop-qc` agent (`.claude/agents/loop-qc.md`) | inherits the session's model (no override pinned) | ~5–15k tokens per clip; ~20–40k per small batch | convenience wrapper that runs `loop_qc.py` and interprets numbers |
| Claude running a full convert batch interactively | session model | ~50–150k tokens per batch today | `convert.py` + reports are designed to shrink this: one command + one readable report instead of many tool round-trips |

Cost-control decisions baked into this plan: QC verdicts live in `exchange/reports/`
markdown so future sessions read a summary instead of re-running tools; `validate_fbx.py`
fails bad takes in seconds before any expensive step; and the `loop-qc` agent SHOULD be
pinned to a small model (`model: haiku`) in its frontmatter during P1 — it only runs a
script and relays numbers, which doesn't need a frontier model.

## 8. Phases, gates, deliverables

| Phase | Work | Entry gate | Exit gate (must pass) | Est |
|---|---|---|---|---|
| **P0 Baseline review** | Run CURRENT tools on Windows (explicit Blender path) on `hakadance1.fbx`; curve-diff vs shipped `war_dance_01.glb`; write `plans/fbx-pipeline-baseline-2026-07-21.md` incl. per-tool audit table | plan approved | Windows/Mac parity verdict documented; blockers listed | 0.5d |
| **P1 Move & wire** | `git mv` per §5; `blender_env.py`; fix kevin hardcoded path; sweep docs/skills (grep checklist: `tools/`, `/Applications/Blender`) | P0 pass | P0 run repeats green from new paths; grep sweep clean | 0.5d |
| **P2 Unified retargeter** | `retarget.py` + rigs registry; port maps from legacy tools | P1 done | Regression matrix §9 green; legacy tools moved to `tools/legacy/` | 1d |
| **P3 iClone export script** | Verify RLPy FBX-export API coverage of §4.4 FIRST; implement `export_take.py`; else checklist-fallback mode; T-pose QA on first export | P2 done; bridge pings | One take exported from iClone passes `validate_fbx.py` with `t_pose: true`; arm QA eyeballed in test bed | 1d |
| **P4 Driver + gates** | `convert.py`, `validate_fbx.py`, reports, registry-id logic (FR-7) | P2 done (parallel w/ P3) | `convert.py` on a cc_base take: green end-to-end, report emitted | 0.5–1d |
| **P5 Docs + pilot** | `pipeline/README.md`; skills point at it; pilot batch: 4× `HumanM@Mining*.fbx` (bstyle) | P2–P4 done | Pilot converted + QC'd + staged; registry updated | 0.5d |
| **P6 Retire legacy** | Delete `tools/legacy/` in a single-purpose commit | 2 real batches through new pipeline w/o legacy fallback | commit merged | 0.1d |

Total ≈ 4 days. P3 and P4 can run in parallel.

## 9. Regression & test matrix (P2 exit gate)

| Case | Family | Legacy tool | Comparison |
|---|---|---|---|
| `aloha1.fbx` | cc_base | `cc_retarget.py` | per-bone quaternion curves, max abs delta ≤ 1e-4; pelvis translation ≤ 1e-4 m; frame count equal |
| `HumanM@Mining01 - Loop Ground.fbx` | bstyle | `kevin_retarget.py` (target path fixed) | same |
| `UAL1_Standard.fbx` (one clip) | quaternius_ue | n/a (new capability) | output plays on pinned rig; bone set ⊆ canonical 65 |
| mixamo | mixamo | `mixamo_retarget.py` | **no source FBX on disk** — gate on any newly downloaded Mixamo take; if none by P2, mark family "ported, unverified" in README |
| Negative: unknown rig | — | — | reject w/ unmapped-bone list (FR-2) |
| Negative: 2-frame TempMotion | cc_base | — | IC-2 reject message |
| `verify_target.py` | — | — | pinned rig == ariki-game rig |

Float-tolerance rationale: Blender 5.1 (Win) vs 5.1.2 (Mac) may differ in low-order
bits; 1e-4 is far below visible thresholds but catches real math changes.

## 10. Risk register

| # | Risk | L×I | Mitigation / trigger |
|---|---|---|---|
| R1 | RLPy export API lacks §4.4 settings | M×M | P3 verifies API surface before coding; fallback = scene-prep + human-clicked checklist; contract still enforced by `validate_fbx.py` |
| R2 | T-pose-bind switch shifts cc_base output vs shipped clips | M×M | Deliberate: new takes only; shipped GLBs untouched; first T-pose take gets arm QA (P3 exit gate) |
| R3 | Blender Win/Mac version drift | L×M | P0 curve-diff gate; pin "Blender 5.1.x" in README; upgrade both sides together |
| R4 | Pinned target rig drifts from game skeleton | L×H | `verify_target.py` auto-run by driver (hard fail) |
| R5 | Doc/skill path rot after move | H×L | P1 grep checklist is a named exit gate |
| R6 | Unified retargeter regression on edge takes | M×H | §9 matrix + NFR-4 legacy window + P6 requires 2 clean real batches |
| R7 | Registry parse (FR-7) breaks on table format change | M×L | Parser tolerant to column additions; on parse failure driver degrades to WARN, never blocks |
| R8 | Mixamo family unverifiable at P2 | M×L | Explicitly labeled "unverified" until a take arrives; not a launch blocker |

## 11. Acceptance criteria (project-level)

1. `python pipeline/convert.py --src exchange/incoming-fbx/<take>.fbx --id nd_##` on
   this Windows machine produces STAGE 1+2 GLBs, provenance sidecars, and a QC report —
   zero Mac involvement, zero manual Blender paths.
2. All rig families in §4.2 convert; §9 matrix green (mixamo allowed "unverified").
3. Unknown-rig and no-motion FBX fail fast with actionable messages.
4. `export_take.py` (or its documented fallback) yields an intake-valid, T-pose iClone
   FBX with settings-recording sidecar.
5. Legacy tools deleted (P6) after two clean production batches.
6. Skills/docs reference only `pipeline/` paths; `pipeline/README.md` is the runbook.

## 12. Open questions (tracked, non-blocking)

- Q1 RLPy FBX-export settings coverage (resolves in P3).
- Q2 Should `loop_qc.py` gain FBX input for pre-retarget QC? (defer; not needed for launch)
- Q3 Option-2 spike (Godot import-time retarget of FBX) — separate plan if/when wanted.
- Q4 `hakadance1_static.glb` orphan in dancegen — cleanup during next ariki-game touch.