feat: clothing lane, character sources, and DCC bridges

Bulk import of the working lanes that were living untracked on the PC. Content: - characters/ Lena/male body lanes, bakes, texture work, run logs - clothing/ garment pipeline, configs, gates, contract docs - garments/ MD-authored garment sources (.zprj/.zpac) - UAL-Lib/ Universal Animation Library 2 source (.blend/.fbx/.glb) - tools/ blender_bridge, iclone_bridge, md_bridge, tailor, glm_agent - docs/, plans/, dev/, .agents/plans/ Repo hygiene: - .gitattributes: LFS now covers .blend, .zprj, .zpac, .obj, .npy and the Reallusion .iAvatar/.ccAvatar/.ccRestore containers. Without this the ~3.8 GB in this commit would land as raw blobs. .png/.jpg are left out on purpose — ~250 are already tracked raw and converting them would rewrite every one without shrinking history. - .gitignore: exclude /accurig/ (~1 GB AccuRig program files, redistributable from Reallusion, nothing authored here) and /dev/null/ (git-lfs hook copies dropped by a `>/dev/null` redirect on Windows). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 15:55:43 -07:00
parent 3363209cac
commit 3ba86b2ea8
558 changed files with 68622 additions and 8 deletions
@@ -0,0 +1,304 @@
+# FBX → Quaternius Pipeline — Formal Plan & Specification
+
+**Doc id:** fbx-pipeline-plan-2026-07-21 · **Status:** APPROVED direction, pre-implementation
+**Owner:** Jeremy · **Author:** Claude (5-agent research audit, 2026-07-21)
+**Supersedes:** ad-hoc `tools/*` pipeline docs scattered across skills/READMEs.
+
+---
+
+## 1. Purpose & scope
+
+After the FBX-vs-GLB study (2026-07-21) we adopted **Recommendation 1**: the game keeps
+GLB clips on the shared Quaternius skeleton; the conversion pipeline gets consolidated.
+
+**In scope**
+- S1. Relocate the FBX→Quaternius pipeline into a single folder `pipeline/` at the
+  animation-repo root, runnable end-to-end on this Windows machine and on the Mac.
+- S2. Accept *any* registered FBX source ("regular fbx"), not only iClone exports.
+- S3. House the iClone integration in the pipeline, driven by an export script over the
+  TinqsBridge API (localhost 18800).
+- S4. Define and enforce a standardized bone/rig contract for all incoming FBX.
+- S5. A reviewed, regression-gated migration (baseline before/after parity).
+
+**Out of scope (non-goals)**
+- N1. No change to ariki-game runtime code, ClipCatalog, dance JSON, or shipped GLBs.
+- N2. No skeleton change (CC-skeleton option was evaluated and rejected).
+- N3. No FBX-in-Godot ingestion (Option 2 spike may be scheduled separately).
+- N4. Video-mocap tools (`mocap_retarget.py`) and the reverse prop bridge
+  (`export_boat_prop.py`) stay in `tools/` unchanged.
+
+**Asset flow decision:** all assets continue to run through **`exchange/`** (§6). The
+pipeline reads and writes only `exchange/` stage folders; no new asset roots.
+
+---
+
+## 2. Definitions
+
+| Term | Meaning |
+|---|---|
+| **Canonical skeleton** | The game's Quaternius 65-bone UE4-named rig (§4.1). Sole retarget target. |
+| **Rig family** | A named source-skeleton convention with a registered bone map (§4.2). |
+| **Take** | One FBX file containing one baked animation clip + skinned mesh. |
+| **Pack / Clip** | GLB basename / animation name baked into the GLB; game ref is `pack/Clip`. |
+| **Intake contract** | The validity rules every FBX must satisfy before retargeting (§4.3). |
+| **Stage folder** | A subfolder of `exchange/` representing one pipeline stage (§6). |
+| **Bridge** | TinqsBridge iClone plugin + `iclone_bridge.py` client (verified 2026-07-21). |
+
+---
+
+## 3. Requirements
+
+### Functional
+- **FR-1** One command converts a take: intake validation → retarget → ping-pong bake →
+  loop QC (both variants) → per-take report. Partial failures stop at the failing gate.
+- **FR-2** Rig family is auto-detected; a take on an unregistered rig is rejected with
+  the complete list of unmapped bones and a nonzero exit code.
+- **FR-3** Adding a rig family requires only a new bone-map file + registry entry
+  (no retargeter code change).
+- **FR-4** iClone export is scriptable: `export_take.py` applies the standard export
+  block (§4.4) and lands the FBX + metadata sidecar in `exchange/incoming-fbx/`.
+- **FR-5** All existing outputs remain reproducible: for each rig family, the unified
+  retargeter's output matches the legacy tool's output within tolerance (§9).
+- **FR-6** The pipeline runs identically from Windows and macOS; Blender executable is
+  resolved via `$BLENDER` → per-OS default paths, never hardcoded in commands.
+- **FR-7** The driver surfaces the next free `nd_##` id by parsing
+  `docs/dances/REGISTRY.md`, and refuses output names that collide with registry rows.
+
+### Non-functional
+- **NFR-1 Fail loud:** every gate exits nonzero on failure; no silent partial output.
+  A failed stage must not leave a half-written GLB in a stage folder (write to temp,
+  rename on success).
+- **NFR-2 Provenance:** every produced GLB gets a sibling JSON recording source file
+  hash, rig family, tool versions (pipeline git rev, Blender version), settings, and
+  QC verdicts.
+- **NFR-3 Determinism:** same input + same pipeline rev ⇒ byte-stable animation curves
+  (float tolerance per §9).
+- **NFR-4 Compatibility window:** legacy tools remain runnable (moved to
+  `tools/legacy/`) until the Phase-2 regression matrix passes; deleted in a separate,
+  single-purpose commit.
+- **NFR-5 3.8 constraint:** any code executed *inside* iClone stays Python 3.8.8
+  compatible. Pipeline-side code targets the repo's standard Python 3.
+- **NFR-6 LFS hygiene:** no new large-binary patterns; `.fbx/.glb` already ride LFS.
+
+---
+
+## 4. Standards (the contract)
+
+### 4.1 Canonical skeleton
+Quaternius 65-bone rig, UE4-style names: `root, pelvis, spine_01..03, neck_01, Head`
+(capital H); per side `clavicle_, upperarm_, lowerarm_, hand_, thigh_, calf_, foot_,
+ball_, ball_leaf_`; fingers `thumb_01..03`, `index/middle/ring/pinky_01`, `*_04_leaf`.
+
+- Pinned snapshot committed at `pipeline/retarget/rigs/target/quaternius_ue.gltf`
+  (mesh-stripped copy of ariki-game `Superhero_Male_FullBody.gltf`).
+- `verify_target.py` compares pinned rig vs the local ariki-game checkout (bone names +
+  rest transforms, tolerance 1e-5) and is run automatically by the driver; drift ⇒ hard
+  fail with instructions to re-pin deliberately.
+
+### 4.2 Rig-family registry — `pipeline/retarget/rigs/*.json`
+One JSON per family:
+```json
+{
+  "family": "cc_base",
+  "version": 1,
+  "probe_bones": ["CC_Base_Hip", "CC_Base_L_Upperarm"],
+  "char_frame": ["CC_Base_Hip", "CC_Base_Head", "CC_Base_L_Upperarm"],
+  "bone_map": { "CC_Base_Hip": "pelvis", "...": "..." },
+  "notes": "iClone 8 / CC4 exports; twist bones intentionally unmapped (ride parent)"
+}
+```
+Registered at launch:
+
+| Family | Probe | Source | Seed map |
+|---|---|---|---|
+| `cc_base` | `CC_Base_Hip` + `CC_Base_L_Upperarm` | iClone/CC exports (primary) | `cc_retarget.py` BONE_MAP |
+| `bstyle` | `B-hips` + `B-upperArm.L` | Kevin Iglesias packs (`HumanM@*`, `BasicMotions@*`) | `kevin_retarget.py` BONE_MAP |
+| `mixamo` | `mixamorig` prefix (strip-variants handled) | Mixamo | `mixamo_retarget.py` BONE_MAP |
+| `quaternius_ue` | `pelvis` + `spine_01` | Quaternius vendor FBX (UAL) | identity pass-through |
+
+Detection order: most-specific probe first; ambiguous or no match ⇒ reject (FR-2).
+Bone-map edits bump `version`; provenance sidecars record it (NFR-2).
+
+### 4.3 Intake contract (validated by `validate_fbx.py`, pre-Blender)
+| # | Rule | Check | On violation |
+|---|---|---|---|
+| IC-1 | Registered rig family | probe-bone scan of FBX node names | REJECT, list unmapped bones |
+| IC-2 | Baked motion present, >5 frames | anim-stack scan (`TempMotion` guard retained) | REJECT: "exported WITHOUT motion" |
+| IC-3 | T-pose bind preferred | `cc_base`: sidecar says `t_pose: true`, else WARN "A-pose — arm QA required" | WARN (blocks nothing; flags QA) |
+| IC-4 | In-place motion | not statically checkable; enforced downstream by loop-QC root-drift metric | n/a (QC gate) |
+| IC-5 | One take per file | anim-stack count == 1 | REJECT |
+| IC-6 | Mesh + skin weights present | at least one skinned mesh node | REJECT (bind pose needs weighted mesh) |
+| IC-7 | Textures absent or ≤512px | embedded-texture scan | WARN (size hygiene) |
+| IC-8 | Metadata sidecar present for iClone takes | `<take>.json` next to FBX | WARN |
+| IC-9 | Output name valid + free | registry parse (FR-7) | REJECT on collision |
+
+### 4.4 iClone standard export block (enforced by `export_take.py`)
+Target Tool Preset **Blender** · Export Range **All** · FPS **60** ·
+**Preserve Bone Names (CC Base)** ON · **Use T-Pose As Bind Pose** ON ·
+**Reset Bone Scale** ON · Embed Textures OFF/≤512 ·
+precondition: `Animation > Flatten All Motion with Constraint`.
+The script writes the settings actually used into the metadata sidecar (IC-8), so QA
+can distinguish "contract says" from "export did".
+
+---
+
+## 5. Target layout
+
+```
+pipeline/
+  README.md                  # operator doc; contract §4 + runbook. Source of truth.
+  convert.py                 # FR-1 driver
+  validate_fbx.py            # IC-1..IC-9 gate (no Blender; raw FBX scan; seconds)
+  blender_env.py             # FR-6 Blender resolver ($BLENDER → per-OS defaults)
+  retarget/
+    retarget.py              # unified retargeter (replaces cc/kevin/mixamo_retarget.py)
+    rigs/                    # §4.2 registry
+      cc_base.json  bstyle.json  mixamo.json  quaternius_ue.json
+      target/quaternius_ue.gltf   target/verify_target.py
+  qc/loop_qc.py              # moved unchanged (then FBX-input tolerant later if needed)
+  fix/loop_fix.py            # moved
+  fix/pingpong_bake.py       # moved
+  post/rename_clip.py        # moved
+  post/dance_profile.py      # moved
+  iclone/
+    export_take.py           # FR-4 (bridge client side)
+    bridge/                  # moved from tools/iclone_bridge/ (plugin src + installer)
+tools/
+  legacy/                    # NFR-4: cc_retarget.py, kevin_retarget.py,
+                             #   mixamo_retarget.py until Phase-2 matrix passes
+  export_boat_prop.py  mocap_retarget.py  bake_run_punch.mjs   # out of scope (N4)
+```
+
+## 6. Exchange data-flow contract (assets run through `exchange/` — decided)
+
+```
+exchange/
+  incoming-fbx/     STAGE 0  raw takes + .fbm textures + <take>.json sidecars
+  converted-glb/    STAGE 1  retargeted GLB (+ provenance .json)      [convert.py]
+  pingpong-glb/     STAGE 2  <pack>_pp.glb loop variants (+ provenance)[convert.py]
+  looped-glb/       STAGE 2b optional loop_fix outputs (recreated on demand)
+  reports/          STAGE 3  per-take markdown QC report               [convert.py]  ← NEW
+  reference-video/  QA reference footage (unchanged)
+  workingfiles-iclone/ iClone .iProject staging (unchanged)
+  outgoing-props/   reverse direction, game→iClone (unchanged, out of scope)
+```
+Rules:
+- E-1 Stages only move forward; a stage folder never contains hand-edited files.
+- E-2 Ship step remains manual+reviewed: copy STAGE 1/2 GLBs into ariki-game
+  `assets/quaternius/dancegen/` with dance JSON, per existing skill workflow.
+- E-3 Cleanup only after Jeremy confirms in-game (existing etiquette, unchanged).
+- E-4 Everything in `exchange/` is LFS-tracked already; sidecars/reports are plain text.
+
+## 7. Component contracts
+
+| Component | CLI (canonical) | Exit codes |
+|---|---|---|
+| `validate_fbx.py` | `python pipeline/validate_fbx.py <fbx> [--name <Clip>]` | 0 pass / 1 reject / 2 warn-only / 3 error |
+| `retarget.py` | `<blender> --background --python pipeline/retarget/retarget.py -- --src <fbx…> --out <glb> --name <Clip> [--family <f>] [--target <gltf>]` | 0 ok / 1 reject (unmapped/no-motion) / 2 error |
+| `loop_qc.py` | unchanged: `-- --src <glb>` | 0 SMOOTH / 1 NOT SMOOTH / 2 error (unchanged) |
+| `pingpong_bake.py`, `loop_fix.py`, `rename_clip.py` | unchanged flags, new paths | unchanged |
+| `convert.py` | `python pipeline/convert.py --src exchange/incoming-fbx/<take>.fbx --id nd_02` | 0 all gates green / 1 QC not smooth / 2 rejected at intake / 3 error |
+| `export_take.py` | `python pipeline/iclone/export_take.py --take <name> [--project <.iProject>]` | 0 exported+valid / 1 bridge down / 2 export invalid vs §4.3 |
+
+Driver behavior (FR-1): temp-write + atomic rename (NFR-1); provenance sidecars
+(NFR-2); report to `exchange/reports/<id>-<date>.md` containing IC results, QC metrics
+for base and `_pp`, worst-offending bones, and the ship checklist.
+
+### 7.1 QC architecture (the QC element)
+
+QC is layered, script-based, and mandatory — `convert.py` will not emit a shippable
+GLB past a red gate:
+
+| Gate | Tool | Runs | Catches |
+|---|---|---|---|
+| **QC-0 Intake** | `validate_fbx.py` (IC-1..9) | before Blender, seconds | wrong rig, missing motion, A-pose flag, name collisions |
+| **QC-1 Target integrity** | `verify_target.py` | every `convert.py` run | pinned skeleton drifting from the game's rig |
+| **QC-2 Loop seam** | `qc/loop_qc.py` on STAGE 1 GLB *and* the `_pp` variant | every take | pose gap / velocity gap / root drift at the loop seam, worst-offending bones |
+| **QC-3 Report + human eyeball** | `exchange/reports/<id>.md` + test-bed viewing | every take | everything numbers can't catch (arm orientation, style, reversal reading as moonwalk) |
+| **QC-R Regression** | §9 matrix | on retargeter/bone-map changes | math/output drift vs known-good clips |
+
+QC-3 stays human by policy: the boat-prop lesson (all asserts green, boat upside down)
+is why no automated verdict ships a clip without an eyeball in the test bed.
+
+### 7.2 Model & token usage
+
+**The pipeline itself calls no AI models and consumes zero tokens.** Every component in
+§7 is deterministic Python/Blender (required by NFR-3 — determinism and model calls are
+mutually exclusive here). There are no LLM API calls, no cloud services; the only
+network-adjacent piece is the TinqsBridge localhost socket to iClone.
+
+Tokens are spent only when a **Claude agent operates** the pipeline, and only in the
+session doing the operating:
+
+| Consumer | Model | Typical cost | Notes |
+|---|---|---|---|
+| Scripts (`convert.py`, retarget, QC, bakes, `export_take.py`) | none | 0 tokens | pure Python/Blender |
+| `loop-qc` agent (`.claude/agents/loop-qc.md`) | inherits the session's model (no override pinned) | ~5–15k tokens per clip; ~20–40k per small batch | convenience wrapper that runs `loop_qc.py` and interprets numbers |
+| Claude running a full convert batch interactively | session model | ~50–150k tokens per batch today | `convert.py` + reports are designed to shrink this: one command + one readable report instead of many tool round-trips |
+
+Cost-control decisions baked into this plan: QC verdicts live in `exchange/reports/`
+markdown so future sessions read a summary instead of re-running tools; `validate_fbx.py`
+fails bad takes in seconds before any expensive step; and the `loop-qc` agent SHOULD be
+pinned to a small model (`model: haiku`) in its frontmatter during P1 — it only runs a
+script and relays numbers, which doesn't need a frontier model.
+
+## 8. Phases, gates, deliverables
+
+| Phase | Work | Entry gate | Exit gate (must pass) | Est |
+|---|---|---|---|---|
+| **P0 Baseline review** | Run CURRENT tools on Windows (explicit Blender path) on `hakadance1.fbx`; curve-diff vs shipped `war_dance_01.glb`; write `plans/fbx-pipeline-baseline-2026-07-21.md` incl. per-tool audit table | plan approved | Windows/Mac parity verdict documented; blockers listed | 0.5d |
+| **P1 Move & wire** | `git mv` per §5; `blender_env.py`; fix kevin hardcoded path; sweep docs/skills (grep checklist: `tools/`, `/Applications/Blender`) | P0 pass | P0 run repeats green from new paths; grep sweep clean | 0.5d |
+| **P2 Unified retargeter** | `retarget.py` + rigs registry; port maps from legacy tools | P1 done | Regression matrix §9 green; legacy tools moved to `tools/legacy/` | 1d |
+| **P3 iClone export script** | Verify RLPy FBX-export API coverage of §4.4 FIRST; implement `export_take.py`; else checklist-fallback mode; T-pose QA on first export | P2 done; bridge pings | One take exported from iClone passes `validate_fbx.py` with `t_pose: true`; arm QA eyeballed in test bed | 1d |
+| **P4 Driver + gates** | `convert.py`, `validate_fbx.py`, reports, registry-id logic (FR-7) | P2 done (parallel w/ P3) | `convert.py` on a cc_base take: green end-to-end, report emitted | 0.5–1d |
+| **P5 Docs + pilot** | `pipeline/README.md`; skills point at it; pilot batch: 4× `HumanM@Mining*.fbx` (bstyle) | P2–P4 done | Pilot converted + QC'd + staged; registry updated | 0.5d |
+| **P6 Retire legacy** | Delete `tools/legacy/` in a single-purpose commit | 2 real batches through new pipeline w/o legacy fallback | commit merged | 0.1d |
+
+Total ≈ 4 days. P3 and P4 can run in parallel.
+
+## 9. Regression & test matrix (P2 exit gate)
+
+| Case | Family | Legacy tool | Comparison |
+|---|---|---|---|
+| `aloha1.fbx` | cc_base | `cc_retarget.py` | per-bone quaternion curves, max abs delta ≤ 1e-4; pelvis translation ≤ 1e-4 m; frame count equal |
+| `HumanM@Mining01 - Loop Ground.fbx` | bstyle | `kevin_retarget.py` (target path fixed) | same |
+| `UAL1_Standard.fbx` (one clip) | quaternius_ue | n/a (new capability) | output plays on pinned rig; bone set ⊆ canonical 65 |
+| mixamo | mixamo | `mixamo_retarget.py` | **no source FBX on disk** — gate on any newly downloaded Mixamo take; if none by P2, mark family "ported, unverified" in README |
+| Negative: unknown rig | — | — | reject w/ unmapped-bone list (FR-2) |
+| Negative: 2-frame TempMotion | cc_base | — | IC-2 reject message |
+| `verify_target.py` | — | — | pinned rig == ariki-game rig |
+
+Float-tolerance rationale: Blender 5.1 (Win) vs 5.1.2 (Mac) may differ in low-order
+bits; 1e-4 is far below visible thresholds but catches real math changes.
+
+## 10. Risk register
+
+| # | Risk | L×I | Mitigation / trigger |
+|---|---|---|---|
+| R1 | RLPy export API lacks §4.4 settings | M×M | P3 verifies API surface before coding; fallback = scene-prep + human-clicked checklist; contract still enforced by `validate_fbx.py` |
+| R2 | T-pose-bind switch shifts cc_base output vs shipped clips | M×M | Deliberate: new takes only; shipped GLBs untouched; first T-pose take gets arm QA (P3 exit gate) |
+| R3 | Blender Win/Mac version drift | L×M | P0 curve-diff gate; pin "Blender 5.1.x" in README; upgrade both sides together |
+| R4 | Pinned target rig drifts from game skeleton | L×H | `verify_target.py` auto-run by driver (hard fail) |
+| R5 | Doc/skill path rot after move | H×L | P1 grep checklist is a named exit gate |
+| R6 | Unified retargeter regression on edge takes | M×H | §9 matrix + NFR-4 legacy window + P6 requires 2 clean real batches |
+| R7 | Registry parse (FR-7) breaks on table format change | M×L | Parser tolerant to column additions; on parse failure driver degrades to WARN, never blocks |
+| R8 | Mixamo family unverifiable at P2 | M×L | Explicitly labeled "unverified" until a take arrives; not a launch blocker |
+
+## 11. Acceptance criteria (project-level)
+
+1. `python pipeline/convert.py --src exchange/incoming-fbx/<take>.fbx --id nd_##` on
+   this Windows machine produces STAGE 1+2 GLBs, provenance sidecars, and a QC report —
+   zero Mac involvement, zero manual Blender paths.
+2. All rig families in §4.2 convert; §9 matrix green (mixamo allowed "unverified").
+3. Unknown-rig and no-motion FBX fail fast with actionable messages.
+4. `export_take.py` (or its documented fallback) yields an intake-valid, T-pose iClone
+   FBX with settings-recording sidecar.
+5. Legacy tools deleted (P6) after two clean production batches.
+6. Skills/docs reference only `pipeline/` paths; `pipeline/README.md` is the runbook.
+
+## 12. Open questions (tracked, non-blocking)
+
+- Q1 RLPy FBX-export settings coverage (resolves in P3).
+- Q2 Should `loop_qc.py` gain FBX input for pre-retarget QC? (defer; not needed for launch)
+- Q3 Option-2 spike (Godot import-time retarget of FBX) — separate plan if/when wanted.
+- Q4 `hakadance1_static.glb` orphan in dancegen — cleanup during next ariki-game touch.