From 4c3870336f08bffa5d63fbacb8ee025a38f406f2 Mon Sep 17 00:00:00 2001 From: Jeremy Kashkett Date: Tue, 18 Aug 2026 11:29:54 -0700 Subject: [PATCH] docs(plans): archive the lena handmorph track handover (2026-08-18) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The plan this track executed against. Its weld diagnosis was real but turned out to be 1 of 6 defects, and its stretch-only acceptance bars were satisfiable by a no-op morph — see hand-shapes/README.md and b57b0f2 for what actually happened. Co-Authored-By: Claude Opus 5 --- .../plans/lena-handmorph-track-2026-08-18.md | 220 ++++++++++++++++++ 1 file changed, 220 insertions(+) create mode 100644 .agents/plans/lena-handmorph-track-2026-08-18.md diff --git a/.agents/plans/lena-handmorph-track-2026-08-18.md b/.agents/plans/lena-handmorph-track-2026-08-18.md new file mode 100644 index 0000000..ae47972 --- /dev/null +++ b/.agents/plans/lena-handmorph-track-2026-08-18.md @@ -0,0 +1,220 @@ +# Handover: Lena hand MORPH track (blend-shape lane) — 2026-08-18 + +You own the **morph lane**. A parallel agent owns the **bone lane** (finger weights + +`HandPoseLayer`). Read the scope fence before touching anything — the two lanes share a +git branch and one serialized test bed. + +## Goal + +Lena can hold **fist** and **grip** in-game, on the **shipped** body. Flat already works +via the bone lane, so flat is not your problem — the two poses that never made it are +fist and grip, and this lane is the one that can carry them, because it does not depend +on finger weights at all. + +The blocking bug is known and localized: **coincident (unwelded) duplicate vertices in +the scan mesh are invisible to the solver's convergence gate**, so the right hand's +fist/grip still tear. Fix that, re-solve, get a bed visual, then make the ship-body call. + +## Why this lane exists (do not re-litigate) + +Lena's mesh is a Tripo scan with ~2.6k inter-digit bridge edges (web remnants). Bone +weights only choose *which bone drags a shared vertex* — when adjacent digits curl apart +in a fist, those bridges MUST tear. Five weight-rebake iterations (exp01–exp05) never +converged; the verdict from that lane is that weights alone cannot clear the bar on this +mesh. A morph target IS the final vertex positions, so tearing is impossible by +construction and the residual web stretch becomes one smoothable, *converging* geometric +problem. Morphs also work on the shipped **rigid-mitt** bodies (zero finger weights) and +on Mako, with no rest-space compatibility hacks and no body denylist. + +## What already works (verified, not aspirational) + +- **Runtime driver committed** in ariki-game: `src/Animation/HandMorphLayer.cs` + (commit `40b995e21`). Discovers `hand__` blend shapes on the body's meshes, + per-hand 0..1 blend, `CyclePose()`, logs once and goes inert on bodies without shapes. + `PoseOrder = { flat, relaxed, fist, grip }`. +- **Shape preservation**: `src/Character/BodyMeshShaper.Deform()` carries the shapes + through muscle/fat rebuilds. +- **Solver + verifier** (animation repo, **untracked** — committing them is your job): + `tools/handshape_solve.py` (723 lines), `tools/handshape_verify.py` (109 lines), + plus `hand-shapes/README.md`. +- **In-engine verification happened once** (2026-08-17): fist and grip read as real + fists/grips mid-dance at hand-cam range on the demo body, no fins/shards/spikes + (agent-API screenshots 181806/181808). +- **Two beds can drive it**: `dance_test_bed` (hotkey **K** cycles poses, **J** hand cam, + **1** frames the team) and the newer `anim_hand_test_bed` (commit `d1e6fb0a5`, Lena + + Mako side by side as game-model rigs, button panels for both hand lanes + hand-follow + cams). Prefer `anim_hand_test_bed` — it is the exact colonist build path the game uses. + +## Solver pipeline (so you can navigate 723 lines fast) + +`tools/handshape_solve.py`, all offline pure numpy over **raw GLB bytes** — Blender is +only the interpreter host (numpy), no `bpy`, no scene import. The +"Blender-importer-draws-false-shards" trap therefore does not apply to the solver, but +see the render warning below. + +1. parse GLB, rest skeleton (node globals × IBM = skinning space) — `read_glb`, `Skeleton` +2. hand ROI: verts within 1.8 cm of finger/hand bone segments, grown 3 edge rings — `build_roi:188` +3. ROI graph (CSR adjacency + unique edge list) — `roi_graph:208` +4. per-bone geodesic fields: multi-source Dijkstra over the ROI subgraph — `dijkstra_multi:226` +5. weights: gaussian kernels on geodesic distance, top-4, renormalized, smoothed — `solve_weights:250` +6. pose: parametric curl/spread/thumb-opposition per joint, pivoted at each joint head, + LBS with the solver's OWN weights (the GLB's rigid-mitt weights are irrelevant and + that is fine) — `pose_globals:336`, `lbs:398`; pose parameters live in `DEFAULT_PARAMS` +7. relax: stretch-gated Laplacian diffusion of the DELTA field until the bars pass — `relax:448` +8. gates: `stretch_stats:434` → `handmorph_report.json` + per-pose OBJ dumps +9. emit: splice morph accessors into the GLB (deltas added to base — exactly Godot's + `w = target + base`) with `extras.targetNames = hand__` — `emit_morph_glb:552`. + POSITION **and** NORMAL deltas are emitted (position-only morphs leave lighting on the + rest shape and read as "torn texture"). + +`--selftest-bump` splices one synthetic 3 cm palm bump with no solve — use it to prove the +import + drive path end-to-end on any new body before trusting solver output. + +## THE BUG — weld before solve + +`stretch_stats` (line 434) and the `relax` gate (line 448, `s[lr < 0.001] = 1.0`) both +apply a **1 mm rest-length floor**. The reasoning was sound for decimation slivers, but +this mesh is unwelded chart soup: it carries *coincident duplicate* verts whose rest edge +length is ~0. Those edges are excluded from the stats AND from the convergence gate, so +**the relaxation never even tries to fix them**. Measured 2026-08-18: a population of +sub-mm edges stretches to **4–20 cm** in fist/grip on BOTH hands (~25–60 per hand over 5×) +— hairline needles, sub-pixel in the screenshots that "passed", but really there. + +`grep -niE "weld|coincid|dedup" tools/handshape_solve.py` returns **zero hits** — no weld +pass exists. + +**The fix, and the shape it has to take.** Weld at the *graph* level, not by rewriting the +mesh: build a representative map over coincident positions (hash/round positions to ~1e-6, +or a KD-tree at ~10 µm), solve on the welded ROI, then **scatter each welded vertex's delta +back to every duplicate in its weld group** before `full[idx] = relaxed`. Morph deltas must +stay indexed by *original* vertex id (the GLB's own attribute order) or the splice breaks. +Because all members of a group then receive an identical delta, coincident edges keep +length exactly and the entire failure class dies by construction rather than by tuning. + +Insert the weld between `build_roi` (called in `main`, ~line 228) and `roi_graph`, and make +sure the Dijkstra/adjacency also runs on the welded graph — otherwise geodesic distances +still leak across seams and the weight field stays fragmented. + +**Hypothesis worth testing while you are in there** (state it as a hypothesis, do not +assume): the right hand's tracked ≥1 mm failures may be the same disease. If duplicates +split the delta field across a chart seam, welding should collapse a good share of those +too. Measure before and after; report both. + +## Current numbers — the baseline you must beat + +Body: `Ariki_Female_QuatSkin_LowPoly_40.glb` (restored `lena_leafbikini_base_v01` mesh, so +no yellow-material defect). ROI = **5,548 verts**. From +`characters/work/lena_leafbikini/handmorph/handmorph_report.json` (2026-08-18 08:49), +post-relax, ≥1 mm edges only: + +| shape | max | p99.9 | n>2× | n>5× | iters | +|---|---|---|---|---|---| +| flat_l | 1.94 | 1.60 | 0 | 0 | 11 | +| relaxed_l | 3.60 | 1.49 | 3 | 0 | 60 | +| fist_l | 4.23 | 1.99 | 15 | 0 | 60 | +| grip_l | 4.38 | 1.97 | 13 | 0 | 60 | +| flat_r | 2.41 | 1.59 | 1 | 0 | 8 | +| relaxed_r | 4.08 | 3.22 | 81 | 0 | 60 | +| **fist_r** | **29.78** | **18.94** | 214 | **37** | 60 (hit cap) | +| **grip_r** | **20.38** | **12.10** | 372 | **35** | 60 (hit cap) | + +Left hand is fully clean — *better* than the earlier LENA_rig_v1 body. Right fist/grip are +the failures, and both burn all 60 relax iterations without converging. + +Fingertip travel is healthy and should stay so: fist_l tips 9.0–10.3 cm (thumb 15.3), +relaxed_l 1.7–2.0 cm (thumb 6.1). + +**Gate bars** (per shape, ≥1 mm edges — and after your fix, the sliver population too): +edge stretch p99.9 ≤ 1.6×, **zero** edges > 5×; fingertip travel fist ≥ 2.5 cm/finger, +grip ~2 cm, relaxed 0.5–2 cm; cross-hand independence 0 cm on the other hand's verts. + +## How to run + +Solve (Blender is just the numpy host): + +``` +"C:/Program Files/Blender Foundation/Blender 5.1/blender.exe" --background \ + --factory-startup --python tools/handshape_solve.py -- \ + --body C:/Users/Jeremy/tinqs/ariki-game/assets/quaternius/derived-bodies/Ariki_Female_QuatSkin_LowPoly_40.glb \ + --poses flat,relaxed,fist,grip \ + --out C:/Users/Jeremy/tinqs/ariki-game/scratchpad/lowpoly40_handmorph.glb \ + --workdir characters/work/lena_leafbikini/handmorph/ +``` + +Verify in-engine — **the only honest gate**: + +``` +HANDMORPH_BODY_F=res://scratchpad/lowpoly40_handmorph.glb \ + SCENE=anim_hand_test_bed MOCK_ONLY=1 AGENT_OWNED=1 WAIT=1 bash tools/game.sh spawn +``` + +(`HANDMORPH_BODY` applies to both rigs, `_F`/`_M` per rig — see `DancerRig.cs:40`. +In `dance_test_bed` instead: **1** frames the team, **J** hand cam, **K** cycles +None→flat→relaxed→fist→grip→None.) + +## Two traps that have already cost time + +- **Do not judge the solver's OBJ dumps by clay render.** The un-welded chart soup renders + as black-gap confetti even at REST (flipped per-chart normals). It is dishonest in both + directions. Judge morphs **in-engine only**. The numeric report + the bed are the gates. +- **`handshape_verify.py` reports `own-delta = -1` sentinels on this body** — a mesh/bone + space mismatch in the verifier, not in the solve. Don't trust it here; either fix the + verifier or ignore it and rely on the report + bed. + +## Open decisions you own + +1. **Ship-body decision** — nothing shipped carries the shapes yet, so the layer is a + silent no-op on the real Lena. `LowPoly_40` costs ~**+41 MB** for 8 shapes; full-res + ~**+98 MB**, not shippable as-is. Options: fewer shapes (drop `flat`/`relaxed` — the + bone lane already does flat on finger-weighted bodies, but the *shipped* Lena is a + rigid mitt, so think it through), hand-region remesh, LOD1-only, or quantized deltas. + Bring Jeremy the numbers and a recommendation; do not ship a 128 MB body silently. +2. **Mako male + full-res female** via the same one-command solve — untried. +3. **Pose authoring** is currently "edit `DEFAULT_PARAMS`". Could become JSON + a tuning + scene. Low priority — only if pose tuning becomes the bottleneck. +4. **Commit the tooling.** `tools/handshape_solve.py`, `tools/handshape_verify.py`, + `hand-shapes/README.md` and the `handmorph/` workdir are all untracked in the animation + repo. Commit the tools and README; keep the demo GLB out of git (128 MB, `scratchpad/`). + +## Scope fence — the parallel bone agent + +**Yours** (edit freely): `tools/handshape_solve.py`, `tools/handshape_verify.py`, +`hand-shapes/`, `characters/work/lena_leafbikini/handmorph/`, ariki-game +`src/Animation/HandMorphLayer.cs`, `scratchpad/lowpoly40_handmorph.glb`. + +**Not yours** (the bone agent is actively editing these): `tools/handpose_*.py`, +`tools/edge_stretch.py`, `tools/fin_bones.py`, `hand-poses/`, +`characters/work/lena_leafbikini/08_finger_weights.py` and the `v02/` weight-experiment +bakes, ariki-game `src/Animation/HandPoseLayer.cs`. + +**Shared, coordinate before touching:** +- `src/Testing/AnimHandTestBed.cs` and `src/Testing/Dance/DanceTestBed.cs` — both lanes' + controls live in these files. Announce edits. +- **Only ONE `--agent-api` game instance may run at a time.** The bed is a serialized + resource; check whether the other agent is mid-run before you spawn. +- ariki-game branch **`handpose-flat-runtime`**, currently 4 local commits ahead and + **not pushed** (`40b995e21`, `518b72ac6`, `fa5e8e8de`, `d1e6fb0a5`). Both agents commit + here. Pull/rebase before committing, and never squash across the other lane's work. + +**Hard rules (unchanged):** +- Do not re-export or swap any shipped body outside the registry process; + `characters/female/lena_leafbikini_base_v01/` is frozen. +- Do not touch `ariki-game/tools/make_lena_fullres_quatskin.py`; `LENA_RIGID_FINGERS` + default stays `"1"`. +- **Never commit `csproj` / `project.godot` engine-version lines**, and don't sweep the + repo-wide `.import` churn into your commits — the working tree has hundreds of modified + `.import` sidecars that are not yours. Commit explicit paths only. +- Do not enable the **bone** lane on Mako (corrupt cross-hand finger weights, verts fly + metres). The morph lane on Mako is safe in principle — but prove it with + `--selftest-bump` first. + +## Acceptance for this track + +1. Weld pass lands in `handshape_solve.py`; a fresh solve reports **zero** edges > 5× on + all 8 shapes with the sliver floor removed (report the sliver population before/after). +2. Fingertip travel bars still met (fist ≥ 2.5 cm/finger), cross-hand independence 0 cm. +3. `anim_hand_test_bed` screenshots at hand-cam range: fist and grip read as a real fist + and grip on Lena mid-dance, both hands, no needles/shards/spikes. +4. A written ship-body recommendation with MB costs, for Jeremy's decision. +5. Tools + README committed in the animation repo; no shipped asset or engine-version + line changed.