Files
animation/hand-shapes
jeremy 36fc0494e8 feat(hand-shapes): morph-target hand-pose solver — fist/grip that actually deform
The morph lane's blocking bug was believed to be coincident duplicate verts hidden
from the solver's convergence gate. Welding was necessary but nowhere near
sufficient: the previous relaxation could satisfy every stretch bar without posing
anything, so the numbers it reported were not measuring the thing they claimed.

Measured on Ariki_Female_QuatSkin_LowPoly_40, the tracked "left hand is fully
clean" baseline was a morph that moved fingertips 0.3cm, and the right hand was a
rigid 6.5cm translation of the whole hand with its shape intact. Gated Laplacian
diffusion of a delta field has a null space — constants — and edge stretch cannot
see any of it: a rigid translation stretches no edge and neither does a collapse to
zero. Both exits report perfect bars.

Six defects fixed, each with its measurement in the code comments:

  weld_roi           356 duplicate groups solved twice, deltas up to 2.55cm apart
  stitch_components  hand is 6 overlapping sheets with 1-10mm gaps; Dijkstra
                     cannot cross one, so each bone saw only its own sheet
  refit_fingers      skeleton finger chain ran to 19.8cm; the flesh ends at 14.0cm,
                     so curl_02/curl_03 drove almost no weight (shipped skeleton
                     untouched — this is solver-local scaffolding)
  solve_weights      chain-arc partition of unity; the old 8mm isotropic kernels
                     left ZERO of 17,480 verts owned above 0.85, and a 3-way blend
                     averages the curl away. Plus a support cutoff: outside every
                     kernel, renormalized 1e-81 noise had handed a mid-palm vertex
                     index_02_r=0.50 and flung it 29cm
  build_roi          joint-sphere ROI, so the rim is a wrist band and not a fractal
                     of interior chart holes
  relax              strain-only edge projection + ROI-border seam constraints,
                     replacing the diffusion described above

All 8 shapes now pass every bar with real deformation: p99.9 <= 1.58x, zero edges
over 5x, zero needles with the sliver floor removed, seam <= 3.2mm, cross-hand
independence exactly 0.0000cm, mesh fingertip travel 4.6-6.2cm on fist and
3.5-4.7cm on grip (both hands). Gates now report mesh travel and seam alongside
stretch, because stretch alone cannot gate this lane.

Verified in anim_hand_test_bed: fist and grip read as a real curl on both hands,
static and mid-dance, no fins/shards/needles. It is a loose fist rather than a
clenched one — her fingers are ~4-5cm past the knuckles.

Demo GLB stays out of git (122.6 MB, ariki-game/scratchpad/). Ship decision, the
thumb-axis refit, and the TDR crash from 8 dense targets are open — see README.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-18 11:31:11 -07:00
..

Hand shapes — morph-target hand poses for ariki-game (the "shape" lane)

Where ../hand-poses/ stores poses as bone rotations (blocked on a finger-weight repair that stalled at exp05), this lane bakes poses as surface deformation — glTF morph targets spliced directly into a body GLB. The pose library ships inside the mesh.

Why this exists (the one-paragraph case)

The scan mesh carries ~2.6k inter-digit bridge edges. Weights only choose which bone drags a shared vertex — when adjacent fingers curl apart in a fist, those bridges must tear, which is exactly the fin-stack failure of exp01exp05 (torn-edge counts 8003300 per hand, five iterations, no convergence; the mesh topology is the problem, not the weights). A morph target IS the final vertex positions: tearing is impossible by construction, and the web stretch becomes one geometric fix.

What this mesh actually is (measured 2026-08-18, LowPoly_40)

Every one of these was silently breaking the solve. Read before tuning anything.

Fact Number Consequence
Coincident duplicate verts on chart seams 356 groups / 732 verts solved twice, deltas disagreed by up to 2.55cm → seam cracks
Hand is built from separate overlapping sheets 6 components, gaps 1.110mm Dijkstra cannot cross a gap: each bone's field covers only the sheet its seeds landed on
Skeleton finger chain overshoots the flesh mesh ends 14.0cm from the wrist, _03 joints sit at 19.8cm _02/_03 carried almost no weight, so curl_02/curl_03 did nothing; the 4mm seed radius found no verts for 10 of 15 bones
Detached fragment near the right wrist 134 verts took index_02_r through a seed leak and flew 31cm on fist_r
Verts owned by any single phalanx zero of 17,480 above 0.85 8mm kernels are wider than the gap to the next phalanx; a 3-way blend averages the curl away

Pipeline (tools/handshape_solve.py, pure numpy on raw GLB bytes)

  1. refit the finger chain into the flesh (refit_fingers) — solver-local scaffolding only; the shipped skeleton is never touched, because every clip pins all 65 bone positions. A morph is just final vertex positions, so the pose only needs pivots that lie inside the flesh they bend.
  2. ROI = union of spheres about the wrist + refit joints (build_roi). NOT a tube about bone segments — a tube leaves the ROI riddled with interior chart holes, so its "rim" is a fractal inside the hand rather than a wrist band.
  3. weld coincident verts into single graph nodes (weld_roi); deltas scatter back to every duplicate, so seams cannot crack by construction.
  4. stitch separate sheets within 12mm (stitch_components) — cross-component pairs only, so a stitch can never fake a shortcut inside a sheet.
  5. weights as a partition of unity along each digit's chain arc (solve_weights): narrow handover ramps at each joint (1.0 mid-phalanx, 0.5 at the joint), times digit ownership from lateral distance relative to the nearest chain.
  6. pose parametric curl/spread/thumb-opposition, LBS with the solver's own weights (pose_globals, lbs); parameters in DEFAULT_PARAMS.
  7. relax by strain-only edge projection (relax) plus ROI-border seam constraints.
  8. emit POSITION and NORMAL deltas as morph accessors (emit_morph_glb), names in extras.targetNames as hand_<pose>_<l|r>.

--selftest-bump splices one synthetic 3cm palm bump with no solve — proves the import + drive path on a new body. --no-weld / --no-stitch / --no-refit reproduce the older behaviour for comparison.

The trap that invalidated every earlier gate

The previous relaxation was gated Laplacian diffusion of the delta field. Diffusion has a null space — constants — and edge stretch is blind to every member of it: a rigid translation stretches no edge, and neither does a collapse to zero. So the diffusion always found one of those two exits, and reported perfect bars on the way out. Measured on this body: the left hand decayed to 0.3cm of fingertip travel (max 1.93x, p99.9 1.60x, zero torn edges — a flawless report for a morph that does nothing), and the right hand converged to a near-constant 6.5cm delta at every arc position from wrist to fingertip — the whole hand translated sideways with its shape intact (max 2.43x, p99.9 1.46x, also "passing"). Strain-only projection has no such exit: a conforming edge contributes no correction, so the pose survives wherever it does not tear.

Corollary: edge stretch alone can never gate this lane. Always read mesh fingertip travel (tip_mesh_cm) and seam_max_mm beside it. tip_bone_cm is scaffolding — it read 10cm/finger while the _03 joints floated 5cm outside the mesh.

Bake a body

"C:/Program Files/Blender Foundation/Blender 5.1/blender.exe" --background \
  --factory-startup --python tools/handshape_solve.py -- \
  --body <body.glb> --poses flat,relaxed,fist,grip --out <out.glb> \
  --workdir characters/work/lena_leafbikini/handmorph/

Blender is only the numpy host — no bpy, no scene import (so the importer-draws-false-shards trap does not apply to the solver). ~1 min for 8 shapes. Numbers land in handmorph/handmorph_report.json, OBJ dumps beside it.

Gate bars (per shape)

  • edge stretch over all edges, no rest-length floor: p99.9 <= 1.6x, zero > 5x, zero "needles" (>5x and >1mm of real growth). The old 1mm floor hid a population of sub-mm seam edges that grew to 34cm — hairline spikes, sub-pixel in screenshots.
  • mesh fingertip travel: fist >= 2.5 cm/finger, grip ~2 cm, relaxed 0.52 cm
  • ROI-border seam: <= ~5mm
  • cross-hand independence: 0 cm on the other hand's verts

Current (2026-08-18, Ariki_Female_QuatSkin_LowPoly_40.glb, ROI 19,159 → 17,480 nodes):

shape max p99.9 n>5x needles seam mesh tip cm
flat_l 1.35 1.35 0 0 1.2mm 0.1
relaxed_l 1.36 1.35 0 0 2.0mm 1.0
fist_l 1.63 1.38 0 0 1.3mm 4.64.9
grip_l 1.82 1.46 0 0 1.3mm 3.53.7
flat_r 1.35 1.35 0 0 1.5mm 0.1
relaxed_r 1.66 1.45 0 0 2.6mm 1.11.2
fist_r 1.77 1.54 0 0 3.2mm 6.06.2
grip_r 2.23 1.58 0 0 3.1mm 4.54.7

Cross-hand independence is exactly 0.0000 cm on all 8 shapes. viol_edges_left in the report counts edges still above the soft 1.35x projection target (~6k on fist/grip after 1500 iterations) — not a bar, but the reason max sits near 1.8x rather than 1.35x.

Verify in-engine (the only honest gate)

HANDMORPH_BODY_F=res://scratchpad/lowpoly40_handmorph.glb \
  SCENE=anim_hand_test_bed MOCK_ONLY=1 AGENT_OWNED=1 WAIT=1 bash tools/game.sh spawn
# then: game.sh click 'fist' / 'grip' / 'flat' / 'morph OFF' / 'Cam: Lena hands'

Verified 2026-08-18: [HandMorphLayer] found 4 hand pose(s); fist and grip both read as a real curl on both hands, static and mid-dance_soul, at hand-cam range — no fins, shards, needles or stray geometry. It reads as a loose fist / cupped hand, not a clenched one: her fingers are only ~45cm long past the knuckles, so ~6cm of tip travel is most of the range available.

Two environment notes: this demo body carries the yellow LowPoly-bake defect (yellow with the morph on and off — it is the body, not the lane), and 8 dense morph targets on a 430k-vert mesh crashed the GPU driver (Vulkan device was lost, TDR) after ~12 minutes of bed time. Take screenshots promptly, and treat runtime cost as an open risk.

Relationship to the bone lane (../hand-poses/)

Independent and composable: the bone lane overrides finger-bone rotations (needs finger-weighted bodies); this lane deforms the surface (works on ANY body carrying the shapes, including the shipped rigid-mitt bodies). Hotkeys H bone lane, K shape lane in the dance bed; button panels in anim_hand_test_bed.

Open items

  • Ship decision. Nothing shipped carries the shapes, so the layer is a silent no-op on the real Lena. Dense float32 POSITION+NORMAL deltas over all 430,551 verts cost ~9.9 MB per shape: 43.8 MB → 122.6 MB for 8 (+78.8 MB, not the +41 MB the handover estimated). Only 5.5k9.7k verts per shape are non-zero (2.2%), so glTF sparse accessors are a ~30x lever (~2.5 MB for all 8) — but the engine's glTF module appears to write sparse accessors without reading them, so test one shape before betting on it. Fallbacks: drop flat (max delta 0.330.66cm — nearly a no-op) and relaxed, keeping fist+grip = 4 shapes at ~+39 MB; or LOD1-only; or a hand-region remesh. The TDR crash above says runtime cost needs measuring too, not just bytes.
  • Thumb chain refit is unreliable. Its reach is measured along a wrist→tip axis that passes through the palm, so palm/wrist flesh gets claimed by the thumb (a vertex 5cm from the wrist came out thumb_01_r=0.88 and swung 4.8cm). The right hand still shows 1.4cm of wrist motion on fist; the left shows none. Needs a thumb-specific axis.
  • Projection does not fully converge to 1.35x within 1500 iterations (see above).
  • tools/handshape_verify.py reports own-delta = -1 sentinels on this body — a mesh/bone space mismatch in the verifier, not in the solve. Superseded in practice by the report plus the bed; fix it or retire it.
  • Mako male and full-res female via the same one-command solve — untried.
  • Pose authoring is editing DEFAULT_PARAMS — could become JSON plus a tuning scene.