feat(hand-shapes): morph-target hand-pose solver — fist/grip that actually deform
The morph lane's blocking bug was believed to be coincident duplicate verts hidden
from the solver's convergence gate. Welding was necessary but nowhere near
sufficient: the previous relaxation could satisfy every stretch bar without posing
anything, so the numbers it reported were not measuring the thing they claimed.
Measured on Ariki_Female_QuatSkin_LowPoly_40, the tracked "left hand is fully
clean" baseline was a morph that moved fingertips 0.3cm, and the right hand was a
rigid 6.5cm translation of the whole hand with its shape intact. Gated Laplacian
diffusion of a delta field has a null space — constants — and edge stretch cannot
see any of it: a rigid translation stretches no edge and neither does a collapse to
zero. Both exits report perfect bars.
Six defects fixed, each with its measurement in the code comments:
weld_roi 356 duplicate groups solved twice, deltas up to 2.55cm apart
stitch_components hand is 6 overlapping sheets with 1-10mm gaps; Dijkstra
cannot cross one, so each bone saw only its own sheet
refit_fingers skeleton finger chain ran to 19.8cm; the flesh ends at 14.0cm,
so curl_02/curl_03 drove almost no weight (shipped skeleton
untouched — this is solver-local scaffolding)
solve_weights chain-arc partition of unity; the old 8mm isotropic kernels
left ZERO of 17,480 verts owned above 0.85, and a 3-way blend
averages the curl away. Plus a support cutoff: outside every
kernel, renormalized 1e-81 noise had handed a mid-palm vertex
index_02_r=0.50 and flung it 29cm
build_roi joint-sphere ROI, so the rim is a wrist band and not a fractal
of interior chart holes
relax strain-only edge projection + ROI-border seam constraints,
replacing the diffusion described above
All 8 shapes now pass every bar with real deformation: p99.9 <= 1.58x, zero edges
over 5x, zero needles with the sliver floor removed, seam <= 3.2mm, cross-hand
independence exactly 0.0000cm, mesh fingertip travel 4.6-6.2cm on fist and
3.5-4.7cm on grip (both hands). Gates now report mesh travel and seam alongside
stretch, because stretch alone cannot gate this lane.
Verified in anim_hand_test_bed: fist and grip read as a real curl on both hands,
static and mid-dance, no fins/shards/needles. It is a loose fist rather than a
clenched one — her fingers are ~4-5cm past the knuckles.
Demo GLB stays out of git (122.6 MB, ariki-game/scratchpad/). Ship decision, the
thumb-axis refit, and the TDR crash from 8 dense targets are open — see README.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,156 @@
|
||||
# Hand shapes — morph-target hand poses for ariki-game (the "shape" lane)
|
||||
|
||||
Where `../hand-poses/` stores poses as **bone rotations** (blocked on a finger-weight
|
||||
repair that stalled at exp05), this lane bakes poses as **surface deformation** — glTF
|
||||
morph targets spliced directly into a body GLB. The pose library ships *inside the mesh*.
|
||||
|
||||
## Why this exists (the one-paragraph case)
|
||||
|
||||
The scan mesh carries ~2.6k inter-digit bridge edges. Weights only choose *which bone
|
||||
drags a shared vertex* — when adjacent fingers curl apart in a fist, those bridges must
|
||||
tear, which is exactly the fin-stack failure of exp01–exp05 (torn-edge counts 800–3300
|
||||
per hand, five iterations, no convergence; the mesh topology is the problem, not the
|
||||
weights). A morph target IS the final vertex positions: tearing is impossible by
|
||||
construction, and the web stretch becomes one geometric fix.
|
||||
|
||||
## What this mesh actually is (measured 2026-08-18, `LowPoly_40`)
|
||||
|
||||
Every one of these was silently breaking the solve. Read before tuning anything.
|
||||
|
||||
| Fact | Number | Consequence |
|
||||
|---|---|---|
|
||||
| Coincident duplicate verts on chart seams | 356 groups / 732 verts | solved twice, deltas disagreed by up to 2.55cm → seam cracks |
|
||||
| Hand is built from separate overlapping sheets | 6 components, gaps 1.1–10mm | Dijkstra cannot cross a gap: each bone's field covers only the sheet its seeds landed on |
|
||||
| Skeleton finger chain overshoots the flesh | mesh ends 14.0cm from the wrist, `_03` joints sit at 19.8cm | `_02`/`_03` carried almost no weight, so `curl_02`/`curl_03` did nothing; the 4mm seed radius found no verts for 10 of 15 bones |
|
||||
| Detached fragment near the right wrist | 134 verts | took `index_02_r` through a seed leak and flew **31cm** on fist_r |
|
||||
| Verts owned by any single phalanx | **zero** of 17,480 above 0.85 | 8mm kernels are wider than the gap to the next phalanx; a 3-way blend averages the curl away |
|
||||
|
||||
## Pipeline (`tools/handshape_solve.py`, pure numpy on raw GLB bytes)
|
||||
|
||||
1. **refit** the finger chain into the flesh (`refit_fingers`) — solver-local scaffolding
|
||||
only; the shipped skeleton is never touched, because every clip pins all 65 bone
|
||||
positions. A morph is just final vertex positions, so the pose only needs pivots that
|
||||
lie inside the flesh they bend.
|
||||
2. **ROI** = union of spheres about the wrist + refit joints (`build_roi`). NOT a tube
|
||||
about bone segments — a tube leaves the ROI riddled with interior chart holes, so its
|
||||
"rim" is a fractal inside the hand rather than a wrist band.
|
||||
3. **weld** coincident verts into single graph nodes (`weld_roi`); deltas scatter back to
|
||||
every duplicate, so seams cannot crack by construction.
|
||||
4. **stitch** separate sheets within 12mm (`stitch_components`) — cross-component pairs
|
||||
only, so a stitch can never fake a shortcut inside a sheet.
|
||||
5. **weights** as a partition of unity along each digit's chain arc (`solve_weights`):
|
||||
narrow handover ramps at each joint (1.0 mid-phalanx, 0.5 at the joint), times digit
|
||||
ownership from lateral distance *relative to the nearest chain*.
|
||||
6. **pose** parametric curl/spread/thumb-opposition, LBS with the solver's own weights
|
||||
(`pose_globals`, `lbs`); parameters in `DEFAULT_PARAMS`.
|
||||
7. **relax** by strain-only edge projection (`relax`) plus ROI-border seam constraints.
|
||||
8. **emit** POSITION *and* NORMAL deltas as morph accessors (`emit_morph_glb`), names in
|
||||
`extras.targetNames` as `hand_<pose>_<l|r>`.
|
||||
|
||||
`--selftest-bump` splices one synthetic 3cm palm bump with no solve — proves the
|
||||
import + drive path on a new body. `--no-weld` / `--no-stitch` / `--no-refit` reproduce
|
||||
the older behaviour for comparison.
|
||||
|
||||
### The trap that invalidated every earlier gate
|
||||
|
||||
The previous relaxation was **gated Laplacian diffusion of the delta field**. Diffusion
|
||||
has a null space — constants — and edge stretch is blind to every member of it: a rigid
|
||||
translation stretches no edge, and neither does a collapse to zero. So the diffusion
|
||||
always found one of those two exits, and reported perfect bars on the way out. Measured
|
||||
on this body: the left hand decayed to **0.3cm** of fingertip travel (max 1.93x,
|
||||
p99.9 1.60x, zero torn edges — a flawless report for a morph that does nothing), and the
|
||||
right hand converged to a near-constant **6.5cm delta at every arc position from wrist to
|
||||
fingertip** — the whole hand translated sideways with its shape intact (max 2.43x,
|
||||
p99.9 1.46x, also "passing"). Strain-only projection has no such exit: a conforming edge
|
||||
contributes no correction, so the pose survives wherever it does not tear.
|
||||
|
||||
Corollary: **edge stretch alone can never gate this lane.** Always read mesh fingertip
|
||||
travel (`tip_mesh_cm`) and `seam_max_mm` beside it. `tip_bone_cm` is scaffolding — it
|
||||
read 10cm/finger while the `_03` joints floated 5cm outside the mesh.
|
||||
|
||||
## Bake a body
|
||||
|
||||
```
|
||||
"C:/Program Files/Blender Foundation/Blender 5.1/blender.exe" --background \
|
||||
--factory-startup --python tools/handshape_solve.py -- \
|
||||
--body <body.glb> --poses flat,relaxed,fist,grip --out <out.glb> \
|
||||
--workdir characters/work/lena_leafbikini/handmorph/
|
||||
```
|
||||
|
||||
Blender is only the numpy host — no bpy, no scene import (so the
|
||||
importer-draws-false-shards trap does not apply to the solver). ~1 min for 8 shapes.
|
||||
Numbers land in `handmorph/handmorph_report.json`, OBJ dumps beside it.
|
||||
|
||||
## Gate bars (per shape)
|
||||
|
||||
- edge stretch over **all** edges, no rest-length floor: p99.9 <= 1.6x, zero > 5x, zero
|
||||
"needles" (>5x *and* >1mm of real growth). The old 1mm floor hid a population of sub-mm
|
||||
seam edges that grew to 3–4cm — hairline spikes, sub-pixel in screenshots.
|
||||
- **mesh** fingertip travel: fist >= 2.5 cm/finger, grip ~2 cm, relaxed 0.5–2 cm
|
||||
- ROI-border seam: <= ~5mm
|
||||
- cross-hand independence: 0 cm on the other hand's verts
|
||||
|
||||
Current (2026-08-18, `Ariki_Female_QuatSkin_LowPoly_40.glb`, ROI 19,159 → 17,480 nodes):
|
||||
|
||||
| shape | max | p99.9 | n>5x | needles | seam | mesh tip cm |
|
||||
|---|---|---|---|---|---|---|
|
||||
| flat_l | 1.35 | 1.35 | 0 | 0 | 1.2mm | 0.1 |
|
||||
| relaxed_l | 1.36 | 1.35 | 0 | 0 | 2.0mm | 1.0 |
|
||||
| fist_l | 1.63 | 1.38 | 0 | 0 | 1.3mm | 4.6–4.9 |
|
||||
| grip_l | 1.82 | 1.46 | 0 | 0 | 1.3mm | 3.5–3.7 |
|
||||
| flat_r | 1.35 | 1.35 | 0 | 0 | 1.5mm | 0.1 |
|
||||
| relaxed_r | 1.66 | 1.45 | 0 | 0 | 2.6mm | 1.1–1.2 |
|
||||
| fist_r | 1.77 | 1.54 | 0 | 0 | 3.2mm | 6.0–6.2 |
|
||||
| grip_r | 2.23 | 1.58 | 0 | 0 | 3.1mm | 4.5–4.7 |
|
||||
|
||||
Cross-hand independence is exactly 0.0000 cm on all 8 shapes. `viol_edges_left` in the
|
||||
report counts edges still above the *soft* 1.35x projection target (~6k on fist/grip after
|
||||
1500 iterations) — not a bar, but the reason `max` sits near 1.8x rather than 1.35x.
|
||||
|
||||
## Verify in-engine (the only honest gate)
|
||||
|
||||
```
|
||||
HANDMORPH_BODY_F=res://scratchpad/lowpoly40_handmorph.glb \
|
||||
SCENE=anim_hand_test_bed MOCK_ONLY=1 AGENT_OWNED=1 WAIT=1 bash tools/game.sh spawn
|
||||
# then: game.sh click 'fist' / 'grip' / 'flat' / 'morph OFF' / 'Cam: Lena hands'
|
||||
```
|
||||
|
||||
Verified 2026-08-18: `[HandMorphLayer] found 4 hand pose(s)`; fist and grip both read as
|
||||
a real curl on both hands, static and mid-`dance_soul`, at hand-cam range — no fins,
|
||||
shards, needles or stray geometry. It reads as a **loose fist / cupped hand**, not a
|
||||
clenched one: her fingers are only ~4–5cm long past the knuckles, so ~6cm of tip travel
|
||||
is most of the range available.
|
||||
|
||||
Two environment notes: this demo body carries the **yellow LowPoly-bake defect** (yellow
|
||||
with the morph on *and* off — it is the body, not the lane), and 8 dense morph targets on
|
||||
a 430k-vert mesh **crashed the GPU driver** (`Vulkan device was lost`, TDR) after ~1–2
|
||||
minutes of bed time. Take screenshots promptly, and treat runtime cost as an open risk.
|
||||
|
||||
## Relationship to the bone lane (`../hand-poses/`)
|
||||
|
||||
Independent and composable: the bone lane overrides finger-bone rotations (needs
|
||||
finger-weighted bodies); this lane deforms the surface (works on ANY body carrying the
|
||||
shapes, including the shipped rigid-mitt bodies). Hotkeys **H** bone lane, **K** shape
|
||||
lane in the dance bed; button panels in `anim_hand_test_bed`.
|
||||
|
||||
## Open items
|
||||
|
||||
- **Ship decision.** Nothing shipped carries the shapes, so the layer is a silent no-op
|
||||
on the real Lena. Dense float32 POSITION+NORMAL deltas over all 430,551 verts cost
|
||||
**~9.9 MB per shape**: 43.8 MB → 122.6 MB for 8 (**+78.8 MB**, not the +41 MB the
|
||||
handover estimated). Only 5.5k–9.7k verts per shape are non-zero (2.2%), so glTF
|
||||
**sparse accessors** are a ~30x lever (~2.5 MB for all 8) — but the engine's glTF
|
||||
module appears to *write* sparse accessors without reading them, so test one shape
|
||||
before betting on it. Fallbacks: drop `flat` (max delta 0.33–0.66cm — nearly a no-op)
|
||||
and `relaxed`, keeping fist+grip = 4 shapes at ~+39 MB; or LOD1-only; or a hand-region
|
||||
remesh. The TDR crash above says runtime cost needs measuring too, not just bytes.
|
||||
- **Thumb chain refit is unreliable.** Its reach is measured along a wrist→tip axis that
|
||||
passes through the palm, so palm/wrist flesh gets claimed by the thumb (a vertex 5cm
|
||||
from the wrist came out `thumb_01_r`=0.88 and swung 4.8cm). The right hand still shows
|
||||
1.4cm of wrist motion on fist; the left shows none. Needs a thumb-specific axis.
|
||||
- Projection does not fully converge to 1.35x within 1500 iterations (see above).
|
||||
- `tools/handshape_verify.py` reports `own-delta = -1` sentinels on this body — a
|
||||
mesh/bone space mismatch in the verifier, not in the solve. Superseded in practice by
|
||||
the report plus the bed; fix it or retire it.
|
||||
- Mako male and full-res female via the same one-command solve — untried.
|
||||
- Pose authoring is editing `DEFAULT_PARAMS` — could become JSON plus a tuning scene.
|
||||
Reference in New Issue
Block a user