tinqs/animation
5
0
Fork 0
Code Pull Requests Actions Packages Projects Releases Wiki Activity Issues
Files
main
animation/.claude/skills/iclone-video-mocap/references/video-mocap-plugin.md
T
jeremy 31ba2911df init: animation pipeline hub — Mac↔PC bridge for converting and implementing animations 
Skills (.claude/skills/): animation-creation, iclone-video-mocap,
pose-estimation (MediaPipe models via LFS), retarget-animations (deprecated).
Tools: cc/mixamo/kevin/mocap retargeters + composite baker.
Plans: animation-gen-pipeline + skills-adoption.
exchange/: incoming-fbx, converted-glb, reference-video (LFS for fbx/glb/mp4).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 10:29:48 -07:00

16 KiB
Raw Permalink Blame History

Reallusion AI Video Mocap for iClone 8 — Reference (researched 2026-07-15)

1. Product Disambiguation (Reallusion mocap ecosystem, mid-2026)

Product What it is Body/Face Input Pricing model
iClone Video Mocap (plug-in) AI body mocap from ordinary video, powered by QuickMagic cloud AI. THE "video mocap plugin". Launched Oct 29–31, 2025 Body + fingers; NO face Recorded video file Free plugin; pay-per-use points per generation (cloud)
AccuFACE (Motion LIVE gear) AI facial tracker from webcam (live) or video file (offline). Local NVIDIA RTX inference Face only Webcam or video file Perpetual plugin, $499 MSRP (launch promo $250, Nov 2023)
Motion LIVE Hub plugin aggregating real-time mocap hardware/gear (Xsens, Rokoko, Perception Neuron, Leap Motion, iPhone LIVE FACE, Faceware, AccuFACE) into one unified real-time recording panel Body+face+hands (via gear) Live hardware streams Hub + per-gear profile purchases
Motion Director Gameplay-style interactive character locomotion/control system inside iClone (WASD-style driving, not video mocap) Body locomotion Keyboard/gamepad Built into iClone 8

Video Mocap deliberately does NOT do faces — official workflow layers AccuFACE (or iPhone LIVE FACE) on top for talking animations. Sources: https://www.cgchannel.com/2025/10/reallusion-launches-ai-motion-capture-service-video-mocap/ , https://www.reallusion.com/iclone/motion-capture/default.html

2. Video Mocap — Identity, Versions, Requirements

  • Exact name: "iClone Video Mocap" / "Video Mocap Plug-in for iClone". Tagline: "Precise AI Mocap from Any Video".
  • Technology partner: QuickMagic (AI mocap provider); processing is cloud/online (upload video → AI conversion on QuickMagic cloud → download motion). Not local; no GPU requirement for generation itself.
  • Requirements: iClone 8.63 or above, Windows 10 & 11 only. Plugin itself is a free download, installed/updated via Reallusion Hub.
  • Version history: launch ~v1.0 (Oct 2025, alongside iClone 8.63); v1.02 (announced 2026-02-26) — urgent fix for task-status-refresh bug that caused re-submission and redundant point charges after closing the app; also added diagnostic logs and auto-recovery of task statuses from local video/motion files. Users on iClone 8.71 + plugin 1.02 reported in forum (Mar 2026+). Manual page shows revision date June 8, 2026.
  • Official manual root: https://manual.reallusion.com/iclone-video-mocap-plug-in/content/welcome.htm (sections: Workflow, Full Body, Upper Body, Import, Edit, Generate (purchase.htm), Apply, Save, Adding Facial Expression (talking-animation.htm), Video Guidelines (rule.htm)).

Sources: https://www.reallusion.com/iclone/video-mocap/download.html , https://discussions.reallusion.com/t/urgent-update-video-mocap-1-02-fixes-task-status-refresh-bug-to-prevent-redundant-charges/16659

3. Pricing

  • Current metering (product page + manual, mid-2026): 5 points per second of processed video (AI / Bonus / DA Points), 100 points = US$1.00 → $0.05/sec, max $3.00 for a 60 s task. Deduction order: AI Points → Bonus Points → DA Points.
  • At launch (Oct 2025) press reported a flat 250 DA Points (US$2.50) per task up to 60 s; the Apr 2026 beginner article still says "$2.50 per motion". Flat-250 vs 5/sec discrepancy: pricing appears to have moved to per-second metering; treat exact figure as ~$2.50–3.00/generation [UNVERIFIED which applies at any given moment].
  • Points sources: DA Points ($10 / $50 / $100 packs, min $10 spend) or Reallusion AI Cloud subscription AI Points (FREE $0 = 500 pts; STARTER $11/mo = 1,100; STANDARD $35/mo = 3,500; PRO $95/mo = 9,500; annual −20%). AI Points are universal across AI Studio, Video Mocap, Headshot image gen. DA:AI = 1:1 value.
  • Fully licensed iClone users get complimentary Video Mocap generation coupon(s); one coupon = one task regardless of duration. (Launch promo advertised "100 free AI mocap generations" limited-time; one forum user mentions losing "5 complimentary generations" — exact freebie count varies by promo [UNVERIFIED].)
  • Failed tasks: Reallusion support has refunded points for failed tasks case-by-case (forum evidence). Verify tasks/transactions: https://kb.reallusion.com/Print53250.aspx

Sources: https://manual.reallusion.com/iclone-video-mocap-plug-in/content/purchase.htm , https://ai.reallusion.com/plans.html , https://www.cgchannel.com/2025/10/reallusion-launches-ai-motion-capture-service-video-mocap/

4. Workflow (official 5 stages: Import Video | Edit Video | Generate Motion | Apply Motion | Save Motion)

4.1 Import Video

  • Open the plugin from iClone's Plugins menu after Hub install [menu path UNVERIFIED, standard for RL plugins].
  • Import via "+ Add Video" button or drag-and-drop into the panel.
  • 13 supported formats: .avi .flv .m4v .mkv .mov .mp4 .mpeg .ogv .webm .wmv .3gp .asf .dv .mxf.
  • Source video duration: 1 s – 15 min accepted; processing region must be 1–60 s (longer videos auto-trimmed or must be trimmed). Max file size 400 MB (else reduce bitrate/rescale/compress/trim). Resolution: ≥720p recommended; 4K accepted but may hurt performance. FPS not specified.
  • Task queue: up to 100 videos; batch generation: up to 10 tasks at once. Statuses: Unprocessed → Trimmed → (processing) → Applicable / Error.

4.2 Edit Video panel

  • Drag playhead to preview; drag Start Time / End Time handles to set the processing region (timecode MM:SS:FF); drag the whole region to shift it with fixed duration; Reset Range restores full span; Apply Settings confirms → status "Trimmed".
  • Detect Actors button: when multiple people are in frame, click it and pick the single target actor in a frame where they are clearly visible. One actor per task; for group scenes, duplicate the task per performer and select a different detection each time.
  • Trim out segments where no person appears (hurts AI quality).

4.3 Generate Motion

  • Choose tracking mode: Full Body (default) or Upper Body; optional "Includes Fingers" checkbox in both modes for finger capture.
  • Full Body: whole body must be clearly visible throughout. Upper Body: designed for framing cut off below the pelvis (sign language, instrument playing, speech); everything below the hip is ignored/not generated.
  • Submit → upload → cloud processing, typically 5–8 minutes per task; progress monitored in the task list. Insufficient points → "Add Funds" / "Cancel" dialog.

4.4 Apply Motion

  • Select completed task → Apply Motion button (bottom-right), or double-click the task, or right-click → Apply Motion.
  • Motion is saved as rlMotion to a preset folder and applied to the currently selected character; if none selected, a dummy character is auto-loaded. Auto-retargeting adapts the motion to any iClone/CC4/AccuRIG standard character body type.
  • "Apply reference video" checkbox: loads the source video as a 2D background object in the viewport behind the character for frame-by-frame comparison during cleanup.

4.5 Save Motion

  • Auto-saved to Content Manager Animation > Motion > VideoMocap; default disk path C:\Users\Public\Documents\Reallusion\Reallusion Custom\Animation\Motion\VideoMocap. Folder button changes path; Reset restores default. Right-click task → Find Motion File. Thumbnail from Edit Video frame (Quick Thumbnail on) or posed character (off).

Sources: https://manual.reallusion.com/iclone-video-mocap-plug-in/content/video-mocap-workflow.htm , .../import.htm , .../edit.htm , .../purchase.htm , .../video-mocap-full-body.htm , .../video-mocap-upper-body.htm , .../apply.htm , .../save.htm

5. Official Video Guidelines (rule.htm) — filming rules

  • Camera angle: flat/eye-level. Overhead or elevated angles make the result lean forward/backward.
  • Camera motion: static camera, single continuous shot, no cuts/switched shots; only minor shake tolerated ("Counteracts camera shakes" is marketing — keep it steady).
  • Framing: entire actor (or pelvis-up for Upper Body) visible for the whole clip; "any body part that moves out of the frame will present a problem". Actor not too small in frame; size consistent (no zooms/dollies).
  • Clothing: avoid solid colors and colors matching the background; patterns/texture are fine.
  • Background/lighting: clean and bright preferred, though the engine tolerates dim light, busy backdrops, patterned clothing.
  • Motion restrictions: avoid very fast motion, violent shaking, acrobatic flips, vertical actions (pull-ups, jumps), and ungrounded-feet activity (swimming).
  • Occlusion: single person only; avoid limbs intersecting other actors; avoid hand-occluding poses ("running fingers through hair").

Source: https://manual.reallusion.com/iclone-video-mocap-plug-in/content/rule.htm

6. Cleanup / Post-Processing in iClone (official recommended tools)

Artifact → tool mapping (from Reallusion Magazine + 80.lv + tutorials):

  • Foot sliding → Motion Correction tool (Motion Correction for Sliding Issues, iClone 8 manual) + Reach Target to lock/release foot positions; also Foot Contact tool + iClone footstep visualization. Official tutorial: "Mocap Motion Fix – Eliminating Foot Sliding | iClone 8 Tutorial" (https://www.youtube.com/watch?v=t8yeA-DJ0ds).
  • Sudden elevation changes / abrupt vertical pops → dedicated fix tutorial "Fixing Sudden Elevation Change"; edit hip height in Curve Editor.
  • Depth misalignment / motion offset → Auto Motion Alignment / Root Alignment.
  • Jitter/noise → Curve Editor smoothing.
  • Sudden body twists / rotating limbs → Motion Layer editing with HumanIK controls.
  • Hand/gesture errors → iClone gesture tools / hand pose library.
  • Missing or garbage frames → rotoscope against the 2D reference video and bridge with transition curves.
  • The auto-loaded reference-video background is the key rotoscoping aid — compare character to footage in real time.
  • Gotcha: if the applied clip has Loop enabled, motion-layer edits "spring back" (looped frames reference the source); Flatten the motion clip (or disable Loop) before keyframe editing (forum-confirmed staff answer).
  • Tutorial course hub: https://courses.reallusion.com/home/iclone/pipelines/video-mocap-plug-in

Sources: https://magazine.reallusion.com/2025/10/29/iclone-video-mocap-converts-videos-into-editable-3d-motion/ , https://80.lv/articles/getting-ai-mocap-ready-for-production-with-reallusion-s-iclone-8 , https://discussions.reallusion.com/t/cant-adjust-motions-after-applying-from-video-mocap/16711 , https://manual.reallusion.com/iClone-8/Content/ENU/8.0/51-Animation-Timeline-Editing/Motion-Correction-for-Sliding-Issues.htm

7. Adding Face (talking animation) — official 7-step layering with AccuFACE

  1. Apply Video Mocap body motion + reference video. 2. In AccuFACE VIDEO mode, load a speech video framed on the actor's head (RL strongly recommends separate recordings for body and face). 3. Connect AccuFACE in Motion LIVE, enable "Audio for Viseme Track" (auto voice clip). 4. Preview + record facial clips via Motion LIVE. 5. Align viseme/expression clips to the motion clip on the timeline using the reference video. 6. Verify sync in playback. 7. Save to Content Manager custom library. Source: https://manual.reallusion.com/iclone-video-mocap-plug-in/content/talking-animation.htm

8. AccuFACE (face-from-video) — key facts

  • Motion LIVE gear plugin; iClone 8.6+, Windows 10/11 64-bit, 8 GB RAM, NVIDIA RTX only (20/30/40/50 series + RTX Pro; driver 520.46+ or 572+ for RTX 50) — local GPU inference, no cloud, no AMD/Intel support. $499 MSRP (frequent discounts).
  • Input: live webcam (min 1280×720@30fps; 720p/30 optimal) or video files (AVI, WMV, MP4, MKV, MOV). Captures up to 60 fps facial data; single-pass audio + face capture; AccuLIPS refines lip/tongue; output framerate-independent (24/30/60 fps).
  • Calibration: guided process (live mode) or pick calibration poses across video frames (video mode). Refinement controls: Strength Balance (per-region), Smooth Filter, Denoise, Anti-Interference (cross-region triggering). Source: https://www.reallusion.com/iclone/motion-capture/accuface.html

9. Known Limitations & Community Issues

  • Hard limits: 60 s/task, 1 actor/task, no face capture, cloud-only (needs internet + points), 400 MB file cap, 10-task batch cap.
  • AI-typical artifacts acknowledged by Reallusion itself: foot sliding, depth misalignment, unnatural twists, jitter — cleanup is an expected part of the workflow, not an edge case.
  • Failure modes: subject leaving frame, background-matching clothing, fast/occluded motion, spins with limb occlusion, airborne/ungrounded feet.
  • Forum-reported bugs: (a) pre-1.02 task-status refresh bug → duplicate charges (fixed in 1.02); (b) failed tasks bricking the queue — Error tasks with retry/delete greyed out and Generate Motion disabled for all new videos; support refunded points but the user had to hand-edit plugin task files to recover (https://discussions.reallusion.com/t/video-mocap-problem/17173); (c) Loop-enabled clips blocking motion-layer edits (see §6).
  • Community consensus: results are solid for grounded, single-person, moderate-speed motion; twitchy feet and torso rotation remain the main manual-cleanup costs, consistent with other AI mocap (DeepMotion, Plask).

10. Best-Practice Tips (Reallusion Magazine "Video Mocap for Beginners", Apr 2026)

  • A mid-range phone (Samsung A54) or 1080p webcam (Logitech C922) on a tripod is fully sufficient; ordinary home lighting worked.
  • Rehearse, then overact mundane motions; record multiple takes.
  • Stack multiple motions in one 60 s clip with ~2 s gaps between them — one generation fee yields several clips to split in iClone.
  • Pre-trim in an external editor; use real props to anchor hand positions and prevent hand drift.
  • Use Upper Body mode + Includes Fingers for seated/talking/instrument performances. Source: https://magazine.reallusion.com/2026/04/13/video-mocap-for-beginners/

11. Brief Comparison

  • vs Motion LIVE hardware: Motion LIVE = real-time, suit/camera hardware ($–$$), no per-use fee, best fidelity and long takes; Video Mocap = offline, zero hardware, ~$2.50–3/clip, 60 s cap, cleanup required. Reallusion positions them as hybrid-complementary (ActorCore packs + hardware for hero shots, Video Mocap for bulk/reference-driven motion).
  • vs free/DIY (Rokoko Video free tier, DeepMotion, Plask, open-source VIBE/4DHumans pipelines): those export FBX/BVH requiring manual retarget; Video Mocap's differentiator is native rlMotion output, auto-retargeting to CC4/AccuRIG characters, the in-viewport reference-video rotoscope, and direct access to iClone's correction toolset — then FBX/USD export from iClone to Unreal, Unity, Blender, Maya, MotionBuilder.

12. Primary Sources

Reference in New Issue View Git Blame Copy Permalink