Add faceFollow, fix face contour rendering, add default eyes/ears
Three changesets bundled together: - faceFollow (Camera/Neck) for the ARKit-blendshape face: it can now rotate with the mapped Body_Neck joint's own local axes instead of always billboarding toward the camera. - Fixed a real correctness bug: the face was rendering as isolated dots. Verified against OpenPose's own published keypoint diagrams and source (FACE_PAIRS_RENDER_GPU, 63 pairs) that it should render connected jaw/eyebrow/nose/eye/mouth contour lines; full_limb_data() was also silently dropping face limbs entirely. Both fixed. - Default eyes/ears: most rigs (HumanIK included) have no REye/LEye/REar/LEar joints, only Head/Nose, leaving those 4 slots permanently invisible. They now get a synthetic head-relative position when unmapped but Body_Nose is mapped, anchored at and rotating with the Nose joint's own orientation (not the camera) -- toggleable via useDefaultEyesEars, always overridden by an explicit joint mapping. Also drops the RShoulder->REar / LShoulder->LEar lines from BODY_25 and COCO connectivity by request (a detection-robustness quirk of OpenPose's original network, not real anatomy) so ears stay leaf points -- a disclosed, deliberate deviation from upstream's otherwise-verbatim connectivity. All verified against real Maya (2022/2023/2024), not just logic: rotation math checked exact, explicit-mapping-overrides-fallback checked, shoulder-ear removal checked against full_limb_data() output, rendered test scenes visually confirmed against OpenPose's own reference diagrams. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
@@ -81,7 +81,29 @@ rendered output for that character:
|
||||
pelvis point. Slots you mapped for MidHip/feet are simply not drawn in
|
||||
this mode; nothing needs to be remapped to switch.
|
||||
|
||||
Hands and face are unaffected by this setting either way. Rendered from the
|
||||
Hands and face are unaffected by this setting either way.
|
||||
|
||||
### Default eyes/ears
|
||||
|
||||
Most rigs (including HumanIK) have no `REye`/`LEye`/`REar`/`LEar` joints --
|
||||
only a Head/Nose. By default (`useDefaultEyesEars`, on unless turned off),
|
||||
whenever any of those 4 slots is unmapped but `Body_Nose` *is* mapped, it
|
||||
gets a synthetic head-relative position instead of being invisible --
|
||||
anchored at and rotating with the joint mapped to `Body_Nose` itself (sized
|
||||
by `faceScale`), so they turn with the head's actual rotation rather than
|
||||
billboarding toward the camera. An explicit joint mapped to any of the 4
|
||||
slots always overrides this fallback. The offsets themselves are a
|
||||
reasonable approximation of human proportions, not from a published
|
||||
source.
|
||||
|
||||
Note also: BODY_25/COCO's `RShoulder->REar` / `LShoulder->LEar` limb lines
|
||||
(present in OpenPose's own spec -- a detection-robustness feature of the
|
||||
original network, not a real anatomical link) are intentionally **not**
|
||||
drawn here; ears connect only via eye->ear, staying leaf points. This is a
|
||||
deliberate deviation from upstream, by request -- see the note in
|
||||
`constants.py`.
|
||||
|
||||
Rendered from the
|
||||
same mapped HumanIK-named test rig (`test/openpose_humanik_test.ma`):
|
||||
|
||||
| BODY_25 | COCO |
|
||||
@@ -108,9 +130,22 @@ keypoints come from:
|
||||
ARKit Blendshapes from Selected* in the mapping editor -- it connects
|
||||
every one of the 52 canonical names that exists as an attribute on the
|
||||
selected node. The resulting face is anchored at the character's mapped
|
||||
`Body_Nose` joint and billboards to always face whichever camera is
|
||||
drawing (viewport or render camera), sized by the `faceScale` attribute
|
||||
(world units; default 15 -- tune to your scene's unit scale).
|
||||
`Body_Nose` joint, sized by the `faceScale` attribute (world units;
|
||||
default 15 -- tune to your scene's unit scale).
|
||||
|
||||
A `faceFollow` enum (mapping editor's *Face Follow* dropdown, or
|
||||
`cmds.setAttr("myCharacter.faceFollow", 1)`; 0 = `Camera`, 1 = `Neck`)
|
||||
controls the face plane's orientation:
|
||||
- **Camera** (default): always faces whichever camera is drawing
|
||||
(viewport or render camera) -- works regardless of rig conventions,
|
||||
since it never has to guess which way a joint's local axes point.
|
||||
- **Neck**: rotates *with* the character's mapped `Body_Neck` joint
|
||||
instead, using that joint's own local X/Y axes as the face plane's
|
||||
right/up -- so the face turns with the head. This is rig-dependent (it
|
||||
assumes local X = right, local Y = up, which is common but not
|
||||
universal); if the face looks rotated relative to the head, adjust the
|
||||
neck joint's rotate axis, or use `Camera` instead. Falls back to
|
||||
`Camera` if no joint is mapped to `Body_Neck`.
|
||||
|
||||
**Accuracy caveat:** there is no published, authoritative "which ARKit
|
||||
weight moves which OpenPose landmark, by how much" table anywhere (unlike
|
||||
@@ -126,9 +161,9 @@ representable effect on a frontal 2D landmark set and are no-ops.
|
||||
Rendered from `test/openpose_arkit_face_test.ma` (a control node with all 52
|
||||
ARKit-named attributes, auto-connected -- see that scene for the pattern):
|
||||
|
||||
| Neutral | `jawOpen` + smile + raised brows |
|
||||
| --- | --- |
|
||||
|  |  |
|
||||
| Neutral | `jawOpen` + smile + raised brows | `faceFollow=Neck`, neck tilted 25° |
|
||||
| --- | --- | --- |
|
||||
|  |  |  |
|
||||
|
||||
### 3. Render OpenPose images
|
||||
|
||||
@@ -147,9 +182,9 @@ ARKit-named attributes, auto-connected -- see that scene for the pattern):
|
||||
Each frame is projected through the given camera at the scene's render
|
||||
resolution (`defaultResolution` node), rasterized as a black-background,
|
||||
OpenPose-colored image (limbs under joints, canonical BODY_25 palette,
|
||||
per-finger rainbow hand colors, white face dots), written to disk, and
|
||||
pushed into Maya's Render View so you can watch it render like any other
|
||||
renderer.
|
||||
per-finger rainbow hand colors, white face contour lines), written to
|
||||
disk, and pushed into Maya's Render View so you can watch it render like
|
||||
any other renderer.
|
||||
|
||||
## Debugging (VS Code)
|
||||
|
||||
@@ -172,14 +207,20 @@ ARKit-named attributes, auto-connected -- see that scene for the pattern):
|
||||
## Notes / known limitations
|
||||
|
||||
- Colors/connectivity: BODY_25 and COCO keypoint layout and limb
|
||||
connectivity are taken verbatim from OpenPose's own
|
||||
connectivity are taken from OpenPose's own
|
||||
`POSE_BODY_25_BODY_PART_PAIRS` / COCO section of `POSE_BODY_PART_PAIRS`
|
||||
(`src/openpose/pose/poseParameters.cpp`). BODY_25's colors are likewise
|
||||
verbatim from `POSE_BODY_25_COLORS_RENDER_GPU`. COCO limb colors, and all
|
||||
hand/face colors, are generated from an HSV hue wheel (rainbow per limb
|
||||
or finger, white face dots) reproducing OpenPose's own visual scheme
|
||||
rather than a hardcoded transcription of its longer, generated gradient
|
||||
tables.
|
||||
(`src/openpose/pose/poseParameters.cpp`), with BODY_25's colors likewise
|
||||
verbatim from `POSE_BODY_25_COLORS_RENDER_GPU` -- with one deliberate,
|
||||
by-request deviation: the `RShoulder->REar` / `LShoulder->LEar` lines
|
||||
upstream includes are dropped, so ears stay leaf points connected only
|
||||
via eye->ear (see "Default eyes/ears" above). The face's 63 contour
|
||||
lines (jaw/eyebrows/nose/eyes/mouth) and solid-white color are equally
|
||||
verbatim, from `FACE_PAIRS_RENDER_GPU` / `FACE_COLORS_RENDER_GPU`
|
||||
(`include/openpose/face/faceParameters.hpp`) -- independently confirmed
|
||||
against OpenPose's own published keypoint diagrams. COCO limb colors and
|
||||
all hand colors are generated from an HSV hue wheel (rainbow per limb or
|
||||
finger) reproducing OpenPose's own visual scheme rather than a hardcoded
|
||||
transcription of its longer, generated gradient tables.
|
||||
- Projection does not attempt to replicate Maya's film-gate/overscan/fit
|
||||
reconciliation pixel-for-pixel -- for correct results, set the render
|
||||
camera's film aspect ratio to match the scene's render resolution, as you
|
||||
|
||||
Reference in New Issue
Block a user