indigo/PoseRenderer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add faceFollow, fix face contour rendering, add default eyes/ears

Browse Source 
Three changesets bundled together:

- faceFollow (Camera/Neck) for the ARKit-blendshape face: it can now
  rotate with the mapped Body_Neck joint's own local axes instead of
  always billboarding toward the camera.

- Fixed a real correctness bug: the face was rendering as isolated
  dots. Verified against OpenPose's own published keypoint diagrams
  and source (FACE_PAIRS_RENDER_GPU, 63 pairs) that it should render
  connected jaw/eyebrow/nose/eye/mouth contour lines; full_limb_data()
  was also silently dropping face limbs entirely. Both fixed.

- Default eyes/ears: most rigs (HumanIK included) have no
  REye/LEye/REar/LEar joints, only Head/Nose, leaving those 4 slots
  permanently invisible. They now get a synthetic head-relative
  position when unmapped but Body_Nose is mapped, anchored at and
  rotating with the Nose joint's own orientation (not the camera) --
  toggleable via useDefaultEyesEars, always overridden by an explicit
  joint mapping. Also drops the RShoulder->REar / LShoulder->LEar
  lines from BODY_25 and COCO connectivity by request (a
  detection-robustness quirk of OpenPose's original network, not real
  anatomy) so ears stay leaf points -- a disclosed, deliberate
  deviation from upstream's otherwise-verbatim connectivity.

All verified against real Maya (2022/2023/2024), not just logic:
rotation math checked exact, explicit-mapping-overrides-fallback
checked, shoulder-ear removal checked against full_limb_data() output,
rendered test scenes visually confirmed against OpenPose's own
reference diagrams.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
indigo
2026-07-14 10:12:10 +08:00
parent 417ff72850
commit 67b3deedbd
14 changed files with 471 additions and 178 deletions
Download Patch File Download Diff File Expand all files Collapse all files
python/openpose_renderer/constants.py
+79 -33
View File
@@ -5,21 +5,23 @@ points, in that order, giving a flat 0..136 global keypoint index used
everywhere else in this plugin (mapping node attribute slots, draw override,
projector, rasterizer).
Body part names, indices and limb pairs are verbatim from OpenPose's own
``POSE_BODY_25_BODY_PARTS`` / ``POSE_BODY_25_BODY_PART_PAIRS`` (see
CMU-Perceptual-Computing-Lab/openpose, include/openpose/pose/
poseParametersRender.hpp). Hand and face keypoint layouts follow the
standard, widely-interoperable 21-point hand (wrist + 5x4-joint fingers,
identical ordering to OpenPose/MediaPipe hand models) and 70-point face
(iBUG 68-point facial landmark scheme + 2 pupils, OpenPose's own face model)
conventions.
Colors are generated from an HSV hue wheel per part rather than
hardcoded from upstream source: OpenPose's own hand/face color tables are
long generated gradients (not simple literals) that could not be reliably
transcribed byte-for-byte here, so we reproduce the same *visual* scheme
(rainbow limbs, distinct hue per finger, white face dots) programmatically,
which is verifiably correct instead of a guess.
Body part names, indices, limb pairs and colors are verbatim from
OpenPose's own ``POSE_BODY_25_BODY_PARTS`` / ``POSE_BODY_25_BODY_PART_PAIRS``
/ ``POSE_BODY_25_COLORS_RENDER_GPU`` (CMU-Perceptual-Computing-Lab/openpose,
include/openpose/pose/poseParametersRender.hpp). Face keypoint layout, limb
pairs, and color are likewise verbatim from OpenPose's own
``FACE_PAIRS_RENDER_GPU`` / ``FACE_COLORS_RENDER_GPU``
(include/openpose/face/faceParameters.hpp) -- 70 points (iBUG 68-point
facial landmark scheme + 2 pupils) connected into jaw/eyebrow/nose/eye/mouth
contour lines, all solid white. Hand keypoint layout follows the standard,
widely-interoperable 21-point convention (wrist + 5x4-joint fingers,
identical ordering to OpenPose/MediaPipe hand models); unlike body and face,
hand and COCO-limb colors are generated from an HSV hue wheel rather than
transcribed from upstream, since OpenPose's own hand color table is a long
generated gradient (not simple literals) that could not be reliably
transcribed byte-for-byte here -- this reproduces the same *visual* scheme
(rainbow limbs, distinct hue per finger) programmatically, which is
verifiably correct instead of a guess.
"""
import colorsys
@@ -66,12 +68,16 @@ BODY_25_NAMES = [
]
assert len(BODY_25_NAMES) == BODY_COUNT
# Verbatim from OpenPose's POSE_BODY_25_BODY_PART_PAIRS_RENDER_GPU (local
# indices 0..24, i.e. relative to BODY_START).
# From OpenPose's POSE_BODY_25_BODY_PART_PAIRS_RENDER_GPU (local indices
# 0..24, i.e. relative to BODY_START), minus (2,17) RShoulder-REar and
# (5,18) LShoulder-LEar -- upstream includes these, but they're a
# detection-robustness quirk of the original network (a long cross-body
# line), not a real anatomical link; ears are otherwise leaf points
# (connected only via eye-ear) and are kept that way here by request.
BODY_25_PAIRS_LOCAL = [
(1, 8), (1, 2), (1, 5), (2, 3), (3, 4), (5, 6), (6, 7), (8, 9), (9, 10),
(10, 11), (8, 12), (12, 13), (13, 14), (1, 0), (0, 15), (15, 17), (0, 16),
(16, 18), (2, 17), (5, 18), (14, 19), (19, 20), (14, 21), (11, 22),
(16, 18), (14, 19), (19, 20), (14, 21), (11, 22),
(22, 23), (11, 24),
]
@@ -83,6 +89,22 @@ BODY_25_LIMB_COLORS = [
BODY_POINT_RADIUS = 4.0
BODY_LIMB_THICKNESS = 4.0
# Approximate head-relative fallback offsets (local plane: +X = one side,
# +Y = up -- same convention as face_blendshapes' neutral template, and
# scaled by the same faceScale attribute) for REye/LEye/REar/LEar. Used only
# when a rig has no explicit eye/ear joints -- true of most Maya rigs,
# including HumanIK, which typically only goes as far as a Head joint --
# but does have Nose mapped. An explicit joint mapping for any of these
# always takes priority over this fallback. Not derived from any published
# proportion table -- a reasonable approximation, documented as such (same
# honesty standard as face_blendshapes.py).
DEFAULT_EYE_EAR_LOCAL_OFFSETS = {
BODY_25_NAMES.index("REye"): (-0.5, 0.3),
BODY_25_NAMES.index("LEye"): (0.5, 0.3),
BODY_25_NAMES.index("REar"): (-1.0, 0.05),
BODY_25_NAMES.index("LEar"): (1.0, 0.05),
}
# --- Body: COCO-18 (legacy OpenPose COCO model) ------------------------------
#
@@ -116,13 +138,14 @@ assert all(BODY_25_NAMES[COCO_TO_BODY25_INDEX[i]] == COCO_NAMES[i] for i in rang
COCO_ACTIVE_BODY25_INDICES = sorted(COCO_TO_BODY25_INDEX)
# Verbatim (COCO section of POSE_BODY_PART_PAIRS), expressed in COCO-local
# 0..17 indices, then translated to BODY_25-local/global indices so it can
# be used directly against the same joint mapping as BODY_25 mode.
# COCO section of POSE_BODY_PART_PAIRS, expressed in COCO-local 0..17
# indices, then translated to BODY_25-local/global indices so it can be
# used directly against the same joint mapping as BODY_25 mode. Minus
# (2,16) RShoulder-REar and (5,17) LShoulder-LEar for the same reason as
# BODY_25's equivalent pairs -- see BODY_25_PAIRS_LOCAL above.
_COCO_PAIRS_COCO_SPACE = [
(1, 2), (1, 5), (2, 3), (3, 4), (5, 6), (6, 7), (1, 8), (8, 9), (9, 10),
(1, 11), (11, 12), (12, 13), (1, 0), (0, 14), (14, 16), (0, 15), (15, 17),
(2, 16), (5, 17),
]
COCO_PAIRS_LOCAL = [
(COCO_TO_BODY25_INDEX[a], COCO_TO_BODY25_INDEX[b]) for (a, b) in _COCO_PAIRS_COCO_SPACE
@@ -189,13 +212,34 @@ FACE_NAMES = (
)
assert len(FACE_NAMES) == FACE_COUNT
# OpenPose's default renderer draws the face as keypoints only (no limb
# lines), all white.
FACE_PAIRS_LOCAL = []
# Verbatim from OpenPose's FACE_PAIRS_RENDER_GPU
# (include/openpose/face/faceParameters.hpp): jaw is an open chain, both
# eyebrows are open chains, the nose bridge/nostril-arc are open chains, and
# both eyes + both mouth contours (outer, inner) are closed loops. All face
# limbs render in solid white (FACE_COLORS_RENDER_GPU), same as the points.
FACE_PAIRS_LOCAL = [
(0, 1), (1, 2), (2, 3), (3, 4), (4, 5), (5, 6), (6, 7), (7, 8), (8, 9),
(9, 10), (10, 11), (11, 12), (12, 13), (13, 14), (14, 15), (15, 16),
(17, 18), (18, 19), (19, 20), (20, 21),
(22, 23), (23, 24), (24, 25), (25, 26),
(27, 28), (28, 29), (29, 30),
(31, 32), (32, 33), (33, 34), (34, 35),
(36, 37), (37, 38), (38, 39), (39, 40), (40, 41), (41, 36),
(42, 43), (43, 44), (44, 45), (45, 46), (46, 47), (47, 42),
(48, 49), (49, 50), (50, 51), (51, 52), (52, 53), (53, 54), (54, 55),
(55, 56), (56, 57), (57, 58), (58, 59), (59, 48),
(60, 61), (61, 62), (62, 63), (63, 64), (64, 65), (65, 66), (66, 67), (67, 60),
]
assert len(FACE_PAIRS_LOCAL) == 63
FACE_POINT_COLORS = [(255, 255, 255)] * FACE_COUNT
FACE_LIMB_COLORS = [(255, 255, 255)] * len(FACE_PAIRS_LOCAL)
FACE_POINT_RADIUS = 2.0
FACE_LIMB_THICKNESS = 0.0
# OpenPose's source does not specify a face line thickness (only color);
# this is a reasonable, thinner-than-body/hand choice for a fine facial
# wireframe, not a value taken from upstream.
FACE_LIMB_THICKNESS = 1.5
# --- Flattened 0..136 global schema ------------------------------------------
@@ -236,10 +280,11 @@ def part_of(global_index):
raise IndexError(global_index)
# Hand limb pairs, offset into the flat 0..136 range -- unaffected by body
# model choice, so precomputed once.
# Hand/face limb pairs, offset into the flat 0..136 range -- unaffected by
# body model choice, so precomputed once.
_HAND_LEFT_PAIRS_GLOBAL = [(a + HAND_LEFT_START, b + HAND_LEFT_START) for (a, b) in HAND_PAIRS_LOCAL]
_HAND_RIGHT_PAIRS_GLOBAL = [(a + HAND_RIGHT_START, b + HAND_RIGHT_START) for (a, b) in HAND_PAIRS_LOCAL]
_FACE_PAIRS_GLOBAL = [(a + FACE_START, b + FACE_START) for (a, b) in FACE_PAIRS_LOCAL]
def _body_limb_data(body_model):
@@ -255,17 +300,18 @@ def _body_limb_data(body_model):
def full_limb_data(body_model=BODY_MODEL_BODY_25):
"""(pairs, colors, thickness) for the whole rig: the chosen body model
plus both full hands, global-indexed into the flat 0..136 range. Faces
render as dots only and contribute no limbs.
"""(pairs, colors, thickness) for the whole rig: the chosen body model,
both full hands, and the face contour, global-indexed into the flat
0..136 range.
"""
body_pairs, body_colors, body_thickness = _body_limb_data(body_model)
pairs = list(body_pairs) + _HAND_LEFT_PAIRS_GLOBAL + _HAND_RIGHT_PAIRS_GLOBAL
colors = list(body_colors) + HAND_LIMB_COLORS + HAND_LIMB_COLORS
pairs = list(body_pairs) + _HAND_LEFT_PAIRS_GLOBAL + _HAND_RIGHT_PAIRS_GLOBAL + _FACE_PAIRS_GLOBAL
colors = list(body_colors) + HAND_LIMB_COLORS + HAND_LIMB_COLORS + FACE_LIMB_COLORS
thickness = (
list(body_thickness)
+ [HAND_LIMB_THICKNESS] * len(HAND_PAIRS_LOCAL)
+ [HAND_LIMB_THICKNESS] * len(HAND_PAIRS_LOCAL)
+ [FACE_LIMB_THICKNESS] * len(FACE_PAIRS_LOCAL)
)
assert len(pairs) == len(colors) == len(thickness)
return pairs, colors, thickness