Add faceFollow, fix face contour rendering, add default eyes/ears

Three changesets bundled together:

- faceFollow (Camera/Neck) for the ARKit-blendshape face: it can now
  rotate with the mapped Body_Neck joint's own local axes instead of
  always billboarding toward the camera.

- Fixed a real correctness bug: the face was rendering as isolated
  dots. Verified against OpenPose's own published keypoint diagrams
  and source (FACE_PAIRS_RENDER_GPU, 63 pairs) that it should render
  connected jaw/eyebrow/nose/eye/mouth contour lines; full_limb_data()
  was also silently dropping face limbs entirely. Both fixed.

- Default eyes/ears: most rigs (HumanIK included) have no
  REye/LEye/REar/LEar joints, only Head/Nose, leaving those 4 slots
  permanently invisible. They now get a synthetic head-relative
  position when unmapped but Body_Nose is mapped, anchored at and
  rotating with the Nose joint's own orientation (not the camera) --
  toggleable via useDefaultEyesEars, always overridden by an explicit
  joint mapping. Also drops the RShoulder->REar / LShoulder->LEar
  lines from BODY_25 and COCO connectivity by request (a
  detection-robustness quirk of OpenPose's original network, not real
  anatomy) so ears stay leaf points -- a disclosed, deliberate
  deviation from upstream's otherwise-verbatim connectivity.

All verified against real Maya (2022/2023/2024), not just logic:
rotation math checked exact, explicit-mapping-overrides-fallback
checked, shoulder-ear removal checked against full_limb_data() output,
rendered test scenes visually confirmed against OpenPose's own
reference diagrams.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-14 10:12:10 +08:00
parent 417ff72850
commit 67b3deedbd
14 changed files with 471 additions and 178 deletions
+38 -1
View File
@@ -74,6 +74,32 @@ class OpenposeRenderOverride(omr.MRenderOverride):
return self._current_operation < len(self._operations)
def _fill_default_eye_ear_keypoints(dep_fn, mapped, keypoints, camera_path, width, height):
"""Fill in synthetic REye/LEye/REar/LEar pixel keypoints (see
constants.DEFAULT_EYE_EAR_LOCAL_OFFSETS) for whichever of those slots
aren't already explicitly mapped, anchored at and oriented by the
joint mapped to Nose (the head) -- so they turn with the head's actual
rotation rather than facing the camera. No-op if Nose isn't mapped.
"""
missing_offsets = {
index: offset
for index, offset in constants.DEFAULT_EYE_EAR_LOCAL_OFFSETS.items()
if index not in mapped
}
if not missing_offsets:
return
nose_index = constants.NAME_TO_INDEX["Body_Nose"]
if nose_index not in mapped:
return
anchor = projector.world_position(mapped[nose_index])
face_scale = mapping_node.get_face_scale(dep_fn)
computed = projector.default_head_relative_points(anchor, mapped[nose_index], face_scale, missing_offsets)
for index, world_point in computed.items():
px, py, in_front = projector.project_point(world_point, camera_path, width, height)
keypoints[index] = (px, py, 1.0 if in_front else 0.0)
def render_frame(camera_path, width, height, output_path):
"""Project + rasterize every mapped openposeCharacter to a single PNG."""
characters = []
@@ -90,6 +116,9 @@ def render_frame(camera_path, width, height, output_path):
mapped = mapping_node.get_mapped_joint_paths(dep_fn)
keypoints = projector.project_character(mapped, camera_path, width, height)
if mapping_node.get_use_default_eyes_ears(dep_fn):
_fill_default_eye_ear_keypoints(dep_fn, mapped, keypoints, camera_path, width, height)
if mapping_node.get_face_source(dep_fn) == mapping_node.FACE_SOURCE_ARKIT_BLENDSHAPES:
nose_index = constants.NAME_TO_INDEX["Body_Nose"]
nose_x, nose_y, nose_confidence = keypoints[nose_index]
@@ -98,7 +127,15 @@ def render_frame(camera_path, width, height, output_path):
weights = mapping_node.get_blendshape_weights(dep_fn)
face_scale = mapping_node.get_face_scale(dep_fn)
local_points = face_blendshapes.compute_face_local_points(weights)
world_points = projector.billboard_world_points(anchor, camera_path, local_points, face_scale)
neck_index = constants.NAME_TO_INDEX["Body_Neck"]
if mapping_node.get_face_follow(dep_fn) == mapping_node.FACE_FOLLOW_NECK and neck_index in mapped:
world_points = projector.oriented_world_points(
anchor, mapped[neck_index], local_points, face_scale
)
else:
world_points = projector.billboard_world_points(anchor, camera_path, local_points, face_scale)
for offset, world_point in enumerate(world_points):
px, py, in_front = projector.project_point(world_point, camera_path, width, height)
keypoints[constants.FACE_START + offset] = (px, py, 1.0 if in_front else 0.0)