Add faceFollow, fix face contour rendering, add default eyes/ears
Three changesets bundled together: - faceFollow (Camera/Neck) for the ARKit-blendshape face: it can now rotate with the mapped Body_Neck joint's own local axes instead of always billboarding toward the camera. - Fixed a real correctness bug: the face was rendering as isolated dots. Verified against OpenPose's own published keypoint diagrams and source (FACE_PAIRS_RENDER_GPU, 63 pairs) that it should render connected jaw/eyebrow/nose/eye/mouth contour lines; full_limb_data() was also silently dropping face limbs entirely. Both fixed. - Default eyes/ears: most rigs (HumanIK included) have no REye/LEye/REar/LEar joints, only Head/Nose, leaving those 4 slots permanently invisible. They now get a synthetic head-relative position when unmapped but Body_Nose is mapped, anchored at and rotating with the Nose joint's own orientation (not the camera) -- toggleable via useDefaultEyesEars, always overridden by an explicit joint mapping. Also drops the RShoulder->REar / LShoulder->LEar lines from BODY_25 and COCO connectivity by request (a detection-robustness quirk of OpenPose's original network, not real anatomy) so ears stay leaf points -- a disclosed, deliberate deviation from upstream's otherwise-verbatim connectivity. All verified against real Maya (2022/2023/2024), not just logic: rotation math checked exact, explicit-mapping-overrides-fallback checked, shoulder-ear removal checked against full_limb_data() output, rendered test scenes visually confirmed against OpenPose's own reference diagrams. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
@@ -55,20 +55,18 @@ def project_point(world_point, camera_path, width, height):
|
||||
return pixel_x, pixel_y, in_front
|
||||
|
||||
|
||||
def billboard_world_points(anchor_world_point, camera_path, local_points_xy, scale):
|
||||
"""Place 2D local (x, y) points on a plane centered at ``anchor_world_point``
|
||||
that always faces ``camera_path`` (camera-space X/Y used as the plane's
|
||||
right/up axes), scaled by ``scale``.
|
||||
|
||||
Used to draw/render the procedural ARKit-blendshape face without needing
|
||||
any facial joint orientation -- it billboards toward whichever camera is
|
||||
currently drawing (the active viewport camera for the live overlay, the
|
||||
render camera for output), sidestepping the rig-specific, unanswerable
|
||||
question of "which way does this head joint's local frame face".
|
||||
"""
|
||||
world_matrix = camera_path.inclusiveMatrix()
|
||||
def local_axes_world(dag_path):
|
||||
"""World-space (right, up) unit vectors from a DAG node's local X/Y axes."""
|
||||
world_matrix = dag_path.inclusiveMatrix()
|
||||
right = (om.MVector(1.0, 0.0, 0.0) * world_matrix).normal()
|
||||
up = (om.MVector(0.0, 1.0, 0.0) * world_matrix).normal()
|
||||
return right, up
|
||||
|
||||
|
||||
def plane_world_points(anchor_world_point, right, up, local_points_xy, scale):
|
||||
"""Place 2D local (x, y) points on a plane centered at ``anchor_world_point``
|
||||
using the given world-space ``right``/``up`` axes, scaled by ``scale``.
|
||||
"""
|
||||
anchor = om.MPoint(anchor_world_point)
|
||||
return [
|
||||
om.MPoint(anchor + right * (x * scale) + up * (y * scale))
|
||||
@@ -76,6 +74,51 @@ def billboard_world_points(anchor_world_point, camera_path, local_points_xy, sca
|
||||
]
|
||||
|
||||
|
||||
def billboard_world_points(anchor_world_point, camera_path, local_points_xy, scale):
|
||||
"""``plane_world_points`` using ``camera_path``'s own local X/Y as
|
||||
right/up, so the plane always faces whichever camera is drawing.
|
||||
|
||||
Used to draw/render the procedural ARKit-blendshape face without needing
|
||||
any facial joint orientation -- it billboards toward whichever camera is
|
||||
currently drawing (the active viewport camera for the live overlay, the
|
||||
render camera for output), sidestepping the rig-specific, unanswerable
|
||||
question of "which way does this head joint's local frame face". This is
|
||||
the "Camera" faceFollow mode.
|
||||
"""
|
||||
right, up = local_axes_world(camera_path)
|
||||
return plane_world_points(anchor_world_point, right, up, local_points_xy, scale)
|
||||
|
||||
|
||||
def oriented_world_points(anchor_world_point, orientation_dag_path, local_points_xy, scale):
|
||||
"""``plane_world_points`` using ``orientation_dag_path``'s own local X/Y
|
||||
as right/up, so the plane rotates *with* that joint (e.g. the mapped
|
||||
Neck) instead of always facing the camera. This is the "Neck" faceFollow
|
||||
mode -- unlike "Camera", it is rig-dependent: it assumes the joint's
|
||||
local X is the character's right and local Y is up, which matches
|
||||
common convention but is not guaranteed for every rig. If the face
|
||||
appears rotated relative to the head, adjust the neck joint's rotate
|
||||
axis/orient, or use "Camera" mode instead.
|
||||
"""
|
||||
right, up = local_axes_world(orientation_dag_path)
|
||||
return plane_world_points(anchor_world_point, right, up, local_points_xy, scale)
|
||||
|
||||
|
||||
def default_head_relative_points(anchor_world_point, orientation_dag_path, scale, local_offsets):
|
||||
"""Compute head-relative fallback world points (e.g. the default
|
||||
REye/LEye/REar/LEar positions in constants.DEFAULT_EYE_EAR_LOCAL_OFFSETS)
|
||||
anchored at ``anchor_world_point`` (the mapped Nose), oriented by
|
||||
``orientation_dag_path``'s own local X/Y axes -- always the joint
|
||||
mapped to Nose itself (the head), so these points turn with the head's
|
||||
actual rotation rather than billboarding toward the camera.
|
||||
|
||||
``local_offsets``: {index: (x, y)}. Returns {index: MPoint}.
|
||||
"""
|
||||
indices = list(local_offsets.keys())
|
||||
offsets = [local_offsets[i] for i in indices]
|
||||
world_points = oriented_world_points(anchor_world_point, orientation_dag_path, offsets, scale)
|
||||
return dict(zip(indices, world_points))
|
||||
|
||||
|
||||
def project_character(mapped_joint_paths, camera_path, width, height):
|
||||
"""Project one character's mapping to OpenPose_full pixel keypoints.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user