PoseRenderer/README.md

OpenPose Renderer for Maya

Turns an animated Maya skeleton into OpenPose-style stick-figure images (up to 25 BODY_25 body/foot + 21+21 hand + 70 face = 137 keypoints), for use with ControlNet and similar tools. Built entirely on Maya's Python API 2.0. Targets Maya 2022-2024 (Maya 2022 = Python 3.7, 2023 = Python 3.9, 2024 = Python 3.10; PySide2 -- PySide6 also detected automatically if present).

Body output supports two OpenPose skeleton formats, selectable per character (see Body model below): BODY_25 (25 points, includes MidHip + feet) and COCO (18 points, no MidHip/feet, Neck connects directly to each hip -- OpenPose's original/legacy body model). Both use the exact same joint mapping; switching formats does not require remapping.

Install

Option A -- Maya module (recommended): copy (or symlink) this whole PoseRenderer folder somewhere on disk, then add its parent directory to MAYA_MODULE_PATH (e.g. in Maya.env):

MAYA_MODULE_PATH = C:\path\to\parent\folder

PoseRenderer.mod (at the repo root) handles the rest: it registers the module as PoseRenderer, adds plug-ins/ to MAYA_PLUG_IN_PATH and python/ to PYTHONPATH automatically. In Maya: Windows > Settings/Preferences > Plug-in Manager, find openposeRenderer.py, and check Loaded (and Auto load if desired) -- it's now loadable by its short name (cmds.loadPlugin("openposeRenderer")) from anywhere.

Option B -- plug-in path only: skip the module system and add plug-ins/ directly to MAYA_PLUG_IN_PATH instead:

MAYA_PLUG_IN_PATH = C:\path\to\PoseRenderer\plug-ins

The plugin adds its own python/ folder to sys.path automatically at load time either way (via MFnPlugin.loadPath()), so PYTHONPATH never needs setting by hand for the plugin itself to work -- Option A's PYTHONPATH entry only matters if you want python/openpose_renderer (or the bundled debugpy) importable outside of loading the plugin, e.g. from a shelf script.

Usage

1. Map a character

• Run cmds.openposeCreateCharacter(name="myCharacter") (or use the mapping editor's Create Character button) to create one openposeCharacter node per person you want to output. Scenes with multiple openposeCharacter nodes render as multi-person OpenPose output.
• Open the mapping editor: cmds.openposeShowMappingEditor().
• Select the character node in the dropdown, select a joint in the viewport, click Set from Selected next to the OpenPose_full slot it corresponds to. Repeat for as many of the 137 slots as your rig has joints for -- unmapped slots simply render with zero confidence (no point/limb drawn), matching how OpenPose itself represents undetected keypoints. Face and hand slots are optional; body-only rigs work fine.
• Mappings are plain Maya connections (joint.message -> openposeCharacter.targetJoint[i]), so they save with the scene and survive joint renames/reparenting.

2. Viewport preview

With an openposeCharacter node in the scene, a colored OpenPose-style skeleton overlay is drawn live over the mapped joints in every viewport (toggle per-character via the node's enableDisplay attribute; tune with jointRadiusScale / limbThicknessScale). This is for QA of the mapping, independent of rendering.

Body model (BODY_25 / COCO)

Each openposeCharacter node has a bodyModel enum attribute (BODY_25, the default, or COCO), settable in the mapping editor's Body Model dropdown or directly via cmds.setAttr("myCharacter.bodyModel", 1) (0 = BODY_25, 1 = COCO). It controls both the live viewport overlay and the rendered output for that character:

• BODY_25: all 25 body/foot points -- MidHip, hips/knees/ankles, eyes, ears, and the 6 foot points (heels + big/small toes).
• COCO: the original 18-point OpenPose body model -- no MidHip, no feet, and the neck connects directly to each hip instead of through a pelvis point. Slots you mapped for MidHip/feet are simply not drawn in this mode; nothing needs to be remapped to switch.

Hands and face are unaffected by this setting either way.

Default eyes/ears

Most rigs (including HumanIK) have no REye/LEye/REar/LEar joints -- only a Head/Nose. By default (useDefaultEyesEars, on unless turned off), whenever any of those 4 slots is unmapped but Body_Nose is mapped, it gets a synthetic head-relative position instead of being invisible -- anchored at and rotating with the joint mapped to Body_Nose itself (sized by faceScale), so they turn with the head's actual rotation rather than billboarding toward the camera. An explicit joint mapped to any of the 4 slots always overrides this fallback. The offsets themselves are a reasonable approximation of human proportions, not from a published source.

Note also: BODY_25/COCO's RShoulder->REar / LShoulder->LEar limb lines (present in OpenPose's own spec -- a detection-robustness feature of the original network, not a real anatomical link) are intentionally not drawn here; ears connect only via eye->ear, staying leaf points. This is a deliberate deviation from upstream, by request -- see the note in constants.py.

Rendered from the same mapped HumanIK-named test rig (test/openpose_humanik_test.ma):

BODY_25	COCO

Face source: facial joints, or ARKit 52 blendshapes

Most facial rigs have no joints at all -- they're driven entirely by blendShape targets, often named per Apple's ARKit convention (from an iPhone/TrueDepth mocap pipeline, Live Link Face, or a hand-keyed rig built to match it). Each openposeCharacter has a faceSource enum (mapping editor's Face Source dropdown, or cmds.setAttr("myCharacter.faceSource", 1); 0 = Joints, 1 = ARKitBlendshapes) controlling where its 70 face keypoints come from:

• Joints (default): unchanged from above -- map Face_* slots to facial joints if your rig has them.

• ARKitBlendshapes: the 70 face keypoints are instead computed procedurally from 52 connectable blendshapeWeight[0..51] float slots (in Apple's canonical ARFaceAnchor.BlendShapeLocation order/naming -- eyeBlinkLeft, jawOpen, mouthSmileLeft, etc.). Select your blendShape node (or an ARKit-mocap-driven control) and click Auto-Connect ARKit Blendshapes from Selected in the mapping editor -- it connects every one of the 52 canonical names that exists as an attribute on the selected node. The resulting face is anchored at the character's mapped Body_Nose joint, sized by the faceScale attribute (world units; default 15 -- tune to your scene's unit scale).

  A faceFollow enum (mapping editor's Face Follow dropdown, or cmds.setAttr("myCharacter.faceFollow", 1); 0 = Camera, 1 = Neck) controls the face plane's orientation:

  • Camera (default): always faces whichever camera is drawing (viewport or render camera) -- works regardless of rig conventions, since it never has to guess which way a joint's local axes point.
  • Neck: rotates with the character's mapped Body_Neck joint instead, using that joint's own local X/Y axes as the face plane's right/up -- so the face turns with the head. This is rig-dependent (it assumes local X = right, local Y = up, which is common but not universal); if the face looks rotated relative to the head, adjust the neck joint's rotate axis, or use Camera instead. Falls back to Camera if no joint is mapped to Body_Neck.

Accuracy caveat: there is no published, authoritative "which ARKit weight moves which OpenPose landmark, by how much" table anywhere (unlike BODY_25/COCO, which OpenPose's own source defines exactly). The neutral face template and per-blendshape displacement rules in face_blendshapes.py are a hand-authored, geometrically-reasoned approximation -- each shape nudges only the anatomically obvious nearby landmarks in an intuitive direction. It produces a recognizable, responsive OpenPose-style face suitable for ControlNet conditioning, but is not a biomechanically precise simulation. jawForward and tongueOut have no representable effect on a frontal 2D landmark set and are no-ops.

Rendered from test/openpose_arkit_face_test.ma (a control node with all 52 ARKit-named attributes, auto-connected -- see that scene for the pattern):

Neutral	jawOpen + smile + raised brows	faceFollow=Neck, neck tilted 25°

3. Render OpenPose images

• Switch a viewport panel's Renderer menu to OpenPose to make it the active/selectable renderer for that panel (this is the Viewport 2.0 render override integration point).
• To actually generate the OpenPose PNG sequence, run:

  cmds.openposeRenderSequence(
      startFrame=1, endFrame=48,
      camera="renderCam",              # optional; defaults to the active view's camera
      outputDirectory="C:/out/openpose",  # optional; defaults to <workspace>/images/openpose
      showInRenderView=True,
  )
  

  Each frame is projected through the given camera at the scene's render resolution (defaultResolution node), rasterized as a black-background, OpenPose-colored image (limbs under joints, canonical BODY_25 palette, per-finger rainbow hand colors, white face contour lines), written to disk, and pushed into Maya's Render View so you can watch it render like any other renderer.

Debugging (VS Code)

.vscode/launch.json provides:

• Maya: Attach (debugpy) -- attaches to a running, interactive Maya. debugpy is bundled in python/ (no separate pip install needed -- verified importable under all three target Maya Python versions, 3.7/3.9/3.10). Each session, run scripts/start_debug_server.py inside Maya's Script Editor first (see that file's docstring), then launch this config from VS Code to hit breakpoints anywhere in plug-ins/ or python/openpose_renderer/ -- including the draw override and render override callbacks, which only run inside a real Maya viewport.
• mayapy 2022/2023/2024: Run current file -- runs the file you have open through that Maya version's own mayapy.exe interpreter (with python/ on PYTHONPATH), for headless scripts that call maya.standalone.initialize() themselves. Useful for quick iteration on constants.py/projector.py/rasterizer.py without opening Maya's UI.

Notes / known limitations

• Colors/connectivity: BODY_25 and COCO keypoint layout and limb connectivity are taken from OpenPose's own POSE_BODY_25_BODY_PART_PAIRS / COCO section of POSE_BODY_PART_PAIRS (src/openpose/pose/poseParameters.cpp), with BODY_25's colors likewise verbatim from POSE_BODY_25_COLORS_RENDER_GPU -- with one deliberate, by-request deviation: the RShoulder->REar / LShoulder->LEar lines upstream includes are dropped, so ears stay leaf points connected only via eye->ear (see "Default eyes/ears" above). The face's 63 contour lines (jaw/eyebrows/nose/eyes/mouth) and solid-white color are equally verbatim, from FACE_PAIRS_RENDER_GPU / FACE_COLORS_RENDER_GPU (include/openpose/face/faceParameters.hpp) -- independently confirmed against OpenPose's own published keypoint diagrams. COCO limb colors and all hand colors are generated from an HSV hue wheel (rainbow per limb or finger) reproducing OpenPose's own visual scheme rather than a hardcoded transcription of its longer, generated gradient tables.
• Projection does not attempt to replicate Maya's film-gate/overscan/fit reconciliation pixel-for-pixel -- for correct results, set the render camera's film aspect ratio to match the scene's render resolution, as you would for any Maya render.
• The "OpenPose" Renderer-menu entry uses a standard scene/HUD/present Viewport 2.0 operation chain (so switching to it always leaves the panel usable); the actual OpenPose image generation happens via openposeRenderSequence, not via GPU render-target pixel substitution inside the viewport override.