Add OpenPose_full renderer plugin for Maya

Maya API 2.0 plugin that maps a skeleton to OpenPose_full keypoints
(BODY_25 or COCO body, 21+21 hand, 70 face) via an openposeCharacter
node, with a live Viewport 2.0 draw override, an "OpenPose" viewport
renderer plus openposeRenderSequence for PNG output, and a PySide2
mapping editor. Face can also be driven procedurally from ARKit's 52
blendshape weights instead of facial joints, for rigs with no facial
skeleton. Includes VS Code debug configs and Maya test scenes/renders.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-14 08:41:57 +08:00
commit e2f2e6668a
23 changed files with 2999 additions and 0 deletions
+176
View File
@@ -0,0 +1,176 @@
# OpenPose Renderer for Maya
Turns an animated Maya skeleton into OpenPose-style stick-figure images (up
to 25 BODY_25 body/foot + 21+21 hand + 70 face = 137 keypoints), for use
with ControlNet and similar tools. Built entirely on Maya's Python API 2.0.
Targets Maya 2022-2024 (Python 3.9/3.10, PySide2 -- PySide6 also detected
automatically if present).
Body output supports two OpenPose skeleton formats, selectable per
character (see *Body model* below): **BODY_25** (25 points, includes
MidHip + feet) and **COCO** (18 points, no MidHip/feet, Neck connects
directly to each hip -- OpenPose's original/legacy body model). Both use
the exact same joint mapping; switching formats does not require remapping.
## Install
1. Copy (or symlink) this whole `PoseRenderer` folder somewhere on disk.
2. Add its `plug-ins` folder to `MAYA_PLUG_IN_PATH`, e.g. in
`Maya.env`:
```
MAYA_PLUG_IN_PATH = C:\path\to\PoseRenderer\plug-ins
```
(The plugin adds its own `python/` folder to `sys.path` automatically at
load time -- you do not need to set `PYTHONPATH` yourself.)
3. In Maya: **Windows > Settings/Preferences > Plug-in Manager**, find
`openposeRenderer.py`, and check *Loaded* (and *Auto load* if desired).
## Usage
### 1. Map a character
- Run `cmds.openposeCreateCharacter(name="myCharacter")` (or use the
mapping editor's *Create Character* button) to create one
`openposeCharacter` node per person you want to output. Scenes with
multiple `openposeCharacter` nodes render as multi-person OpenPose output.
- Open the mapping editor: `cmds.openposeShowMappingEditor()`.
- Select the character node in the dropdown, select a joint in the
viewport, click *Set from Selected* next to the OpenPose_full slot it
corresponds to. Repeat for as many of the 137 slots as your rig has
joints for -- unmapped slots simply render with zero confidence (no
point/limb drawn), matching how OpenPose itself represents undetected
keypoints. Face and hand slots are optional; body-only rigs work fine.
- Mappings are plain Maya connections (`joint.message ->
openposeCharacter.targetJoint[i]`), so they save with the scene and
survive joint renames/reparenting.
### 2. Viewport preview
With an `openposeCharacter` node in the scene, a colored OpenPose-style
skeleton overlay is drawn live over the mapped joints in every viewport
(toggle per-character via the node's `enableDisplay` attribute; tune with
`jointRadiusScale` / `limbThicknessScale`). This is for QA of the mapping,
independent of rendering.
### Body model (BODY_25 / COCO)
Each `openposeCharacter` node has a `bodyModel` enum attribute (`BODY_25`,
the default, or `COCO`), settable in the mapping editor's *Body Model*
dropdown or directly via `cmds.setAttr("myCharacter.bodyModel", 1)` (0 =
BODY_25, 1 = COCO). It controls both the live viewport overlay and the
rendered output for that character:
- **BODY_25**: all 25 body/foot points -- MidHip, hips/knees/ankles, eyes,
ears, and the 6 foot points (heels + big/small toes).
- **COCO**: the original 18-point OpenPose body model -- no MidHip, no
feet, and the neck connects directly to each hip instead of through a
pelvis point. Slots you mapped for MidHip/feet are simply not drawn in
this mode; nothing needs to be remapped to switch.
Hands and face are unaffected by this setting either way. Rendered from the
same mapped HumanIK-named test rig (`test/openpose_humanik_test.ma`):
| BODY_25 | COCO |
| --- | --- |
| ![BODY_25 output](test/openpose_humanik_test_preview.png) | ![COCO output](test/openpose_humanik_test_preview_coco.png) |
### Face source: facial joints, or ARKit 52 blendshapes
Most facial rigs have no joints at all -- they're driven entirely by
blendShape targets, often named per Apple's ARKit convention (from an
iPhone/TrueDepth mocap pipeline, Live Link Face, or a hand-keyed rig built
to match it). Each `openposeCharacter` has a `faceSource` enum (mapping
editor's *Face Source* dropdown, or `cmds.setAttr("myCharacter.faceSource",
1)`; 0 = `Joints`, 1 = `ARKitBlendshapes`) controlling where its 70 face
keypoints come from:
- **Joints** (default): unchanged from above -- map `Face_*` slots to
facial joints if your rig has them.
- **ARKitBlendshapes**: the 70 face keypoints are instead computed
procedurally from 52 connectable `blendshapeWeight[0..51]` float slots
(in Apple's canonical `ARFaceAnchor.BlendShapeLocation` order/naming --
`eyeBlinkLeft`, `jawOpen`, `mouthSmileLeft`, etc.). Select your
blendShape node (or an ARKit-mocap-driven control) and click *Auto-Connect
ARKit Blendshapes from Selected* in the mapping editor -- it connects
every one of the 52 canonical names that exists as an attribute on the
selected node. The resulting face is anchored at the character's mapped
`Body_Nose` joint and billboards to always face whichever camera is
drawing (viewport or render camera), sized by the `faceScale` attribute
(world units; default 15 -- tune to your scene's unit scale).
**Accuracy caveat:** there is no published, authoritative "which ARKit
weight moves which OpenPose landmark, by how much" table anywhere (unlike
BODY_25/COCO, which OpenPose's own source defines exactly). The neutral
face template and per-blendshape displacement rules in
`face_blendshapes.py` are a hand-authored, geometrically-reasoned
approximation -- each shape nudges only the anatomically obvious nearby
landmarks in an intuitive direction. It produces a recognizable, responsive
OpenPose-style face suitable for ControlNet conditioning, but is not a
biomechanically precise simulation. `jawForward` and `tongueOut` have no
representable effect on a frontal 2D landmark set and are no-ops.
Rendered from `test/openpose_arkit_face_test.ma` (a control node with all 52
ARKit-named attributes, auto-connected -- see that scene for the pattern):
| Neutral | `jawOpen` + smile + raised brows |
| --- | --- |
| ![Neutral face](test/openpose_arkit_face_test_neutral.png) | ![Expression face](test/openpose_arkit_face_test_expression.png) |
### 3. Render OpenPose images
- Switch a viewport panel's **Renderer** menu to **OpenPose** to make it
the active/selectable renderer for that panel (this is the Viewport 2.0
render override integration point).
- To actually generate the OpenPose PNG sequence, run:
```python
cmds.openposeRenderSequence(
startFrame=1, endFrame=48,
camera="renderCam", # optional; defaults to the active view's camera
outputDirectory="C:/out/openpose", # optional; defaults to <workspace>/images/openpose
showInRenderView=True,
)
```
Each frame is projected through the given camera at the scene's render
resolution (`defaultResolution` node), rasterized as a black-background,
OpenPose-colored image (limbs under joints, canonical BODY_25 palette,
per-finger rainbow hand colors, white face dots), written to disk, and
pushed into Maya's Render View so you can watch it render like any other
renderer.
## Debugging (VS Code)
`.vscode/launch.json` provides:
- **`Maya: Attach (debugpy)`** -- attaches to a running, interactive Maya.
One-time setup: `"<Maya install>\bin\mayapy.exe" -m pip install debugpy`.
Each session, run `scripts/start_debug_server.py` inside Maya's Script
Editor first (see that file's docstring), then launch this config from
VS Code to hit breakpoints anywhere in `plug-ins/` or
`python/openpose_renderer/` -- including the draw override and render
override callbacks, which only run inside a real Maya viewport.
- **`mayapy 2022/2023/2024: Run current file`** -- runs the file you have
open through that Maya version's own `mayapy.exe` interpreter (with
`python/` on `PYTHONPATH`), for headless scripts that call
`maya.standalone.initialize()` themselves. Useful for quick iteration on
`constants.py`/`projector.py`/`rasterizer.py` without opening Maya's UI.
## Notes / known limitations
- Colors/connectivity: BODY_25 and COCO keypoint layout and limb
connectivity are taken verbatim from OpenPose's own
`POSE_BODY_25_BODY_PART_PAIRS` / COCO section of `POSE_BODY_PART_PAIRS`
(`src/openpose/pose/poseParameters.cpp`). BODY_25's colors are likewise
verbatim from `POSE_BODY_25_COLORS_RENDER_GPU`. COCO limb colors, and all
hand/face colors, are generated from an HSV hue wheel (rainbow per limb
or finger, white face dots) reproducing OpenPose's own visual scheme
rather than a hardcoded transcription of its longer, generated gradient
tables.
- Projection does not attempt to replicate Maya's film-gate/overscan/fit
reconciliation pixel-for-pixel -- for correct results, set the render
camera's film aspect ratio to match the scene's render resolution, as you
would for any Maya render.
- The "OpenPose" Renderer-menu entry uses a standard scene/HUD/present
Viewport 2.0 operation chain (so switching to it always leaves the panel
usable); the actual OpenPose image generation happens via
`openposeRenderSequence`, not via GPU render-target pixel substitution
inside the viewport override.