indigo
ec5f52c7c6
Bundles debugpy 1.7.0 directly in python/ so "Maya: Attach (debugpy)" works with no per-machine pip install step. Installed via Maya 2022's own pip so it resolved a version actually compatible with Python 3.7 (Maya 2022's interpreter), then verified import + listen() succeeds under all three target Maya Python versions (3.7/3.9/3.10). Also fixes a latent __file__-under-exec() bug in start_debug_server.py (same pitfall as Maya's own plugin loader, never hit until this exercised it) and corrects the README's Maya Python version claim -- 2022 ships Python 3.7, not 3.9 as previously stated, which was never independently verified until now. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
178 lines
9.0 KiB
Markdown
178 lines
9.0 KiB
Markdown
# OpenPose Renderer for Maya
|
|
|
|
Turns an animated Maya skeleton into OpenPose-style stick-figure images (up
|
|
to 25 BODY_25 body/foot + 21+21 hand + 70 face = 137 keypoints), for use
|
|
with ControlNet and similar tools. Built entirely on Maya's Python API 2.0.
|
|
Targets Maya 2022-2024 (Maya 2022 = Python 3.7, 2023 = Python 3.9, 2024 =
|
|
Python 3.10; PySide2 -- PySide6 also detected automatically if present).
|
|
|
|
Body output supports two OpenPose skeleton formats, selectable per
|
|
character (see *Body model* below): **BODY_25** (25 points, includes
|
|
MidHip + feet) and **COCO** (18 points, no MidHip/feet, Neck connects
|
|
directly to each hip -- OpenPose's original/legacy body model). Both use
|
|
the exact same joint mapping; switching formats does not require remapping.
|
|
|
|
## Install
|
|
|
|
1. Copy (or symlink) this whole `PoseRenderer` folder somewhere on disk.
|
|
2. Add its `plug-ins` folder to `MAYA_PLUG_IN_PATH`, e.g. in
|
|
`Maya.env`:
|
|
```
|
|
MAYA_PLUG_IN_PATH = C:\path\to\PoseRenderer\plug-ins
|
|
```
|
|
(The plugin adds its own `python/` folder to `sys.path` automatically at
|
|
load time -- you do not need to set `PYTHONPATH` yourself.)
|
|
3. In Maya: **Windows > Settings/Preferences > Plug-in Manager**, find
|
|
`openposeRenderer.py`, and check *Loaded* (and *Auto load* if desired).
|
|
|
|
## Usage
|
|
|
|
### 1. Map a character
|
|
|
|
- Run `cmds.openposeCreateCharacter(name="myCharacter")` (or use the
|
|
mapping editor's *Create Character* button) to create one
|
|
`openposeCharacter` node per person you want to output. Scenes with
|
|
multiple `openposeCharacter` nodes render as multi-person OpenPose output.
|
|
- Open the mapping editor: `cmds.openposeShowMappingEditor()`.
|
|
- Select the character node in the dropdown, select a joint in the
|
|
viewport, click *Set from Selected* next to the OpenPose_full slot it
|
|
corresponds to. Repeat for as many of the 137 slots as your rig has
|
|
joints for -- unmapped slots simply render with zero confidence (no
|
|
point/limb drawn), matching how OpenPose itself represents undetected
|
|
keypoints. Face and hand slots are optional; body-only rigs work fine.
|
|
- Mappings are plain Maya connections (`joint.message ->
|
|
openposeCharacter.targetJoint[i]`), so they save with the scene and
|
|
survive joint renames/reparenting.
|
|
|
|
### 2. Viewport preview
|
|
|
|
With an `openposeCharacter` node in the scene, a colored OpenPose-style
|
|
skeleton overlay is drawn live over the mapped joints in every viewport
|
|
(toggle per-character via the node's `enableDisplay` attribute; tune with
|
|
`jointRadiusScale` / `limbThicknessScale`). This is for QA of the mapping,
|
|
independent of rendering.
|
|
|
|
### Body model (BODY_25 / COCO)
|
|
|
|
Each `openposeCharacter` node has a `bodyModel` enum attribute (`BODY_25`,
|
|
the default, or `COCO`), settable in the mapping editor's *Body Model*
|
|
dropdown or directly via `cmds.setAttr("myCharacter.bodyModel", 1)` (0 =
|
|
BODY_25, 1 = COCO). It controls both the live viewport overlay and the
|
|
rendered output for that character:
|
|
|
|
- **BODY_25**: all 25 body/foot points -- MidHip, hips/knees/ankles, eyes,
|
|
ears, and the 6 foot points (heels + big/small toes).
|
|
- **COCO**: the original 18-point OpenPose body model -- no MidHip, no
|
|
feet, and the neck connects directly to each hip instead of through a
|
|
pelvis point. Slots you mapped for MidHip/feet are simply not drawn in
|
|
this mode; nothing needs to be remapped to switch.
|
|
|
|
Hands and face are unaffected by this setting either way. Rendered from the
|
|
same mapped HumanIK-named test rig (`test/openpose_humanik_test.ma`):
|
|
|
|
| BODY_25 | COCO |
|
|
| --- | --- |
|
|
|  |  |
|
|
|
|
### Face source: facial joints, or ARKit 52 blendshapes
|
|
|
|
Most facial rigs have no joints at all -- they're driven entirely by
|
|
blendShape targets, often named per Apple's ARKit convention (from an
|
|
iPhone/TrueDepth mocap pipeline, Live Link Face, or a hand-keyed rig built
|
|
to match it). Each `openposeCharacter` has a `faceSource` enum (mapping
|
|
editor's *Face Source* dropdown, or `cmds.setAttr("myCharacter.faceSource",
|
|
1)`; 0 = `Joints`, 1 = `ARKitBlendshapes`) controlling where its 70 face
|
|
keypoints come from:
|
|
|
|
- **Joints** (default): unchanged from above -- map `Face_*` slots to
|
|
facial joints if your rig has them.
|
|
- **ARKitBlendshapes**: the 70 face keypoints are instead computed
|
|
procedurally from 52 connectable `blendshapeWeight[0..51]` float slots
|
|
(in Apple's canonical `ARFaceAnchor.BlendShapeLocation` order/naming --
|
|
`eyeBlinkLeft`, `jawOpen`, `mouthSmileLeft`, etc.). Select your
|
|
blendShape node (or an ARKit-mocap-driven control) and click *Auto-Connect
|
|
ARKit Blendshapes from Selected* in the mapping editor -- it connects
|
|
every one of the 52 canonical names that exists as an attribute on the
|
|
selected node. The resulting face is anchored at the character's mapped
|
|
`Body_Nose` joint and billboards to always face whichever camera is
|
|
drawing (viewport or render camera), sized by the `faceScale` attribute
|
|
(world units; default 15 -- tune to your scene's unit scale).
|
|
|
|
**Accuracy caveat:** there is no published, authoritative "which ARKit
|
|
weight moves which OpenPose landmark, by how much" table anywhere (unlike
|
|
BODY_25/COCO, which OpenPose's own source defines exactly). The neutral
|
|
face template and per-blendshape displacement rules in
|
|
`face_blendshapes.py` are a hand-authored, geometrically-reasoned
|
|
approximation -- each shape nudges only the anatomically obvious nearby
|
|
landmarks in an intuitive direction. It produces a recognizable, responsive
|
|
OpenPose-style face suitable for ControlNet conditioning, but is not a
|
|
biomechanically precise simulation. `jawForward` and `tongueOut` have no
|
|
representable effect on a frontal 2D landmark set and are no-ops.
|
|
|
|
Rendered from `test/openpose_arkit_face_test.ma` (a control node with all 52
|
|
ARKit-named attributes, auto-connected -- see that scene for the pattern):
|
|
|
|
| Neutral | `jawOpen` + smile + raised brows |
|
|
| --- | --- |
|
|
|  |  |
|
|
|
|
### 3. Render OpenPose images
|
|
|
|
- Switch a viewport panel's **Renderer** menu to **OpenPose** to make it
|
|
the active/selectable renderer for that panel (this is the Viewport 2.0
|
|
render override integration point).
|
|
- To actually generate the OpenPose PNG sequence, run:
|
|
```python
|
|
cmds.openposeRenderSequence(
|
|
startFrame=1, endFrame=48,
|
|
camera="renderCam", # optional; defaults to the active view's camera
|
|
outputDirectory="C:/out/openpose", # optional; defaults to <workspace>/images/openpose
|
|
showInRenderView=True,
|
|
)
|
|
```
|
|
Each frame is projected through the given camera at the scene's render
|
|
resolution (`defaultResolution` node), rasterized as a black-background,
|
|
OpenPose-colored image (limbs under joints, canonical BODY_25 palette,
|
|
per-finger rainbow hand colors, white face dots), written to disk, and
|
|
pushed into Maya's Render View so you can watch it render like any other
|
|
renderer.
|
|
|
|
## Debugging (VS Code)
|
|
|
|
`.vscode/launch.json` provides:
|
|
|
|
- **`Maya: Attach (debugpy)`** -- attaches to a running, interactive Maya.
|
|
`debugpy` is bundled in `python/` (no separate pip install needed --
|
|
verified importable under all three target Maya Python versions,
|
|
3.7/3.9/3.10). Each session, run `scripts/start_debug_server.py` inside
|
|
Maya's Script Editor first (see that file's docstring), then launch this
|
|
config from VS Code to hit breakpoints anywhere in `plug-ins/` or
|
|
`python/openpose_renderer/` -- including the draw override and render
|
|
override callbacks, which only run inside a real Maya viewport.
|
|
- **`mayapy 2022/2023/2024: Run current file`** -- runs the file you have
|
|
open through that Maya version's own `mayapy.exe` interpreter (with
|
|
`python/` on `PYTHONPATH`), for headless scripts that call
|
|
`maya.standalone.initialize()` themselves. Useful for quick iteration on
|
|
`constants.py`/`projector.py`/`rasterizer.py` without opening Maya's UI.
|
|
|
|
## Notes / known limitations
|
|
|
|
- Colors/connectivity: BODY_25 and COCO keypoint layout and limb
|
|
connectivity are taken verbatim from OpenPose's own
|
|
`POSE_BODY_25_BODY_PART_PAIRS` / COCO section of `POSE_BODY_PART_PAIRS`
|
|
(`src/openpose/pose/poseParameters.cpp`). BODY_25's colors are likewise
|
|
verbatim from `POSE_BODY_25_COLORS_RENDER_GPU`. COCO limb colors, and all
|
|
hand/face colors, are generated from an HSV hue wheel (rainbow per limb
|
|
or finger, white face dots) reproducing OpenPose's own visual scheme
|
|
rather than a hardcoded transcription of its longer, generated gradient
|
|
tables.
|
|
- Projection does not attempt to replicate Maya's film-gate/overscan/fit
|
|
reconciliation pixel-for-pixel -- for correct results, set the render
|
|
camera's film aspect ratio to match the scene's render resolution, as you
|
|
would for any Maya render.
|
|
- The "OpenPose" Renderer-menu entry uses a standard scene/HUD/present
|
|
Viewport 2.0 operation chain (so switching to it always leaves the panel
|
|
usable); the actual OpenPose image generation happens via
|
|
`openposeRenderSequence`, not via GPU render-target pixel substitution
|
|
inside the viewport override.
|