While standard text-to-image prompting in Stable Diffusion allows you to control the subject and theme, it often fails at directing spatial details. If you want a character to strike a specific dynamic pose, or keep the exact geometric layout of a building, you need **ControlNet**. ControlNet is a neural network structure that controls Stable Diffusion by adding extra conditions like edges, depth maps, or human skeletons.
1. The Key ControlNet Models
Depending on your project's visual goals, you must choose the appropriate ControlNet preprocessor and model:
- OpenPose: Detects and copies human skeleton points (limbs, head, hands). Essential for action scenes, character sheets, and specific poses.
- Depth: Extracts the 3D depth of a reference image (closer items are lighter, background is darker). Perfect for maintaining room layouts, landscapes, and spatial relationships.
- Canny: Extracts clean outline edges of an image. Best for coloring sketches, logo modifications, and technical architecture renders.
2. How to Align Prompts with ControlNet
ControlNet controls the structure, but your text prompt still controls the styling, colors, and textures. You must keep your prompt aligned with the reference input:
Tip: If you upload a pose reference of a running athlete via OpenPose, your prompt must explicitly contain "a man running" or "dynamic action pose". If the text prompt contradicts the ControlNet structure, the image will display weird distortions or low quality.3. Adjusting Control Weights
If you want the AI to follow the reference image structure loosely, reduce the **Control Weight** slider to 0.6 or 0.7. This gives Stable Diffusion the creative freedom to add natural lighting and textures. Conversely, set it to 1.0 or higher for strict logo templates or technical line-art renderings.