StencilGL 1.0

glStencilOp

Says what happens to the stored stencil value in each of the three outcomes of the stencil and depth tests.

Signature

void glStencilOp(GLenum sfail, GLenum dpfail, GLenum dppass);

How it works

glStencilFunc decides whether a fragment passes; glStencilOp decides what is written into the stencil buffer as a side effect. There are three cases, one per parameter:

sfail: the stencil test failed. dpfail: the stencil test passed but the depth test failed. dppass: both passed (or there is no depth test).

The default is GL_KEEP for all three, so the stencil buffer never changes. To paint a mask you usually set dppass to GL_REPLACE, so every pixel the object really covers gets the ref value.

Every fragment meets the stencil test first, then the depth test. glStencilFunc sets the first test; glStencilOp chooses what is written into the stencil buffer at each of the three exits: sfail, dpfail and dppass. A discarded fragment draws nothing, but its stencil action still runs.

Parameters

sfailGLenum

Action when the stencil test fails.

Common values

GL_KEEP: Leave the stored value as it is (default for all three).
GL_REPLACE: Write ref (from glStencilFunc). The usual way to paint a mask.
GL_ZERO: Write 0.
GL_INCR / GL_DECR: Add or subtract 1, stopping at the maximum (255 with 8 bits) or at 0.
GL_INCR_WRAP / GL_DECR_WRAP: Add or subtract 1, wrapping around (255 + 1 = 0). Shadow volumes count with these.
GL_INVERT: Flip every bit of the stored value.

dpfailGLenum

Action when the stencil test passes but the depth test fails.

dppassGLenum

Action when both tests pass.

Tips & pitfalls

Writes still go through glStencilMask: with mask 0x00 none of these actions changes anything.
To treat front and back faces differently (shadow volumes), use glStencilOpSeparate.

Errors

When the call is invalid it does nothing and sets an error, which glGetError returns:

GL_INVALID_ENUM: Any argument is not one of the eight actions.

Example

example.cpp

1glEnable(GL_STENCIL_TEST);
2glStencilOp(GL_KEEP, GL_KEEP, GL_REPLACE); // write ref where the object is drawn

Official Khronos documentation