Tessellation And Displacement

Tessellation is a technique that can take a coarse, low-polygon mesh and render it completely smooth. This is achieved with polygonal subdivision at render-time. Working with low-polygon meshes and letting Redshift subdivide at render time has certain advantages:

It can be more memory-efficient (which is important for GPUs) when combined with view-dependent and/or adaptive subdivision. Small or distant objects, for example, will look smooth enough with fewer subdivisions than objects that are close to the camera. Redshift supports screen-space adaptive and world-space tessellation for improved memory usage.

Displacement is a feature typically combined with tessellation. It allows the user to add extra detail to a mesh through shader networks, i.e. textures, noise shader nodes, etc.

Redshift supports both height field (displacing along the vertex normal) and vector displacement maps (in object or tangent space). Importantly, any displacement detail that can't be represented with the current tessellation settings can instead be represented using bump mapping – therefore a good level of surface detail can be present even in fairly low-quality tessellation settings.

Getting Started

If you don't care about adaptive tessellation or displacement and prefer to work with Maya's "Smooth Mesh" properties, you can!

If, on the other hand, you do care about adaptive tessellation and/or displacement, you'll need to use the Redshift-specific tessellation/displacement properties. You have two options for that.

The easiest way is to use the object's Redshift properties, as shown below.

Please note that once you enable Redshift's tessellation, the equivalent Maya Smooth Mesh options will be overridden by the Redshift ones!

Or, alternatively, you can create a "Redshift Mesh Parameters" node for your mesh(es). You do that through the menu Redshift -> Object Properties -> Create mesh parameter node for selection.

The benefit of this method is that you can use a single mesh parameter node for a hierarchy/group of objects, which is useful when multiple objects need to share the same tessellation/displacement options.

Tessellation Settings

Subdivision Rule

Redshift supports two algorithms for polygon subdivision: Catmull-Clark and Loop. Catmull-Clark can be used with quads or triangles but the Loop method only works with triangles.

The Subdivision Rule parameter controls how Redshift subdivides your mesh from the following options.

Screen Space Adaptive

The Screen Space Adaptive parameter controls how polygons are subdivided by changing the unit of measurement used by the Minimum Edge Length parameter. It might be helpful to think of the Screen Space Adaptive parameter as a subdivision mode selector.

When enabled, polygons are subdivided based on their measured edge length in pixels in the final render — hence, Screen Space Adaptive. When disabled, polygons are subdivided based on their measured edge length in scene units — in this case, subdivision takes place in World Space.

In Screen Space mode polygons are subdivided while their edge length measures longer than the number of pixels set by the Minimum Edge Length. For example, if the Minimum Edge Length is set to 10 then all polygon edges that measure longer than 10 pixels will be subdivided until the Maximum Subdivision level is hit or the newly subdivided edge length measures less than 10 pixels — whichever comes first.

In World Space mode polygons are subdivided while their edge length measures longer than the number of scene units set by the Minimum Edge Length parameter up to the Maximum Subdivision level. For example, if the Minimum Edge Length is set to 2 then all polygon edges that measure longer than 2 scene units will be subdivided until the Maximum Subdivision level is hit or the newly subdivided edge length measures less than 2 scene units — whichever comes first.

Screen Space subdivision is adaptive where as World Space subdivision is effectively static — the only time subdivisions will change in World Space mode is if the Minimum Edge Length is changed or the actual geometry changes in size. However, in Screen Space mode there are many ways to change how long an edge measures in pixels. For example, moving an object closer or farther from the camera, changing the render resolution, the camera zooming in or out, and an object scaling up or down could all trigger subdivision changes.

There are upsides and downsides to this adaptivity. On the positive side screen space can be much more efficient, like only adding more detail to objects close to the camera while far off objects have less detail, leading to savings on GPU memory and processing time that can be significant. Screen Space subdivision is also uniquely able to limit the amount of tessellation on objects outside of the camera frustum which can heavily reduce GPU memory usage. This is an optimization World Space subdivision is not able to do, instead, World Space mode will always tessellate objects — onscreen or not — as long as they meet the requirements set by the Minimum Edge Length and Maximum Subdivisions. However, since the subdivision level of an object can change as it moves across the screen the transition between different subdivision levels may be too noticeable and undesirable — for this reason screen space subdivision is frequently better suited to still images or animations where the transition is not as easy to see.

Tessellation and Instancing

When a mesh is instanced in Redshift, adaptive tessellation is no longer supported. However, you can use fixed-rate World Space tessellation (and, subsequently, displacement too) by using the following settings:

Use World Space subdivision (Disable the Screen Space Adaptive parameter)
Set Minimum Edge Length to 0
Set the Maximum Subdivisions to something reasonable

Please pay particular attention to the third point, as the default setting (6 subdivisions) is generally too high for fixed rate subdivision.

Smooth Subdivision

When enabled, objects are smoothly subdivided using the selected Subdivision Rule algorithm. This is great for smoothing out coarse low poly objects.

When disabled, objects are subdivided without smoothing using simple linear subdivision. This increases the polycount while retaining the original shape and silhouette of the object. If you are adding displacement on simple angular meshes (such as walls or a box) and don't want them turned into smooth objects, disabling smooth subdivision might be the right option for you.

In the example below, note how the handle on the cup and the edges of the spoon are rounded off when Smooth Subdivision is enabled. When disabled, the objects are still subdivided but the actual shape does not change.

UV Smoothing

When "Smooth subdivision" is enabled, Redshift will smooth the vertex positions as well as the UV coordinates and tangent space vectors. Smoothing UV coordinates shifts the UVs to remove any 'zig-zagging' and 'UV breaks' during tessellation in order to maintain smooth UV-space curves. In the majority of cases, this is the desirable way to treat UVs. However, there are cases where strict UV layouts (such as when UV tiles are aligned to quads) need to be preserved and not smoothed. For this reason, Redshift supports enabling/disabling UV smoothing.

Minimum Edge Length

Minimum Edge Length (M.E.L.) controls when polygons are subdivided based on their edge length. If a quad or triangle has an edge that is longer than the Minimum Edge Length then it will be subdivided either until the Maximum Subdivision level is hit or until the edge is shorter than the M.E.L.. When the Screen Space Adaptive option is enabled (Screen Space Adaptive subdivision), the edge length is measured in pixels. When the Screen Space Adaptive option is disabled (World Space subdivision), the edge length is measured in scene units.

Lower values result in more subdivisions while higher values result in fewer subdivisions. When the M.E.L. is set to 0 tessellation will continue until it hits the maximum allowed subdivision level. When using Screen Space Adaptive tessellation there is an internal minimum limit of 0.5 pixels, this can result in polygons not actually reaching the Maximum Subdivision level. However, when using World Space subdivision, a M.E.L. of 0 will always hit the Maximum Subdivision level. Regardless of the subdivision mode, it is extremely important that the Maximum Subdivision level is set carefully otherwise render times may increase exponentially. Determining the right balance between polygonal density and render time by optimizing the Minimum Edge Length for your scene is an important step.

In the examples below you can see the difference in behavior between Screen Space Adaptive and World Space subdivision. The scene is comprised of five identical cubes measuring 4x4x4 scene units, each with only 6 polygons before any subdivision takes place. Since the Minimum Edge Length in Screen Space mode is measured in pixels note how polygons closer to the camera are consistently subdivided more often than polygons in the distance — this is common behavior as polygons nearest to the camera tend to have longer edges. However, as the Minimum Edge Length gets closer to 0 even the distant polygons reach the same level of subdivision since most edges now measure longer than the very short M.E.L..

By comparison, in World Space mode, the cubes changes subdivision levels at the same time since each cube is exactly the same size and Minimum Edge Length is calculated in scene units. World Space mode is the best way to hit specific subdivision levels consistently regardless of where the object is in the scene but this behavior also makes it easy to subdivide distant objects more than necessary, potentially wasting memory and render time in the process.

Next is a demonstration of Screen Space Adaptive and World Space subdivision with a ground plane displaced by a texture map. On the right side of the screen is a wireframe overlay and a pink object that matches the current size of the Minimum Edge Length. In the Screen Space example there is a pink square that matches the M.E.L. in pixels from 0 - 30, and in the World Space example there is a pink cube that matches the M.E.L. in scene units from 0 - 0.5. The transition between subdivision levels is clearly visible in the Screen Space example, it moves up and down the ground plane from the large polygons in the foreground to the small ones in the background due to perspective. In the World Space example all of the geometry updates at the same time since the ground plane is made up of equally sized quads.

Maximum Subdivisions

Sets the limit on the highest number of subdivision passes that can be applied to a tessellated object.

Subdividing a mesh happens in 'passes.' Each pass turns a single quad or triangle into four quads or triangles respectively and each pass is applied to the result of the previous pass.

The number of polygons can grow very quickly so the Maximum Subdivisions parameter should only be set as high as necessary for the look that you need. Sometimes increasing Maximum Subdivisions by just one level can make a frame take four times longer to render than the previous level or you might run out of system memory and not being able to finish rendering the frame at all. So if you took a mesh made up of 1,000 quads with a Maximum Subdivisions level of 8 it would become a 65 million quad mesh. (warning)

This could take an extremely long time to generate and would consume lots of memory! For this reason, great care has to be applied when setting the Maximum Subdivisions and balancing the results with the Minimum Edge Length.

Out Of Frustum Tessellation Factor

Out of Frustum (O.O.F.) Tessellation Factor allows objects that are outside the camera frustum (i.e. objects outside the camera view that are not directly visible) to be subdivided to a lesser degree. Higher values result in less subdivision outside of the camera frustum, while lower values result in more subdivision outside the camera frustum. As a result increasing the O.O.F. Tess. Factor can help save memory and reduce render times by performing fewer subdivisions on objects that are potentially less visually important. However, sometimes an object might be outside the camera frustum but still perfectly visible in a reflection or the object might be casting a well-defined shadow within the camera frustum. For such objects, lower values should be used and an Out of Frustum Tessellation Factor of 0.0 disables this optimization entirely. Additionally, the Minimum Edge Length parameter has an impact on O.O.F. subdivision too — lower Minimum Edge Length values push O.O.F. subdivision closer to maximum subdivision levels which reduce the effectiveness of this option, in such a case consider capping the O.O.F. subdivision level with the Limit Out Of Frustum Tessellation options.

In the examples below, a blue cube placed just outside of the camera view can be seen reflected in a mirror, note how subdivisions increase as the O.O.F. Tess. Factor gets closer to 0 in the first example but don't change at all while the Minimum Edge Length is set to 0 in the second example. Generating fewer subdivisions is more efficient as long as the result does not negatively impact the look that you are after.

In the previous examples you may notice how the upper left part of the cube always remains more highly subdivided, this is because that part of the cube is closer to the camera frustum bounds than the rest of it. In the example below you can see how moving the cube farther away from the camera reduces the subdivision level.

This effect can also be illustrated on a tessellated ground plane. Take the following scene with a single quad and texture displacement, the camera is tilted down toward the plane from a close position. In the example images below the scene was rendered once from the camera, then the tessellation was frozen in the Render View, and the camera was pulled back to see the still frozen effect of the tessellation outside the camera frustum. As you can see, the polygons inside the camera frustum, outlined by a pink border, are tessellated the most. Polygons that are farther away from the camera frustum are subdivided less based on the Out Of Frustum Tessellation Factor.

When the camera is close to an object with displacement you should be careful with the Maximum Displacement value because it is used to determine if a polygon is inside the camera frustum or outside of it. This is necessary because a polygon with high displacement might start outside the camera frustum before displacement but end up inside the frustum after displacement. Without this consideration the polygon would not receive the tessellation it needs to look right. When Maximum Displacement is too high Redshift might incorrectly "think" too many polygons are inside the camera frustum leading to a lot of subdivision outside the frustum.

In the example below, the Maximum Displacement value is animated from 10 to 100. 10 is the highest value necessary for this scene and raising it any higher than that results in needless additional tessellation outside the camera view. This is a waste of resources and render time since those polygons are not visible which is why it is important to set the Maximum Displacement value only as high as is necessary. If you need a high Maximum Displacement but are generating lots of unnecessary subdivisions off screen the Out Of Frustum Limit can be enabled and the polygons outside the camera view can be capped at a lower subdivision level as demonstrated in the last image.

Limit Out Of Frustum Tessellation

When Limit Out-Of-Frustum Tessellation is enabled, you can specify the highest number of subdivision passes that can be applied to objects outside the camera frustum with the Max Out-Of-Frustum Subdivs option. This setting is useful when increasing the Out-Of-Frustum Tessellation Factor still results in excessive subdivision. This condition can happen when the mesh is using large displacements. For a more detailed discussion on this, please see the relevant section below.

In the examples below, a blue cube placed just outside the camera view can be seen reflected in a mirror and the camera pans over until the cube comes partially into view. In both examples the cube has a Maximum Subdivision level of 4 and a Max Out of Frustum Tess. Subdivs level of 2. Note how the subdivision level does not change when Limit Out of Frustum Tessellation is disabled. However, when Limit Out of Frustum Tessellation is enabled the subdivision level changes depending on whether the cube is visible in the camera frustum or not. When the cube is directly visible the reflection of the cube also illustrates the higher subdivision amount and when it disappears from direct view the cube is limited by the lower maximum subdivision level

In the next example, you can see the result of applying an Out of Frustum Limit to an off-screen object casting a shadow on a plane in the camera view. When Limit O.O.F. Tessellation is enabled a five-sided torus is limited to 0 subdivisions resulting in a very jagged shadow compared to the smooth torus when it is disabled and allowed to subdivide up to the Maximum Subdivision level of 4.

Max Out of Frustum Tess. Subdivs

Sets the limit on the highest number of subdivision passes that can be applied to objects outside of the camera view.

In the examples below, a blue cube placed just outside the camera view can be seen reflected in a mirror and the camera pans over until the cube comes partially into view. In both examples the reflected cube can be seen matching the subdivision level of the part of the cube when it is directly visible, however, the off-screen part of the cube is subdivided less based on the Max Out of Frustum Tess. Subdivs value.

Displacement Settings

Enabled

Maximum Displacement

Sets the limit on the highest distance in scene units that a vertex can be displaced.

For example, if you are adding two displacement textures together and each Displacement Shader has a scale of 1 scene unit, combined they can displace the vertices by 2 scene units, so a setting of 2.0 should be used for Maximum Displacement. If Maximum Displacement is set too low, the displacement result will be clamped and it will be cut off at a 'ceiling'.

On the other hand, if this value is set too high, there won't be any visual artifacts but performance could suffer because the Maximum Displacement value also has an impact on tessellation results. Increasing the Maximum Displacement value is similar to increasing an objects bounding box, the larger an object's bounding box becomes the more likely it is to enter the camera frustum which would trigger tessellation to occur. This might not be necessary as Redshift does not know if the bounding box actually needs to be that large so it will err on the side of caution and tessellate the object. Therefore, it is recommend to use a value that is as low as possible. Unfortunately, due to flexible nature of shaders, it's not currently possible to compute this value automatically. Settings similar to this can be found on other renderers, too, where they might be called "bounds padding" or "min/max bound".

In the example scenes below, one is a smooth black to white gradient ramp displacing a small yellow plane and the other is a paving stones texture displacing a large ground plane — in both examples the Maximum Displacement value is animated from 0 to a value higher than strictly necessary. The Displacement Scale listed under each example represents the lowest value necessary to displace the objects without clamping. In the scene with the yellow plane, the three blue bars are 1, 2, and 3 scene units tall. Note how the displacement appears cut off, it plateaus until Maximum Displacement meets or exceeds the Displacement Scale and once it exceeds the 3 unit bar there is no benefit to be gained. If your Displacement Scale is animated then Maximum Displacement would need to be set high enough to allow for the full range of displacement.

Displacement Scale

Scales the displacement results by increasing or decreasing the displacement.

Displacement Scale can also be controlled in the Displacement shader node itself, as a matter of fact, the final displacement result is the product of the Displacement Scale in the Redshift Object settings multiplied by the Scale set in the shader node. For example, a Redshift Object Displacement Scale of 3 with a shader Displacement Scale of 1 results in a total scale of 3 and the Maximum Displacement would also need to be set to 3 in order to accommodate for this.

Being able to adjust the displacement scale at the object level allows you to have different scales for different objects that use the same material. It is generally recommended to leave one of these values at 1 for simplicity, be careful not to double up your displacement scale.

In the examples scenes below, one is a smooth black to white gradient ramp displacing a small yellow plane and the other is a paving stones texture displacing a large ground plane — in both examples the Maximum Scale value is animated from 0 to a value higher than the Maximum Displacement limit. Note how the displacement retains its overall form, looking more or less extreme depending on the scale, it isn't until it hits the Maximum Displacement limit that it gets cut off. In the scene with the yellow plane you can see displacement is capped at a Maximum Displacement of 2.9 scene units, just under the top of the third blue bar which is 3 scene units tall.

Enable Auto Bump Mapping

Fine surface details require very high tessellation levels in order to look nice and sharp, otherwise the surface is likely to look soft and blurry. However, this means higher memory usage and frequently longer render times.

The Auto Bump Mapping option can be used to get around this by essentially using the displacement texture map to automatically drive bump mapping for increased detail. It emulates what would happen if you were to tessellate your geometry to a sub-pixel level and modifies the surface normals accordingly.

In the examples below, note how the first two images use the same number of subdivisions but when Auto Bump Mapping is enabled the detail level looks much higher. The third image is provided for reference, it has a higher subdivision level that results in greater detail but it uses more then double the amount of memory since it has to generate and displace many more vertices.


Ground plane with tessellation and displacement	Ground plane without tessellation and displacement


Screen Space Adaptive: Enabled (Screen Space mode)	Disabled (World Space mode)


Smooth Subdivision: Enabled	Disabled
Teacup model by Raphael Rau


Minimum Edge Length: 0 - 35 (pixels) Screen Space Adaptive: Enabled (Screen Space Subdivision)	0 - 2 (scene units) Screen Space Adaptive: Disabled (World Space Subdivision)


Minimum Edge Length: 0 - 30 (pixels) Screen Space Adaptive: Enabled (Screen Space Subdivision)	0 - 0.5 (scene units) Screen Space Adaptive: Disabled (World Space Subdivision)


O.O.F. Tessellation Factor: 0 - 50 Minimum Edge Length: 30	0 - 50 0


O.O.F. Tessellation Factor: 0 - 40	Test scene render (for reference)


Max Displacement: 10 - 100 Out Of Frustum Limit: Disabled Max O.O.F. Tess Subdivs: 3	100 Enabled 3


Limit O.O.F. Tessellation: Disabled (default) Maximum Subdivisions: 4 Max O.O.F. Tess. Subdivs: 2	Enabled Maximum Subdivisions: 4 Max O.O.F. Tess. Subdivs: 2


Limit O.O.F. Tessellation: Disabled (default) Max O.O.F. Tess. Subdivs: 0 Maximum Subdivisions: 4	Enabled Max O.O.F. Tess. Subdivs: 0 Maximum Subdivisions: 4


Maximum Displacement: 0 - 4 Displacement Scale: 3	0 - 1 0.5
Paving Stones Substance 003 textures from AmbientCG


Displacement: Enabled with Bump Mapping	Disabled with Bump Mapping	Disabled without Bump Mapping


Displacement Scale: 0 - 4 Maximum Displacement: 2.9	0 - 2 1.5


Auto Bump Mapping: Enabled Max Subdivisions: 5	Disabled 5	Disabled 7 (for reference)