Optimised local post-processing FX in Unreal Engine

LinkedIn
Optimised Local Post-Processing FX in UnrealEngine

Attention technical artists

I have something important to share with technical artists, the real-time community and anyone in the constant battle for more frames per second. In this article, I’ll let you in on a pretty neat method I developed for optimising localised post-processing FX, plus a little bonus treat at the end.

Most people probably haven’t tried making localised post-processing FX, nor know what it is. But not to worry, I assure you, the journey I went through making one will be worth your time! The challenges it brought forced me to come up with solutions which may help solve other people’s problems.

To give some context, it all started when I was inspired by Tom Looman’s Blog on Local Outline FX. I decided to make my own localised effect using a transparent surface shader, this time with a sobel effect that draws lines across sharp edges.

Getting started

To start, I made a sobel effect: a simple shader that draws lines along edges. To make it, I used the world normal buffer and sampled the surrounding pixels to detect a change in direction using dot product.

The node setup for a 4 sample sobel looks like this:

Node setup for a 4 sample sobel in Unreal Engine

And the result it produces looks like this:

Basic example of sobel set up in Unreal Engine.

Like a window, everything behind the surface of the spherical mesh has the sobel effect. Unfortunately this breaks the concept of the effect being localised. Alas, my first challenge.

I faced three major challenges making this localised effect. But through that, I came up with three major discoveries (spoiler: the last one is my favourite).

World position behind transparency

I wanted the effect to only apply itself within the spherical mesh, but the objects in the background receive the sobel effect because they’re simply behind the surface of the spherical mesh.

To localise the effect inside the spherical mesh, I knew I could do that with just a simple 3D sphere mask. The challenge, however, was that I needed to know the world positions of the objects behind the surface of the spherical mesh. Unfortunately, UE4 didn’t have a simple node for this, so I made my own.

With some thought, I figured out that I could use depth fade as a distance through the surface, then multiply it with the direction vector from the camera to the surface and add it to the surface position. That’s a mouth full so I’ll show the node setup and what it produces:

Node setup for transparent world position

With this I successfully created the sphere mask to contain the sobel effect within the spherical mesh.

Here’s the node setup:

Sphere mask node setup in UE4

Here’s the result:

Sphere mask example with rocket in UE4

The box problem

When testing the sobel effect on different objects, I found that on a cube, the top facing normal is the same as the ground, so in some cases, a line isn’t drawn where it should be.

Box problem with sobel effect on cube in UE4

To get around this, I used a similar node setup to the sobel world normals. But instead of world normals I used scene depth and detected changes in values of surrounding pixels.

Scene depth fixing sobel effect box problem in UE4

This allowed me to fill in any missing lines, just by using the depth buffer.

Sobel effect box problem in UE4 fixed

Transparent object woes

There are two major problems with using transparent objects for projecting localised post-processing effects: viewing the effect up close, and performance.

When I tried moving the camera inside the spherical mesh, the sobel effect disappeared and it dawned on me the harsh limitations of the effect’s practical use.

Transparency is also expensive and it only gets worse when you overlay multiple sheets of it. Here is an image of shader complexity; green is good, red is bad and white is downright irresponsible:

Example of shader complexity in UE4

As you can see, it’s adding a lot of complexity to a lot of pixels that don’t really need it. But fear not, for I found a solution to both these problems…

Deferred decals for post processing

Deferred decals have been in the engine for a while now, here’s an image that describes what they do:

Image describing deferred decals in UE4

As you can see, all the surfaces within the volume receive the decal material.

By using the sobel effect as a decal, it only applies it to the relevant surfaces inside the decal volume. This means that we save on a lot of shader complexity. And more impressively, allow us to view the sobel effect up close on the surface!

Sobel decal used on rocket ships in UE4
Close up of sobel decal used on rocket ships in UE4

Findings

While challenged with the task of making a localised post-process effect, I learned a lot through its limitations. But through that I also discovered a new method that I’m excited to share with the community and I’m even more excited to see what comes next.

Bonus! Content pack with experiments

If you don’t want to recreate the sobel effect yourself, you can grab my master materials along with all the successful experiments I tried on the UE4 Marketplace here.

All materials are full of settings you can tweak to get the desired look. Furthermore, each component of the material is broken down into modular material functions so you can create your own effects.

Here’s a video showcasing all my experiments I developed along the way:

And finally, here are some key examples of effects the material content pack can achieve:

Blending different sobel edges together.

Blending different sobel edges together

Using a separate colour for normal/depth sampling.

Using separate colours for normal/depth sampling

Applying a linear gradient using curves.

Applying a linear gradient using curves

Applying a sphere gradient using curves.

Applying a sphere gradient using curves

Using depth-stencil to apply sobel to selected objects.

Using depth-stencil to apply sobel to selected objects

Black edges using transparent decals, limited to depth sampling only.

Black edges using transparent decals

Read more articles

Explore our featured articles.

Stay in the loop

Subscribe to our newsletter to receive more insights, how-to’s and updates from David Grouse and the S1T2 team.

Welcome to the party