Realistic Basics Unreal Engine ICVFX Optimization
Hello everyone
I am Dohwan Lee, and I’ll be delivering this Unreal Engine ICVFX lecture
Today, we will discuss how to utilize web control
to optimize the performance of Unreal Engine
and improve workflow efficiency in an ICVFX environment
In the ICVFX environment, simple configuration settings are not enough
It is crucial to control and optimize
various elements in real time
For this, the web control tool can be a very useful asset
In this lecture, I will explain how to set up web control
and use it to optimize real-time lighting
camera settings, and various other engine elements
Optimizing Unreal Engine for ICVFX
When conducting ICVFX
achieving realistic representation of assets on an LED wall
requires high-resolution textures
complex lighting, and fine details
However, these elements impose a heavy burden on real-time operation
often causing performance degradation and lag
Conversely, prioritizing optimization for performance
can lead to compromises in quality
or even sacrificing the quality
If you lower texture resolution or reduce details
during optimization
it can become challenging to fully realize
the intended realism and immersion
To overcome these practical limitations
and balance performance and quality
applying various optimization strategies becomes essential
For this lecture, we will use
what Megascans provides for free
It's the abandoned apartment project
When working on projects
lag is usually caused by an excessive number of objects
within the level, such as numerous meshes
or high polygon counts
including many triangles and polygons
Complex lighting is another thing
Complex lighting with multiple placements
consumes significant resources
To address these issues
we will explore console commands and optimization methods
that can help resolve such challenges
First, let’s discuss profiling
Profiling is an important tool for identifying how resources
like the CPU, GPU, and memory are being utilized
Unreal Engine provides various console commands
such as Stat Unit
which can be used to pinpoint where bottlenecks are occurring
I’ll start by explaining
the most basic commands used in this context
Specifically, I’ll show you how to use Stat Unit
which I mentioned earlier
Since I already have it activated, let me turn it off
Search for Stat Unit in the command line
and you’ll see a display like the one shown here on the right
This option allows you to check the time spent
on the current frame, the time consumed by the game thread
the time taken by the rendering thread
and the time spent on the current frame by the GPU
Let me address each, starting with the first option
The first option lets you view the total time it takes
to generate a single frame in the game
The option below it shows
the time used by the game thread
If this value is similar to the frame time
it means that a bottleneck has occurred
in the rendering thread
The next option displays
the time spent by the rendering thread
If this value is similar to the frame time
the game’s performance bottleneck
is occurring due to the rendering thread
which works inversely to the game thread
The option below that
shows the time consumed by
the rendering hardware interface thread
This is related to systems such as OpenGL
Direct3D 12, or Direct3D 11
and things like those, so I’ll skip this part
The GPU time option
shows the time consumed by the GPU
in rendering the current scene in real time
This is also synchronized with the frame time
so it has a similar duration
The DynRes option below
stands for Dynamic Resolution
It’s a feature that automatically adjusts resolution
in real time within the game to optimize performance
Simply put, when a scene becomes too complex and the GPU slows down
it temporarily reduces resolution to improve performance
This feature is usually disabled
but can be enabled in the project settings
I’ll demonstrate this in the project settings section later
The next option represents
the number of draw calls
A draw call is a type of command where
the CPU tells the GPU to render an object
The number of draw calls is directly related to the number of objects
that the GPU needs to render
There are around 2,500 draw calls here
The last option below that
indicates the number
of triangles in the objects currently visible
In other words, it shows the number of triangles
in the objects being viewed right now
These commands allow you to identify
where performance drops occur and pinpoint areas causing lag
Let me now talk about the Stat command
To remove the current Stat options displayed
simply re-enter the Stat Unit command, which will hide them
Another frequently used command is Stat FPS
which displays the current frame rate, or FPS
Higher FPS values indicate better performance
This command is useful for quickly
assessing overall performance
but lacks detailed information
Let me remove it, and next is Stat UnitGraph
This option visualizes Stat Unit metrics
so frame, game, draw, and GPU time
So this option visualizes them in a graph
This enables users to see real-time fluctuations
and determine
bottlenecks in system performance
As you can see
it is changing real-time
The color to the right changes too
which is reflected in the final result
To remove this, do the same
The next option is Stat Game
Stat Game focuses on time allocation
within the game thread
analyzing components like AI, physics calculations
and blueprint execution
to identify potential optimization areas
Since we didn't do much
as in making any blueprint logic
it is not responding too much Let's turn it off again
Stat Slow highlights tasks
causing performance delays
identifying time-consuming or repetitive processes
This is useful for troubleshooting specific problem areas
Literally, it checks
for the parts that are particularly slow
Next is Stat SceneRendering
It provides detailed insights into the rendering process
including draw call counts, shadow computations
and lighting details
It helps pinpoint
performance bottlenecks during rendering stages
As I showed you before
Multiple commands
can be activated simultaneously
such as Stat Unit, Stat SceneRendering, and Stat Slow
All these can be activated like so
Stat GAME, just like this
To disable all active at once, use Stat None
which is a convenient way to clear all overlays
Let's now move on to optimization view modes
Unreal Engine provides various view modes
designed for optimization
To access these view modes
click on this section
and navigate to Optimization Viewmodes
Let's start with the first option
As you see, there is red, white
green, orange, and many more
We see some overlaps here
This view mode visualizes
the complexity of the lights in a scene
It uses colors to indicate
how much performance cost
each light incurs
Colors closer to the left
represent lighter costs
while colors like yellow and red on the right
indicate higher performance demands
Areas with overlapping lights
often appear as red, purple, or white
signaling areas requiring optimization
Let's move on
The next mode is Lightmap Density
which visualizes the density of lightmaps
It's a visualization, or a view mode
with colors like green, red, and purple
There are many colors
It helps easily identify areas
where the density is too high or too low
allowing for adjustments to achieve an optimal balance
between lighting quality and performance
Now we can check for the lightmaps density
Now it's Stationary Light Overlap
You can't really see, but it is
what visualizes overlapping zones of stationary lights
These lights provide fixed illumination
but when multiple stationary lights overlap in the same area
it can significantly increase performance costs
Let's move on to the option below that
Shader Complexity
It visualizes the complexity of shaders calculated by the GPU
It shows how computationally intensive each pixel is
Particles are red in this case
with simpler shaders displayed in green
and more complex shifting to red or white
Areas with red or white shaders
are highly complex
and should be optimized to prevent lag
Particles are another common source of such issues
The cracked window line and areas on the floor
are actually replaced with textures
making them less complex, maintaining this quality
Let's move on to the next option
This mode visualizes both shader complexity
and quad overdraw simultaneously
combining information on shader complexity
and pixel-based quad overdraw, allowing users
to identify areas with high rendering inefficiencies
Regions with complex shaders or high overdraw levels
require optimization to improve performance
Let's move on to Quad Overdraw mode
This view mode
highlights areas where overdraw occurs
Overdraw happens when the same pixel is rendered multiple times
which can lead to significant performance drops
This mode helps visualize such areas
enabling developers to minimize unnecessary rendering
Techniques like instancing or mesh merging
can reduce overdraw
making it an effective optimization strategy
By using these view modes, developers can
visually identify and address performance bottlenecks
and lag-inducing areas
Coupled with Stat commands
it becomes easier to pinpoint where optimization efforts should focus
We talked about identifying performance bottlenecks
and lag-inducing areas
using various methods
Now, let’s discuss optimization methods
The first method is using LOD
which is a short form of Level of Detail
LOD adjusts the level of detail in scenes
or individual meshes
based on distance or significance
By reducing detail for distant objects
for example
from LOD0 at close range to LOD4 at a distance
developers can lower GPU and memory usage
As seen in the example, there are five LOD levels
from LOD0 to LOD4
They progressively decrease triangle counts
simplifying objects
For example, here at object
LOD0 contains 16,128 faces
but by the time you get to LOD4 it only has 48
Using LODs allows for high detail at close range, like LOD0
and lower detail when farther away, such as LOD4
At intermediate distances, LOD2
or LOD3 may be used at a distance
reducing unnecessary resource usage
and improving performance
Below are the same LOD0, LOD1, and LOD2
As shown in the image, LOD0 through LOD4 are configured
with high-resolution LOD0 applied for close-up views
and simpler LOD4 for distant views
To create LODs, tools like Maya, Blender, or 3D Max
are used to design LOD groups
containing meshes for LOD0, LOD1
LOD2 Mesh, LOD3 Mesh, LOD4 Mesh, and so on
These in an LOD group
are exported as FBX files
at Unreal Engine, when importing the FBX file
we have the Import Mesh LODs option
Once this option is ticked
this can automatically
load these LODs
This process optimizes performance
while retaining the necessary level of detail
Another optimization method is reducing polygon counts
Unnecessary polygons on each mesh are eliminated to ensure efficient rendering
Simplifying high-polygon models
helps balance
rendering efficiency and visual quality
This optimization method
unnecessary faces, edges
or elements in the modeling
The next step is removing unnecessary objects
In Unreal Engine
objects or unused elements in areas
not visible to the player can
be removed from the scene
to reduce rendering overhead
For example, in scenes for ICVFX and camera-based rendering
eliminating unused lighting or meshes
and such can significantly
reduce performance drops and lag
Let's go to Stat Unit
See the option Draws?
We are looking at 3,000 triangles now
An additional approach is merging objects
It is all separate as of now
Merging smaller meshes into a single mesh reduces draw calls
and improves performance
For instance, multiple individual meshes
can be combined
to lower draw call counts
simplifying the rendering process
Let's move on to the next method
It's a sub-level, which I will create now
Make a folder named sub_level in the content section
Create a level under this folder
then make a lighting level and save it
Once you place the lighting into this level
turning it on or off will not make any noticeable changes yet
because nothing exists in the level at this point
First, go back to the persistent level
and search for lighting
Turning the five lights will show you this
Let's try this here
Currently, the lightmap is baked, so the lighting appears this way
We will place these lights into this level
To do this
select the lights
then right-click on the lighting level’s sub-level
and select Move Selected Actors to Level
The lighting objects belong to the lighting level
If you turn off the lighting level
you can adjust it like so
Use sub-levels to separate lighting into distinct levels
When creating levels, you can divide the space
into different zones, like here and there
assigning them to sub-levels
Moving on, let me explain another method
First, if you open up the Static Mesh settings
you’ll see an option related to the LOD group
Below that, there’s an LOD group setting
which automatically applies the appropriate LOD settings
calculated by Unreal Engine
when a mesh belongs
to a specific group
This feature simplifies optimization
by aligning LOD settings with object size
and characteristics predefined by Unreal
eliminating the need for manual LOD setup
For example, when using Foliage
selecting the Foliage option
will automatically calculate suitable LOD values for it
While this grouping
helps organize and optimize
there may still be unnecessary lag in some cases
requiring manual LOD adjustments
To manually adjust LOD
toggle the options above
where you’ll find settings
to modify the Percent Triangles value
to determine things like what percentage of triangles to use
This allows you to optimize by reducing
polygon counts in LOD1 or LOD2 appropriately
For instance, if the current setting is at 100%
As you see, it is 100%
You can adjust it to 50%
After making changes, be sure to click Apply Changes
This means that the triangle count is
effectively reduced to 50%, as displayed
Another optimization method involves using Nanite
Nanite is a technology developed by Unreal Engine
to efficiently process detailed models
It handles only perceptible details
which optimizes its performance
Conventional LOD settings required distance-specific configurations
But Nanite automates this process
eliminating the need for prior LOD adjustments
It automatically converts distant objects
into low-poly models to reduce rendering load
while close objects are rendered with high-poly meshes
for detailed expression as well as optimization and precision
There are two ways to enable Nanite
You can either right-click on the asset in the content browser
and select Enable Nanite
or double-click to enter detail mode
and locate the Nanite settings in the details panel
Tick the Enable Nanite Support box
and click Apply Changes
to switch the mesh to Nanite and make use of this feature
To view Nanite settings
click the Lit button, and under the Nanite section
select Triangles here
to see the meshes configured with Nanite
This is what we see now
The triangle count decreases as you move further away
reducing the GPU’s computational load
This time, we will discuss
polygon optimization using texture maps
First, let’s talk about optimizing polygons using a normal map
If you look at the image above
you will see spheres with varying polygon counts
To express high-resolution details, the polygon count increases
which can add performance stress
The lower image demonstrates the principle of mesh normal
These normals can be baked
to record the direction of each face’s normal
allowing low-poly models
to also appear detailed
model with fewer polygons can look like a high-resolution mesh
with a more three-dimensional appearance
Looking at the left image
you’ll see a model with relatively fewer faces
where the normal vectors extend from the center of each face
Conversely, in the right image
more faces have been added
resulting in a more detailed representation of the normals
While increasing the number of faces
provides finer details, it can also increase
the polygon count, causing performance strain
To address this
normal maps are effective for detailed representation
without adding extra polygons
By storing normal information in a texture, you can add
details to low-poly models without increasing the number of faces
For instance
if you compare two spheres, both with 288 polygons
the left sphere is a low-poly model with visible individual faces
while the right one uses a normal map
to give the illusion of a high-resolution model
Although the right sphere has a low polygon count
the normal map
creates a high-poly-like appearance
To understand this further
the diagram illustrates
a low-poly model with simple normals at the top
lacking details
And a high-poly model with more complexity
has more defined curves in the middle
In the baking process
the detailed normal information from the high-poly model
is applied to the low-poly model
making it look as detailed as a high-poly mesh
Applying normal maps to low-poly models
enables you to create
detailed meshes without adding extra faces
When baking a high-poly mesh
applying its normal to a lower-poly model
can result in visually similar details
Next, let’s discuss the next optimization method
which involves mask maps
The left image shows
a fence-like mesh
that is also transparent
Modeling such a structure individually
would increase polygon count and require significant time
Representing the mesh entirely through polygons
involves complex modeling
which could severely impact performance
However, as shown in the right image
Unreal Engine’s material editor allows you to use a mask map
to display only the fence parts while making other areas transparent
This technique avoids additional polygons
while realistically representing empty spaces in the fence
Using mask maps
can significantly save work time
and reduce polygon count for optimization
Mask maps are also frequently used
for visual elements, especially
like torn fabric or leaves
Modeling individual leaves as meshes
would require many faces
to capture the necessary details
And we will need a lot of leaves for the project
So using mask maps
becomes crucial for optimization
Next, we will cover material optimization
By reducing unnecessary textures and lowering texture resolution
you can minimize memory usage
Removing unused materials
also improves performance
as material complexity has a significant impact
on GPU load
For example, the mesh in the image
has seven material slots
for wood, concrete, paper, and more
Each slot can hold a different material
but having many slots
means multiple draw calls for each material
If a single mesh has seven material slots
it generates a big number of seven draw calls
The number of slots affect
the draw calls, which can strain the performance
Reducing the number of slots can optimize the performance
The next method is using material instances
Based on a master material
we created two instances here
With these instances
you can apply different colors or textures
while sharing the same base material
For example, the two leaves in the image
both use the same master material
but are displayed in different colors
This is also an optimization approach
Generating multiple instances from a single master material
reduces memory usage draw calls
compared to using separate materials for each variation
The next optimization method
involves decals
Decals allow you to add graphics or textures
to specific parts of a scene without
directly modifying the mesh or applying additional texture
For instance, you see a manhole detail on the floor
If this detail were modeled as a separate mesh
and assigned textures
it would increase memory usage and draw calls
adding a significant performance burden
However, with decals
you can apply graphics only where needed
avoiding extra meshes or textures
and enhancing performance
Decals are especially useful
for efficiently adding details
while maintaining optimization
It's especially great in complex scenes
Now let’s discuss
level streaming optimization using sub-levels
as explained earlier, sub-levels can be used to optimize lighting
but here we’ll elaborate further
By utilizing sub-level streaming
you can load and unload only the necessary levels for environment creation
This reduces memory usage and loading times
while improving performance
For example, when working on a scene
we won't be working on the grass here
we'll work on these big ones
But for small props like grass
they may consume significant resources
And for this, I separated
this leaf
Toggle its visibility, and it is removed
You can achieve higher frame rates
and fewer draw calls with this
Similarly, lighting can also be turned off using the Unlit mode
When you're working in large-scale scenes
it is common to divide sections into sub-levels
such as lighting, foliage, and mesh
to manage resources effectively
Let's move on to shader optimization
Let's go to our master material
Complex shaders consume a lot of GPU resources
so they need optimization
by simplifying intricate calculations
or removing unnecessary operations
Using the view mode
the Shader Complexity view mode
you can visualize the complexity of shaders
to identify and streamline problematic areas
Here at material
it looks pretty clean now
But it could look much more complex, like this
Unnecessary mathematical calculations
or logical complexity
can be simplified for shader optimization
Let’s now discuss code optimization
Although this is not a central focus
and may be challenging
code optimization is essential for reducing CPU resource usage
It is important to remove unnecessary function calls
Let me show you
In Blueprints
we could create a function like
Get All Actors of Class, which can retrieve
all Static Mesh Actors
Retrieve all Static Mesh in the scene
and subsequent actions can be added
We should remove unnecessary function calls
and improve inefficient loop structures
For example, this loop
Get All Actors Of Class
So retrieving all Static Mesh Actor
will end up with so many actors
This will read all that Static Mesh in that array
One loop, one process, one loop, two processes
That's a loop
And this structure could be improved
Profiling can help identify bottlenecks
when using Blueprint scripts
Another area to optimize is the tick graph in event graphs
Tick graphs
respond on every frame
and it's better to minimize the use of tick events
and activate them only when needed
Next, let’s move on to project settings
Go to project settings through Settings
Here at project settings
we will configure a fixed frame rate
Why do we need this?
It configures a fixed frame rate
for projects involving ICVFX
So I recommend using Frame Rate
So let's say Frame Rate
We can adjust the animation
time code, and more here
This can sync to
the external time codes
The second setting is Motion Blur Disable
In an ICVFX environment, it is critical for the real camera
and virtual background to synchronize perfectly
When Motion Blur is enabled, rapid camera movements
can cause the blur applied to the virtual background
to mismatch with the actual camera
creating a visually awkward effect
To prevent this discrepancy
disabling Motion Blur in the project settings
ensures that the virtual background and live-action scenes
are represented more naturally and consistently
To disable it, search for Motion Blur in the settings
and untick the box
The next setting is Auto Exposure Disable
Auto Exposure refers to automatic exposure adjustments
and it is crucial
to disable this for ICVFX work
When Auto Exposure is enabled
the brightness of the virtual background displayed on the LED wall
can fluctuate with camera movements
This discrepancy between the brightness of real lighting
and the virtual background disrupts
the visual consistency of the scene
So untick the box
The next adjustment concerns Auto Save
Auto Save is located not in the project settings
but in Editor Preferences, so search for Auto Save
We untick this because
ICVFX work requires real-time synchronization
It involves multiple systems such as camera data
and lighting data transmitted over a network
This real-time synchronization
requires stability
Enabling Auto Save during a shoot
can disrupt multi-user connections
live links, or other real-time data connection
This can lead to interruptions
or errors in real-time workflows
To maintain stability, it is best to
disable Auto Save
We work with ICVFX
with Auto Save option unticked
Unreal Engine Settings to Keep in Mind for ICVFX
When working in ICVFX, care must also be taken
While shooting live, you should not alter critical settings
in project configuration or display settings
This impact the quality of the virtual background displayed on the LED wall
or result in frame drops and system instability
It is crucial to avoid modifying significant project settings
during a shoot to prevent such issues
The next point is maintaining accurate camera settings
The virtual camera in Unreal
with the real-world physical camera
in terms of Field Of View, lens settings, film back, and focus settings
If these settings do not match
the virtual background might appear distorted
To ensure smooth alignment between camera movements and the virtual background
these settings must be reviewed and confirmed before filming
Lastly, regarding multi-user session precautions
there is a multi-user browser
When utilizing ICVFX, the main computer running Unreal
and the LED computer displaying the output
both join the multi-user session
This ensures that movements of the camera or objects
are reflected on the LED wall simultaneously
or in a multi-user session
However, during a multi-user session
if one user updates or changes their work in real time
other users connected to the session
might experience disconnections
When transferring a large amount of data or making significant changes
they can experience disconnection
due to network delays or buffering issues
For minor adjustments, it’s fine to proceed
but for substantial changes
it’s best to avoid doing so
The Web Control feature in ICVFX
We've covered optimization methods
and things to look out for
while working on your ICVFX work
Let me give you some tips about Web Control
The purpose of Web Control
is to allow users to remotely manage and monitor
various elements of their Unreal Engine project in real time
This tool is especially critical for implementing
in-camera visual effects
enabling adjustments to lighting
camera settings
and sequence management
while efficiently managing
multiple virtual production elements
Users can test desired scenes in real time
and apply immediate feedback in live environments
To set up Web Control, search for Web in the plugins menu
and activate Remote Control Web Interface and Remote Control API
After enabling these features, restart the program
Turn off the control and then back on
the activation status can be confirmed in the output log
where the Remote Control Web Interface should appear
Next, in the project settings
search for Remote Control to adjust settings
such as the port number that
Remote Control Web Interface uses
This can prevent network conflicts
or tailor configurations for specific environments
I will keep it as it is now
Now, at Content
we will create Remote Control Preset
by right-clicking
and selecting it under the Remote Control category
Click Remote Control Preset here
Leave the name
Now we have a preset, which we will save
Now we can control it with an Internet browser
We're only doing the local testing
so let's proceed
The local address is 127.0.0.1 and the port number 30000
This will give us this page
Click on CONTROL and this button to find the design
You can switch mode for
real-time adjustments for the project
First, we need to add a variable
For this demonstration
I’ll use Directional Light’s color
and implement a feature to adjust its color
after opening the preset, you’ll see icons like these
indicating that they will be added to the web control
When you click this button, it will be included
Add Color as the ID
and when you return to the web interface
you will see that Light Color
From here, select the color in the design section
choose Build Your Own UI, and drag it over
You’ll see that the UI is now implemented
To confirm it’s working
I’ll minimize the window
Now, if you change the color
you can see it updating in real time
The color is changing in real time
This way, you can input and adjust the desired parameter values
observing and using the changes in real-time
So far, we’ve gone over
how to set up the Unreal Engine environment for ICVFX
As I mentioned earlier
during practical work, the program tends to get heavier
resulting in longer processing times
Therefore, always keep optimization in mind when setting up the environment
This can significantly reduce work time
We’ll wrap up this session by summarizing what we’ve learned
Thank you
Unreal Engine Optimization for ICVFX
Stat Unit Elements Frame: Total time taken to generate a single frame in the game
Game: Time spent on the game thread Draw: Time spent on the rendering thread
RHIT: Time spent on the rendering hardware interface thread
GPU: Time spent on real-time scene rendering using the GPU
DynRes (Dynamic Resolution): Automatically adjusts resolution in real-time to optimize performance
Draws: Number of draw calls Primes: Current number of triangles on objects
Stat Commands Stat FPS: Displays current frame rate (FPS)
Stat UnitGraph: Visualizes the values of each Stat Unit item as a graph
Stat GAME: Shows time details for various tasks on the game thread
Stat Slow: Identifies tasks causing performance degradation
Stat SceneRendering: Provides detailed information about scene rendering
Stat None: Disables all currently active Stat commands
Optimization View Modes Light Complexity: Visualizes light complexity in the scene
Lightmap Density: Visualizes the density used in lightmaps
Stationary Light Overlap: Visualizes areas where stationary lights overlap
Shader Complexity: Visualizes the complexity of calculations
performed by the shader for each pixel
Shader Complexity & Quads: Visualizes both shader complexity and quad overdraw simultaneously
Quad Overdraw: Visualizes areas with overdraw
Various Optimization Methods
LOD Optimization Nanite Optimization
Polygon Optimization Using Texture Maps Material Optimization
Level Streaming Optimization Shader Optimization
Code Optimization Project Settings
Precautions for Unreal Engine Settings for ICVFX
Precautions During ICVFX Work Avoid changing project settings during real-time rendering
Refrain from making large changes in multi-user sessions
Web Control Functionality in ICVFX
Web Control Allows users to remotely control various elements of the project
and monitor in real-time
Enables real-time scene testing and immediate feedback in live environments