Reality Advanced Media Server Disguise Utilization (2)
GCC Academy
Hello, I am Jiyong Park, and I am
lecturing on the use of Disguise today
This time, we will learn how to use
Disguise through hands-on practice
In particular, we will cover three important topics
3D mapping, 2.5D settings, RenderStream, and ICVFX
XR technology is an innovative technology
that is quickly taking hold in broadcasting, film and various media industries
Through this lecture, you will experience
the main technologies used in the XR environment
and learn how to apply them
in production
Today's lecture focuses on practice rather than theory
So, you will have time to try out the tools yourself, set various settings
and learn the technology firsthand
In the process, I hope you will feel
a new sense of the possibilities of
XR and virtual production technology
2D Mapping Method
Let's find out how to do Mapping
within the project
and how to do Mapping by type
First, let's create an LED screen
After creating it, we'll adjust the size and location
Then, we'll change the Resolution to 1920×1080
Then, we'll create another LED screen to place next to it
We'll adjust the location and size
If you can't see it well, you can rotate the viewport
to adjust it so that the LED screen can be placed well
We'll also change the Resolution of the LED screen
to 1920×1080
Once the LED screen placement is complete
add a video layer and specify the video
When you add a video layer,
Direct Mapping is automatically set
to the last LED screen
Let's modify this
We'll name the Mapping, direct
and select Direct Mapping
We'll add the two LED screens we created for the screen
To repeat the video, we'll shorten the video layer length
and divide the timeline
Direct Mapping is the easiest way
to map directly
to the LED screen
It's a method of Mapping the entire screen to fit the LED screen
Next, let's create a Feed Mapping
Set the mapping name to feed
and select the mapping type as Feed
Then, add two LED screens to the screen
Change the Feed Mapping Resolution
to 1920×1080
To make it easier to understand Feed Mapping
let's freely change the part
mapped to the LED screen
As you can see, if you set it up
it will look the same on the LED screen
Feed can be mapped to any arbitrary rectangle
on the screen
Feed Mapping lets you
easily turn all screens into one canvas
by applying pixel-perfect content
You can individually move, scale, crop, or flip
the sample rectangles
Next, let's create a CameraPlate Mapping
Name the mapping cameraplate
and select the mapping type as CameraPlate
Then, let's add two LED screens to the screen
If you play the video, you can see that
the video is not output to the LED screen
CameraPlate is a method of
Mapping to a camera
not directly to the LED screen
Let's create a camera on the stage
and adjust its position
Let's add the camera we created
in the Mapping settings screen
You can see the video output in the camera view
Let's display the camera view on the monitor in the feed window
This is a 3D camera-based mapping fixed to the CameraPlate
When adjusting the color in the set extension
you should use the CameraPlate Mapping
Next, let's create a Special Mapping
Let's set the mapping name as special
and select the mapping type as Special
Then, let's add 2 LED screens to the screen
If you play the video, you can see that
the video is not output to the LED screen
Special Mapping needs to add
MR Set to the screen
Let's create an MR Set in the Stage menu
Then, in the Mapping settings window
we'll add the MR Set we just created
We'll change the Camera plate to the Back Plate
With 3D camera-based mapping, you can move around the 3D environment
by having a space or location
Next, let's create Cylindrical Mapping
We'll name the mapping cylindrical
and select Cylindrical as the mapping type
Then, we'll add two LED screens to the screen
Cylindrical Mapping is similar to Parallel Mapping
except it outputs content to the LED screen
based on a cylinder
Even if the screen moves or expands
the Cylindrical Mapping feature
remaps the content in real-time
Next, let's create Mesh Mapping
We'll name the mapping mesh
and select Mesh as the mapping type
Then, we'll add two LED screens to the screen
Mesh Mapping technology allows you
to map content based on the UV coordinates
of a specific mesh in a 3D environment
This allows you to fill the entire mesh with content
and efficiently implement movable locations
or spaces within the 3D space
This technology enables precise content mapping
even on complex structures or surfaces
providing immersive 3D experiences
Next, let's create Perspective Mapping
Name the mapping as perspective
and select Perspective as the mapping type
Then, we will add two LED screens to the screen
Perspective Mapping
has the characteristic that it starts from the point
where the content is output and gets bigger as it gets farther away
from the output point, just like an actual project
Except for this point, it is similar to Parallel Mapping
Perspective Mapping lets you precisely
map the desired 3D content at a specific point,
enabling more immersive visual expression
Next, let's create Radial Mapping
Name the mapping as radial
and select Radial as the mapping type
Then, we will add two LED screens to the screen
It is similar to Cylindrical Mapping
except that it extends from the center
of the radial to the end of the cylinder
Therefore, the pixels inside the cylinder are mapped
according to the height and distance from the center axis
Next, let's create Spherical Mapping
Name the mapping as spherical
and select Spherical as the mapping type
Then, we will add two LED screens to the screen
Spherical Mapping
works similarly to cylindrical mapping,
except it takes the form of a sphere
In the latest version, Disguise has a solution
that makes it easy to work in 2.5D
Let's take a look at it
First, run the d3 manager
and open the desired project folder
I will put the 2.5D file I prepared
in the object subfolder
Then, I will run the project
When the project is run, I will create one LED screen
change the settings, and arrange it
I will add a 2.5D layer to the timeline
I will adjust the length
and select the file I put in the 2.5D file folder in the asset
The 2.5D data must be mapped
based on the MR Set, not the general mapping
Therefore, I will create an MR Set in the Stage menu
Then, I will set the mapping to the MR Set I created earlier
and set the Camera plate to the Back Plate
Next, I will adjust the camera position
to check that it is mapped to the LED screen
If you move the camera, each image will appear
to move in response to the camera's movement
To change the arrangement of the background image
right-click the asset, select Plates
and adjust the position as it is now
Place the plate in the desired location
to take more natural shots
Even if you move the camera, you can see it is mapped
to the appropriate size within the camera view
Render Stream and ICVFX
Before proceeding with RenderStream
we will download the plugin to be applied to Unreal Engine
First, visit the Disguise homepage
Click LEARN - Download Hub on the top menu
You will be taken to the download site related to Disguise
Click Resources in the middle menu
If you select the RenderStream plugins menu on the left menu
a URL menu that allows you to access the GitHub site will open
You can check the d3 manager
and Unreal versions used on GitHub
and download the appropriate plugin
First, we will proceed with the Unreal Engine settings
on the RenderStream device, RX2
Save the Unreal project to be used
in the RenderStream project
and create a Plugins folder in the project folder
Then, use the RenderStream plugin folder
you downloaded
and run the Unreal project
Create a folder in the Outliner
create an actor to be used as the origin
and change the name to Origin
We will create two Cine Camera Actors
The actor and the camera will move to the (0, 0, 0) position
We will also rename the Cine Camera Actor
and make it subordinate to Origin
Then, we will move the camera position to the desired location
To apply the RenderStream plugin to the camera
go to the Edit - Plugins menu
search for Disguise, and enable it
Then, select each camera, add the plugin
and make sure it is checked
Edit - Project Settings - Maps & Modes
select the level to render stream
save it, and exit Unreal Engine
Let's proceed with the disguise settings
in VX4, which acts as the master
Before setting the render stream
we will create one LED screen, one camera
and one MR Set
This is because the render stream
needs to be mapped to the LED screen through the MR Set
Let's add a RenderStream layer to the timeline
Right-click Workload to open the settings window
set the project file for Asset
the Default assigner for the settings for the render stream to be mapped
set Full frame in Splitting strategy
and set the Transport format to YUV 4:2:0 8bit
compressed for light render stream
Create a new ClusterPool
and select the equipment to render
Determine the camera mapping method
through Channel mappings
Assigner is the same as the Default assigner above
Set the Inner and create a new mapping
to set the Outer
The Outer will proceed with Mesh Mapping
in a different way from the Inner
Since the Outer is an image that gives light effects
in the area outside the Inner
it is to output low quality with Mesh Mapping
Then, in Actions, press Start
to output the Render Stream
While looking at the output image
create an origin to change
the origine of the MR color and Mesh Mapping
and apply the same origin to each
Then, move the origine
to match the location set
in Unreal Engine
There is a part where the inner image
is not being transmitted
You can check that the inner image
is transmitted properly by adjusting
the Channel mappings and Load weight values
Next, you can set it by moving the origine
and camera to fit the desired screen
Since the render stream receives data rendered from other equipment
and transmits it, it is also important to
view the information through the details window
You can turn off each setting
to check only the data of the part you want to see
or check it all
If another problem occurs, you can check the log
to identify the cause
Now, let's go to the VX Studio site
and apply what we've learned
VX Studio Site Demonstration
This is the VX Studio inside
the Gwangju Real Content Cube GCC
I will briefly explain each piece of equipment
we learned in the previous theory lecture
while showing you directly
Before applying the various mappings
we learned today
let's briefly look at the basic structure
and setting method
It would be good to think of it as
reviewing everything we learned in the previous class
on the LED stage
I hope you've seen the disguise mapping method
we saw earlier applied to an actual LED wall
and I hope this helps you understand it more easily
If you look, there's a master sync device
called SPG8000A
This device is very small
and responsible for tying all of these systems
into a single signal
Here, it provides the same clock to all of the devices,
such as each media server, Brompton,
and Mode SD, to synchronize them
All of these can be controlled
separately from the computer
so let's go to the computer and take a closer look
Before we go to the computer
let's quickly examine how it's configured
Let's go to the back of our master sync generator equipment
If you look closely, it's really simple
but you can see that these BLC cables send a sync signal
to our main equipment
If you look here, the world clock signal is coming in
and the black signal
sends a signal to each piece of equipment
One of the key points is that
all of these signals are electrical signals
so a stable power supply is essential
That's why there are two power supplies
for each of them
If one has a problem, the other supports it
and maintains a stable signal
Let's take a closer look at the equipment
Looking in front, you can see
1, 2, 3, 4, and 5 Disguise VX series equipment
Next to them, you can see the RX series
This RX series is responsible for
rendering and streaming Unreal Engine projects
If you look here, the rendered images
are streamed and sent to these
These VXs are ultimately responsible for
mapping the LEDs and outputting the images
If you look at it, one is in the directing role
and the rest are in the acting role
So if the director manages each actor together
the actors just render
and output the images
Let's start with the basic project settings
First, let's create a project
The most basic thing is that we need an LED Wall Mesh
Let's put it in
You can put the mesh in the object's Mesh folder
You have to put everything in the same way
in the actor computer
that you will export from the directing computer
In this way, you can directly enter
the folders of each machine in the d3 manager
In this way, we'll put the project
in each machine
Since we've set it up lightly,
we moved it quickly, but if you have a lot of media
you can see that this whole process takes a little longer
When you run the project
you can see that
only the main machine is running
but each actor machine needs to run
so that we can work smoothly
Let's do the basic settings first
We'll delete the projector
and space in the default
and the person in the default
In Project Settings
you have to change the refresh rate by default
The default is 60, but we'll change it to 59.94
As I said before, the director is set up like this
but each actor machine must be connected
to output images to the LED
so let's set that up
We'll designate which machine will be the director
and set up the actor machines
Each machine has a hostname
which is the serial number initially assigned to each machine
With our current configuration, one is the main
another is the ceiling, and another is the floor
and each has a separate role
Now that all the settings are complete
you can check that the connections are all correct
If you run the project on each machine like this
the basic preparations are complete
If you look at the status window, the light is red
If you change it like this
you can see they are connected
Now, when you are done here, if you go to the Feed settings
as we mentioned earlier
we can see the status windows of each actor
Let's go back to the stage and continue with
the basic settings
We will create the most basic LED screen
Let's install
the LED on the wall
First, we start by inserting the LED mesh
that we inserted earlier
Insert the prepared OBJ file like this
and enter the Resolution
for our LED panel
In addition, we will proceed with the floor
This way
we have completed installing the LEDs on the wall and floor
Additionally, we will add the ceiling part
In the case of the ceiling, we created it
using the mesh provided by Disguise
When we are done here
we can see that the LEDs installed on-site
and in our virtual world
are installed the same way
Now, to check if each LED is outputting properly
we will proceed with a test pattern
This is how we create layers and map them
so that we can put test patterns on the LED
If you go to the next Feed
and look here, you can see that the Refresh Rate reference point
is the Input part
and the bottom part is the Output
You can think of the main, floor, and ceiling
as the expanded form of what we just created
and the actors are the devices we will export
Let's export it
This way, you can export the video to the point you want
by Alt + right-clicking the mouse
For the resolution, I will export it as 3840×1920
So, I will adjust the wall's LED wall
to use 3840×2160 in total
Here, I will move the source position
You can move it by dragging it with the mouse
but to be precise
it is most accurate to move it by entering numbers
I did it this way
and if you look at it, there is a little bit of resolution left at the bottom
I will apply these as separate scrap resolutions
and put them in the final output
little by little
The horizontal size is the same, 3840
I will only use 384 pixels for the vertical size
Likewise, I recommend entering the source position
as the exact location
This basically completes the Output settings
for the main wall
If this is not done exactly
there is a slight possibility that the grid lines of the test pattern will be distorted
You can check it by adjusting it like this
Additionally, I will set the floor
Since the resolution of the floor is a little lower than before
I will only use two outputs like this
and I will only use one for the ceiling like this
Now, after setting the resolution with the Feed setting
you need to set the color setting
If you go into Project Settings
or Basic Color Management
the default value is Gamma Space
But we will click on this and change it to ACES
We chose ACES because
many of the results and contents
that come after working on VFX
are in ACES
Originally, you could set 8-bit or 10-bit
in the feed
If you do ICVFX
or Unreal, you have to do it in 10 bits
but today, we will do it in 8 bits
And before I tell you about the additional settings
let's insert an image
Now, the media is one
Let's simply load
an Egyptian video
When we load the media
we have to set the color space
of this media
Right-click on the input image
and set the color section
If you look, it's set to sRGB by default
If you look at it, we're going to do 8 bits today
so you can just use this setting, but to show you simply
if you do ACES-2065-1
you can do it as pass-through
If not, if you do it in 10 bits
you often go with Rec. 2020 in the color section
and ST2084 (PQ)
in the transfer function
Today, we will go with sRGB
This means that you have set
the color space for the input image
We also need to set the output's Colour Space
You can do this
by right-clicking on the mesh
If you look, you can set the Output Transform
Colour here
Now, before we finish
let's go back to the beginning
and change it to Gamma Space
If you change ACES to Gamma
in Project Settings
right-click here again and look at Colour Space
There aren't many values ​​you can set like before
It is confusing if you see it sometimes and you can't set it
You can set it like this by changing
Gamma Space to ACES in Project Settings
After setting Output Colour Transform in Disguise
you need to input it in the same way
in Brompton and Image Processor
Normally, you can set
the input like this
and since we will use 709 today, set it to SDR
and press Rec.709 to proceed with the setting
If you have done everything up to this point
you can see that the color for the source, the color for the export
and the color of the Image Processor
that is exported at the end
are all set the same
Now that the color setting is complete
let's create the mappings we learned earlier one by one
and check how they are applied
to the LED. Among the various mappings
Let's start with the simplest
2D method
There are many different ones in 2D
but let's start with the simple Direct Mapping
As soon as you input an image, the mapping is applied right away
The default is Direct Mapping
In the case of Direct Mapping, you can see that the image I have
is directly applied to the LED mesh
Now, if you right-click on the mapping
you can see which screen this direct is connected to
and how the resolution is applied
It's really simple
For example, let's add a floor here
The advantage is that we can apply it
to the mesh we want right away
Next, let's take a look at Perspective Mapping
If you look at it like this
you can see that the image is only in the center of the LED
n the case of Perspective Mapping
the mapping is a bit different from Direct
because you can see that the image changes position when the camera moves
You can think of the area
the camera looks at as the LED image
If you right-click on the camera, you can see it like this
I'll rotate the camera
so that it's on this side
If you look at it, the LED has a curve
but the image from the camera looks flat
These parts are one of the biggest advantages
of Perspective Mapping
This is used in many cases
but the most common use in Korea is
when filming driving scenes
Let's take a look at how it's used
We have a layer that we prepared
Let's take a look at that layer to make sure
If you look at the image on the right
you can see it's roughly connected
and the image on the left is slightly off
Although it is not perfect
you can stitch multiple videos
into one video within Disguise
Let's play it
This is how we utilize Perspective Mapping
If necessary, you can see that
the left and right cameras move separately
Of course, we can stitch the footage
taken on-site in our post-processing
but there are also many cases where Disguise
simply stitches and uses it
And one last tip
If you move each camera at once, it is a bit cumbersome
so if you create a specific mesh and group all the cameras under it
you can control three simultaneously
If you control it this way, you can quickly place
the image in the desired location on site
Next, let's look at another mapping
Sphere Mapping
First, let's control
the mapping that we have already installed
The source we have now is a 360-degree source
There are two ways to move the position left and right like this
or if you go into the mapping itself
you can rotate it with Rotation
If you look at this
you can see a specific circular shape in the middle
and that it is connected radially
This is Sphere Mapping, which is additionally
used in 2D mapping
Up to this point, you can think of it as a mapping
that anyone can easily use with 2D mapping
The next mapping we will look at
is the 2.5D mapping method
Now, the 2D mapping that we've seen so far
is a single 2D image played on LED
so there's no perspective change
and no parallax effect that we commonly call it
To get more advantages from these
we use 2.5D
Let's take a quick look at 2.5D
Now, if you look, there's an LED in front
and we've stacked up many layers behind it
If you stack layers like this, including
the sky layer, the building, and the foreground
and then all of these are combined and combined into one LED,
you can see that this is 2.5D
Let's take a look at the camera
If the camera moves like this
the sky will move slightly because it's far away
and the foreground will move more because it's close
We can say that we can implement
the 2.5D effect because of these differences
Let's set these up
Everything we've done so far was made on an LED screen
but to do 2.5D
we need to add Projection Surfaces
Let's make one as an example first
For the rest, we'll use the ones we've already made
If you make an example like this
the size has to be significantly larger than the LED
so there's no space
The LED we're using now
is 30m wide
so we'll make it 40m, which is larger
and we'll place it behind the LED
In this way, we'll divide the Surfaces
we'll use into background, foreground, and midground
This time, we'll place all the ones
we've already made
If we place it now, it won't be visible on the screen
We can see this if we change
all the Render Stages to Onstage
I'll erase the back
In this way, we can see the layers
we've stacked exactly
As you can see, we've prepared a separate image
with alpha for the image
We've created two more layers
that can be divided into two stages
The foreground and the midground
And we've created a sky
that can be used as the background
All mappings for this are done in
a Direct Mapping manner for each Surface
and when all of this is ready
to combine all of this
a layer called StageRender is created
However, after creating StageRender
mapping is needed here
and this mapping creates an MR Set
Click to take a look
As you can see, we need to select
the On-stage for compositing that is, the LED to be connected
We have set it up now, but let's remove it
and put it back in
When we take it out, nothing is drawn on the LED
We will proceed in this way, and additionally
we will create an MR Set like this
and then add the MR Set
to the StageRender mapping to finish
If we have reached this point
we are ready to use basic 2.5D
The advantage of this 2.5D is that images created in Unreal Engine
are a game engine, so they are a little awkward
However, since they are based on real images
the quality is high, but the perspective effect
can be maintained as it is
Now, to do this, we need to do
additional camera tracking
The basic 2D and 2.5D preparations are complete
Now, to utilize this
we need an Inner Camera
Let's try setting up the Inner Camera
We'll try camera tracking
For tracking, we'll use
what we've created before
You can track the camera like this
You can see that the Inner
changes every time you move the camera
To greatly improve the effect of this method
it's essential to receive camera tracking data in real-time
and apply it to the virtual camera
As you can see, the incoming data
is in real-time
Let's move the camera
and see how it is applied
Moving the camera to the right and left
and applying it this way
is a big advantage of 2.5D
We have briefly shown you
how to apply the parts we have learned
in theory to LED in practice
The biggest difference between pre-visualization on your computer
and in the field is
the connection settings of various existing equipment
Everything above is very accurate and the same as
what you see on the main computer
and what appears on the LED wall
Therefore, even if you don't necessarily
test it on the LED stage
you can do a lot of pre-vis and tech-vis in the office
sitting at your computer
You can see that the biggest advantage of Disguise is that
you can prepare everything in the office
and just run it in the studio
with confidence in this area
This is the second lecture on utilizing Disguise
and the last lecture on media servers
Since it is not easy to directly access
the actual VX studio or media server
I hope this lecture has allowed you to indirectly experience it
and increase your basic understanding
hen, I will finish by summarizing what you learned today
Thank you
2D Mapping Method Creating Mapping by Mapping Type Direct Mapping: A method of directly mapping the entire screen to fit the LED screen
Feed Mapping: A method of mapping to the desired location on the screen by specifying an arbitrary rectangle CameraPlate Mapping: A method of mapping to a camera rather than an LED screen
Special Mapping: A 3D camera-based mapping that has a space or location so that it can move around the 3D environment
Cylindrical Mapping: A method of remapping content in real time even if the screen moves or expands
Mesh Mapping 3D: A method of mapping content according to the UV coordinates of a specific mesh in the environment
Perspective Mapping: A method of mapping that starts from the point where the content is output and gets larger as it gets farther away from the output point
Radial Mapping: A method of mapping from the radial center to the end of the cylinder Spherical Mapping: A method of mapping in the shape of a sphere
2.5D Setting Import 2.5D project Move prepared 2.5D file to objects folder Add 2.5D layer to timeline
Set 2.5D Set Mapping to MR set and Camera plate to Back Plate Each separate image moves according to the movement of the camera
Render Stream and ICVFX Setting up Unreal Engine Create a Plugins folder in the project folder Create a folder in the Outliner and create an actor to use as the origin
Select a level to Render Stream in Edit - Project Setting - Maps & Modes
Setting up Disguise Create one LED screen and camera, then set one MR set Right-click on Workload and set according to the task
Decide the camera mapping method in Channel mappings Set Outer to Mesh Mapping
VX Studio on-site demonstration SPG8000A that connects all systems with a single signal Each BLC cable sends a sync signal to the main device
RX Series that renders and streams Unreal Engine projects One Disguise device directs, the rest act
Setting up the project The data to be exported from the directing computer must be entered in the actor computer in the same way Change the global refresh rate to 59.940Hz
Run the project for each machine Run the Test Pattern to check the output of each LED If the output is not accurate, the grid lines of the Test Pattern will be distorted
Change the value of Color management to ACES Set the Project Bit Depth to 8 Bit
After setting the Color Transform, the same input must be entered in the Image Processor
2D Mapping Direct Mapping where the image you have is directly applied to the LED Mash
Perspective Mapping where you can adjust the position of the image by moving the camera Spherical Mapping where the image appears radially through a circle in the center
2.5D Mapping Stack numerous layers behind the LED You need to add a Surface to set up 2.5D Mapping
You need to set the Surface value larger than the LED to avoid creating empty space Create a Stage Render to merge all layers
You need to select On-stage for layer compositing Insert the MR set into the Stage Render Additionally, camera tracking is required