Realistic Basics Realistic contents and new technologies
GCC Academy
Learning Goals Can explain hologram types and materializing technologies Can explain what is needed for XR content production through content application cases
Learning Content Hologram materializing technology XR content application cases
CEO Jeong Jiwon Career Current board member of T.OZ corporate Former board member of Wysisyg Studios corporate General director of Lotte World theme park TaekwonV VR General director of Gyeonggi Content Agency Gyeonggi VR AR
General director of Jeju National Museum realistic videography General director of Sokcho Museum realistic content production
Hello this is Jeong Jiwon for realistic content lectures
Today we will learn about holograms
As you took the realistic content lectures, you saw many technologies
and I think that hologram is the final stage of realistic contents
When hologram is well shown in a real space
the immersiveness will be very big
But the hologram now
are not yet shot on actual spaces
So that's too bad but
I think that when hologram is well produced
it will be the perfect realistic content
and so today, we are going to take a look into hologram materializing technologies among realistic contents
And XR content, that is, extended reality,
and cases of contents made with extended reality is another thing we will learn in this lecture
Hologram materializing technologies
Hologram can be seen on Star Wars
When first seeing that movie, you can see a lot of hologram
Hologram technology is
most effective when it is shot in a space
I've mentioned that 3D video is due to stereoscopy
3D is produced using stereoscopy
so when making these 3 dimensional videos distortion or
dizziness happens
And also you had to put glasses on for the stereoscopy method
But if hologram is perfectly done, no glasses are needed
and there will also be no dizziness since stereoscopy isn't used
So these 3 dimensional video technology is called holographic technology
and this holographic technology is, as I mentioned,
free of the issues that stereo 3D or multi-point of view 3D videos had
from the discordance of focus and convergence angle
So there will be no dizziness from visual fatigue
making the 3D video watching much more comfortable
In the filming process as well using object wave and reference wave
all information wave and phase of the light logging these in a wave interference format
visual fatigue can be eliminated, which is a strength
There are analog holography and digital holography in hologram
Analog holography, to put it easily,
is 3 dimensional video image generating technology filmed with lasers in general
and digital holography is made with 3 dimensional digital data
acquired through CCD camera or CMOS sensor image
referring to technology generating and saving hologram in such methods
So hologram is recording media via holography
and a case of the use of holography technology would be hologram
Hologram, looking into the term,
'holos' means 'whole' in Greek
and meaning and information
so 'gramma' which means 'message'
So these together makes 'record the whole's information'
and in a way it is a technology that records all information about the subject
Hologram considers the light that is reflected off an object from the laser source
through reflection and rotation as a wave
So all information about this wave is recorded as wave interference
As I kept mentioning, there will be no issues with glasses that the existing 3D videos had
or visual fatigue like dizziness
and with the space distortion due to stereoscopic camera filming being resolved
a complete 3 dimensional video can be produced
making it a perfect 3 dimensional method in any logical background
Analog hologram
separates light into two paths
The object beam that was reflected from the object
and reference beam reflected from the mirror interfere each other
The pattern made with the interference is
recorded on the photographic plate
and with the same reference beam shot on the photographic plate
the object image is played as a hologram video
which is the analog hologram method
Analog hologram uses optical phenomenon through light
which makes it very similar to photos
So analog hologram is also referred to as laser 3 dimension photography
Analog optical holography uses laser as the light source
and uses hologram film or photo polymer as the recording medium
So it's the method of recording the object's wave interference
and playing this
Analog hologram is
usually used in exhibitions, advertisement, and arts
Viewing the recording content
Photos can be developed ans scanned from films
and optical hologram just needs to be developed from the holography film
If the same light source is shot on the developed film
the 3 dimensional video will be generated
and only parts of the film could be attained as 3 dimensional videos as well
While analog hologram used light source, digital hologram
plays the image using computers
So instead of photographic plates, it uses SLM, spacial light modulator
that can control light's wave's amplitude or phase
to generate 3 dimensional videos
This method uses computer
calculation to record the aimed 3D object indirect pattern image
on an SLM
Then this is played as a hologram image
SLM can continuously change the pixel information
so it can generate not only images but also videos
and LCD is an example of SLM
LCD can control the light's phase or color information by controlling the liquid crystal molecules
Digital holography, as previously mentioned,
generates hologram by acquiring 3 dimensional digital data filmed with CCD camera or CMOSS image sensor
These are also able to be saved and edited as well
While analog hologram recorded on film with optical holography
cannot be compressed or sent
but digital holography is good with compressing and sending
So what we saw in movies
and what we experience analog and digital in everyday life
the digital data itself can be sent in emails
and such
Another strength of digital hologram is that it is possible to record the phase information of the object and
3 dimensional information and to analyze the object
Another thing is that there is no developing required
making it very convenient in the production
Also, digital hologram can make a not existing object
with computer graphic technology
and film these and generate these
Digital hologram's objective is
making 3D video data that is identical to the real one in real time
And through vast calculations, turn this into an algorithm
and making a sped-up graphic processing algorithm
would be the final goal of digital hologram generation
Next is volumetric display
Volumetric display is a technology that
creates image right onto a 3 dimension space
So this uses voxel,
volumetric piel
to make a 3 dimensional image in a physical 3 dimensional space
There are 3 to 4 ways to make this 3 dimensional image
There are screen movement method, plasma luminescence method,
changeable focus method and particle trap method
Volumetric display is
in a way, the complete version of hologram
Volumetric display is,
taking Star Wars as an example
in Star Wars, when sending a message to the princess
an image is shot in the space
These being materialized is the final goal in my view
because in VR, AR, or 3D movies
there are screen surfaces where the images are screened
There is a surface where a video is shot
or the video is played so that we can sense an object
and acquire information
but this volumetric display
generates image in an actual space
So this by itself is a very complex technology
and it will take much more time to actually make this
Currently in many labs in and out of Korea
volumetric display technologies are being developed
but the videos made have short running time
The running time is short
and the videos made are
For example a star forming or a sphere forming
Like these, simple things
are generated in a short time mostly
But these videos
even through Youtube or other materials
gives much 3 dimensional and spatial senses
and if this was a longer content
it will be ground breaking
I think it will be ground breaking
and it will turn all the previous hologram methods
into very outdated ones
with it
And for this volumetric hologram display to be done
currently, the devices needed are very large in sizes
and many technological equipment are needed
So I think that it will take more time to
make these devices smaller
Volumetric display
and especially screen movement method
moves the screen in rotation or back and forth in high speed
and a 3D object's section images synchronized to the movement speed are
continuously shot to the screen, making it possible
to view a 3 dimensional video due to afterimage effect
And particle trap method
traps a very small and light particle
with a laser and move them
Then external light is irradiated in sync to the movement creating an afterimage effect
to make an 3 dimensional video
in a space
This particle trap method is the hologram screening method
seen in Star Wars
Volumetric display has no limit in viewing angle
and there is no dizziness in the natural 3 dimensional video
but there will be spatial limits and noises
which are big weaknesses
of volumetric display
The next hologram method is
pseudo hologram
Pseudo hologram is also called pseudo(in Korean) hologram
and also as floating hologram
Floating hologram started from
Pepper's ghost by Henry Dircks,
an English inventor in 1862
I am sure that you have seen pseudo hologram a lot
Where you saw it is Psy's hologram show at Dongdaemun
and YG Entertainment singers' hologram shows
and SM Entertainment hologram shows
These were all done using the
pseudo hologram method
I also thought that pseudo hologram was a new technology when it was out
But as I did some research
it was easy to find out that it started from a stage illusion
in England in 1862
and thought that we were using it well now
Examples of pseudo hologram in Korea
are the mentioned hologram performance of Psy
or SM Entertainment's show
Let's see how it worked
How it was done is, you put a projector above
and play a pre-recorded video
on the floor where there is a reflective surface
Once it is screened, the reflected video is
reflected on the set-up transparent film
in a 45 degree angle
Then the pseudo hologram is
just like it was done in England
screened as if it is floating to the audience
So when you think about it
you will remember that the background
of Psy's hologram show being very dark
This is because only when the background is dark, when the object is screened
it looks like it is floating in the air
and in a way, the layers are separated
There's the 45 degree transparent film in the front
and the background in the back
creating layers in the space itself
making the audience feel perspective and floating effect
So it looks like it is 3 dimensional
If you try to remember
in the 45 degree one,
there were a lot of objects or effects
and the back layer had the background projected
having two layers
combined into one
So the floating method, pseudo hologram method
makes it possible for
the hologram video and the performer and host
to interact
So the 3D objects or effects are
screened on the 45 degree film
and actual performers stand on the stage
And lastly at the back
a background is screened
separating the stage into three layers
which creates perspective
And also from the actual performer and the background only
to another video in the front being screened
more perspective and 3 dimensional feeling is created
So if you re-watch
videos from that time
you will be able to see
what I had just explained
XR content application cases
Next we'll learn about XR contents
XR is extended reality
It isn't far different from VR, AR, and MR
VR, AR, MR, XR are better
explained as one whole concept
XR, extended reality is
referred to as extended reality(in Korean)
Some also call it MR, mixed reality
Recently XR was more commonly used
Let's see how XR contents were used
and how these are being incorporated
these days
XR is used a lot in Korean medicine
or in the medical field
QuadRutcher made a Korean medicine XR
What it is is, students do acupuncture
When they do it on real person
it could cause a medical accident
when it is done wrong
so they make a part of a human body into a sculpture,
a dummy
And make this sculpture's texture
identical to actual human skin
And with this dummy, and putting on hololens
you can see the muscles in the leg
And when you do acupuncture on the muscles
there are what looks like needles
and it shows how deep the
needles go in
So in a way, to maximize the effect of
XR contents
there must be an analog object
And when digital information is added
on top of that analog object
the effect is maximized
So at that time, when the company made this content
they first made a dummy leg
With that dummy leg
they screened the images of the
muscles in the leg
And what's important here is that
when doing the XR experience
when you just put in on the muscle
you know that the needle is going in
But it is hard to see
how deep or how shallow it is reaching
But this content
showed how deep it went in
and how this affects the muscles and blood vessels
So if you want to know more about this
you can search Quadrature Korean Medicine on Youtube
and it will help a lot
Also when producing these
extended reality conent
it would be good to incorporate it into many fields
Especially when creating this Korean medicine content
what's more important is cooperation with experts
The companies making contents
cannot do this all
So at the end, how the information from the experts
are materialized is very important
So these contents are
developed into a cloud based system,
so that students can study it
anytime, anywhere
Next is
XR bus
XR bus is what Suwon-si made
Suwon-si made XR bus
XR bus was made with the title of 1795
Jeongjo, on his mother Lady Hyegyeong's 60th birthday,
marched to the grave of Crown Prince Sado
for 8 days
which is called Eulmyowonhaeng
This was recreated realistically
So this,
starting at Hwaseong Yeonmudae, a military training ground of Joseon
for 30 minutes, it went from the south gate Paldalmun,
to west gate Hwaseomun, and main gate Janganmun,
then the north gate, Buksumun Hwahongmun
coming back to Yeonmudae at the end
XR bus had
12 55 inch T OLED on both sides
they also made a different seating plan
So instead of having
seats on both sides,
there were seats in the middle
Because that way,
people would be able to see the images in the T OLED
so they put the seats that way
And the transparent OLED
going a step ahead of a normal OLED
had been divided into a self-luminous WRGB pixel area
and a transparent area
and these were closely intersected with each other
The strength of a transparent OLED is
First, when riding the bus,
you can see out the window
But if the screen switches to the video
the real outside and the video becomes translucent
being able to see both
So when the pixel area lights up
it acts as a display
but normally, it acts as a window with 40% transparency
being able to see outside as well
So this XR bus' production itself
actually is considered
to be made after seeing the content Field Trip to Mars
which will be explained next
This XR bus is in a way
the process of marching through the Suwon Hwaseong
being screened on both left and right
while moving in the bus
It is screened
If the XR bus was to give a bit more realistic feeling
then when moving
if it showed images that matched the spaces
the effect of that experience would have been maximized
but at the time it was played on both sides
like a video being
seen on both sides
So with the seats in the middle
audience, the experiencers were sat there
and watching the walls on both sides, they would see
how the Eulmyohaengcha was done
At that time, this content was produced
as a part of the smart city project
with a lot of companies in consortium
This XR bus
can be used a lot in other local governments
and the real space and content
being matched interactively
maximizes the effect of
delivering the information
or the things you can see in the real space
So XR bus,
in the case of Italy
had VR bus, which when you ride this bus
moving around the tourist attractions
you could see the information there
Another example I often give for realistic content is
a project called Field Trip to Mars
This project was
done by Lockheed Martin
with marketing purposes
For Field Trip to Mars
the production process and
promotion video for Field trip to Mars is on Youtube
First of all, it's a school bus
Students get on the school bus
And the students are told
that they are going on a field trip by their teacher
We're going here for a field trip
Let's have fun
The kids are excited getting on the school bus
As it goes through the road
the screen suddenly changes
and shows the perspective of a car running on the surface of Mars
So the title Field trip to Mars means
a field trip to Mars(in Korean)
So at first, the kids
were excited thinking they were going to play outside
They were happy just by that thought
but with the image of Mars
on both sides
And they are not just still images
Videos are playing
and they interact with the real environment
So how it works is
if the bus shakes while it takes a turn or goes over something
the Mars space also shakes
And there are also scenes where an agent in the
Mars station talks to them
When Lockheed Martin did this project
it was they that paid and
the company who made it was a separate company
And they bought a school bus
They bought it and completely renovated it
Because as I told you before
when they're running on the road
the images would show accordingly
So GPS, 3-axis accelerometer, magnetometer
or laser surface velocimeter was needed to be installed
So the bus was completely remodeled
and 4K resolution transparent LCD display
was installed on both sides
So what the students didn't realize was
when they were getting on the bus
they could see the window
and see what's outside,
the things passing by outside
So they thought 'we're passing by here'
but when it suddenly changes to Mars, they were very surprised
But if this was just played as a video
then the shock wouldn't have been as big
But with real magnetometer or 3-axis accelerometer
and surface velocimeter,
everything was coordinated
so the experience of the children would have been maximized
So this Field trip to Mars,
I consider it to be the best case among realistic contents that were
done in real physical spaces while being on an object
Because how immersed the kids were were
pretty good
So through this
the kids could resolve their curiosities about Mars
or how the surface of Mars looks
These things would have been limited when
it was taught just in class
but since they're feeling like they are
actually going to that place in a bus
the educative effect would have been bigger
This educative effect was generated
with the screen showing them taking a turn
when the bus took a turn, and when the bus shook
the screen also shook making the experience
more effective
Next, where the concept of XR is most used is
XR studios recently
XR studios are
before, they would film with
blue screens or silver screens as a background
And then they would add the background
Recently, having LED display on the background and on the floor
or sometimes even on the ceilings
so even without going to a specific location
they would be able to film
with high quality end product
XR studios are actively used not only in Korea
but also in other countries as well
Especially when COVID-19 hit
when it was hard to film abroad
XR studios could make it possible to make it look like
it was filmed abroad and the costs would be cut as well
so that the production was much more effective
These things are possible with
the videos screened on LED,
materializing them,
the camera that captures it,
and real-time rendering
becoming one
which makes this possible
In The Mandalorian
they used this XR studio
to minimize the process
and cut costs
So the strengths of XR studio is
using real time engine
and enlarging the display itself
the space is covered with it
So in a way, a scene that was impossible to make before,
constantly changing the space and environment
has become possible to do in real time
and that way, when actors are acting
the flow would also be
much more connected
Taking a look into cases of use,
XR studios have
used unreal engines as well
and in there, virtual scenes,
lighting, effecting would be done all at once
So in a way
post process is shortened
And this part can
also be used in bringing up the density
So this virtual production workflow
can eliminate outside fliming
and complex VFX scenes
can be generated within the camera
which are strengths of it
And making 3 to 4 sets simultaneously
actors could move from one to another
and film at different locations at the same time
which is another strength
The last thing is
where XR technology is used
These days, XR technology is
also used in mobility field a lot
Sony and Honda
working together in Sony Honda Mobility
made the car AFEELA
In this car, AI technology
and entertainment VR AI
were equipped
so that in that mobility car
one could experience all
visual entertainments
experiences
So here
they integrated 5G, WiFi, GPS,
cloud, wireless updating
and everything like this
so that everything could be done
in the car
And not only Sony
but many domestic car brands
like Hyundai and Kia
are incorporating this XR technology
to make it possible to do these
various experiences
Today we learned about hologram
and XR
Hologram,
especially volumetric display
did not have limitations in the viewing angle
and was free of dizziness and natural 3 dimension screening was possible
which are its strengths
but also had limitations in spaces and generated noise
XR technology was used in Korean medicine
filming sites and in cars
This technology was incorporated
in these fields
Thank you for taking this lecture
Summary Hologram materializing technology Analog hologram Laser-filmed 3-dimensional video image-generating technology A method of recording and playing the wave interference of the object
Digital hologram Generating holograms with the 3-dimensional digital data acquired with image sensor filming, a technology that can be saved and edited
Generate 3-dimensional videos using spacial light modulator that controls light's amplitude or phase
Hologram materializing technology Volumetric display Technology that generates image right into a 3 dimension space Using voxel to make 3 dimensional image in physical 3 dimension space
No limit in viewing angle and generates natural 3 dimensional image free of dizziness Has limits in spaces, generates noise Pseudo hologram Floating method
Started from Pepper's ghost in 1862 Interaction possible between the hologram image and the actual performer on stage
XR content application cases Korean medicine Make a body part into a dummy sculpture Effect maximized when digital information is screened on an analog object
Quadrature XR showed the depth needles went it, reactions of blood vessels and muscles XR bus Suwon-si XR bus to 1795: recreated the 8-day-long Eulmyowonhaeng where Jeongjo marched to the grave of
Crown Prince Sado for Lady Hyegyeong's 60th birthday Transparent OLED: intersect self-luminous WRGB pixel area and transparent area
Field trip to Mars Renovated a school bus to show as if they were on Mars Installed GPS, 3-axis accelerometer, magnetometer, laser surface velocimeter on the bus
XR content application cases XR studios Installing LED display in the place, possible to film without moving to locations Production cost is cut and production efficiency is maximized
Sony Honda Mobility's AFEELA Everything can be done in a car