For this lecture we will take a look at some motion capture videos
to see how to install a motion capture,
how it is filmed
and the developing
technology flows
We will also look at how to
work together with other equipments or other fields
while shooting motion capture
Also we will look into how contents are
being expanded and used these days
Progress of Motion Capture Technology
This video is about the Magnetic system
Nowadays, there are lots of equipment
using gyro sensors
Gyro sensors can measure not only
rotation data, but also speed and acceleration
It's also wallet-friendly
So it is trending as of now
The video you are about to see is
a magnetic system using gyro sensors
You can see it utilizes a large space
I will explain after the video
Magnetic System - Xsens MVN
Magnetic sensors in the past was
vulnerable to high voltage currents and rebar material
Filming done in
wooden buildings were not a
big problem
However it was very difficult
to film near concrete buildings with rebar
to use the magnetic system
Many of these issues have been
resolved by using the gyro sensor
but we still have to deal with some data errors
regarding current flow and reinforcing bar material
However, as software resolves these issues
the frequency of use is increasing
In the video
you saw that we were able to use a large space freely
And the data comes out seemlessly
even though there are steel material all around the set
Magnetic System - Xsens MVN
Here you can see that the data for
driving a car and riding a bike
have been extracted well
Like this, the magnetic system's data
have been greatly improved by the use of gyro sensors
and we can use it with less difficulties
There are still some areas where
the legs are blurred and the hands are out of sync
while shooting in real time
but it is still widely used in SNS content using
virtual humans and games and animation production
that can extract a large amount of data in smaller amounts of time
Games, for example
We apply a technique called Motion Matching
This technique
make the object run 20 to 30 meters
and extracts the data
for the required section
When these techniques use Optical systems
We need area for installation
and it will take a lot of time
But as for Gyro systems
If we do not have an indoor area
we can shoot at outdoor areas
So the Gyro systems are
preferred in the game scene
Next we will look in to the Optical system
This video shows the Optical systems
default settings, software installations
network settings and camera installations
Also briefly shows the process of calibration
attaching body and tool markers on the character
and converting them to third-party tools
Ultimately, you can see how it is implemented
in Motion Builder, Unity, Unreal Engine, etc.
Let's go ahead to the video
Optical System - Optitrack
Optical systems require the hassle of installing cameras
because data must be created by the camera
But once installed and calibrated
you can produce optimal data, as long as you do not withdraw
For most studios you can
continue to use after the first installation
But in the instance of overseas or large space shooting
you would usually rent a large studio
about 30 to 50 meters
and use it for a month or two
and move out
Because it would be very costly
to keep the space and equipment for longer
These types of projects are
gradually emerging in Korea as well
The chart here shows
the difference between Gyro and Optical systems
the Gyro system does not have space limitations
and can be used indoors
so it has a big advantage in terms of location
The optical system has space limitations
it should be shot indoors
and also needs time for installations
gyro systems do not need post-edits
It has the advantage of saving
a lot of time and money
Gyro systems have the disadvantage of
lower data quality compared to optical systems
due to low accuracy
Optical systems need post-edits
so it can cost you both time and money
but compared to the gyro it has
the upper-hand in data quality
with the 0.1mm accuracy
Therefore it is suitable for works requiring
quick movements, detailed acting and accurate measurements
The video about to be played uses motion capture
it is a 2014 film called Big Match
It was used as a previs
This was probably the first time in Korean cinema
that motion capture has been used from previs
Let's watch the video first
Previs and Motion Capture used in 'Big Match'
I took the role in this film for
the martial arts, special effects, CG, etc.
to be more intensely and elaborately
designed into the film
I attempted to add precision to the movements by
motion-capturing the action sequences in advance
those techniques were used for things like Ikho's digital character
At the police station scene
Ik-ho was simply curious about his brother
and he had to do as Ace told him to do so
and get out without harming the police
I think this can be
the mission and fun of this movie
Perfected by 3D Mission Game in the Middle of the City
The setting with most excitement and CG effert
is of course the Sangam Stadium
Because we were trying to use the entire stadium
previs and motion capture technology was incorporated
to design each sequence more accurately
including the precision of movements, actors' routes
and camera angles for the main character
Ik-ho's action scenes
At the time, the concept of
previs was not well known in Korea
So this might be the very first Korean film to
try Previs using motion capture
In other countries, previs technology
has been used for over 20 years
Previs is gradually earning its
popularity recently
Motion capture technology also
is being used more and more
The next video is a Korean video
The children in the first scene
are digital characters made by motion capture
The setting is impossible to find in real life
and even if found, it would be too dangerous
to bring the children there to shoot
Hence it was made digital
This probably made in the early 2000
Motion capture technology would be necessary
in order to create videos where characters
must move in a background that is not real
Next is an edited video of the
digital characters in a Korean drama
Digital characters are being used efficiently
in the large crowd scene
Let's watch the video
Motion Capture Uses in Korea
For the drama, digital characters were created first
then the movements were made with motion capture
and then it was turned into
Database for use
The first scene is an action scene from the drama ‘Gu Family Book’
Enemies would turn in to ash when defeated
so all of them had to be made from digital characters
and motion capture was used for more realistic movements
Also in dramas like ‘The Moon Embracing the Sun’
‘The King's Doctor’, and ‘The God of War’
the smaller characters at the outer angle
are made with digital characters in means of
reducing costs efficiently
Like for the performance scene in 'Light and Shadow'
Supporting cast were placed in the front of the audience
and for the back
digital characters were used
We collected about 10 cheering gestures during the motion capture
ran a crowd simulation
to create an audience over 1,000 people
Collaboration and Application with Other Fields
Let's move on to the next case
This is a 2007 american film called Beowulf
It is almost 17 years old
but had already incorporated today's hi-technology
It used 240 cameras
There were about 200 of markers
per person on the body, face, fingers and etc.
It filmed 10 people simultaneously
It also used various of different structures
and also did motion capture on horses
You can also spot that
virtual camera was used
Let's watch the video
Motion Capture used in 'Beowulf'
This movie has left many records at the time of production
For example, filming was completed in 30 days
It is very unusual to finish shooting
a movie in 30 days
It usually takes 3 to 6 months
but this one only took 30 days
Another one is that the main cast
lived in a trailer next to the set during his shoot
He also limited the scene transitions to
15 minutes
So all scene transitions were done in 15 minutes
Which is probably why the whole shoot
was able to be completed in 30 days
Virtual Camera used in 'Beowulf'
you could see an example of using a virtual camera
In the past, virtual cameras were used
while watching the action in the post studio
But now we can apply motion capture
in real time during the actual filming
I will explain this part later in a different video
The movie 'Beowulf'
used yet another hi-technology
Which is tracking the movement of the pupil using a sensor
This is called a EGO sensor
and it tracks the muscles around the pupil and generates a data
This is a medical sensor
but was applied in movie making
So you can see the slightest shakings
just like real pupils
Let's watch the video
EOG Sensor Used in 'Beowulf'
Next is a 2009 film, 'Avatar'
160 motion capture cameras were used in this movie
The facial captures were filmed
using camera tracking rather than a marker method
It used virtual cameras in real time
and also used a fusion camera that
produces 3D images
'Avatar' not only uses motion capture
but also used Simulcam in real time which is
the combination of virtual and 3D cameras
Let's watch the video
Motion Capture Application Example 'Avatar'
When 'Avatar' was released
there was a huge boom of 3D movies
There was also a lot of recognition of motion captures
Before then it was not too popular in Korea
but the movie 'Avatar' played a big role in its popularity
Interest in Face has increased as well
The marker method had good quality data
but it was very difficult to
apply to a digital face
this was resolved using the camera tracking equipment
In fact, the face camera is
more outdated than the marker method
But its quality has been verified through the 'Avatar'
and it was also easier to apply to digital characters
Hence we use this more and more
and has become established through continuous technological advancement
So now the marker method is gone
as the camera tracking methods develop
now it has evolved into 4D capture method
using multiple cameras
The 4D capture method uses 4 or more cameras
to track the facial expressions
This allows to extract modeling data
of the facial expressions
I will touch on 4D captures
later once again
This video shows the latest
motion capture shooting methods
You will be able to see optical system
and face captures using markers
and making high quality games
using various of structures
Let's watch the video
Performance Capture 'Goodbye Kansas Studio'
In the next video you will see a good example of
virtual production and motion capture usage
This will be a comprehensive collection of technologies that are currently in use
Let's continue after the video
Case of Virtual Production and Motion Capture 'ABBA Virtual Concert'
This is a 4D capture method that reproduced the youthful appearances
and expressions of members over 70 years old
It shapes up natural gestures and movements
using motion capture technology
And it used LED and holograms
to build a virtual production for this performance
So we moved away from the old method of
production content through post-editing
to a real-time filming with the advancement in technology
In addition, by incorporating various technologies
it is changing into an efficient production method
called virtual production
The key point of virtual productions is
to minimize location shooting
and to do all shooting in real time
making efficient use of time and reducing costs
This concludes the motion capture lecture
Magnetic system Gyro sensor can measure rotation data, speed and acceleration Gyro sensors are inexpensive and can be used efficiently
In Korea, most concrete building had rebar, which was difficult to use magnetic System Recent software solves those problems, resulting in increased usage
Optical System Can continuously produce data once installed One-off Optical system projects are trending
Collaboration and Application with Other Fields 'Beowulf': used 240 cameras, 200 markers per person
Able to track eye movements and even small tremors with EOG sensors
'Avatar' Case 160 motion capture cameras used, face capture using marker camera tracking Virtual cameras used in real time, incorporated fusion camera that produces 3D images
Real-time production using a method called Simul Cam through collaboration between virtual camera and 3D Fusion Camera