In this lecture I will explain
overall production process of motion capture
What preparations you need to make to film motion capture
and what you should be paying attention to
And also
What you need to do after filming
Learning contents will be, first, preparation for filming
Second will be actual filming of motion capture
and third will be data clean-up and converting
Lesson objective will be to differentiate between
regular filming and motion capture shooting
and to learn the
overall elements of motion capture filming
As an example of mine
We used motion capture for the movie, 'The Admiral: Roaring Currents'
Even though I worked on it, I couldn't tell
which parts of the movie used motion capture
when I saw the end result
I found out later
First, the bird's eye view of this ship
which is the view of the ship from the sky
Obviously it is not easy to find dozens of actual ships
or even make them
So the ships were made with computer graphics
and because the ships were processed
actual people would not be able to be inside them
Then all the characters here
has to be digital characters as well
so we used motion capture here
If I remember correctly we gathered some
rocking-on-the-boat movements
We took about 5 cuts
5 cuts of different movements
and we ran it through crowd simulation
and made hundreds and thousands of people
That is what we used there
And the difference between real shooting and motion capture shooting is
For this movie as an example
we didn't shoot at an actual ship
We made a fake model ship and
the backgrounds were all
green and blue screen
and did a separate task of
masking the main character's face
However, motion capture
must be filmed in a studio
Other than for the main character
the parts for the supporting actor
such as falling into the water or getting stabbed
were all motion captured
Two tracks were filmed
in this way and later
the CG part of the motion capture data
was synthesized
For some countries
this has been a thing for 10 to 20 years
Even in Korea, this method of filming is
being implemented now
As I mentioned in the previous lesson
We no longer use acoustic or
mechanical systems
So I my explanations will be based on
Optical and Magnetic systems
What makes motion capture shooting different from regular
is that it extracts data
For normal filming, we need
to capture the actual casts faces on camera
Motion capture filming extracts data
for the remaining supporting cast
For movies or drama shooting
If you need to capture a character on screen
Since motion capture is basically
making data for digital characters
There are no restrictions on space or structure
So even if we don't find or make
and actual ship
If we have a simple
steel or wire frame type structure
we can shoot motion capture
with no issues
That is an advantage
So inside the studio
the shooting would not be restricted
by the weather as well
That is why filming using a virtual studio
is most trending now
and one of them is motion capture
There is a 2007 film by Zack Snyder
called 300
All of the movie
was shot indoors except for one scene
Even though it was filmed in a studio
it managed to make it very unnoticeable
And again, the advantages of
studio shooting is no restrictions on weather and time
You can choose to shoot
any time on any day
However, you do need thorough preparations
and contents may be limited due to
the lack of background sources
but still this type of shooting is in the limelight
I think it will continue to develop
The motion capture production process
can be largely divided to 4 sections
In the past, production was done step by step
starting with the pre-production
then shoot and then post-production
Now we can shoot real time
which greatly reduces the risk on post-production work
I will talk about real time shooting later
and for now explain what
steps we have here
Preparations Before Shooting
The most important element during preparation
is a previous meeting
It is important to thoroughly explain the motion capture filming
because not all the crew members know very well
Motion capture filming requires
different shooting methods
and need to separate the body, face and fingers
We need to check equipment need for each of the shoot
and secure all the markers needed depending on the number of actors
You would also need to decide
if you would place markers on the
fingers or wear gloves
From time to time you need to
move to a different set because the size doesn't fit
When this happens if you are doing a magnetic system shoot
sets with steel materials
should be avoided as they affect the sensor
Because the devices are wireless
we could be using the same frequencies
as other wireless devices, like sound devices
In that case, you need to check
and pay attention to interference
Magnetic systems are not limited to space
so it is possible to shoot outdoors
On the other hand, optical systems are restricted
so it is crucial to talk about it during the meeting
For example if we wanted a stairs scene
For optical, we need to build stair-type structure
in order to shoot, but for magnetic
we can simply go to a staircase and shoot there
So this could be a huge
disadvantage of optical systems
because you would have to build the stairs
install them to the scene to shoot
and remove them again for the next sceene
Whereas for magnetic system
you can just shoot at the place you have in mind
Therefore it is important to
talk and plan things out
thoroughly before shooting
Optical system is sensitive to light
so we need a noise check caused by light
and also check if the lights reflect on the floor
If it reflects, we need reconstruct the floors
and consider its visuals on the screen
If we own both Optical and
Magnetic systems
It is better to incorporate both into filming
For example, optical system is weak
in movements that cover the markers, like falling
so if we could secure this data by shooting in magnetic system
the cleanup time would be significantly reduced
So the use of both systems
will result in more efficient motion capture shooting
During the meeting you should also talk about
agenda scheduling and actor recruitment
and also check and prepare props
For optical shooting, props and structures
need to be in a wire-framed form
That is because the cameras
are installed almost everywhere
top and bottom as well
they will be installed at any angle as needed
If the prop or structure is filled inside
The camera on the other side
will not be able to find the markers
Smaller props are fine
but bigger structures like a door
or a car
the camera on the opposite side
will not pick up the markers and therefore no data will be extracted
So this is a very important part
We also need to check reference cameras and monitoring system
The monitoring system is for the director's confirmation purposes
and they may request videos taken from various of angles
It applies to not only the body
but also the facial shooting
Directors also consider the front
and side views to be very important
They may request images taken from multiple angles
for the body, and the face as well
So for those, we would need reference cameras
as well as extra man-power
which needs to be confirmed during the meeting
Motion Capture Shooting
Once the preparations are finished, we will go into shooting
Although the shooting sequences have been decided during the meeting
it is best to double check on the shooting day of
A thorough rehearsal may
solve any issues
but if no rehearsal took place
It will be better to adjust the schedule according to the
condition of the actors, equipment situation and movement difficulty
For example running will cause stamina loss
so it could be placed towards the end
You need to make decisions like
If it is partial face shooting
place it in the morning
and body shooting in the afternoon
After these things are sorted out
we start with Calibration first
This is substituting the actual
equipment's settings into a computer
For the Gyro system, after wearing the suit
specify the position value for each sensor
measure and write down the measurements of each joint
Then a skeleton will be created to match the actor's height and joints
Then, you specify the axis
For Optical systems
Set up the cameras in specific locations
and allot the axis
Next, wear the suit and markers
ROM, short for Range of Motion
It creates the skeleton by entering the joint's range of motion
and doing the T pose
For Optical systems
Appointing each camera's position is very important
Most of the time of
Calibration will be used here
So it is best to do them before shooting
Redo the Calibrations as needed
during the shoot when
data is not coming through
or too much noise is processed
Magnetic system sets the axis
after wearing the suit and sensors
while for Optical, the axis is set
after assigning
camera location
These are important differences
When these preparations are done, you would
go through final rehearsal before motion capture shooting
Motion capture shooting crew would
make close communications with the director
Install props and structures appropriately
and respond well to the scene
At this time, you must check whether
cleanup is possible while checking the data
to reduce post-production time
Figuring out the channel of movements is also needed
Since the space is limited
We might need separate shooting if long space is needed
Data may not come out well
for movements made at the edges
So we need to figure out how to use the space
and also reallocate the structures accordingly
Crew should also make sure we have enough markers
and double check that they are all at the right places
Crew should always be mindful of
the equipment and its well being as well
Data Clean-up and Converting
The data gathered from shooting
needs a post-edit called Clean up
Real time shooting is important
but clean up and converting are as well
An amazing shoot with not so great Clean-up
will result in
decreased quality results when applied to characters
Both systems go through the ReProcess
It applies more detailed options
to the data extracted to make them
in a state that is good to work with during clean up
For Magnetic system, only ReProcess is needed
but for Optical systems
Marker Data Clean up is also necessary
Marker Clean up may differ by equipment manufacturers
But it could be done efficiently with
some basic rules set
First is the Labeling process
Placing names or ID for the marker data
to help them not get mixed up
One actor wears about 50 markers
If these are mixed up
Then the skeleton will eventually be twisted as well
so the first label process needs to be well done
Next is Cut Key
Due to the nature of the optical system, calculating on a frame-by-frame
may result in errors depending on the filming environment
For example, movements that cover the markers
That will be considered
data calculation error if the
camera looses track of the markers
So if we have many movments as such
errors will occur continuously
So, after the Cut Key is completed
t is necessary to connect the broken sections
This is called the Fill Gap
It is recommended to
apply a filter to smoothen the data before
running Fill Gap
If not, the graph would not be
correct
It's good to make a habit of smooth work
before doing the fill gap
This completes the work on the marker data
You can extract the marker data here
and then use Solve Bone to
convert it to a skeleton
and extract the Bone data
Solve Bone is the actor's skeleton
during motion capture filming
So it needs a separate
3D converting job
Now I will explain converting
We can process converting in 2 ways
Actorize, which uses markers to convert
and Characterize, which uses
Solve Bone
They both have their pros and cons
Characterize that uses markers
is straightforward and easy to use
so it could save you time
So this is preferred in Freevis where data must be
used immediately
This is because you receive the data quickly
make temp edits, and quickly determine camera angles or
animation modifications
Actorize is based on marker data
to convert
and takes significant amount of time
installing the detailed settings
But it has higher data accuracy
than Characterize
I recommend going through Actorize processing
Motion capture personnel must convert the animation
to a format appropriate for each tool
such as Maya, Max, or Unreal
and also should be able to make suggestions
on how to convert and secure
better quality data
Now as a review, let's summarize the
technologies required for motion capture
Shooting location is
probably the most important in Motion Capture
Spaces need to be designed according to
different movements
The range of the capture depends on
what system is used and
how much space is available
In addition to various marker settings
It is good to set markers considering the cleanup aspect
for restoring lost items
when shooting motion capture
For example if there are lots of movements
lying down, move the markers on the back to the fronts
Or we can reposition the camera lower
So we can see the bottom part as well
These kind of decisions all have
data cleanup in mind
Of course, if you use the magnetic system
you won't have to do that
So you should use what is more
appropriate to your shooting
Motion capture usually works on humans
but sometimes sensors are attached to cameras
and extracts the camera's movements
and they could also be attached to animals
and tools
like any weapons or structures
to extract their data
So a diverse of Bone setting techniques
are necessary
We need to offer
Bone that is applicable to humans
animals, cameras
and structures
Also, for animators who use different tools
he data must be transmitted by converting it
into a format suitable for each software
Lastly, we need to respond well to incorporating
virtual studios through real-time shooting
in line with today's trends
There will be more explanations on
Virtual studios on the third lecture
This wraps up the second lecture
Overview
Preparation for filming 4 steps in the motion capture production process
Preparation for filming: schedule, quotation, contract, preliminary meeting, recruiting actors, preparing props. Filming: Wearing a suit, attaching markers (sensors), Stage-Coordinate, communication, field response, structure
Cleanup: ReProcess, Lable, Cut Key, Filter, Fill Gap, Solve Bone Converting: Actorize, Characterize, Motionbuild
Motion Capture Filming
Calibration Substituting the settings of equipment in a real space into a computer
Essential Elements of an Optical System Camera positioning and axis designation Save post-production time by checking whether cleanup is possible
Data Cleanup and Conversion ReProcess Convert data extracted by markers or sensors into 3D data
Mark data cleanup process Lable: Specifies about 50 marker data with a given name. Cut Key: Cutting out protruding frames
Fill Gap: The task of connecting broken sections of the graph. Solve Bone: The task of converting to skeleton data
Application skills required for motion capture Expansion of motion capture shooting space (Motion Capture Setting) Various marker settings (Motion Capture Maker)
Lost data restoration technology (Motion Capture Editing) Various bone settings (Motion Data Bone)
Various format conversion technologies (3D Tool Converting) Virtual studio integration (Real Time Motion Capture)