WEBVTT

1
00:00:27.513 --> 00:00:28.553
Hello

2
00:00:28.553 --> 00:00:32.700
My name is Hee-seok Jeong, and I will be giving a lecture on Motion Capture

3
00:00:32.700 --> 00:00:39.299
I have been working on motion capture for 24 years since 1999

4
00:00:39.299 --> 00:00:44.459
I've worked on projects in various fields such as

5
00:00:44.459 --> 00:00:47.540
movies, games, music videos, and motion analysis

6
00:00:47.540 --> 00:00:52.619
I currently run a motion capture studio for Westworld

7
00:00:52.619 --> 00:00:56.820
Nowadays, movies like 'Avatar'

8
00:00:56.820 --> 00:01:02.020
and 'Planet of the Apes' use motion capture

9
00:01:02.020 --> 00:01:05.660
When I first started motion capture

10
00:01:05.660 --> 00:01:11.059
There was no manual or barely any data on it

11
00:01:11.059 --> 00:01:14.180
I had a hard time

12
00:01:14.180 --> 00:01:17.180
Now you could learn them from

13
00:01:17.180 --> 00:01:22.220
online or even Youtube

14
00:01:22.220 --> 00:01:25.860
Still, I think it is very meaningful

15
00:01:25.860 --> 00:01:28.940
to have this time

16
00:01:28.940 --> 00:01:33.820
to have this opportunity where I can

17
00:01:33.820 --> 00:01:37.339
share my knowledge of motion capture

18
00:01:37.339 --> 00:01:41.940
My lesson on Motion Capture

19
00:01:41.940 --> 00:01:45.379
will be divided in 3 lectures

20
00:01:45.379 --> 00:01:50.139
Rather than talking about motion capture's history, theories, or manuals

21
00:01:50.139 --> 00:01:53.059
I plan to give a lecture based on my experiences

22
00:01:53.059 --> 00:01:57.300
for your overall understanding

23
00:01:57.300 --> 00:02:01.739
Also the concept, types, and areas of use of motion capture

24
00:02:01.739 --> 00:02:05.820
and how the field of motion capture has expanded

25
00:02:05.820 --> 00:02:09.259
due to technological advancements

26
00:02:09.259 --> 00:02:12.419
The first and second lectures

27
00:02:12.419 --> 00:02:17.059
will be heavy on theory and explanation

28
00:02:17.059 --> 00:02:21.179
they third one would be based on videos

29
00:02:21.179 --> 00:02:25.300
Once you've been through all 3 lectures

30
00:02:25.300 --> 00:02:29.779
I think you would be able to gain general understanding of

31
00:02:29.779 --> 00:02:34.740
how motion capture works, the advancements of technology

32
00:02:34.740 --> 00:02:38.419
how filming is done in Korea and abroad these days

33
00:02:38.419 --> 00:02:43.339
and how data is produced

34
00:02:43.339 --> 00:02:47.020
The first part is understanding motion capture

35
00:02:47.020 --> 00:02:50.940
This time, I will explain and help you understand

36
00:02:50.940 --> 00:02:53.979
the types of motion capture, the fields of use

37
00:02:53.979 --> 00:02:58.419
and the technologies required for them

38
00:02:58.419 --> 00:03:04.339
Lesson objective is to understand the concept of motion capture

39
00:03:04.339 --> 00:03:06.779
by learning the basics

40
00:03:06.779 --> 00:03:09.500
they types of motion capture, and its fields of use

41
00:03:09.500 --> 00:03:12.589
We will learn application techniques and basic skills

42
00:03:12.589 --> 00:03:15.790
required for each type of motion capture

43
00:03:15.848 --> 00:03:19.398
Understanding of Motion Capture

44
00:03:20.490 --> 00:03:26.139
Motion capture refers to recording all moving objects

45
00:03:26.139 --> 00:03:30.699
including people, animals, tools, and cameras

46
00:03:30.699 --> 00:03:32.899
in a computer-usable format

47
00:03:32.899 --> 00:03:35.799
It is used when you want to apply the actual movement

48
00:03:35.799 --> 00:03:39.300
of a person or object as is

49
00:03:39.300 --> 00:03:42.899
or when we have to film a difficult movement

50
00:03:42.899 --> 00:03:44.530
or a dangerous scene

51
00:03:44.619 --> 00:03:48.740
Most motion capture involves attaching

52
00:03:48.740 --> 00:03:52.339
markers or sensors to people

53
00:03:52.339 --> 00:03:56.860
That is to apply it to 3D characters

54
00:03:56.860 --> 00:03:59.779
We also attach sensors to

55
00:03:59.779 --> 00:04:02.079
animals or tools that people hold

56
00:04:02.079 --> 00:04:05.339
weapons, like knives or guns

57
00:04:05.339 --> 00:04:09.300
to extract their data

58
00:04:09.300 --> 00:04:12.460
Sometimes we attache them to actual cameras

59
00:04:12.460 --> 00:04:17.100
to use as virtual cameras and extract camera angles

60
00:04:17.100 --> 00:04:20.899
Motion capture was first created for medical reasons

61
00:04:20.899 --> 00:04:24.449
and was advanced greatly in fields of

62
00:04:24.449 --> 00:04:27.059
virtual reality, movies and games

63
00:04:27.059 --> 00:04:30.700
Recently, most of the content using computers

64
00:04:30.700 --> 00:04:33.140
is created as 3D images

65
00:04:33.140 --> 00:04:37.700
The implementation of virtual reality and virtual characters have emerged

66
00:04:37.700 --> 00:04:41.739
These things created an opportunity for us

67
00:04:41.739 --> 00:04:44.339
 to easily access them in real life through

68
00:04:44.339 --> 00:04:46.374
the use of motion capture systems

69
00:04:46.374 --> 00:04:49.774
Types of Motion Capture

70
00:04:50.579 --> 00:04:54.779
Next we will look into the types of motion capture

71
00:04:54.779 --> 00:04:59.260
Motion capture works by attaching markers or sensors

72
00:04:59.260 --> 00:05:01.899
to various parts of the actor's body

73
00:05:01.899 --> 00:05:06.059
determining the position values of them

74
00:05:06.059 --> 00:05:10.420
then animating the movements of the 3D character, or digital character

75
00:05:11.100 --> 00:05:14.260
Motion capture is usully distinguished

76
00:05:14.260 --> 00:05:17.339
by the type of sensors

77
00:05:17.339 --> 00:05:19.589
The number of sensors

78
00:05:19.589 --> 00:05:22.059
determine the expression of

79
00:05:22.059 --> 00:05:26.779
natural movements

80
00:05:26.779 --> 00:05:32.779
It is largely divided in to 4 types of systems:

81
00:05:32.779 --> 00:05:36.299
Acoustic, Mechanical, Magnetic and Optical

82
00:05:36.299 --> 00:05:39.980
I will explain them one by one

83
00:05:39.980 --> 00:05:43.299
First is the Acoustic system

84
00:05:43.299 --> 00:05:48.579
It consists of multiple ultrasonic generators and three receivers

85
00:05:48.579 --> 00:05:54.140
Ultrasound waves are generated from a device attached to each joint of the actor

86
00:05:54.140 --> 00:05:56.820
and this system calculates the time and distance

87
00:05:56.820 --> 00:06:01.619
compared to the speed of ultrasonic waves

88
00:06:01.619 --> 00:06:05.140
The position of each device in 3 dimensional space

89
00:06:05.140 --> 00:06:09.299
can be implemented by the triangulation principle

90
00:06:09.299 --> 00:06:13.179
using values ​​calculated from each of the three receiving devices

91
00:06:13.179 --> 00:06:16.579
The downside is that it is difficult to simultaneously measure

92
00:06:16.579 --> 00:06:19.940
the positions of all ultrasonic generators

93
00:06:19.940 --> 00:06:22.059
attached to the actor's body

94
00:06:22.059 --> 00:06:25.109
and the size of the generator device is

95
00:06:25.109 --> 00:06:26.579
big and has cables

96
00:06:26.579 --> 00:06:30.540
which may result in unnatural and awkward movements of the actor

97
00:06:30.540 --> 00:06:34.980
The second is the Mechanical system

98
00:06:34.980 --> 00:06:41.059
This method attaches articulated steel equipment to actors

99
00:06:41.059 --> 00:06:45.700
This steel device has variable resistance

100
00:06:45.700 --> 00:06:48.779
attached to each joint

101
00:06:48.779 --> 00:06:52.079
allowing the mechanical device to measure

102
00:06:52.079 --> 00:06:53.420
how much a person bends their arm or leg

103
00:06:53.420 --> 00:06:56.059
Simply put, the variable resistance is

104
00:06:56.059 --> 00:07:00.220
a sensor, like an electromagnet

105
00:07:00.220 --> 00:07:04.579
Installation and operation is easy and is not limited to space

106
00:07:04.579 --> 00:07:07.660
so we could move far away to capture images

107
00:07:07.660 --> 00:07:09.760
and has advantage of

108
00:07:09.760 --> 00:07:13.540
very high sampling frequency

109
00:07:13.540 --> 00:07:18.100
making it easy to use in real time

110
00:07:18.100 --> 00:07:25.339
These are actually from 20 years ago

111
00:07:25.339 --> 00:07:28.089
Now we have better equipment

112
00:07:28.089 --> 00:07:31.940
and can collect real-time data

113
00:07:31.940 --> 00:07:34.339
with higher qualities

114
00:07:34.339 --> 00:07:40.140
The mechanical system is actually not very much used now

115
00:07:40.140 --> 00:07:42.640
Mechanical equipment was used

116
00:07:42.640 --> 00:07:48.540
in games liek Tekken because it was very difficult

117
00:07:48.540 --> 00:07:49.980
to hold the keys

118
00:07:49.980 --> 00:07:55.140
for such movements in the past

119
00:07:55.140 --> 00:07:58.820
The downside is that the equipment

120
00:07:58.820 --> 00:08:01.459
going on the actor's body is very uncomfortable

121
00:08:01.459 --> 00:08:03.779
that we cannot produce natural movements

122
00:08:03.779 --> 00:08:07.299
and has lower accuracy

123
00:08:07.299 --> 00:08:10.859
This is the steel equipment used

124
00:08:10.859 --> 00:08:16.260
This actually weighs about 10 to 15kg

125
00:08:16.260 --> 00:08:18.700
Which makes it hard to do some movements

126
00:08:18.700 --> 00:08:22.700
The acoustic and mechanical systems together

127
00:08:22.700 --> 00:08:24.140
have lots of disadvantages

128
00:08:24.140 --> 00:08:27.140
so it is not very much used now

129
00:08:27.140 --> 00:08:30.019
But about 20 years ago

130
00:08:30.019 --> 00:08:35.739
when we had to use motion capture

131
00:08:35.739 --> 00:08:38.419
for extreme movements

132
00:08:38.419 --> 00:08:43.140
Acoustic systems were almost never used because of their difficulties

133
00:08:43.140 --> 00:08:47.780
So we had to use the mechanical system, although equipment weighed 10 kg

134
00:08:47.780 --> 00:08:51.739
we had to make up for its short comings

135
00:08:51.739 --> 00:08:57.500
and extracted data of the movements we needed

136
00:08:57.500 --> 00:09:00.099
Third is the Magnetic system

137
00:09:00.099 --> 00:09:04.380
This is a method where a magnetic field generator is placed

138
00:09:04.380 --> 00:09:09.219
in a fixed location and a person attaches small magnetic sensors

139
00:09:09.219 --> 00:09:14.340
to each joint to measure the position and angle of each joint

140
00:09:14.340 --> 00:09:17.979
In this method, the magnetic sensor is

141
00:09:17.979 --> 00:09:19.879
connected to the collector

142
00:09:19.879 --> 00:09:23.900
and then from the collector to the computer

143
00:09:23.900 --> 00:09:27.780
Magnetic system is easy to operate

144
00:09:27.780 --> 00:09:30.659
There is no need for investment in studios other than equipment

145
00:09:30.659 --> 00:09:34.659
real-time processing is convenient as the amount of calculation is not large

146
00:09:34.659 --> 00:09:39.460
and there are no shielding issues

147
00:09:39.460 --> 00:09:43.010
In Optical systems, the camera

148
00:09:43.010 --> 00:09:44.859
may block the sensors

149
00:09:44.859 --> 00:09:49.380
Gyro does not have those issueus

150
00:09:49.380 --> 00:09:53.179
post processing is also not needed

151
00:09:53.179 --> 00:09:58.500
One downside is that the extracted data may be a bit inaccurate

152
00:09:58.500 --> 00:10:01.100
As you can see for Magnetic systems

153
00:10:01.100 --> 00:10:04.419
the sensors are already mounted on the suits

154
00:10:04.419 --> 00:10:08.059
so it is easier for actors to make movements

155
00:10:08.059 --> 00:10:11.460
Gyro sensors are used mostly

156
00:10:11.460 --> 00:10:15.500
and finger data could also be extracted by wearing gloves

157
00:10:15.500 --> 00:10:19.600
This gloves have

158
00:10:19.600 --> 00:10:23.099
small gyro sensors in every inch of the fingers

159
00:10:23.099 --> 00:10:27.460
So this equipment can gather

160
00:10:27.460 --> 00:10:33.099
detailed data of the finger joints' movements

161
00:10:33.099 --> 00:10:36.099
I've pointed out that the data extraction may be

162
00:10:36.099 --> 00:10:38.299
a bit inaccurate

163
00:10:38.299 --> 00:10:43.739
Magnetic system sensors only extracts rotation data

164
00:10:43.739 --> 00:10:46.059
For animation, there must be translation

165
00:10:46.059 --> 00:10:48.209
or position value in the note part

166
00:10:48.209 --> 00:10:49.940
but the magnetic system

167
00:10:49.940 --> 00:10:54.179
does not provide position data

168
00:10:54.179 --> 00:10:58.059
So, we have to forcefully make that with the rotation data

169
00:10:58.059 --> 00:11:04.059
For example every time we move from

170
00:11:04.059 --> 00:11:08.900
the pelvis to the knee, from the knee to the ankle, etc.

171
00:11:08.900 --> 00:11:12.979
the error is taken as a square rather than a plus

172
00:11:12.979 --> 00:11:16.529
Therefore the data is bound to be inaccurate

173
00:11:16.529 --> 00:11:19.900
at the toes and fingertips

174
00:11:19.900 --> 00:11:22.179
The more bones we have

175
00:11:22.179 --> 00:11:26.299
In the case of feet, it is about stage 4 or 5

176
00:11:26.299 --> 00:11:29.380
the hands are from root to the waist

177
00:11:29.380 --> 00:11:33.020
The waist has about 3 joints

178
00:11:33.020 --> 00:11:37.059
then the shoulders, elbows and wrists

179
00:11:37.059 --> 00:11:41.460
This takes you through steps 6 and 7

180
00:11:41.460 --> 00:11:46.140
In that case, situations like the feet flowing

181
00:11:46.140 --> 00:11:50.140
or hands going out of sync

182
00:11:50.140 --> 00:11:53.099
may occur

183
00:11:53.099 --> 00:11:56.979
This is why additional processes

184
00:11:56.979 --> 00:11:59.299
for secondary key work are required

185
00:11:59.299 --> 00:12:03.140
This may be the difficult part

186
00:12:03.140 --> 00:12:07.739
in animation after processing

187
00:12:07.739 --> 00:12:10.820
The last one is the Optical system

188
00:12:10.820 --> 00:12:16.059
The Optical system attaches highly reflected markers to actor's body

189
00:12:16.299 --> 00:12:19.539
and captures the image with a camera that emits infrared light

190
00:12:19.539 --> 00:12:23.700
You are able to move your body more freely compared to other systems

191
00:12:23.700 --> 00:12:25.800
and it could almost perfectly capture movements

192
00:12:25.800 --> 00:12:28.619
even the fast fighting movements

193
00:12:28.619 --> 00:12:31.419
However, the equipment is very costly

194
00:12:31.419 --> 00:12:35.419
and is sensitive to surrounding lighting

195
00:12:35.419 --> 00:12:39.179
It is still widely used for its

196
00:12:39.179 --> 00:12:41.419
accuracy in animation data

197
00:12:41.419 --> 00:12:43.900
It has lots of space limitations

198
00:12:43.900 --> 00:12:47.020
and does need extra time for post-processing

199
00:12:47.020 --> 00:12:49.460
The post-processing

200
00:12:49.460 --> 00:12:54.020
is not just a few hours step

201
00:12:54.020 --> 00:12:56.780
It includes calibrations of the captures

202
00:12:56.780 --> 00:13:00.739
and if the camera is

203
00:13:00.739 --> 00:13:05.099
not installed correctly

204
00:13:05.099 --> 00:13:06.979
It could take days or even months to fix

205
00:13:06.979 --> 00:13:10.020
So the optical system requires

206
00:13:10.020 --> 00:13:14.059
a huge amount of effort in the installation process

207
00:13:14.059 --> 00:13:17.900
The optical cameras detects infrared rays

208
00:13:17.900 --> 00:13:23.380
but the sunlight also has infrared lights so we need to over them

209
00:13:23.380 --> 00:13:27.900
So in case of studio shooting, even if they are underground

210
00:13:27.900 --> 00:13:31.539
we need to make sure to cover the windows

211
00:13:31.539 --> 00:13:33.900
If not, they could generate lots of noise

212
00:13:33.900 --> 00:13:36.179
that results in inaccurate data

213
00:13:36.179 --> 00:13:40.219
and thus the post-processing clean up time will increase

214
00:13:40.219 --> 00:13:43.260
In other words, the optical camera's principle

215
00:13:43.260 --> 00:13:45.059
is to emit infrared rays

216
00:13:45.059 --> 00:13:50.539
receive the infrared rays again

217
00:13:50.539 --> 00:13:54.260
Sunlight contains small amount of infrared

218
00:13:54.260 --> 00:13:58.380
and when this is exposed

219
00:13:58.380 --> 00:14:02.979
The cameras will pick up both

220
00:14:02.979 --> 00:14:05.739
the sunlight and markers and thus create noise

221
00:14:05.739 --> 00:14:12.739
In the past the noise level was not twice or three times

222
00:14:12.739 --> 00:14:15.780
but it was about 10 times the noise we have now

223
00:14:15.780 --> 00:14:20.580
The clean up time was

224
00:14:20.580 --> 00:14:22.940
significantly lengthy

225
00:14:22.940 --> 00:14:24.820
Now we have more experiences with the equipment

226
00:14:24.820 --> 00:14:29.020
and also lots of advancement in technology

227
00:14:29.020 --> 00:14:31.419
that we have less noise

228
00:14:31.419 --> 00:14:34.380
but still the clean up time is inevitable

229
00:14:35.099 --> 00:14:38.619
Therefore, installation must be very sensitive

230
00:14:38.619 --> 00:14:40.769
to things such as

231
00:14:40.769 --> 00:14:43.380
exposure to sunlight

232
00:14:43.380 --> 00:14:48.739
As you can see here the markers are attached on top of the suits

233
00:14:48.739 --> 00:14:51.089
and we can also add the head mount

234
00:14:51.089 --> 00:14:53.539
for facial capture

235
00:14:53.539 --> 00:14:57.099
Recently, technology has emerged to extract

236
00:14:57.099 --> 00:15:01.340
motion capture data using video cameras

237
00:15:01.340 --> 00:15:03.700
Multiple cameras are installed

238
00:15:03.700 --> 00:15:06.619
and these are not an optical, but cameras that

239
00:15:06.619 --> 00:15:09.020
record everyday videos like a GoPro

240
00:15:09.020 --> 00:15:13.500
about 4 to 8 cameras are set up in a circle

241
00:15:13.500 --> 00:15:16.859
From here, we don't extract data from

242
00:15:16.859 --> 00:15:19.059
gyro sensors or markers

243
00:15:19.059 --> 00:15:21.979
but extract the person's movements

244
00:15:21.979 --> 00:15:25.500
from the video to make a skeleton

245
00:15:25.500 --> 00:15:28.659
Of course, this method

246
00:15:28.659 --> 00:15:33.099
may lack accuracy compared to those that use sensors and markers

247
00:15:33.099 --> 00:15:37.700
but this part can be made up by the AI technology

248
00:15:37.700 --> 00:15:40.979
Securing deep data for deep learning is key

249
00:15:40.979 --> 00:15:44.020
In the future, it is expected that these technologies

250
00:15:44.020 --> 00:15:45.870
will continue to develop

251
00:15:45.870 --> 00:15:49.685
so that motion data can be more easily obtained

252
00:15:49.685 --> 00:15:53.135
Fields of Use - Motion Capture

253
00:15:53.700 --> 00:16:01.739
Before motion capture was used in film, animation and games

254
00:16:01.739 --> 00:16:06.140
they were widely used in medicine, sports

255
00:16:06.140 --> 00:16:09.140
education and ergonomics

256
00:16:09.140 --> 00:16:12.659
Walking, for example

257
00:16:12.659 --> 00:16:17.739
We would film steps of a person with joint discomfort

258
00:16:17.739 --> 00:16:20.939
from the front, side, from behind

259
00:16:20.939 --> 00:16:23.219
and from above

260
00:16:23.219 --> 00:16:27.219
and then analyzed while watching the video

261
00:16:27.219 --> 00:16:31.619
However, it was lacking in detail

262
00:16:31.619 --> 00:16:37.940
in terms of time and accuracy

263
00:16:37.940 --> 00:16:42.219
By using marker data while using the optical system

264
00:16:42.219 --> 00:16:45.780
you can measure rotation values ​​and position data for joints

265
00:16:45.780 --> 00:16:50.059
and furthermore, speed and acceleration

266
00:16:50.059 --> 00:16:54.259
Therefore, very detailed and highly accurate data

267
00:16:54.259 --> 00:16:57.260
can be calculated

268
00:16:57.260 --> 00:17:02.760
Rehabilitation doctors

269
00:17:02.760 --> 00:17:05.780
or those who manage these data

270
00:17:05.780 --> 00:17:10.459
can receive data on rotation or position values ​​in Excel

271
00:17:10.459 --> 00:17:15.159
easily measure them

272
00:17:15.159 --> 00:17:19.060
and calculate the data

273
00:17:19.060 --> 00:17:22.739
And we developed from there

274
00:17:22.739 --> 00:17:28.099
to film making and games

275
00:17:28.099 --> 00:17:32.979
For movie production, motion capture is  mainly used

276
00:17:32.979 --> 00:17:35.420
in scenes where acting may be difficult

277
00:17:35.420 --> 00:17:38.380
or where a large number of people is needed

278
00:17:38.380 --> 00:17:42.459
in the film

279
00:17:42.459 --> 00:17:46.359
Sports analysis is an important field

280
00:17:46.359 --> 00:17:49.060
of motion capture use

281
00:17:49.060 --> 00:17:52.420
Motion capture data has been carefully categorized

282
00:17:52.420 --> 00:17:57.500
so that different elements of each movement can be studied

283
00:17:57.500 --> 00:18:02.699
Sports and activities such as track, golf swings

284
00:18:02.699 --> 00:18:06.459
swing analysis for tennis

285
00:18:06.459 --> 00:18:10.109
and even swimming in water

286
00:18:10.109 --> 00:18:12.699
all use motion data

287
00:18:12.699 --> 00:18:19.099
to analyze movements and improve the performance of athletes

288
00:18:19.099 --> 00:18:23.249
In medicine, it initially began to be used as a gait analyzer in orthopedics

289
00:18:23.249 --> 00:18:26.140
and rehabilitation departments

290
00:18:26.140 --> 00:18:30.790
Currently, it is used for spinal analysis, design of prosthetic hands and limbs

291
00:18:30.790 --> 00:18:34.540
calculation of biomechanical data

292
00:18:34.540 --> 00:18:37.690
and these data are widely used for physical therapy

293
00:18:37.690 --> 00:18:40.099
and rehabilitation treatment of patients

294
00:18:40.099 --> 00:18:42.349
In the industrial field

295
00:18:42.349 --> 00:18:44.999
it is used in the field of ergonomics

296
00:18:44.999 --> 00:18:47.260
necessary for robot development and automobile development

297
00:18:47.260 --> 00:18:50.260
For example when making a car

298
00:18:50.260 --> 00:18:53.260
the height, handle distance, etc.

299
00:18:53.260 --> 00:18:55.510
The level of comfort for the passenger is

300
00:18:55.510 --> 00:18:58.459
measured and developed

301
00:18:58.459 --> 00:19:02.099
using motion capture equipment

302
00:19:02.099 --> 00:19:07.020
Markers are also attached to the car's bonnet and doors

303
00:19:07.020 --> 00:19:10.720
to measure how much the gap fat vibrates

304
00:19:10.720 --> 00:19:12.420
to help with development

305
00:19:12.420 --> 00:19:16.660
I actually tried measuring that

306
00:19:16.660 --> 00:19:22.910
and I found that the vibrations last

307
00:19:22.910 --> 00:19:25.060
5 to 10 seconds

308
00:19:25.060 --> 00:19:30.020
These measurements help with the development of cars

309
00:19:30.020 --> 00:19:33.260
I also had an experience of measuring a robot arm

310
00:19:33.260 --> 00:19:35.210
When assembling a car

311
00:19:35.210 --> 00:19:37.660
there are times when you need to measure a robot arm

312
00:19:37.660 --> 00:19:41.619
that must always maintain the same motion

313
00:19:41.619 --> 00:19:45.519
In the past they made camera measurements

314
00:19:45.519 --> 00:19:48.020
by attaching small bristles

315
00:19:48.020 --> 00:19:52.819
In this case, if we measure in one direction a day for one month

316
00:19:52.819 --> 00:19:58.500
we would only end up with 30 direction data

317
00:19:58.500 --> 00:20:06.380
But with motion capture, you can obtain 360 degrees

318
00:20:06.380 --> 00:20:11.180
of data with a single measurement by attaching a small, 10cm marker

319
00:20:11.180 --> 00:20:15.140
The accuracy is about 0.1mm

320
00:20:15.140 --> 00:20:19.619
It's completely different from video extraction

321
00:20:19.619 --> 00:20:22.419
Like this

322
00:20:22.419 --> 00:20:24.739
Motion capture can replace

323
00:20:24.739 --> 00:20:28.939
a year's worth of measurements

324
00:20:28.939 --> 00:20:31.979
I've prepared two videos

325
00:20:31.979 --> 00:20:35.929
In this video, motion capture technology

326
00:20:35.929 --> 00:20:38.180
was used in developing hiking wear

327
00:20:38.180 --> 00:20:42.330
Sensors are attached in 5cm distances on the body

328
00:20:42.330 --> 00:20:45.739
then data is gathered by doing climbing motions

329
00:20:45.739 --> 00:20:48.239
This is an example of how

330
00:20:48.239 --> 00:20:51.540
the data on how much the arms and legs stretch

331
00:20:51.540 --> 00:20:55.699
is quantified and used in climbing clothing

332
00:20:55.699 --> 00:20:59.180
The data is made of

333
00:20:59.180 --> 00:21:02.900
measurements on the bare body

334
00:21:02.900 --> 00:21:06.380
and measurements after clothing

335
00:21:06.380 --> 00:21:11.180
and the difference of those two

336
00:21:11.180 --> 00:21:15.979
As a result, we can put into numbers

337
00:21:15.979 --> 00:21:18.540
the stretches of the bare body

338
00:21:18.579 --> 00:21:22.180
and then with clothing

339
00:21:22.180 --> 00:21:27.680
and use the error between numbers to

340
00:21:27.680 --> 00:21:29.780
make the developments

341
00:21:29.780 --> 00:21:31.780
Let's watch the video

342
00:21:32.226 --> 00:22:12.330
Developing Climb-wear Using Motion Capture Technology

343
00:22:12.392 --> 00:22:17.979
In the next video, we motion captured some ski movements

344
00:22:17.979 --> 00:22:20.979
after installing the optical system in the ski park

345
00:22:20.979 --> 00:22:27.380
The purpose of this experiment was to measure

346
00:22:27.380 --> 00:22:32.300
 the ski turning movement of normal people

347
00:22:32.300 --> 00:22:35.500
and that of disabled people

348
00:22:35.500 --> 00:22:39.260
The difficult part here was

349
00:22:39.260 --> 00:22:43.579
filming in optical system in day time

350
00:22:43.579 --> 00:22:47.780
As said before, noise will be created in sunlight

351
00:22:47.810 --> 00:22:51.180
and the snow accumulated on the ground has reflections

352
00:22:51.180 --> 00:22:54.660
which results in more noise

353
00:22:54.660 --> 00:22:57.699
So filming had to be done at night time

354
00:22:57.699 --> 00:23:02.180
The next difficulty was the coldness

355
00:23:02.180 --> 00:23:04.979
because it was even colder in the night there

356
00:23:04.979 --> 00:23:08.739
It felt like negative 15-20 degrees Celcius

357
00:23:08.739 --> 00:23:10.889
Shooting in that weather

358
00:23:10.889 --> 00:23:15.459
attaching all the markers was very difficult to do

359
00:23:15.459 --> 00:23:19.180
We had to keep the cameras running

360
00:23:19.180 --> 00:23:23.540
during daytime so it would not freeze

361
00:23:23.540 --> 00:23:28.339
We managed to finish them and gather good data

362
00:23:28.339 --> 00:23:33.140
I will explain after watching the video

363
00:23:33.802 --> 00:23:53.604
Measuring Ski Movements Using Motion Capture Technology

364
00:23:53.660 --> 00:23:59.129
As you see, motion capture technology is being used in various fields

365
00:23:59.129 --> 00:24:01.779
Simulations using VR are also being developed

366
00:24:01.779 --> 00:24:04.900
using motion capture

367
00:24:04.900 --> 00:24:08.739
Now I will talk about the technical parts of motion capture

368
00:24:08.739 --> 00:24:14.020
Nowadays acoustic and mechanical systems are not being used

369
00:24:14.020 --> 00:24:17.619
but Magnetic and Optical systems are

370
00:24:17.619 --> 00:24:21.380
So this will be mainly based on those two

371
00:24:21.380 --> 00:24:24.979
The first common factor is space

372
00:24:24.979 --> 00:24:28.180
For magnetic, 30m or up to 50m of space

373
00:24:28.180 --> 00:24:33.780
can be secured, depending on the specifications of the equipment

374
00:24:33.780 --> 00:24:37.939
There are no major space restrictions, but as for height

375
00:24:37.939 --> 00:24:41.380
you should be mindful that data does not come out well

376
00:24:41.380 --> 00:24:44.500
for movements in the air

377
00:24:44.500 --> 00:24:46.660
What this means is that

378
00:24:46.660 --> 00:24:50.180
For the magnetic system, only rotation data is produced

379
00:24:50.180 --> 00:24:54.180
 by the gyro sensor and the position value cannot be secured

380
00:24:54.180 --> 00:24:58.660
So movements like pull ups or

381
00:24:58.660 --> 00:25:02.900
flying in air on karts

382
00:25:02.900 --> 00:25:05.100
they have no position values

383
00:25:05.100 --> 00:25:08.260
so the data becomes floating as well

384
00:25:08.260 --> 00:25:13.219
The gyro system can sensor short jumps

385
00:25:13.219 --> 00:25:16.420
that land soon after

386
00:25:16.420 --> 00:25:20.099
but cannot gather data from movements

387
00:25:20.099 --> 00:25:21.540
that stay in air for a long time

388
00:25:21.540 --> 00:25:24.260
This is to take note of

389
00:25:24.260 --> 00:25:29.780
So positions would need to be on the ground

390
00:25:29.780 --> 00:25:33.780
For optical, the quantity of cameras is very important

391
00:25:33.780 --> 00:25:36.020
it requires capturing the markers with cameras

392
00:25:36.020 --> 00:25:38.900
More cameras mean more space

393
00:25:38.900 --> 00:25:42.739
But that may be restricted due to

394
00:25:42.739 --> 00:25:45.219
financial issues

395
00:25:45.219 --> 00:25:49.060
Therefore, securing space with limited number of cameras is

396
00:25:49.060 --> 00:25:51.540
one of the important skill

397
00:25:51.540 --> 00:25:55.060
Next is marker settings

398
00:25:55.060 --> 00:25:59.859
Depending on how you set the marker in the optical system

399
00:25:59.859 --> 00:26:03.859
cleanup time can be greatly reduced during post-production

400
00:26:03.859 --> 00:26:08.500
You may need to change the settings if some markers are covered

401
00:26:08.500 --> 00:26:12.739
We also need installations on fingers and other structures

402
00:26:12.739 --> 00:26:16.339
we need to be flexible and mindful of the placements

403
00:26:16.339 --> 00:26:19.219
Even after the marker settings were done well

404
00:26:19.219 --> 00:26:22.739
The data may be cut off or have errors

405
00:26:22.739 --> 00:26:24.979
during the acting

406
00:26:24.979 --> 00:26:27.780
We need a detailed skill in orderto

407
00:26:27.780 --> 00:26:29.699
restore these lost data

408
00:26:29.699 --> 00:26:32.180
This sometimes takes more than 50%

409
00:26:32.180 --> 00:26:37.060
of Optical system processing

410
00:26:37.060 --> 00:26:39.300
Some movements or postures

411
00:26:39.300 --> 00:26:42.180
like crossing one's arms

412
00:26:42.180 --> 00:26:44.260
or hugging each other

413
00:26:44.260 --> 00:26:46.900
they cover the markers a lot

414
00:26:46.900 --> 00:26:50.180
It is bound to take a lot of work process

415
00:26:50.180 --> 00:26:54.900
To put it simply, simple movements

416
00:26:54.900 --> 00:26:58.420
makes easy of a proccess

417
00:26:58.420 --> 00:27:01.780
but for movements like squatting

418
00:27:01.780 --> 00:27:05.140
which covers a lot of the markers

419
00:27:05.140 --> 00:27:10.739
these will result in a lot of clean up processing

420
00:27:10.739 --> 00:27:14.260
When the marker data is restored

421
00:27:14.260 --> 00:27:19.219
we need to convert it to skeleton data made of bones

422
00:27:19.219 --> 00:27:23.060
We need to set up not only human-form on two legs

423
00:27:23.060 --> 00:27:26.900
but also in 4 legged animal form

424
00:27:26.900 --> 00:27:31.300
and also in the form of inverted joins

425
00:27:31.300 --> 00:27:34.660
When the skeleton data is made

426
00:27:34.660 --> 00:27:38.420
we also need to convert it again

427
00:27:38.420 --> 00:27:42.819
into various formats for animator to use

428
00:27:42.819 --> 00:27:46.099
It must be converted to be used in

429
00:27:46.099 --> 00:27:50.420
Maya, Max, and Unreal

430
00:27:50.420 --> 00:27:55.459
If necessary, you may need to convert directly to the desired character

431
00:27:55.459 --> 00:27:59.219
And lastly, incorporating a virtual studio

432
00:27:59.300 --> 00:28:02.260
This is a real time technique

433
00:28:02.260 --> 00:28:05.810
It requires basic knowledge of network technology

434
00:28:05.810 --> 00:28:09.519
or interworking with motion capture equipment and

435
00:28:09.519 --> 00:28:14.290
Motion Builder or Unreal to show

436
00:28:14.290 --> 00:28:16.180
as the final output

437
00:28:16.180 --> 00:28:21.380
The primary task is considered finished by

438
00:28:21.380 --> 00:28:24.130
extracting the data using the application technology

439
00:28:24.130 --> 00:28:26.020
required for motion capture

440
00:28:26.020 --> 00:28:29.219
based on a general understanding

441
00:28:29.219 --> 00:28:31.939
of the motion capture system

442
00:28:31.939 --> 00:28:37.380
Furthermore, applying it to Motion Builder or Unreal in real time

443
00:28:37.380 --> 00:28:42.300
to show directly on the character while shooting

444
00:28:42.300 --> 00:28:44.819
will be the key skill

445
00:28:44.819 --> 00:28:49.380
This will wrap up the first lecture on Motion capture

446
00:28:50.374 --> 00:28:58.024
Understanding Motion Capture
What is motion capture? 
A computer-readable record of all moving objects
Used to create scenes where the application or execution of actual movements is dangerous

447
00:28:58.024 --> 00:29:00.024
Types of motion capture
Acoustic system
A system that calculates the time and distance it takes compared to the speed for ultrasonic waves generated from a device attached to the actor's joints to be received

448
00:29:00.024 --> 00:29:02.024
Mechanic system
Method of capturing by attaching iron devices to the actor's joints

449
00:29:02.024 --> 00:29:04.024
Magnetic system
A method of measuring the position and angle of a human joint by attaching a small magnetic sensor to the joint

450
00:29:04.024 --> 00:29:06.024
Optical system
A method of attaching a highly reflective marker to a person's body and capturing it with a camera that emits infrared light

451
00:29:06.024 --> 00:29:10.024
Areas of use for motion capture
Entertainment: Movies, games, commercials, music videos, animation
Movement analysis: rehabilitation, physical therapy, gait analysis, spine analysis, sports

452
00:29:10.024 --> 00:29:14.024
Industrial: Ergonomics, automotive development, robot development
Others: VR, simulation