WEBVTT

1
00:00:23.720 --> 00:00:28.470
Hello, this is Ju Yeong Lim, the director of Westworld technology research

2
00:00:28.470 --> 00:00:32.170
In this GCC realistic content curriculum

3
00:00:32.170 --> 00:00:37.440
I will tell you about virtual production during the fourth session

4
00:00:37.440 --> 00:00:40.290
Among virtual productions

5
00:00:40.290 --> 00:00:44.120
I would like to talk about movies, dramas, or XR content

6
00:00:44.120 --> 00:00:48.840
First, in the first session, I would like to introduce the concept of virtual production

7
00:00:48.840 --> 00:00:54.499
And provide a general overview of how it is used

8
00:00:54.907 --> 00:00:58.256
Virtual Production's Concept, Entertainment Industry, and Animation Use Cases

9
00:00:58.759 --> 00:01:01.809
Some of you may have heard of virtual production

10
00:01:01.809 --> 00:01:03.560
While others may be unfamiliar

11
00:01:03.560 --> 00:01:07.110
And even those who have heard of virtual production

12
00:01:07.310 --> 00:01:12.119
May think it is difficult to approach because it is a technical topic

13
00:01:12.119 --> 00:01:16.469
In reality, there are indeed many technical elements among the various elements

14
00:01:16.469 --> 00:01:18.969
That make up virtual production

15
00:01:19.119 --> 00:01:22.219
However, it seems that the most important factor is not simply operating with technology

16
00:01:22.219 --> 00:01:26.439
But what kind of content will be created with this technology

17
00:01:26.439 --> 00:01:28.139
So, the virtual production

18
00:01:28.139 --> 00:01:30.720
Rather than focusing on the word virtual production

19
00:01:30.720 --> 00:01:33.520
This is a part of the production process

20
00:01:33.520 --> 00:01:38.360
Virtual Production Concept 1
And is a set of technologies and used to create content

21
00:01:38.360 --> 00:01:39.910
It involves not only technology

22
00:01:39.910 --> 00:01:46.559
But also filming, post-production, lighting, art, and everything else maintained before

23
00:01:46.559 --> 00:01:54.090
Virtual Production Concept 2
So that you can think of it as a fusion of existing and new virtual technology

24
00:01:54.440 --> 00:01:58.240
First, you are looking at the picture you see now

25
00:01:58.240 --> 00:02:00.360
About the virtual production pipeline

26
00:02:00.360 --> 00:02:04.559
Unlike the existing VFX pipeline

27
00:02:04.559 --> 00:02:06.259
Virtual production is organically related

28
00:02:06.259 --> 00:02:10.400
And content is produced as it continues to develop

29
00:02:10.400 --> 00:02:13.750
Suppose the existing VFX pipeline or production method

30
00:02:13.750 --> 00:02:18.320
Goes through a series of processes from shooting to post-production

31
00:02:18.320 --> 00:02:19.870
Virtual production is

32
00:02:19.870 --> 00:02:24.480
A series of processes from shooting to producing final content that are organically connected

33
00:02:24.480 --> 00:02:29.040
And content is continuously updated

34
00:02:29.040 --> 00:02:34.140
In XR-related content, animation, drama, and movie content

35
00:02:34.140 --> 00:02:36.679
This virtual production technology is used mainly

36
00:02:36.679 --> 00:02:41.399
I will explain how it is used in each field

37
00:02:41.399 --> 00:02:47.240
First, let me tell you about the use cases of virtual production in the entertainment industry

38
00:02:47.240 --> 00:02:52.479
I think virtual production is probably actively used in the entertainment industry

39
00:02:52.479 --> 00:02:55.229
Considering the number of

40
00:02:55.229 --> 00:02:59.080
Films or dramas produced and the production period

41
00:02:59.080 --> 00:03:02.430
Even if actual virtual production technology is used

42
00:03:02.430 --> 00:03:07.919
It takes time for viewers or content enjoyers to consume them

43
00:03:07.919 --> 00:03:12.019
Through entertainment performances or YouTube content, these entertainments

44
00:03:12.019 --> 00:03:16.559
A market seems to have been formed where XR content can be created and consumed immediately

45
00:03:16.559 --> 00:03:20.409
As you can see, auditions using digital characters

46
00:03:20.409 --> 00:03:25.279
Performances using characters such as digital idols

47
00:03:25.279 --> 00:03:28.329
And YouTube contents are continuously being produced

48
00:03:28.329 --> 00:03:32.530
Virtual production is being used for this production technology

49
00:03:32.930 --> 00:03:35.030
First is the XR extended performance

50
00:03:35.030 --> 00:03:38.680
So, to know about examples of virtual production being used in XR

51
00:03:38.680 --> 00:03:41.600
You need some knowledge of motion capture

52
00:03:41.600 --> 00:03:43.750
This motion capture is probably

53
00:03:43.750 --> 00:03:48.240
So deep that you must create a separate class to take it

54
00:03:48.240 --> 00:03:49.740
But generally speaking

55
00:03:49.740 --> 00:03:54.590
Motion capture refers to converting human or animal movements

56
00:03:54.590 --> 00:03:57.039
Into digital data

57
00:03:57.039 --> 00:03:59.039
There can be many ways

58
00:03:59.039 --> 00:04:02.139
As you can see, using an optical camera

59
00:04:02.139 --> 00:04:04.839
Or a gyro sensor

60
00:04:04.839 --> 00:04:07.539
Such as the gyro sensor contained in a cell phone

61
00:04:07.539 --> 00:04:10.039
That uses an axis sensor receiving motion

62
00:04:10.039 --> 00:04:11.839
Or by using a video

63
00:04:11.839 --> 00:04:15.839
Using only bone data of a person or animal in a video

64
00:04:15.839 --> 00:04:18.459
To get motion capture, etc.

65
00:04:18.559 --> 00:04:21.059
Using this motion capture data

66
00:04:21.059 --> 00:04:26.409
Animations that had to be done manually by humans

67
00:04:26.409 --> 00:04:30.479
Can be alleviated by 70 to 80%

68
00:04:30.479 --> 00:04:31.929
So it makes quick production possible

69
00:04:31.929 --> 00:04:33.429
This quick production is

70
00:04:33.429 --> 00:04:35.879
Being used to create content quickly

71
00:04:35.879 --> 00:04:39.920
For performances or YouTube content

72
00:04:39.920 --> 00:04:42.720
What you are seeing now is a Real-Time Motion Capture

73
00:04:42.720 --> 00:04:46.070
A function that changes actual movements

74
00:04:46.070 --> 00:04:49.200
Into 3D data right on the spot

75
00:04:49.200 --> 00:04:50.400
The most important thing here is that

76
00:04:50.400 --> 00:04:54.250
I think it would be good to know the term Retargeting

77
00:04:54.250 --> 00:04:55.950
In fact

78
00:04:55.950 --> 00:04:58.640
The dancing actor in the middle

79
00:04:59.040 --> 00:05:01.390
And the 3D characters moving on either side

80
00:05:01.390 --> 00:05:03.440
May appear to be moving in unison

81
00:05:03.440 --> 00:05:07.600
However, if the ratio of the prepared 3D character

82
00:05:08.000 --> 00:05:12.000
Or the actual actor acting is incorrect

83
00:05:12.000 --> 00:05:13.900
Issues arise, such as

84
00:05:13.900 --> 00:05:16.679
The actual movement is torn or appears strange

85
00:05:16.679 --> 00:05:21.379
These can be corrected through a function called Retargeting

86
00:05:21.379 --> 00:05:23.829
Rather than a difficult technology

87
00:05:23.829 --> 00:05:25.829
It would be better to think of retargeting

88
00:05:25.829 --> 00:05:30.710
As a technology that actually reattaches the position of each bone to a new bone

89
00:05:31.609 --> 00:05:34.359
In these XR contents entertainment

90
00:05:34.359 --> 00:05:37.859
The method of producing content using a game engine

91
00:05:38.559 --> 00:05:43.659
Has the advantage of allowing one content to be used in many different forms

92
00:05:43.759 --> 00:05:49.000
As you can see, creating a digital idol and using it not only for performances

93
00:05:49.000 --> 00:05:53.750
But also for creating interactive content that can communicate with the digital idol

94
00:05:53.750 --> 00:05:57.250
Making a game and selling it

95
00:05:57.350 --> 00:05:59.880
Or making and selling another movie also

96
00:05:59.880 --> 00:06:02.930
Utilizes the 3D data that was previously created.

97
00:06:02.930 --> 00:06:04.880
Because it is an area where it is fully possible

98
00:06:04.880 --> 00:06:09.820
It seems that the method of producing through virtual production is emerging

99
00:06:09.820 --> 00:06:12.520
In entertainment, virtual production technology is

100
00:06:12.520 --> 00:06:15.279
Also changing the way entertainment show is produced

101
00:06:15.279 --> 00:06:19.129
The video you are watching is a show made in Hollywood called Alter Ego

102
00:06:19.129 --> 00:06:22.279
The show used real-time motion capture

103
00:06:22.279 --> 00:06:23.879
Performance capture

104
00:06:23.879 --> 00:06:26.590
AR, XR compositing

105
00:06:26.590 --> 00:06:30.990
To create content that features real-life interactions with digital characters

106
00:06:30.990 --> 00:06:34.340
What you see is content called Ava Dream, which was made in Korea

107
00:06:34.340 --> 00:06:38.220
In the same way as Alter Ego, this content also

108
00:06:38.220 --> 00:06:43.020
Used motion capture, facial capture, and real-time AR, XR compositing technology

109
00:06:43.020 --> 00:06:48.119
To produce new entertainment content that had never been seen before

110
00:06:48.119 --> 00:06:50.319
In producing all of these entertainment contents

111
00:06:50.319 --> 00:06:52.269
Motion capture, camera tracking

112
00:06:52.269 --> 00:06:57.119
And AR/XR compositing technologies used in virtual production ere utilized

113
00:06:57.892 --> 00:07:12.640
Korean virtual idol Plave

114
00:07:13.590 --> 00:07:17.240
In the case of the real-time idol performance you see

115
00:07:17.240 --> 00:07:20.040
As the Ava Dream and Alter Ego seen earlier

116
00:07:20.040 --> 00:07:24.559
By using motion capture, facial capture, and real-time AR compositing technology

117
00:07:24.559 --> 00:07:26.659
Making it possible to produce content

118
00:07:26.659 --> 00:07:30.159
Featuring these idols performing in actual live broadcasts

119
00:07:30.509 --> 00:07:33.079
This type of idol content production

120
00:07:33.079 --> 00:07:37.129
Continues to be attempted in many ways

121
00:07:37.829 --> 00:07:41.989
Because it can lead to a constant influx of fans and form a consumer market

122
00:07:41.989 --> 00:07:44.839
Korean virtual idol MAVE:
A bit different from the Plave we saw earlier

123
00:07:44.839 --> 00:07:47.589
You are seeing a video of a live-action digital character being created

124
00:07:47.589 --> 00:07:50.829
And an actual performance being performed

125
00:07:50.829 --> 00:07:54.729
Of course, in existing entertainment

126
00:07:54.729 --> 00:07:59.959
XR content production is pursuing faster speed or more efficient production methods

127
00:07:59.959 --> 00:08:02.209
But by using virtual production technology

128
00:08:02.209 --> 00:08:05.559
By creating digital twin characters seen in movies and dramas

129
00:08:05.559 --> 00:08:11.519
I wanted to show that it is possible to create high-quality digital content

130
00:08:11.519 --> 00:08:15.269
Another thing you need to know about virtual production is

131
00:08:15.269 --> 00:08:17.989
The concept of XR performance expansion

132
00:08:17.989 --> 00:08:19.339
The videos you see were

133
00:08:19.339 --> 00:08:23.389
Shot on a green screen or LED wall

134
00:08:23.389 --> 00:08:27.239
As you can see, the LED wall or green screen was

135
00:08:27.239 --> 00:08:30.160
Used only in the visible area, not the entire area

136
00:08:30.160 --> 00:08:31.910
However, the actual environment shown is

137
00:08:31.910 --> 00:08:37.679
A video created so that the performance can be performed in an expanded form that covers the entire area

138
00:08:37.679 --> 00:08:42.979
These technologies are being used in music video production or entertainment

139
00:08:42.979 --> 00:08:47.519
Emerging as a new video production method that has never been seen before

140
00:08:47.519 --> 00:08:51.419
So, as you can see, this XR content and performance expansion technology

141
00:08:51.619 --> 00:08:55.219
Utilizes a method of transmitting existing sources, produced 3D videos

142
00:08:55.219 --> 00:08:59.469
And people or objects that are actually filmed

143
00:08:59.469 --> 00:09:04.090
Through real-time compositing

144
00:09:04.090 --> 00:09:06.140
The technologies used here are

145
00:09:06.140 --> 00:09:09.999
Green screen compositing, motion tracking using camera tracking

146
00:09:09.999 --> 00:09:12.799
AR compositing, and XR compositing

147
00:09:12.799 --> 00:09:15.149
These technologies are being put to use

148
00:09:15.149 --> 00:09:18.699
What you see is a diagram of the XR performance expansion

149
00:09:18.699 --> 00:09:23.479
As you can see, within an actual LED wall or green screen

150
00:09:23.479 --> 00:09:25.579
Live shooting is proceeded

151
00:09:25.579 --> 00:09:30.929
Adding another new source to the LED wall and green screen

152
00:09:30.929 --> 00:09:35.369
And transmitting a composite image

153
00:09:35.619 --> 00:09:38.669
In entertainment, virtual production technology is

154
00:09:38.669 --> 00:09:43.370
Emerging as a new content creation method for

155
00:09:43.520 --> 00:09:47.070
Performances or idols you have seen before

156
00:09:47.070 --> 00:09:50.960
And for commerce, interactive, and realistic videos

157
00:09:50.960 --> 00:09:55.810
You see a video of an e-commerce transaction

158
00:09:55.810 --> 00:09:59.799
Using an actual LED wall and XR technology

159
00:09:59.799 --> 00:10:01.399
The second thing you see is

160
00:10:01.399 --> 00:10:04.949
Not only a video filmed on the existing LED wall

161
00:10:04.949 --> 00:10:08.149
But also a video made using XR technology

162
00:10:08.149 --> 00:10:11.449
So that you can watch the show as if

163
00:10:11.449 --> 00:10:14.150
The actual idols were performing within the space rather than in front of the LED wall

164
00:10:14.150 --> 00:10:16.500
This XR virtual production technology

165
00:10:16.500 --> 00:10:19.150
The virtual production technology on the XR stage

166
00:10:19.150 --> 00:10:26.040
Enables live communication between the audience and actual content creators

167
00:10:26.040 --> 00:10:27.690
The content we are creating is

168
00:10:27.690 --> 00:10:32.090
Not rendered and played back statically

169
00:10:32.090 --> 00:10:36.139
But can continuously interact interactively in real-time

170
00:10:36.139 --> 00:10:39.089
Whether it is lighting or audience motion

171
00:10:39.089 --> 00:10:42.559
Because continuous communication is possible with any specific field

172
00:10:42.759 --> 00:10:46.209
Creating an environment that can provide a more interactive

173
00:10:46.409 --> 00:10:49.350
And different experiences during actual performances are possible

174
00:10:50.000 --> 00:10:51.750
So far

175
00:10:51.750 --> 00:10:54.800
We have looked at virtual production utilization technologies in entertainment

176
00:10:54.800 --> 00:10:57.050
Generally called the media entertainment field

177
00:10:57.050 --> 00:11:00.200
From now on, I will talk about examples of

178
00:11:00.200 --> 00:11:04.650
How virtual production is used in the field of animation

179
00:11:04.650 --> 00:11:07.300
What you are seeing is an animation called Yumi's Cells

180
00:11:07.300 --> 00:11:15.058
This animation is 100% rendered using a game engine

181
00:11:15.058 --> 00:11:20.159
Animations such as Yumi's Cells, Armored Saurus, and Willbengers are

182
00:11:20.159 --> 00:11:26.159
Animations produced using game engine real-time rendering technology

183
00:11:26.159 --> 00:11:30.509
Existing animation overlaps several steps, starting from a storyboard, creating a character

184
00:11:30.509 --> 00:11:33.110
Planting a skeleton on the character

185
00:11:33.110 --> 00:11:34.460
Animation

186
00:11:34.460 --> 00:11:39.210
Then working on textures and lighting to suit the animation

187
00:11:39.210 --> 00:11:42.080
And finally compositing

188
00:11:42.080 --> 00:11:46.230
Each of these steps requires enormous resources

189
00:11:46.230 --> 00:11:49.580
And a lot of time

190
00:11:49.580 --> 00:11:51.430
Confirmation was required for each step

191
00:11:51.430 --> 00:11:55.239
And it is impossible to know 100% perfectly how it will look

192
00:11:55.239 --> 00:11:59.760
Until you go through the actual final rendering process and see the image

193
00:11:59.760 --> 00:12:01.560
However, real-time rendering engine

194
00:12:01.560 --> 00:12:04.060
Especially animation production using game engines

195
00:12:04.060 --> 00:12:09.799
Has the advantage of being able to check all of these processes at once

196
00:12:09.799 --> 00:12:12.949
Although it is impossible to check the final lighting with just one button

197
00:12:12.949 --> 00:12:18.200
It has the advantage of checking the output immediately within the actual working time

198
00:12:18.200 --> 00:12:19.500
This advantage

199
00:12:19.500 --> 00:12:21.850
Allows you to get faster feedback

200
00:12:21.850 --> 00:12:26.700
Which will help you make faster revisions and produce the final product

201
00:12:27.250 --> 00:12:30.150
In animation production

202
00:12:30.150 --> 00:12:35.239
When using a game engine and motion capture or performance capture

203
00:12:35.239 --> 00:12:38.939
It can be implemented in real-time

204
00:12:38.939 --> 00:12:42.489
To create the picture desired by the creator and director

205
00:12:42.489 --> 00:12:45.350
And confirm it to the final quality

206
00:12:45.350 --> 00:12:49.200
The diagram you see is a diagram of the entire process of

207
00:12:49.200 --> 00:12:54.679
Implementing animation through motion capture in real-time

208
00:12:54.679 --> 00:12:56.329
The video you are watching is

209
00:12:56.329 --> 00:13:00.430
About something called performance capture or facial capture

210
00:13:00.430 --> 00:13:02.980
As you can see, on real actors

211
00:13:02.980 --> 00:13:06.930
To Acting actors, headgear is put on

212
00:13:06.930 --> 00:13:09.280
And the animation is

213
00:13:09.280 --> 00:13:14.359
Created live as they act in the headgear

214
00:13:14.359 --> 00:13:16.259
Even if we had previously received such animation

215
00:13:16.259 --> 00:13:19.009
Especially facial data

216
00:13:19.009 --> 00:13:22.109
Many processes were required

217
00:13:22.109 --> 00:13:25.320
To import this facial data, process it, and proceed to actual lighting

218
00:13:25.320 --> 00:13:28.720
However, we can now see that it is possible

219
00:13:28.720 --> 00:13:33.320
To confirm it in a form equivalent to the final quality as soon as it is filmed on-site

220
00:13:33.320 --> 00:13:35.770
This real-time animation creation technology is

221
00:13:35.770 --> 00:13:38.970
Not only used in animation

222
00:13:38.970 --> 00:13:41.189
But can also be used in live-action filming

223
00:13:41.189 --> 00:13:45.239
What you are seeing is a scene from Jingle Jangle, one of Netflix's shows

224
00:13:45.239 --> 00:13:49.589
Where actual footage and 3D animation characters were

225
00:13:49.589 --> 00:13:52.439
Live composited through performance capture

226
00:13:52.439 --> 00:13:54.289
Through the live composite data

227
00:13:54.289 --> 00:13:57.230
The final result was produced

228
00:13:57.380 --> 00:13:59.630
As you can see, in the past

229
00:13:59.630 --> 00:14:03.430
The actual animated 3D characters and the live-action

230
00:14:03.430 --> 00:14:06.080
Would have been filmed separately and later composited

231
00:14:06.080 --> 00:14:09.480
However, if you utilize live virtual production technology

232
00:14:09.480 --> 00:14:14.180
By viewing live-action shooting and live characters simultaneously

233
00:14:14.180 --> 00:14:17.960
You can achieve faster confirmation and production efficiency

234
00:14:17.960 --> 00:14:20.510
Animation production using virtual production has

235
00:14:20.510 --> 00:14:22.239
Also been attempted in Korea

236
00:14:22.239 --> 00:14:26.039
If you look at the work called Armored Saurus, which aired in 2022

237
00:14:26.389 --> 00:14:30.539
Real characters, 3D environments

238
00:14:30.539 --> 00:14:34.589
And all moving characters filmed on a green screen were

239
00:14:34.589 --> 00:14:36.239
Composited live on-site using a game engine

240
00:14:36.239 --> 00:14:39.559
And were finally aired after post-processing

241
00:14:39.559 --> 00:14:41.759
Without these technologies

242
00:14:41.759 --> 00:14:43.659
To shoot green screens

243
00:14:43.659 --> 00:14:45.409
Create real environments

244
00:14:45.409 --> 00:14:46.909
And dress up characters

245
00:14:46.909 --> 00:14:48.909
It would have taken a long time and a lot of money

246
00:14:48.909 --> 00:14:51.209
But by using these virtual production technologies

247
00:14:51.209 --> 00:14:55.880
I think we can create content of decent quality in a short period

248
00:14:55.880 --> 00:14:57.630
Almost real

249
00:14:57.630 --> 00:15:00.430
Of course, we cannot say 100% that it is of real-life quality

250
00:15:00.430 --> 00:15:04.630
You can see that it is possible to produce 3D animation equivalent to live-action

251
00:15:04.630 --> 00:15:08.440
Through this Armored Saurus case

252
00:15:08.440 --> 00:15:09.890
The diagram you see

253
00:15:10.090 --> 00:15:15.990
Shows a pipeline that uses a real-time game engine and real-time motion capture

254
00:15:15.990 --> 00:15:20.000
To produce photorealistic composite animation

255
00:15:20.000 --> 00:15:22.350
As in the Armored Saurus example above

256
00:15:22.350 --> 00:15:26.150
The environment and characters prepared for live-action

257
00:15:26.150 --> 00:15:30.050
are rendered in real-time at the live-action shooting site

258
00:15:30.050 --> 00:15:35.479
And the final result, the final plate, is filmed through compositing

259
00:15:35.479 --> 00:15:40.429
This method of producing animation using virtual production

260
00:15:40.429 --> 00:15:43.529
Has the advantage of

261
00:15:43.529 --> 00:15:47.379
Quickly producing things that were previously difficult to attempt in animation

262
00:15:47.379 --> 00:15:50.200
Or were not cost-effective

263
00:15:50.200 --> 00:15:53.700
If you were to produce a CF using that cartoon rendering method

264
00:15:53.700 --> 00:15:57.650
You would have to go through the entire animation process

265
00:15:57.650 --> 00:16:00.750
And many processes, including actual lighting, would be required

266
00:16:00.750 --> 00:16:04.359
Making it difficult to achieve the CF production cost

267
00:16:04.359 --> 00:16:07.159
However, by using game engines, real-time rendering technology

268
00:16:07.159 --> 00:16:09.409
And motion capture technology

269
00:16:09.409 --> 00:16:11.659
In short time, cartoon rendering

270
00:16:11.659 --> 00:16:14.989
Or advertisements with an animation production became possible

271
00:16:15.990 --> 00:16:18.690
Drama and Movie Content Use Case

272
00:16:19.389 --> 00:16:23.139
So far, we have talked about virtual production

273
00:16:23.139 --> 00:16:25.439
In media entertainment or animation

274
00:16:25.439 --> 00:16:27.439
Now we will talk about dramas and movies

275
00:16:27.439 --> 00:16:30.839
In general, I will talk about

276
00:16:30.839 --> 00:16:34.239
Virtual production, an area combined with live action

277
00:16:34.239 --> 00:16:37.439
If virtual production technology is responsible

278
00:16:37.439 --> 00:16:42.789
For the final output in XR content or animation content

279
00:16:42.839 --> 00:16:44.689
In drama or movie content

280
00:16:44.689 --> 00:16:50.439
It would be good to think of it as being used in a form that helps with pre-visualization

281
00:16:50.439 --> 00:16:52.719
or filming method production

282
00:16:52.719 --> 00:16:54.819
Such as ICVFX

283
00:16:54.819 --> 00:16:58.369
There are ways to produce actual final plates

284
00:16:58.369 --> 00:17:02.219
Still, it would be good to know that

285
00:17:02.219 --> 00:17:06.519
Not only those methods but also each technology of virtual production is

286
00:17:06.519 --> 00:17:08.559
Used throughout the entire movie or drama production process

287
00:17:08.949 --> 00:17:23.798
The Silent Sea

288
00:17:24.598 --> 00:17:27.698
You are watching a video of

289
00:17:27.698 --> 00:17:33.348
An actual final plate filmed using an LED wall in a project called The Silent Sea

290
00:17:33.598 --> 00:17:35.348
As I mentioned

291
00:17:35.348 --> 00:17:36.448
This is an example of

292
00:17:36.448 --> 00:17:40.560
How virtual production technology can utilize the final plate

293
00:17:40.860 --> 00:17:42.460
In addition to these examples

294
00:17:42.460 --> 00:17:45.060
Through the various scenarios explained later

295
00:17:45.060 --> 00:17:47.360
You will see how virtual production is used

296
00:17:47.360 --> 00:17:50.560
In movies and dramas

297
00:17:50.750 --> 00:17:52.750
You are looking at

298
00:17:52.750 --> 00:17:57.250
A pipeline where virtual production is used in movie and drama content

299
00:17:57.250 --> 00:18:01.500
You can think of it as similar to animation or virtual production

300
00:18:01.500 --> 00:18:03.650
Pipeline and picture above

301
00:18:03.650 --> 00:18:06.700
Virtual production technology is being used

302
00:18:06.700 --> 00:18:12.000
In all areas of content creation, from the pre-production, production

303
00:18:12.000 --> 00:18:14.600
And post-production stages

304
00:18:14.650 --> 00:18:16.100
We wanted to show

305
00:18:16.100 --> 00:18:20.489
How this leveraged virtual production streamlines how content is created in the real world

306
00:18:20.939 --> 00:18:22.289
To explain briefly

307
00:18:22.289 --> 00:18:24.039
In reality

308
00:18:24.039 --> 00:18:28.089
Existing scenarios were produced, and before making a video of it

309
00:18:28.089 --> 00:18:29.089
With the concept of a storyboard

310
00:18:29.089 --> 00:18:35.079
A specific image, or a keyframe image

311
00:18:35.079 --> 00:18:37.879
By using current virtual production technology

312
00:18:37.879 --> 00:18:40.229
All of these images can be created

313
00:18:40.229 --> 00:18:43.579
Into 3D images called actual sequence images

314
00:18:43.579 --> 00:18:47.079
And actual shooting is performed using this image data

315
00:18:47.079 --> 00:18:49.179
And the data used in the shooting

316
00:18:49.179 --> 00:18:52.579
And the image 3D data prepared previously are combined

317
00:18:52.579 --> 00:18:56.349
It explains the process of creating the final output using

318
00:18:56.613 --> 00:19:16.613
〈The Silent Sea〉

319
00:19:17.079 --> 00:19:18.179
In the video you saw earlier

320
00:19:18.179 --> 00:19:20.079
By using virtual production

321
00:19:20.079 --> 00:19:23.579
Previs and actual ICVFX

322
00:19:23.579 --> 00:19:26.479
And post-production throughout the entire process

323
00:19:26.479 --> 00:19:29.719
You saw the show reel of The Silent Sea

324
00:19:29.719 --> 00:19:32.369
From now on, I will explain what technologies were used in The Silent Sea

325
00:19:32.369 --> 00:19:35.479
And how they are used

326
00:19:35.479 --> 00:19:38.329
First, as explained earlier, virtual production

327
00:19:38.329 --> 00:19:42.839
Allows new content to be created in the pre-production stage

328
00:19:42.839 --> 00:19:45.789
Previously, based on the storyboard concept

329
00:19:45.789 --> 00:19:47.789
Storyboards were made into scenes

330
00:19:47.789 --> 00:19:52.040
Or only important scenes were made in full 3D

331
00:19:52.040 --> 00:19:53.790
However, by utilizing this virtual production technology

332
00:19:53.790 --> 00:19:58.090
Especially technology such as motion capture and real-time camera linkage

333
00:19:58.090 --> 00:20:02.079
The entire area of ​​the storyboard can be created in full 3D

334
00:20:02.079 --> 00:20:05.779
The director can change the desired angle anytime

335
00:20:05.779 --> 00:20:09.680
And create a new shooting plan

336
00:20:09.680 --> 00:20:13.080
You are seeing

337
00:20:13.080 --> 00:20:16.720
A real-time previs test video of the Sweet Home project filmed in 2020

338
00:20:17.150 --> 00:20:36.550
Sweet Home Real-time Previs

339
00:20:37.500 --> 00:20:41.150
Previously, when filming such a creature

340
00:20:41.150 --> 00:20:45.439
There was an issue of having to film on a blank green mat or a blank field

341
00:20:45.439 --> 00:20:49.150
And then composite the creature later

342
00:20:49.150 --> 00:20:51.800
Then, on set, actors or directors

343
00:20:51.800 --> 00:20:55.650
Had to look into space and give directions

344
00:20:55.650 --> 00:21:00.700
Or they had no choice but to film based solely on their imagination

345
00:21:00.700 --> 00:21:05.750
However, using real-time compositing and real-time previs technology of virtual production

346
00:21:05.750 --> 00:21:09.189
You can create an environment

347
00:21:09.189 --> 00:21:14.289
Where you can film while checking the character's movement and scale in real time

348
00:21:14.289 --> 00:21:16.739
And because the data captured in this way

349
00:21:16.739 --> 00:21:22.939
And the 3D data synthesized later in post-production are maintained consistently

350
00:21:22.939 --> 00:21:27.839
The actual output quality can also be guaranteed to be high

351
00:21:28.000 --> 00:21:31.500
The video you are seeing is called real-time previs

352
00:21:31.500 --> 00:21:35.320
Before the actual movie or drama is filmed

353
00:21:35.320 --> 00:21:41.170
So you are watching on-site footage that uses motion capture and camera tracking technology

354
00:21:41.170 --> 00:21:45.200
To create a specific scene

355
00:21:45.200 --> 00:21:48.650
When you look at the actual picture

356
00:21:48.650 --> 00:21:52.100
It seems like a person is simply moving and filming it with an iPad

357
00:21:52.100 --> 00:21:55.600
But all the data is being collected from that single action

358
00:21:55.600 --> 00:22:01.300
And 3D environments and characters are being created in real-time through this data

359
00:22:01.300 --> 00:22:03.000
By using this technology

360
00:22:03.000 --> 00:22:04.700
As mentioned earlier

361
00:22:04.700 --> 00:22:08.480
The angles or imagined actions of the director

362
00:22:08.480 --> 00:22:12.880
Can be expressed not only through drawings or writing

363
00:22:12.880 --> 00:22:15.180
But also through moving images

364
00:22:15.180 --> 00:22:17.580
Which not only improves the efficiency of the production method

365
00:22:17.580 --> 00:22:22.950
But also allows for more possibilities during production

366
00:22:23.465 --> 00:23:25.676
Sweet Home Simulation Camera

367
00:23:27.120 --> 00:23:28.920
In the video you saw earlier

368
00:23:28.920 --> 00:23:32.170
The simulation camera  is generally referred to as a Simul Cam

369
00:23:32.170 --> 00:23:35.440
And the term Ncam is also used as a specific term

370
00:23:35.440 --> 00:23:37.990
As you can see, in Sweet Home season 1

371
00:23:37.990 --> 00:23:41.690
This is a video where you can see how it was filmed using the simulation camera technology

372
00:23:41.690 --> 00:23:44.990
And how the filmed data is used

373
00:23:44.990 --> 00:23:48.000
Until the final output

374
00:23:48.000 --> 00:23:49.450
Lastly, virtual production

375
00:23:49.450 --> 00:23:51.650
Among virtual production technologies in movies and dramas

376
00:23:51.650 --> 00:23:53.900
I think the most talked-about

377
00:23:53.900 --> 00:23:57.600
And most useful technologies are

378
00:23:57.600 --> 00:24:01.880
In-camera VFX and 2D Driving Comp

379
00:24:01.880 --> 00:24:06.780
You can think of one area of In-Camera VFX as 2D Driving Comp

380
00:24:06.780 --> 00:24:09.680
The elements that make up in-camera VFX are

381
00:24:09.680 --> 00:24:11.600
Divided depending on whether they are composed of a full 3D environment

382
00:24:11.600 --> 00:24:14.500
A 2D image plate

383
00:24:14.500 --> 00:24:16.400
Or a combination of the two types

384
00:24:16.400 --> 00:24:19.050
To create a 2.5D environment

385
00:24:19.050 --> 00:24:22.919
That combines a full 3D environment and a 2D environment

386
00:24:22.919 --> 00:24:24.969
You can think of it as an In-Camera VFX

387
00:24:24.969 --> 00:24:27.919
That encompasses all those areas

388
00:24:27.919 --> 00:24:30.369
In-camera VFX is a method of erecting an LED wall

389
00:24:30.369 --> 00:24:36.119
And placing the environment or image that requires actual filming on the LED wall and filming

390
00:24:36.119 --> 00:24:38.819
This method is not

391
00:24:38.819 --> 00:24:41.419
A simple background or a stationary background called hole

392
00:24:41.419 --> 00:24:45.960
Because it is an environment that can be interactively interacted with in real-time

393
00:24:45.960 --> 00:24:50.160
Because it helps with immersion and acting on set

394
00:24:50.160 --> 00:24:53.660
And it is not just a light-emitting object

395
00:24:53.660 --> 00:24:56.559
It also performs the role of lighting

396
00:24:56.559 --> 00:24:59.359
This ICVFX technology was first introduced

397
00:24:59.359 --> 00:25:02.279
In 2018 by The Mandalorian

398
00:25:02.279 --> 00:25:05.679
The Mandalorian is a spin-off of Star Wars

399
00:25:05.679 --> 00:25:08.829
In which the environment changes continuously while traveling to various parts of the universe

400
00:25:08.829 --> 00:25:13.320
And the story takes place within this environment

401
00:25:13.320 --> 00:25:15.820
This isn't making a two-hour movie

402
00:25:15.820 --> 00:25:18.270
It's making an episodic drama

403
00:25:18.270 --> 00:25:22.020
Therefore, if you build all sets or film on location

404
00:25:22.020 --> 00:25:23.880
In a constantly changing environment

405
00:25:23.880 --> 00:25:26.680
Production costs are bound to increase exponentially

406
00:25:26.680 --> 00:25:29.330
So, for the first time, they built an LED wall

407
00:25:29.330 --> 00:25:34.199
And filmed by projecting various space environments onto the LED wall

408
00:25:34.199 --> 00:25:40.199
Through this, it became possible to freely move between various environments in one set

409
00:25:40.199 --> 00:25:45.000
And produce high-quality content while minimizing costs

410
00:25:45.200 --> 00:25:47.350
Through this case in Korea

411
00:25:47.350 --> 00:25:51.700
There have been cases of attempting this type of ICVFX

412
00:25:51.700 --> 00:25:57.480
What you see is ICVFX on the moon, which was first attempted in The Silent Sea

413
00:25:57.480 --> 00:25:59.630
The difference between domestic and overseas cases is that

414
00:25:59.630 --> 00:26:01.530
In the domestic case

415
00:26:01.530 --> 00:26:04.680
The actual location area

416
00:26:04.680 --> 00:26:08.600
was not carefully adjusted from the pre-production stage

417
00:26:08.600 --> 00:26:12.450
So that usability may be slightly lower

418
00:26:12.450 --> 00:26:14.550
However, when it was actually filmed

419
00:26:14.550 --> 00:26:18.079
It was a project whose potential was fully confirmed

420
00:26:18.079 --> 00:26:22.829
I can tell you that the most used one is 2D Driving Comp

421
00:26:22.829 --> 00:26:27.920
I put 2D Driving Comp separately under ICVFX

422
00:26:27.920 --> 00:26:31.570
Because it is the most efficient method compared to existing shooting methods

423
00:26:31.570 --> 00:26:33.270
So I wanted to introduce it separately

424
00:26:33.270 --> 00:26:38.559
So, I think I will split the episode and talk about 2D Driving Comp in more detail

425
00:26:38.559 --> 00:26:40.859
Simply put

426
00:26:40.859 --> 00:26:42.809
Any scene in a movie or drama

427
00:26:42.809 --> 00:26:46.309
That takes place in a car or any other vehicle is included

428
00:26:46.309 --> 00:26:48.839
Whether it's the future, the past

429
00:26:48.839 --> 00:26:52.239
Or, of course, not if it's a much past

430
00:26:52.239 --> 00:26:54.789
In contemporary, modern, or future works

431
00:26:54.789 --> 00:26:55.889
Many conversations

432
00:26:55.889 --> 00:26:58.600
Or actions take place on something

433
00:26:58.600 --> 00:27:02.950
Previously, all these scenes were filmed on a painted mat

434
00:27:02.950 --> 00:27:04.400
It was synthesized later

435
00:27:04.400 --> 00:27:06.799
Because of the later compositing

436
00:27:06.799 --> 00:27:09.199
It was not easy to match lighting issues

437
00:27:09.199 --> 00:27:12.049
Actors' acting, or interactions

438
00:27:12.049 --> 00:27:14.599
All objects are made of glass

439
00:27:15.149 --> 00:27:16.849
And if we tried to composite them

440
00:27:16.849 --> 00:27:20.640
By implementing reflections on the glass or moving environments outside

441
00:27:20.640 --> 00:27:23.590
There were problems with time and quality

442
00:27:23.590 --> 00:27:26.690
However, the use of ICVFX

443
00:27:26.690 --> 00:27:27.690
Seems to be optimized

444
00:27:27.690 --> 00:27:32.559
For filming these vehicle scenes, especially dialogue or action scenes

445
00:27:32.559 --> 00:27:36.559
Although it may vary depending on the size or size of the LED wall

446
00:27:36.559 --> 00:27:41.559
If you build a size that suits the vehicle or a specific vehicle

447
00:27:41.559 --> 00:27:45.440
And film it through prepared or created video

448
00:27:45.440 --> 00:27:47.390
You can feel immersed in the space

449
00:27:47.390 --> 00:27:50.840
However, it seems to have a strong advantage

450
00:27:50.840 --> 00:27:56.000
In terms of the characteristics of the window glass and its ability to express reflection

451
00:27:56.079 --> 00:28:01.029
The video you are watching is

452
00:28:01.629 --> 00:28:05.119
A 2D Driving Comp used in Netflix's MANK

453
00:28:05.119 --> 00:28:07.619
The reason why I showed this 2D Driving Comp separately is

454
00:28:07.619 --> 00:28:09.919
Because this work was filmed in color

455
00:28:09.919 --> 00:28:13.200
This work was later converted to black and white

456
00:28:13.200 --> 00:28:16.500
So, in certain areas, some characters appear in color

457
00:28:16.500 --> 00:28:18.959
While others change to black and white

458
00:28:18.959 --> 00:28:22.359
If you look at it now, when viewed in actual color

459
00:28:22.359 --> 00:28:24.959
The 3D environment may look awkward

460
00:28:24.959 --> 00:28:27.559
However, when converted to black and white

461
00:28:27.559 --> 00:28:31.200
You can see that it blends into the scene exactly like this without any sense of incongruity

462
00:28:31.200 --> 00:28:33.150
I wanted to show that this virtual production technology

463
00:28:33.150 --> 00:28:36.600
Depending on how the idea is created or used

464
00:28:36.600 --> 00:28:39.079
Not only reduces production costs

465
00:28:39.079 --> 00:28:43.679
But also has many possibilities in terms of usability

466
00:28:43.679 --> 00:28:46.679
2D Driving Comp is not just about vehicles

467
00:28:46.679 --> 00:28:52.720
It can also be a train, a helicopter, or any space object

468
00:28:52.720 --> 00:28:56.270
As you can see, filming on a train is not easy

469
00:28:56.270 --> 00:28:58.520
Not only is the space of the train small

470
00:28:58.520 --> 00:29:01.600
But it is impossible to know where the train is running, what scene to film

471
00:29:01.600 --> 00:29:05.150
When to pass through the tunnel

472
00:29:05.150 --> 00:29:07.300
Doing the action is impossible

473
00:29:07.300 --> 00:29:09.500
And it is difficult to film

474
00:29:09.500 --> 00:29:13.359
Because the environment can change from time to time as a person operates the train

475
00:29:13.359 --> 00:29:16.559
But by using this train set and LED wall

476
00:29:16.559 --> 00:29:18.609
Exterior background you want

477
00:29:18.609 --> 00:29:20.009
The timing you want

478
00:29:20.009 --> 00:29:23.720
And the perspective you want can be made and filmed

479
00:29:23.720 --> 00:29:27.070
So, to create train scenes

480
00:29:27.070 --> 00:29:28.970
Green mats and sets were built

481
00:29:28.970 --> 00:29:33.040
But now it is possible to create

482
00:29:33.040 --> 00:29:36.640
Content of the same quality through ICVFX technology and sets

483
00:29:36.640 --> 00:29:41.279
Among the ICVFX cases, the most successful one I would like to introduce is

484
00:29:41.279 --> 00:29:43.829
A project called Netflix 1899

485
00:29:43.829 --> 00:29:49.079
This 1899 work is set

486
00:29:49.079 --> 00:29:54.160
During the Enlightenment period, that is, from the late 1800s to the early 1900s

487
00:29:54.160 --> 00:29:57.410
And most of the costumes and environments at this time are

488
00:29:57.410 --> 00:30:02.400
Set, and it takes a lot of money and time to build them

489
00:30:02.400 --> 00:30:05.950
Since most of the action takes place on a ship

490
00:30:05.950 --> 00:30:11.760
The background of the ship consists of water, sky, clouds, etc.

491
00:30:11.760 --> 00:30:17.160
Combining these with ICVFX and art set technology

492
00:30:17.660 --> 00:30:23.210
Through minimal set production and maximum environmental 3D production

493
00:30:23.210 --> 00:30:25.679
Efficient filming was enabled

494
00:30:25.679 --> 00:30:29.979
Such Driving Comp filming continues to take place in domestic as well

495
00:30:29.979 --> 00:30:34.159
You see the Driving Comp shot used in The Queen of Tears

496
00:30:34.159 --> 00:30:37.909
If the two actors filmed this conversation scene

497
00:30:37.909 --> 00:30:40.859
Outside in the rain

498
00:30:40.859 --> 00:30:44.259
It would not be easy to match the shooting schedule and rainy day

499
00:30:44.259 --> 00:30:48.080
Ad it would not be easy to continue filming on a rainy day

500
00:30:48.159 --> 00:30:52.609
However, through the LED wall, captured sources, and ICVFX technology

501
00:30:52.609 --> 00:30:54.009
While continuing to retake

502
00:30:54.009 --> 00:30:59.119
We continued filming until we captured the desired emotion of the actor

503
00:30:59.119 --> 00:31:02.919
As a similar example, in Season 2 of Netflix's DP

504
00:31:02.919 --> 00:31:05.359
You are looking at the Driving Comp shot filmed

505
00:31:05.359 --> 00:31:08.659
As you can see, the first attempt was made by

506
00:31:08.659 --> 00:31:12.959
Putting a truck and a car in front of the LED wall simultaneously

507
00:31:12.959 --> 00:31:17.760
And filming while the two cars were interacting

508
00:31:17.760 --> 00:31:21.760
As you can see, the size of the LED wall is important

509
00:31:21.760 --> 00:31:25.110
But, depending on your shooting method

510
00:31:25.110 --> 00:31:30.719
I wanted to show you that you can shoot the scene you want with ICVFX

511
00:31:30.719 --> 00:31:32.069
And it's the same project

512
00:31:32.069 --> 00:31:40.400
You are watching a scene using the train and LED wall used in DP

513
00:31:40.400 --> 00:31:43.900
This scene also involved a lot of action by real actors

514
00:31:43.900 --> 00:31:46.950
And with so many changes in camera angles

515
00:31:46.950 --> 00:31:49.039
It was impossible to film it on a train

516
00:31:49.039 --> 00:31:52.439
Although the train was got

517
00:31:52.439 --> 00:31:56.339
If all the exterior windows of the train were filmed and composited together

518
00:31:56.339 --> 00:31:59.520
The production cost would inevitably rise significantly

519
00:31:59.599 --> 00:32:02.899
However, by using this LED wall technology and ICVFX technology

520
00:32:02.899 --> 00:32:06.299
We could film all episodes in a short period of time, in two days

521
00:32:06.299 --> 00:32:10.400
And produce the final plate immediately without any special post-processing

522
00:32:10.400 --> 00:32:16.239
So far, I have told you what kind of technology virtual production is and how it is used

523
00:32:16.239 --> 00:32:19.189
If you think of virtual production not simply as a technology

524
00:32:19.189 --> 00:32:22.400
But as a way to create content

525
00:32:22.400 --> 00:32:26.550
You can use it in many ways to create new content in the future

526
00:32:26.550 --> 00:32:31.280
Or in any project you use, I think it has potential

527
00:32:31.280 --> 00:32:33.810
I hope this helps you a lot, thank you

528
00:32:34.482 --> 00:32:35.682
The Concept of Virtual Production and Examples of Use in the Entertainment Industry and Animation
Virtual Production
A set of technologies used to create content

529
00:32:35.682 --> 00:32:36.932
Producing content through organic connection and fusion of existing and virtual technologies
Motion capture, performance capture, facial capture, real-time AR/XR compositing technology, etc.

530
00:32:36.932 --> 00:32:38.182
Entertainment Industry Use Cases
Virtual idol content: PLAVE, MAVE:
Entertainment content: AVA Dream, The Golden Age of Subcharacters, Avatar Singer

531
00:32:38.182 --> 00:32:39.382
LED Wall + performance expansion content
Commerce content
LED Wall + Interactive Concert

532
00:32:39.382 --> 00:32:41.432
Animation Use Cases
Real-time rendering that allows you to check the output while working provides quick feedback and final results

533
00:32:41.432 --> 00:32:43.482
Autonomous confirmation, flexible pipeline, high-quality results in a short period of time
Works: Yumi's Cells, Armored Saurus, WillBengers

534
00:32:43.482 --> 00:32:44.832
Drama and Movie Content Use Example
Real-time Previs
The entire storyboard area can be created in FULL 3D and visualized with data
A production method with real-time feedback, improved quality, increased efficiency, flexibility, and creativity

535
00:32:44.832 --> 00:32:46.182
In-Camera VFX
Creating the environment necessary for the LED Wall and filming
Interactive real-time interactive backgrounds

536
00:32:46.182 --> 00:32:48.882
2D Driving Comp
It is a type of in-camera VFX technology and has the highest utilization and usefulness
Optimized for dialogue scenes and action scenes inside the car