Realistic Advanced Media Facade
GCC Academy
Hello
This is Kim Saetbyeol for realistic media content
Recently, realistic media contents' use has expanded a lot
Not only movies, advertisements, and drama series
media facade using buildings for marketing
or backgrounds for concerts and festivals
also have been produced
If you are in this field
you might start from a certain area
but the connection between
production, design, and engineering is necessary
Through this course, let's get to know
how realistic media content is produced
and delivered
Understanding Media Facade
In this lecture we will learn what media facade is
what steps media facade production takes
and the programs and software
often used in the production
process
Media facade is a word consisting of media
and faced meaning the outer wall of a building
which means a method of visual content
screening a video on the wall of a building
It is a combination of architecture and digital media
Through this, architecture goes further
and gains the role of a communicating media
Media is another word for mass media
and I think media is message
Not just a method of expression of media
but content within media
and the content acting as a media
Media facade started developing
since the late 1990s
and it is still developing with the rapid development of technology
It first started as projector
and now, using LCD, LED, OLED technologies
a more vivid and bright imagery is possible
Especially, recently, the use of anamorphic technology
has become a trend
Beyond simply making a building beautiful
it is used as a media to deliver messages
or produce art in the space
For example, symbolical events of a city
or marketing by firms
delivering cultural messages
are examples of the use of media facade
Before, it was used as advertisement a lot
but recently its function as public art has been highlighted
delivering many artistically significant videos
Media facade is the integration of media art and architecture
It strengthens the visual and audio experience of the audience
Storytelling of architectural elements
or communication with the public
is also its part in media art
Especially, you might have heard of
national heritage art festival in traditional architecture
These festivals can provide a fresh visual and audio experience
without damaging the traditional architecture
Also in newly made modern architecture
with the development of technology
media has become a part of a city
Media facade proposes a potential to expand
with the ability of light expression
integrated with AR and VR
and with technological development and creativity
more variety is expected
to be shown
The types of media facade include LED, LCD display
and projection media using projectors
First, let's see the two according to resolution and brightness
LED and LCD display
provide high resolution and brightness
and can maintain vivid screen display
in various weather conditions
Especially LED display provides high luminance
mkaking it be vividly visible even in bright environments
On the other hand, projectors have limitations in the brightness
so the vividness can be diminished in daytime
or in bright outside environments
It is more suitable for nighttime or dark environments
and bright projectors can be more pricey
Recently the method of using
multiple of these projectors together
has been used
Let's see the cost aspect
LED and LCD display have high initial installation cost
but have relatively lower maintenance cost
Thanks to its long life and good durability
the long term cost can be effective
Projectors have relatively lower initial installation cost
but consistent maintenance
like lamp repair or clean up
and after long time of use the quality can get diminished
so it would be ideal to
choose the suitable display taking these cost differences into account
LED and LCD display also
requires display panels in the installation
and needs structural support due to its size and weight
But the maintenance is relatively simple
On the other hand, projectors have a simpler installation process
Also using many projectors
large screening is possible
but as I previously mentioned, cleaning the lenses, and repairing the lamps
and such various maintenance cost
is one downside
LED and LCD display
are usually constructed of a single panel
and can be made in a curved shaped
but limitations in shape still exist
But projectors can screen on various surfaces
like curved, unstructured surfaces, and complex architectures
which makes creative design possible
Thanks to this advantage, more fluid visual expression
can be delivered
LED display
has relatively higher energy efficiency
and uses less electricity
But projectors,
especially high luminance projectors have high electricity consumption
so checking this criterion would also be crucial
Media Facade Workflow
Next I'll explain the workflow
of creating this media facade
and it will be in five steps
First is idea planning
The goal of
what to express with media facade must be clear
Some examples might include advertisement of certain events or
strengthening brand image, or artistic expression
You must also analyze the environment and architectural characteristics of the location of the art
and of the audience
I also call this placeness
A media facade is considered a good one
if it conveys the place's character well
Also one must plan a story
that can communicate with the architecture
Content with story
provide better immersiveness to the audience
Second is design and visualization
Initial idea is sketched into various visualization methods
using concept sketch
and the overall plan is made
Also, 3D modelling must take place so that
the content can be shown according to the architectural structure
Through this, we can pre-check
how the content would look in the actual projection environment
And the visualized sample content
will be simulated as a digital prototype
considering light, color, screening method, etc.
These days, through various engines,
a step called pre-viz is also sometimes done
Next, to realized the facade
various equipment and technology must be installed
Choosing the needed equipment is crucial
LED panel, projector
and control system and other gears are needed
and according tot he building's size and requirements
suitable equipment must be chosen
Mapping also must take place
which is the step where the content is placed to fit the actual architecture
Adjusting the screen ratio, deciding where to screen it,
and preparing technological accuracy is very important
For content control
control system must be set
Controlling the content for each time horizon or altering the lights
are planned with various software
Fourth is testing and optimizing
One must test and adjust to see
if every element is working fine
After all equipment are installed
the content's resolution, placement, and lighting
must be checked to see if they are just right
Errors like distortion or color mismatch
must also be checked and adjusted
According to the architectural environment, the content must be optimized
so that it is clearly visible no matter day or night
Lastly, the media facade must be run
and long term maintenance must be in place
To maintain the quality of the content and run it smoothly
real time monitoring is necessary
Maintaining the installed equipment's function
and updating the content when needed
and replacing certain equipments is also a crucial step
Lastly, by analyzing the audience's reaction and data
collecting data on what to improve and upgrade
will help produce better content
Media Facade Practical Program and Solution
Third, in media facade practice
I would like to introduce the programs that are used
The first of these is the programs used in
the concept and visualization stage
Provided by Adobe
Photoshop and Illustrator are those
The purposes are creating visual elements and editing images
They are also used for working with vector graphics
Textures, colors, etc. to be used on the facades
on the buildings are edited
In is an essential programs for elements such as
logos and icons in vector format
Next, 3D modeling and animation production
programs include Cinema 4D, Autodesk's 3ds Max
and also Blender etc.
Model the 3D structure of the building
and content to be projected onto actual buildings
are simulated with these
How content will look at different points of view
can be checked in advance which makes them very important programs
and at this stage, the actually 3D scanned
models are used to reconstruct as well
For creating motion graphics and visual effects
Adobe's After Effects program
must not be left behind
This program allows you to add
colorful movement and visual effects to the media
Especially for creating motion graphics tailored to the exterior walls of buildings
it provides excellent features
therefore it is used necessarily
Next comes the programs used
in mapping and simulation phase
At this stage, the content you have created
is placed precisely to fit the building
pre-testing work in a real environment takes place
There are a wide variety of programs.
Three representative programs used in practice
will be introduced
The first one is Resolume Arena
It is a tool developed for media server or video mapping purposes
Resolume Arena lets you manipulate content in real time
It's a very powerful tool for video mapping
Adjusting the ratio of the content to fit a specific part
or distorting the screen is very easy in this program
which makes itvery suitable for media facades
The second program is MadMapper
It is a program specialized in video mapping and lighting control
It has an intuitive interface
which makes video mapping and lighting control easy
Various graphics on the media facade
can be freely placed and is often used in practice
Environments where multiple projects are attached, etc.
is very intuitive and easy to use
The third is TouchDesigner
You can create real-time interactive content
By linking data feeds and sensor inputs in real time,
media facade that interacts with the audience
is often made with this program
Thirdly, development environment that includes
complex interactive element production
First, I would like to introduce Unity and Unreal Engine
Interactive and VR or AR content
can be produced with this engine
Both are game engines, but it is appropriate for
real-time 3D visuals
and those that require interactive elements
Recently, these two engines have been utilized
the most in development environments
Arduino and Raspberry Pi
and such small computer
lets you control hardware
and do sensor based work
Often in interactive art
it can control sensors and electronic components
and it is useful for linking tasks like
movement, light, sound, etc.
Media Facade Practice Program
and solution
First, to proceed with the corresponding media facade
3D scan the scene
3D scanning is the technology that collects in digital form
of three-dimensional data of real objects or spaces
In media facade, 3D scan is performed to clearly understand the structure of the building
and to precisely map the content
By utilizing the curves, textures, and detailed data of the building
the digital content is created
It is useful for simulation and preview
and pre-visualization
To accurately project content onto complex structures
This 3D scan data is used
Even on the restoration or maintenance of cultural heritage buildings
it allows you to create media facades
As for technical equipment, we use Lidar
Equipment that can quickly scan buildings of any size
or complex terrain and photogrammetry is also used
Compositing multiple photos taken with a drone or camera
is also a way of creating 3D models
and with a handheld 3D scanner
we also scan small installations
Bringing the scanned data into Blender or a modeling tool
lets the restoration of the corresponding model possible
So we can take a look at the restored architecture
and see which perspectives can be used
and find vanishing points as well
So the exact location of the corresponding structure and
detailed description of various expressions
are used to finish the work
3D modeling is the process of organizing scan data or
creating 3D objects from scratch without scanning
digitally
In media facade
A digital twin of a building
is made and UV map
for content mapping is produced
And 3D modelling is done to
conduct an accurate simulation that takes into account obstacle factors
like windows, structures, etc. on the media facade
The process involves cleaning up scan data to remove noise
and surface cleaning
Working on surface coordinates is also a part of it
which allows you to place content accurately
This is called UV mapping
Texture production finishes the job which
are applied to the digital content
The workflow involves collecting 3D scans as basic data,
organizing this scan data in modelling software
and modifying the necessary parts
Also through mapping tests
the matching of the content and the building is simulated
So, the part where the scan data is lacking is
supplemented with some of the data with modeling
In the case of curved content, to make sure the mapping results are not distorted
it is tuned precisely
The work that was completed with the modeling is taken through photography and Photoshop
and illustrations to create a mask
So of the building, various photos
from various angles are used
so that this main mask is organized
especially in the frontal mapping
If you proceed with the actual work, now
we utilize various programs
Same work as the previous masking process
and tasks that can create lines
are done with programs called Photoshop and Illustrator
By vectorizing the various components of the corresponding space into lines
they are moved to a program that can work on this again
For tasks like this, After Effects
which is motion graphic effects are then made
Through a simple task called trim pass,
components can be made to glow in lines
Through this work
it is again combined with the mask worked on earlier
so the corresponding content can be
projected well
We work on a timeline with these different footages
As for content that needs to be dragged on for a long time,
very precise adjustments are made
to create transitions by inserting various footage
And among the examples like this
all the footage that are finished working on are collected
and then edited using a program called Premiere
At this time, sound is also added to the corresponding section
to see if the video and sound are well matched
In fact, we proceed by creating various files like this
The trim pass we created earlier
It is also used like this in the beginning
Sometimes the entire thing is used as one side, but
Sometimes we proceed by dividing it into sections
If the building is long horizontally
Especially in the case of cultural heritage like this
Because it is not visible at a glance, securing the field of view
and placing contents that fit that viewing angle
here and there is important
As you can see now
adding various SFX sounds
and matching them to the corresponding content
is also very important
Within the corresponding facade content
a wide variety of programs were used
Unity, Unreal, After Effect, Blender, Premiere, etc.
are all utilized in practice
The reason why we don't stick to just one program
and rather use many different ones
is because for each applicable program
there are different ways to visualize it
and also for each applicable program
the parts that can be emphasized are different
so I highly recommend trying out a variety of programs
These patterns, like Dancheong
by utilizing the assets you have
are great techniques for creating unique shapes
As we use various programs
there are also many different rendering methods
All these rendered footages
must be color graded on Premiere
which is the last and a very important step
As you can see, the part covered by the mask is
Because this is the part where the image is not projected
If this mask work is not done in the beginning,
you cannot estimate how long
or what the ratio the content is showing
so the mask work must come first
Depending on the material surface of the projection target
the depth of the projected image changes
So we are doing mapping
and examining how this architecture and content can be well connected
is also something that requires great care
The characteristics of the building in question
must be preserved in the mapping
As you can see, there are lasers
and other images being mixed in the environment
so one must design how to
best project one's content
The photo below shows the corresponding modeling
performed virtually
Let me show you
scenes were performed in the actual site
When you perform an actual facade
the sense of speed seen on the screen and the sense of speed seen in real life
are very different
I recommend that you try mapping even on small mockups
Also, depending on the time it is shown
the brightness of the projectors varies greatly
Please pay attention to that part and the surrounding brightness level's
effect on the image to be screened
must also be checked
When carrying out the project
the main purpose also needs to be well organized at the planning stage
Whether it is a commercial advertisement, a cultural event, or a work of art, etc.
With the connection between architecture and content
you must plan a project in a creative approach
that utilizes the building's features
The scene you can see in this part is
The corresponding part of the tile roof
Because the material is made of dark colors
You can see something that is somewhat invisible compared to the wall
The characteristics of these materials, the characteristics of the building, etc.
must be thought carefully to create the content
Equipment and environment are also very important
How many projectors are installed and at what angle
If the projector moves even just 1 cm
the projected image looks very different
so installing the projector properly on site
and with HDMI, power, etc. all fixed
the mapping should take place
Also how did the projection distance is set
Using a single focus or ultra-short focus lens, etc.
is also something that happens in projecting
so you should carefully consider the angle of projection and the angle of view
Also, the projection environment on site
are influenced by time like day or night, weather
or near lightings so solution to minimize the effect of these
are also needed
How to secure and align the projector
is also a thing to consider
In the site
there are times when immediate corrections and responses are needed
so real time feedback is very important
With the audience or staff on site
Communication is very important
Lighting, ambient colors, distance, etc. in a real projection environment
must be optimized
and technical issues that may arise when handling the projector
and solutions are adjusted
When planning a project like this
collaboration with various teams is required
3D Modeling Designer, Animation Designer
technical directors and others'
collaboration is essential
Now, through various field installation cases
I would like to explain the media facade
First, installation according to architectural structure, curved surface, flat surface
and large buildings, there are various examples
and depending on the placement of the projector and the projection technology,
the cases are slightly different
There are examples of using single focus or multi-projectors
Also, in content creation strategy,
by utilizing the form and message of the building,
creative content are created
And we need to consider the field environment
Whether it's outdoors or indoors, how much light pollution occurs
How the audience's movement is structured
And finally, there are times when
interactive elements exist
Examples of audience participation or sensor-based technologies
will be introduced
This is a project where the same installation was done indoors and outdoors
In these large spaces
because it has the characteristic that the sound is not actually played
I think it would be good to consider various field situations
There are several cases like this one where multiple LED screens
were screened simultaneously
so it is important to create looping sections that doesn't feel awkward
when played over and over again
Although it is indoors, when light comes in through the window
the screen of the corresponding projector may appear blurry
This is a case of such instance
In the case of this site, it was surrounded on four sides
The projector is installed on the ceiling
Contents that make the four sides interlock
was the part that I focused on the most
The location of the exhibition space is also
one of the main points I consider
Unlike the exterior walls of city buildings, these spaces
can provide even more intense effects
Not for commercial purposes
but instead a facade work that expressed an artistic message
There were also scenes that had interactive elements
On the right, a child is using VR
The virtual space seen by VR users
and the real space in real life visible from the outside are consistent with each other
And we've hidden a small Kinect camera
to give a section where the corresponding audience member appears on the screen
This is a site that combines interactive elements
When a VR user clicks with a hand controller
the screens that are shown to external users
the changes in hand controllers can be detected directly
and seen
For both camera sensors and VR users
by placing them in the same virtual environment
It was content that reacted in real time
Recently, these interactive media facades have also been
becoming very diverse
Interaction at the ICC building in Hong Kong
or Interactive billboards in Tokyo, Japan
are some examples
Audience participation elements include motion sensors and
and touch pads that respond in real time
and using Touch Designer or Unity, etc.
interactions are produced
Also, there are cases where real-time data feeds,
weather or social media data, etc. are reflected in contents as well
Let me explain a little more through various examples.
In the case of large curved structures, such as the Sydney Opera House
or the Louvre Museum pyramid
to solve the problem of projection distortion on curved surfaces
with 3D modelling and UV maps
the curved surfaces are connected smoothly
And we map out the points where there is distortion
Also seamlessly connecting multiple projectors by synchronizing them
edge blending takes place
Like the Times Square Building in New York or the COEX Artium in Seoul
projections are also made on the exterior walls of city buildings
Precision and detail in mapping on large planes are important
Architectural elements such as windows and columns
must be reflected in the content
Also, light pollution in urban environments
and harmony with surrounding structures is also something to consider
There are also small-scale installations
On the exterior walls of installation media art exhibitions or pop-up stores, etc.
impressive effect even in small spaces
require content strategy
and you need to use a short throw projector to project at a close distance
Also commercial purposes or artistic messages
must be combined well
There is also a media facade that is integrated with nature
Projected onto Gaudi's architecture in Barcelona
this nature-themed content is an example
It must include storytelling that fits the history and theme of the building
and natural elements, water, trees, light, etc.
and digital technologies must be harmoniously integrated
Also, this project environment's effect on
media facade is also something to consider
You have to consider things like weather and lighting
Finally, event based
temporary media facade also exist
For Olympic opening ceremony, World Cup, etc.
the large scale media facades take place
Because it needs to have a strong effect in a short period of time
various installation techniques are being considered
Portable projectors and temporary structures are also used
Also, storytelling and the corresponding event theme
being shown through the content
is the most important thing to consider
The programs and solutions introduced today
are essential to complete the entire process of media facade
Each of the programs introduced in the latter half
has its own unique role and function
Depending on the appropriate circumstances and nature of the project
you must select and use it appropriately
This concludes our lecture on media facades
Understanding Media Facade Media Facade A compound word of Media and Facade, meaning the exterior wall of a building, refers to a method of expressing visual content by projecting images onto the exterior wall of a building
Starting with projectors in the late 1990s, they have developed along with cutting-edge technologies such as LCD, LED, and OLED The purpose of media façades goes beyond building aesthetics to present space and convey messages artistically
LED/LCD High resolution and brightness provide clear picture quality regardless of the weather LED provides high brightness, so it is visible even in bright environments
High initial installation cost, but relatively low maintenance cost, long life and durability, efficient in the long term Structural support is required for display panel installation, and maintenance is relatively simple
Integrated panels are common and have shape constraints Relatively high energy efficiency and low power consumption
Projection media Poor clarity in bright environments, suitable for night and dark environments Low initial installation cost, but ongoing maintenance costs, and quality deteriorates over time
Large-scale projection with multiple projectors is possible with simple setup, maintenance like lens cleaning, adjustments, lamp replacement needed
Can be projected onto various surfaces, allowing for creative designs High brightness projectors have high power consumption
Media Façade Workflow Idea Planning Goal Setting: Clearly define what you want to express (promote a specific event, enhance brand image, artistic expression, etc.)
Target Analysis: Examine the location’s architecture, environment, audience Storytelling: Create story that connects with the building for immersion
Design and Visualization Concept Sketch: Visualize ideas with sketches to capture the overall concept 3D Modeling: 3D models to align content with the building’s structure
Digital Prototype: Simulate pre-made sample content into a digital prototype, taking into account light, color, projection method, etc.
Equipment Installation and Technical Implementation Equipment Installation: Choose equipment like LED panels, projectors, and control systems based on the building's size and needs
Mapping: Arrange and align content to fit the building, adjusting screen ratios, projection locations, and ensuring technical accuracy.
Control System Setup: Create time-based content control and lighting transition functions for content adjustment and control.
Installation and technical implementation Quality and location testing: After installing all equipment, check the quality, location, lighting, etc., of the content
Adjust & Correct Errors: Fix screen distortion or color issues Optimize: Ensure content is visible both day and night
Execution and Maintenance Real-time monitoring of execution and maintenance: conducted to maintain the continuous quality of content and smooth operation
Maintenance: Maintain equipment functionality and replace equipment, update content when necessary Data Analysis: Analyze audience response to gather data for future improvements and upgrades
Media Façade Practice Program and Solutions Media Façade Practice_Concept and Visualization Stage Program Adobe Photoshop & Illustrator
Uses: Visual elements, image editing, vector graphics work Editing textures, colors, etc. for use on building facades Working with elements such as logos and icons in vector format
Cinema 4D / Autodesk 3ds Max / Blender Usage: 3D modeling and animation production
Modeling the 3D structure of a building to simulate projected content It allows previewing how the content will appear from different viewpoints
Adobe After Effects Uses: Create motion graphics and visual effects
Used to add colorful movement and visual effects to media façade content Provides excellent capabilities for creating motion graphics tailored to the exterior walls of buildings
Media Facade Practice_Mapping and Simulation Step Program Resolume Arena Usage: Media Server and Video Mapping
A powerful tool for manipulating content in real time and mapping videos Easy to adjust the scale and distort the screen to fit a specific part of the content, making it ideal for media façades
MadMappe Uses: Video mapping and lighting control
A program that allows easy video mapping and lighting control with an intuitive interface It is frequently used in practice because it allows the free placement of various graphics on the media facade
TouchDesigner Uses: Create real-time interactive content
Content can be created with interactive elements Used to create media façades that can interact with the audience by linking data feeds and sensor inputs in real time
Media Facade Practice_Programming with Interactive Elements Unity & Unreal Engine Usage: Creating interactive and VR/AR content
Although it is a game engine, it is suitable for media art work that requires real-time 3D visuals and interactive elements.
Arduino / Raspberry Pi Uses: Hardware control and sensor-based tasks
Useful for controlling sensors and electronic components in interactive art to create real movements, light, sound, etc.