Hello
Today, let's take some time
to analyze new technologies in more detail
Today, we will utilize various sensors and
how they can be used to interact with users
Next, we'll explore the importance and potential applications of XR technology
So, based on these aspects
we aim to utilize new technologies in immersive content
and to discuss what kinds of content
can emerge from these technologies
Definitions and Applications of XR
First, let's discuss the definition and applications of XR
I assume the definitions of XR, VR, AR, and MR
have been covered in many lectures
However, I'll try to explain these concepts
focusing less on theoretical and foundational aspects
in a more simple way
AR is simply based on smartphones
You can think of it as viewing through the camera lens
Perhaps it's easier to think of it that way
Then, what's VR?
VR is based on HMD, which stands for Head-Mounted Display
it's entering a virtual world to enjoy content
VR can be defined as immersing oneself in a virtual environment to experience content
Then, what's MR?
MR can be defined as being based on HMD
But incorporating lens-based technology to simultaneously show the real world
allowing for augmented experiences
Actually, the definitions I just provided for AR, VR, and MR were
to help everyone understand
It might be helpful to blend those examples a bit
for easier understanding
When I approach technology
I often see technology based on content from movies
In fact, if you enjoy sci-fi movies, you'll see all sorts of futuristic technologies in them
So, practicing matching terms with scenes
from such movies might make it easier
for you to encounter new technologies
Thus, we've concluded with the definitions of AR, VR, and MR, returning to the basics
What's the definition of XR?
XR is a definition that combines all definitions of AR, VR, and MR
So, some people actually refer to AR as XR as well
or VR as XR
or even MR as XR
However, it's not an incorrect statement
But if you see XR as an overarching concept
it could be seen as a somewhat incorrect statement in that context
So, going forward, the definition of XR would be
I see it as spatial-based or spatial-content
which is likely to expand
That's not to say that AR, VR, and MR aren't part of XR
But it's important to clearly distinguish the differences between them
Perhaps we could define AR, VR, MR and XR
in terms of their potential for expansion
and extension into these areas
Then, if I were to give a simple definition of XR
let's delve a bit deeper into the definition of AR this time
AR stands for Augmented Reality, which means augmenting the real world with digital elements
Augmented Reality, it's on top of the real world, in physical space
It combines images or videos existing in real space with computer-generated graphics
Therefore, it synthesizes images, whether they exist or not, into the real world
It's a technology that creates virtual spaces within the real world
Where? Where do they create it?
Yes, it's created within a monitor
This kind of augmented reality technology
immerses users in virtual reality
enabling real-time interactions that you can't see in the actual environment
That's how it's defined in the dictionary
Actually, this immersion is incredibly present in both VR and AR
It seems to be present in MR and XR as well
But the idea of immersion seems to involve a similar format to the real environment
It seems to involve implementing that concept
It relates to various industries such as broadcasting, video, and healthcare
It's being applied in various areas
However, AR is one of the most representative
If we're talking about Pokemon GO
the second most prominent thing we encountered after COVID-19
could be considered QR codes
I actually thought QR codes were going to disappear
I find it as ugly and inconvenient
In fact, after COVID-10, there's been a significant increase in the use of QR codes
which have become highly activated
When you go to places like restaurants, you often see keyboards, tablets, or similar devices for ordering, right?
In the future, these kinds of processes will likely
be mostly done through QR codes
And these QR codes
come in a wide variety
serving as a means to promote oneself and express various aspects
Next up is virtual reality
Virtual reality is a completely imagined space
It's a space that doesn't exist in reality
That's why it requires special devices
It's represented by devices like HMDs
Through head-mounted displays
they allow users to access virtual worlds from the real world
and transform the real-world experience
into accessing an artificial environment
In fact, virtual reality can be composed of real places or videos
or it could be 100% computer-generated CG environments
Real environments can be created using 360-degree cameras
and those cameras can produce videos of actual locations
By creating videos of these environments
we try to represent actual places as if you're really there in 360 degrees
So far, these have been represented mainly through HMDs
accessing these realities through head-mounted displays, but
as mentioned in the previous lecture, square and similar areas
extensions of XR are also possible
Extensions into the virtual realm are where
much of the expansion is happening
What are the differences between virtual and augmented reality?
The biggest difference is
whether it utilizes information from the real world
In case of augmented reality, it involves using the camera
by receiving information from the real world
and it operates based on that information
But virtual reality is entering entirely new spaces, spaces that don't exist
Even though it can recreate existing spaces, it also creates entirely new virtual reality spaces
The data for spaces in virtual reality
doesn't measure data from real world
Therefore, the biggest difference between virtual reality and augmented reality is
whether it utilizes information from the real world
And whether it creates entirely new spaces from scratch
it's worth considering that virtual reality can come into existence
For examples of augmented reality
content extensions can be seen in games like Minecraft
It can also be exemplified in experiences like Pokemon GO
Currently, the most promising developments could be
applications in the logistics system
Possible advancements in the logistics system as well
And it can be personalized and tailored to individual needs
It would be beneficial to consider personalized marketing ventures as potential content
for virtual reality and augmented reality in the future
I'm sure you all know what algorithms are
What algorithms do you know?
You've probably heard of
the YouTube algorithm, the Google algorithm, and the Naver algorithm
When you've looked into how algorithms determine what content surfaces,
going forward, personalized content tailored to each individual
will continue to be provided on YouTube
Thus, some contents that you're interested in, such as
if you're interested in politics, you'll see a lot of political content
If you're interested in VR
you'll find a lot of VR content
If you're interested in artificial intelligence, you'll receive a lot of recommendations for AI content
If you're interested in celebrities, you'll find a lot of celebrity content
If you're interested in Hallyu (Korean Wave), you'll also find a lot of content related to that
When you've explored these areas
I believe people can create new algorithms
based on these existing algorithms
Based on the aspects that you've already formed
through your algorithms
I can probably tell what specific colors you like
Can I?
Well, based on the content
through algorithmic analysis, it's possible to determine what specific colors people might like
Color preference and matching outfits: Algorithmic analysis and recommendations If, for example, when you shop online
and looked at clothing related to the specific colors you like
it can be seen as something that could be included in recommendations by the algorithm
I'll imagine this scenario a bit further
This is a product that hasn't been released yet
The phone bends, right?
Through a flexible display, the phone bends
And fiber-based displays are currently being developed
Let me share an imaginative scenario about future content
While preparing for the lecture this morning,
I was considering which suit to wear and looked at various colors
There's navy blue, black
and even charcoal gray suits, among others
I ended up wearing the most classic black suit
However, when I stood in front of the camera
I started to think, "Maybe I should have worn navy blue instead?"
and felt a slight regret
Decisions and dilemmas in everyday life could be solved through algorithms It seems like these aspects could be addressed in the future
I'm imagining content right now
It's a prudent imagination about
whether these aspects might actually come true in 10 years
Flexible displays have already emerged in smartphones
The display bends
And there's a lot of development and
emergence of fiber-based displays right now
So, in the future
it's possible that suits could be made
entirely from fiber materials like this in the future
Someday, it's conceivable that
there could be features on phones where you can say
"Today, I'd like to wear this color," or "Today, I'd like to wear this pattern"
If that were the case, using your phone in the morning
you wouldn't buy clothes based on
patterns like A today, B tomorrow, C the next day
clothing becomes more like a tool
and I think it might change into a form
where people buy designs created by you
In that case, industries like VR and XR, which produce immersive content
could be seen as connected to the fashion industry
Until now, designers and design, as well as content
have been somewhat disconnected from industries
That's why, in fact, design has been
more about my own pride and satisfaction
In reality, many people, especially older generations
have the perception that design is something that can be done in just one day
However, with the emergence of content and immersive experiences, there's been a significant industrial connection
You should be aware of the significant changes
happening in industries, which would be beneficial
Next, we have an example of virtual reality
Currently, the virtual reality we know of is
primarily focused on the entertainment sector
If you look at the cases, you might already be familiar with them
But in this entertainment context
the VR content we create is
now becoming usable
in a wide variety of fields
While the majority of VR content is still based on devices
like head-mounted displays (HMDs)
these applications span across various industries including gaming, film, fine arts, design, architecture, medicine, aviation, military, etc.
VR device Oculus/ Artwork created with VR by Giovanni Nakpil It's reported that there's a lot of utilization happening across various fields
Indeed, in the United States
military personnel are actively using VR technology
for various aspects of military training
Samsung Electronics in South Korea, too
They supply VR training content to the Ministry of National Defense
where personnel wear Gear VR for virtual training
I've analyzed some of these cases
You can see that these technologies are also widely utilized in South Korea
especially in reserve forces training
And these advancements are further developed in the field of MR
Mixed Reality
Mixed Reality is a blend of virtual and real worlds
This mixed reality involves real environments and augmented reality
This includes both augmented reality and virtual reality,
creating a digital world within the real world
that can be integrated with augmented reality and similar technologies
Imagine a scenario where
these real and digital worlds merge
While wearing the MR device, you could navigate the streets and engage in various activities
Mixed Reality (MR) examples: Microsoft HoloLens 2 / Meta Quest 3 Could it be a world similar to the one Iron Man creates in the Avengers?
While using MR technology, you could have you own J.A.R.V.I.S., assisting you with various tasks and providing information
It narrates information like an encyclopedia, based on your knowledge
But what do we have?
We have AI from Google for Android devices, and Siri for iPhones
With these features, currently,
many aspects of future content depiced in Avengers movies
are now reflected in the real world
Apple has released products like the Apple Vision Pro
These developments indicate
a gradual convergence of the real and virtual worlds in our reality
Because I was curious
I also wanted to try out devices for Gear VR and mixed reality
so I bought one and tried using it
When watching movies
you feel like you're completely immersed in the scene, as if you're right there in the square
And when playing games, you experience a tremendous increase in immersion
with a heightened sense of realism
In Korea, there aren't many people
using these devices to walk around the streets just yet
However, I've personally seen many people doing so in the United States
There are even people working in cafes using these devices
Additionally
some people use MR devices
to walk around and view objects in real life
And there are also people who engage in continuous conversation with Siri
I believe that in the future, a world blended with Mixed Reality
Virtual Reality, and XR will unfold before us
I assume that because we anticipate such a world emerging
it's likely why you're listening to this lecture now
Then, if we consider whether XR was always just XR from the beginning
the answer is likely no
If we ask whether VR was always just VR from the beginning, the answer is likely no as well
In fact, the origins of VR can be traced back to simulations and visualizations
used for various purposes
before evolving into the immersive experiences we know today
So it started with simulators
Then it was expressed as VR, AR, MR, XR, and so on
It's being implemented as a time
when we're going back to simulators based on VR and XR
And these aspects
now more people are buying HMDs and using them on their own
And now there's a need for extensions of content
that can be used beyond just wearing HMDs
Let's delve a bit deeper
into the technical aspects of these developments
Now, XR technologies
They encompass a wide range of applications across various industries
Describing these technologies as encompassing a diverse array of capablites would be apt
XR is currently based on tasks such as 3D scanning
motion capture, CGI, and real-time rendering technology in Unreal Engine
Furthermore, deep learning techniques for tactile sensation recognition through VR gloves
are being analyzed as production elements for XR in VR
Indeed, 3D scanning encompasses not only avatar's skeletal structures
but also building architecture
So, the concept of digial twin for buildings and the digital mirror world mentioned in the lecture
can be related to the use of 3D scanning
These days, not only do people work with actual modeling, but
they also express concepts through 3D scanning operations
And along with that, there's motion capture technology to capture movements and actions
All the actions I'm doing
they can be replicated
by a 3D-scanned avatar with natural movements
This capability can be implemented not only with high-end equipment
But also with the smartphones you already have
which is worth mentioning
So, as you can see on your phones
most come equipped with at least two to three built-in cameras
One is known as a depth camera, and the other is an RGB camera
One is based on vision cameras, including IR cameras
Smartphones with three lenses, including depth-sensing and vision cameras
can perform motion capture tasks
In addition, CGI technology, which stands for Computer-Generated Imagery
refers to the process of generating and manipulating visual content using computer software
It's one of the oldest and most foundational technologies in the realm of digital graphics
You could also think of CGI as advanced post-processing for achieving high-quality visual effects
In the past
when creating content based on 3D
All of these elements were primarily developed through rendering technologies
The content created through Unreal Engine and Unity Engine, for example
Live techniques allow for the expression of realistic aspects in real time
even those previously rendered
The following is deep learning
If artificial intelligence technology is incorporated here, it woud enable the creation of higher quality textures
and backgrounds for the technologies that one's developing
Additionally, you can see that gloves and head-mounted can provide
input-sensing that allows for more diverse expressions
through facial expressions and other features
Based on these aspects, it would be good to see that the metaverse platform
it would be good to see that the metaverse platform
can create a wider variety of XR experiences
These aspects allow
for the creation of content in real-time
based on various hardware and motion capture
utilizing diverse inputs
While the performance of the device is also very important
Because device performance has improved significantly
You can see that there are hardly any content
that are being created now that cannot be run
Furthermore
As these aspects become more diverse in terms of devices and sensors
The applications of XR technology are increasingly expanding
Particularly centered around simulators
In fields such as manufacturing, healthcare, education, culture, and defense
Based on these aspects
You can also see that XR can no longer exclude elements
Such as artificial intelligence
These, including artificial intelligence, now involve creating virtual assistants
So, what we were talking about earlier - creating a virtual assistant -
is actually content
However, the technologies that transform what I'm saying into text
and translate into different languages
can be summarized as aspects of artificial intelligence
within the context of virtual assistants
These could lead to a future
where you can naturally utilize everything
by simply wearing earphones
similar to how you might wear earbuds or AirPods now with your Buzz or iPhone
When these aspects become more commercialized
for mixed reality (MR) devices
They can be expressed in a much broader range of applications
Then why would we want to express these aspects of XR?
Why would XR be suitable for industries?
We can actually create real content
Why would industries consider transitioning to XR virtual and augmented reality worlds
transitioning to XR virtual and augmented reality worlds
when they already have existing content to work with?
Would you like to think about it for a moment?
These factors are also evolving in tandem
with our environmental impact
I think you've probably heard a lot about ESG (Environmental, Social, and Governance) issues worldwide
There are indeed many issues related to the environment being addressed
These aspects are also relevant to virtual reality and augmented reality
XR Benefit 1: Waste Reduction And XR helps to reduce waste
Since separate training facilities are not built
It means that training can be conducted multiple times without being discarded after a single use
Because it is a content
XR Benefit 2: Cost Reduction And it can reduce costs
Content related to these aspects
Content that is created once and then discarded
needs to be continually discarded
Therefore, it needs to be continually produced
However, virtual content that is produced once
can be continuously utilized
XR Benefit 3: Improved Understanding Additionally, it can enhance comprehension
XR Benefit 4: Project Efficiency And it allows for efficient project management
Because it's essentially composed of virtual elements
XR Benefit 5: Enhanced Safety Safety is extended and improved
XR Benefit 6: Better Decision Making It can even lead to better decision-making, making a total of six benefits
We anticipate that these aspects will expand further in the future
due to environmental and ESG considerations
Sensor-based interaction research
XR technology encompasses various sensors
User movements, positions, voice, emotions, and more
It involves collecting various information
and reflecting it in the virtual environment
So, because of these aspects
It's often tied closely to the real world
Augmented reality similarly relies heavily on various sensor-based foundations
for its expression
Actually, you could say that
Each of you also has these sensors one by one
Gyroscope and accelerometer sensors Both the gyroscope and the accelerometer sensors are present in VR devices as well
This gyroscope sensor is calibrated based on gravity here
aligning with movements accordingly
It initially started as a sensor primarily for gaming purposes
Various contents are being developed
Based on these sensor foundations
You've probably seen a lot of content on mobile devices
Where you tilt to control the movement
Thus, these contents are expressed
through the interactions with sensors
found in mobile and smartphones, among others
And there's also the accelerometer sensor
Using various sensors
The accelerometer measures
The current position and the next position to determine the acceleration
The current position is determined based on past positions
The sensor also incorporates aspects like velocity
assessing how much movement occurs within a certain time frame
Say, in one second
It allows for a more extensive expression of the interaction
in VR
and XR
Next up is the motion sensor, what is a motion sensor?
It's a sensor that detects gestures and motions to express actions
In the past, motion sensors typically relied on recognizing this kind of shape
(A rectangle cut in half-like shape) to initiate recognition
Nowadays, content based on human skeletons can be extracted
Not only from specialized motion cameras
But also from regular cameras
It recognizes human faces as well
By recognizing the face
It can then extend recognition to the hands, feet, and individual joints
Now, it's possible to recognize human motion
incorporating recognition of various body parts
Radar sensors that utilize electromagnetic waves Next up is the laser sensor
In reality, laser sensors could be based on devices like laser pointers
The ones you see used in presentations or lectures
Actually, these laser sensors can also display colors in various ways
Right now, it's emitting red light
It can also display the red as blue
It can also be expressed as green
And in fact, it can also be represented as sensors
that utilize forms invisible to the naked eye using visible light
So, by positioning the sensor from under the palm of the hand to above it
It can diversify the content accordingly
And these aspects enable the expression of content in a wider variety of forms
Speaking of these laser sensors
They can actually indicate whether a person has passed by
representing the flow of movement
It can also detect speed
The utilization of laser sensors involves
expressing applications in various fields
each focusing on the detection of flow
through the first and last sensors
Next up is the voltage sensor
The Capacitive Touch Sensor detects changes in electrical capacitance to determine user contact These sensors are commonly used in smartphones
In fact, older smartphones used pressure sensors rather than capacitive touch sensors
So, they recognized when pressure applied
Nowadays, it's not the pressure applied
But rather the user's touch sensation based on their electrical capacitance
Therefore, it detects regular changes in capacitance to sense the user's touch sensation
When it's applied to clothing or similar items, it prevents responses based on user contact
and instead detects changes in capacitance
These aspects are expressed through the touch method of capacitive touch sensors
These aspects are utilized in various industries such as homes
Such as homes, smartphones, automotive systems
and electrical control systems
The utilization of capacitive touch is
indeed extensive
In sensor-based interaction, there are also image sensors
An image Sensor reads object information and converts it into electrical image signals This image sensor is a device that
converts visual information into electrical signals, essentially what cameras do
These components convert the light from real-world objects into electrical energy
transforming it into digital signals
which allows physical scenes to be captured and rendered digitally
The size of the image sensor directly determines the quality and resolution of the images you see
Larger sensors typically capture more light and detail
leading to higher-quality images
The image sensor is crucial when transitioning from the physical world to the virtual world
As it serves as the core technology that captures
and converts real-world objects into digital signals
The next component is the GPS sensor
Satellite-based Navigation System: GPS What is a GPS sensor?
GPS sensor receives signals transmitted from the satellites
So, based on the signals sent from these satellites
The current location of the user is determined
Therefore, in a way, one of the most common applications of GPS sensors is
navigation system
Things like your metro card or other similar systems
often rely on GPS-based technology for distance-based integration
These applications are widely used in various fields such as navigation for sailing
aviation, automotive navigation, hiking, and fitness tracking
Recently, there has been a trend towards creating content
that shows outdoor landscapes and environments
As these areas become more diverse
They are based on indoor positioning systems
Contents based on Wi-Fi positioning
The development of technologies that measure
indoor positioning
based on the distance of Wi-Fi signals is also considered
It's likely that services based on these technologies
will become increasingly common in the future
In reality, due to Korea being a country that's technically at war or in a ceasefire situation
Its industries may not have flourished as much as those in peaceful countries
However, in countries like the United States and others
The signals from GPS satellites are being leveraged
for various forward-looking commercial ventures in the private sector
In places like the California desert
You can witness scenes where private satellites are actually being launched
The gyroscope sensor detects and measures rotation speed Next up is the gyroscope sensor
It detects and measures the rate of rotation
It's expressing that a lot of movement can be represented
based on content using gyro sensors
The gyro sensor-based content is widely utilized in drone-related aspects
to represent such activities
In drone sensors
Gyro sensors are extensively used to depict rotation and movement
So, the drone you're seeing now is
based on four gyro sensors
to control its movement
You can consider these sensors as part of a composite sensing system
that contributes to a sense of realism
The convergence of cutting-edge technology and content in the field
In such various sensors
it's fascinating how various emerging technologies are shaping content creation
across different fields
XR technologies are revolutionizing the architecture and engineering construction industries
by integrating sensors
and content creation in immersive environments
For another example
XR in the automotive industry encompasses various aspects
Electric vehicles can be seen as a combination intermediary
for the integrated aspects of these sensors and content in terms of XR
These aspects, before actual construction
are based on virtual simulators
In fact, it's even possible to refine these aspects further
to conduct prototyping on a smaller scale
XR in media and entertainment
VR games, chat apps with augmented reality, AR filters, virtual exhibitions, and more are
increasingly becoming integrated into your daily life
in various recommended forms
And various aspects can be seen as content-convertible from diverse sensing
as content-convertible from diverse sensing
based on motion recognition sensors
mentioned earlier
NEWJAK's Laser Training: Simulation training for avoiding lasers The content you're currently seeing is
an e-sports style experience based on LiDAR sensor technology
See this as an example of
How LiDAR sensor technology is used to recognize users and
How it's utilized to create content for e-sports
without the need for multiple users
to wear sensors
NEWJAK's XR Shooting Simulator: Body movements using gyroscope sensors Another example is using sensors for firearms in XR, based on infrared (IR) technology
and representing various silhouettes of firearms positions as part of XR content
using methods that don't involve wearing traditional HMD VR headsets
users could potentially enjoy content in a format
where they can physically move around
without having to wear a traditional VR headset
These aspects are combined in a way
that effectively expresses the RDW (Realistic Digital World) technology
NEWJAK's RDW applied on Dobong-ro When it comes to VR cave setups
and Redirected Walking (RDW)
we might see a convergence of various forms of content
into more expansive spaces
In fact
VR content, because of wearing goggles, has lines
so you can't move straight ahead
However, in a cave-like structure
if you continue straight ahead in XR format, you'll bump into the wall
In such cases, it would be beneficial
to consider the use of visual illusion effects to subtly redirect the user
making them turn slightly as if circling around
You could observe
a wide variety of technologies related to XR
NEWJAK's Motion Detection Illustration And in another way
interactive forms in galleries could be like
the paintings in Harry Potter that
continuously interact with the audience
The artworks could be adapted to show
the kind of art that each viewer prefers
DIGIMIX's Interactive touch wall These features can be expressed
simply through a touch wall interface
The Wonyeong Flower at the Naju Immersive Content Experience Zone Motion recognition technology is now
commonly used to create interactive content
Such as particles
or flowers that respond to movements
The Light Playground at Jungang Art Center Additionally, particle generation and
content interaction based on LiDAR sensors
could become more prevalent
Besides these, there are sensors and other foundational technologies for content
that haven't been released yet
And I think we could see much more utilization
in various industrial sectors that we haven't discussed yet
So, I've talked about
some challenging sensor-based technologies
and the immersive aspects associated with them
Considering these aspects
I suggest you to imagine
what kind of content you could create
based on these elements?
If you feel a bit lacking in imagination for that creativity
Why not try recreating existing content from movies or other references?
It could be a great way to experiment and get inspired
By doing so
You can explore and expand upon XR immersive content as content planners
It's an exciting prospect for
creating a variety of XR experiences
That's all for now
Definition and application of XR Understanding AR, VR, MR, and XR
AR Technology that creates a virtual space in the real world by combining images or videos with CG Use Medical training / Retail business / Repair and maintenance / Design and modeling / Business distribution / Tourism / Education / Entertainment etc.
VR Immersive spaces that do not exist in reality requiring special devices (HMDs) Use Entertainment(Games, movies, etc.) / Fine art and design / Architecture / Medical / Aviation / Army etc.
MR Showing a combined real world of AR and VR through HMD-based lenses
XR: Technology with the potential to expand into spatial-based content, a hybrid of AR, VR, and MR Use Manufacture / Medical / Culture / Defense field etc.
The benefits of utilizing XR Reduced waste, cost savings, improved understanding, project efficiency, enhanced safety, better decision-making
Sensor-based interaction research Types of sensors Accelerometer Motion Sensor Radar Sensor
Capacitive Touch Sensor Image Sensor Global Positioning System Gyro Sensor
The integration of emerging technologies and content XR in the architecture, engineering, and construction industries Integrate sensors and content to use in the field or environment
XR in product design, manufacturing, and automotive Electric vehicles are a comprehensive medium for sensor fusion and content, etc.
XR in Media and Entertainment VR games, chat apps, augmented reality, AR filters, virtual exhibitions, etc.
Harmony of content and new technologies in practice NEWJAK XR Laser Training NEWJAK XR Shooting Simulator
NEWJAK RDW(Redirected Walking) application on Dobong-ro NEWJAK Motion Detection Illustration DIGIMIX Interactive touch wall
The Wonyeong Flower at the Naju Immersive Content Experience Zone The Light Playground at Jungang Art Center