Let's learn about VR lenses
which operate on different principles
from monitor displays
and how to use Widget Components
to convert screen-based UI
into World UI format
Next, let's implement the effect of clicking buttons on a World UI Widget
using the VR controller
like a remote control
Additionally, let's learn how to apply curved UI
to address visibility issues
caused by the wide
and free viewing angle unique to VR
Create a widget
Hello
today we will be creating
a VR widget
If we've previously created widgets while making other content
many of you might be wondering
what will be different
when creating widgets for VR
Generally, when we create widgets
we use an overlay method
to draw the widget on top of the World
It was drawn to fit the size of the screen
but in the case of VR, it's not a fixed monitor
When we move our head
the camera moves
so drawing on top of it can result in the widget not appearing as intended on the screen
Therefore, in VR
we use World Widgets and different widget pointer interactions to manipulate buttons
which involves some additional steps
Also, because our field of view is wider in VR
there may be UI elements
that are not easily visible within our field of view
so we need to represent the widgets in a slightly curved display format
rather than a flat plane
So, despite several somewhat cumbersome procedures
in VR, we have to consider that due to the nature of VR device mechanisms
we have no choice
but to draw in that way
I'll explain the principle of drawing UI on VR devices
while showing you from behind
Alright, continuing from last time
this session we'll create a widget together
Let's start by creating a basic widget then
Go to the Contents folder
then click Add → New Folder to create a new folder
Name the folder 'UI'
Inside this UI folder
right-click and go to User Interface → Widget Blueprint
to add a new Widget Blueprint
Select User Widget as the parent class
So, I'll create a widget file
named WBP_MainWidget
Double-click to open the settings window
First and foremost
we'll need a canvas, right?
So, we'll add a canvas panel
And then we'll add a text block
to this canvas panel
Add a text block to the canvas panel
Set the anchor of this text block to be centered
Setting the alignment to (0.5, 0.5)
centers the pivot
so placing it at (0, 0)
will position it in the center, like this
I'll increase the size a bit
Just adjust it to fit the font size, right?
I'll make the font stand out more by setting it to about 40%
The content of the first text
will be named
"Select Move Type"
Set the font justification to center
so that it aligns centrally
I'll express the text in a noticeable yellow color
to make it stand out
Then let's set the size to around 500 width
and 80 to 70 height
So, set it to around 500 width and 70 height
Place it slightly above like this from the top
Set X to 0 and Y
to approximately -350 for positioning
And below that, we'll place the buttons
We'll place one button on the canvas
We'll add a text inside this button
I'll add a text block like this as a child of the button
inside its hierarchy
The first text will be named "Normal Move"
The font size will be the same, 40
The color will be set to a yellow shade like this
The alignment will be set to center alignment like this
Next, the button's name
will be "Button_NormalMove"
So, check the "Is Variable" option for it
It's best to adjust the button's size to fit the text perfectly
Set the anchor to center first
then a size of approximately 400 width and 70 height should be sufficient
I'll set the background color
to a slightly reddish hue like this
Let's place this button here
Set its alignment to 0.5, 0.5 for center alignment
with X at 0 and Y
at around -150
If the button seems a bit too low, let's reduce its height a bit more, shall we?
Let's leave it like this for now and proceed
Since there might be more buttons
let's duplicate it by pressing Ctrl+D
Then, set the position with X at 0
and Y around 0
to place it approximately in the center
So, let's name the text
inside it "Teleport Move"
I'll add another button
to display the text "Teleport Move"
Next
the function of this button will be
when you select this button
it will use the Normal Move function
we created last time
for movement
If you select "Teleport Move"
the movement method will change to teleportation mode
implementing this functionality in the UI
First, let's compile and save
To implement this functionality
we need to access the movement method
from the HunetVRPlayer phone
This involves accessing the movement method from the HunetVRPlayer phone
where we had teleportation and normal movement methods
under Thumbstick Axis Right
We'll add another boolean variable
to branch out using Branch and choose between the two methods
teleportation and normal movement
under Thumbstick Axis Right
Press the "+" button
under VARIABLES to add a new variable
I'll create a variable
named "Use Teleport"
I'll change the data type to Boolean
so it can accept
True/False values
Let me compile it briefly
and set the default value to False
I'll set the initial state to False
Bring in "Use Teleport"
to create a branch
based on it
Use the Branch node
to execute the Draw Teleport Line function when Triggered
if Use Teleport is True
If Use Teleport is False
execute Normal Move in this manner
Next
connect the Action Value to here
Then, similarly
when it's completed
add another Branch node here because it needs to be done again
So, when it's completed
set the condition to "Use Teleport"
If it's true in the Branch node when completed
perform the Teleport Action
Next, duplicate it by pressing Ctrl+D
and if it is False, set it to perform a Normal Move
Then, I'll pass the Input Value
here as well
I will use line sorting
I will adjust the position to achieve this look
Compile, Save
Then, the actual movement will change according to this value, right?
Let's test it first
For now, it's in a False state
In the False state, when you press Play
you can see that it moves forward and backward
with the basic Normal Move
right?
However
I will change the variable value of Use Teleport
from False to True
then compile and save it
before testing once more
Then, when moving
you'll see the Teleport line drawn
confirming that it moves in the direction of Teleport
right?
This does not have exception handling for grabbing
but while we're at it, you confirmed that this works well
While we're at it
I'll try adding exception handling for grabbing when it's not handled yet
When performing Grab Action or Release Action
you should only grab the object when there is no actor currently grabbed
So, here before the Grab Action
you'll first use the Is Valid function on CurrentGrabbedActor
to check
whether there is a value
or not
When the value is not present
using Is Not Valid
let's neatly organize it to take action to grab the object
On the other hand, when releasing
When releasing with Release Action
you should detach
only when there is a Grabbed Actor
present
Add an Is Valid node to
so that Detach is performed
only when there is something being held (Grabbed Actor)
and Is Valid is true
So, now even if there's no object
and you attach and detach, there won't be any errors
Therefore, recognizing the importance of such exception handling
is crucial
Alright, let's switch back to the Widget
and return to the Widget Blueprint
We already know that
depending on whether the player's variable Use Teleport
is True or False
the movement method changes accordingly
For creating
a World-type UI
we'll have the player
create this Widget
Let's switch to the Event Graph tab
and turn off other function nodes like this
When we start
at the BeginPlay event
we'll create a Widget here
Here, from the execution pin of the SET node
drag to create a Widget
and make one
The Widget we're creating
will be named WBP_MainWidget
Just to be safe
I'll store this value
as a variable using "Promote to variable"
So, I'll name this variable
MainWidget_instance
Since it's already created, it's an instance now
You can see that the variable has been created here
Then, we need to implement the functionality of the buttons
so we'll pass the owner
which is the VR player
to this Widget
So
let's go to the MainWidget
click on the Graph tab
and switch to the Graph tab
Here, in the My Blueprint panel
under VARIABLES, click the + button
to create a new variable named Player
And the data type of this variable
will be BP_HunetVRPlayer
Let's set it to be of
type Object Reference for VRPlayer
Compile and save it
Now, let's go back to the VRPlayer side
When we initially create this Widget
we have the Player variable here
Set Player
Here, we'll pass in self (or "this") as the Player variable
Self
We'll get a reference to self
and then pass that reference here
So, the Widget will now have
the information about its owner
the Player, once it's created
So, to see
how the Widget appears
let's check this value
in the MainWidget Instance
In the MainWidget Instance
let's use Add to Viewport to display it
following the method
we've used before
with Add to Viewport
I'll add a Reroot node here to tidy up the lines
as they're getting a bit complex
Should we do it like this?
Let's add another Reroot node here
to make it more organized and easier to view like this
Then, when we initially start
the Widget should be displayed like this
When you press Play, the Widget should be displayed
but currently, it's not showing up
even though the Widget should be visible on the screen
I'll explain why the Widget isn't displaying
and the principles behind it
This is explaining
the traditional method of UI display
Usually, we have a camera in the world
that captures scenes either through a Projection or Orthographic method
It captures everything
within its view
and converts it into an actual image
That image is then rendered onto the actual screen, or monitor
following this principle
After capturing the image from the world as described
we first render that world image onto the screen
Then, we overlay the UI image
on top of it
So, in essence
the later drawing (UI image) will be rendered on top of the earlier drawing (world image)
effectively overlaying it
This was the traditional method of UI rendering
However, in the case of VR
the method of rendering the screen is somewhat different
So, here you can see it's labeled Left Eye, Right Eye
In VR, there are separate cameras capturing images
for the left and right eyes
on the display
The two captured images
are then displayed
within the VR device
where each eye corresponds to a lens section
There are separate lenses for the left and right eyes in VR devices
Those lenses distort the slightly rectangular screen
to give a sense of depth by slightly warping the image for each eye
They pass through fisheye lenses
that create a sense of depth by providing a stereoscopic effect, allowing for a sense of distance
However, because the left and right eye lenses
output different images
we effectively see the same screen
duplicated for both eyes
With each lens angled slightly, around 5 degrees
both lenses are projecting the current screen onto the VR display
Therefore, if we were to draw the UI on the screen as usual
actually
since it's not drawn on the Screen World
but on the entire area
the UI ends up
not being displayed
at all
So, even though we used Add to Viewport
it actually wouldn't be drawn on the screen
So, how should we go about solving this issue?
In VR, you can't draw UI on the screen
using the traditional viewport method like this
You need to go with the method of creating
and displaying UI objects in the world
Then let's practice that once
Then I'll first come over to the HunetVRPlayer
So, for the existing part
where you use Create Widget to insert
I'll select all of it and press the Delete key to remove it
Now, this type of screen-based widget
cannot be used in VR
So here in HunetVRPlayer
if you press the Add button
and select Widget
there is a component called Widget
Let's rename it to Main Widget
and add the component here
Next, if you go to the Details panel of this Main Widget
you'll find a User Interface tab
at the bottom
Here, in the drawing space options between World and Screen
we need to select World for our purposes
So, let's choose to draw in the World space then
Next, we'll create a Widget class
that will generate
our previously made
WBP_MainWidget class
Here, I see there's something called "Draw Size"
If you set it up like this
you'll see the widget floating in front of the player
like this when you play
But I think we had something like
Selection Move Type
up here
and right now it looks a bit cropped or cut off
If I were to reduce the Draw Size
If you set it to 100 by 500, it would look like this
If it's set to 100, the horizontal space might be too narrow, causing it to get cut off
So, we should adjust it
according to the Canvas Size
Since we don't know the current Canvas Size
let's go to WBP_MainWidget first
and adjust the canvas in the Designer panel
as it might be
unnecessarily large
So, let's adjust the Fill Screen option here
to make it visible up to this extent
We'll change the Fill Screen option
labeled Custom
to actually specify the size we want
The canvas is currently too small, right?
If it's 100x100, it's too small
So, horizontally, if we set it to about 500
vertically to 600, it should fit here
Set it to about 600 in height as well
600 should be sufficient
Let's create it in a square shape with dimensions of 600x600
I'll just bring the position down a bit
Let's start by adjusting the Button Move position
I'll move Position X and
Position Y down by about 200
I'll just bring it down by 150 instead, since 200 seems a bit too much
Next, I'll set the Y value of Normal Move to 0
Select Move Type will be around -200
With these settings, it should arrange the layout in this kind of arrangement
So, let's compile and save it once
Let's go back to HunetVRPlayer
and this time, we'll adjust the Draw Size of MainWidget
to match the earlier Canvas Size of
600x600
Currently, the screen is being viewed
from the camera's perspective
so it would end up showing the back of the widget
So, if we move the widget a bit forward
and rotate it 180 degrees
it should appear as if it's right in front of our eyes
when viewed from the camera's perspective
So, for the location adjustment
let's position it about 4 meters forward
and rotate it 180 degrees on the Z-axis
to ensure it appears correctly when outputted like this
Let's compile and save to see how it looks
It seems a bit too large
and also a bit sunken down
When converting pixel sizes to world sizes
sometimes the size can become too large like this
So, we should either reduce the overall size a bit
or lift it up slightly
either one should help
Let's try lifting it up first
We'll lift it up a bit first
and then
compile and save
And now, when you look again
the widget should appear
more like an actor
in the world, right?
But since there's
no mouse cursor in VR
you might be unsure how to press the button
If you're using controllers, you might be unsure how to press the button
I'll help you implement the button's functionality
and figure out
how to press it
Widget Interaction
First, let's implement the functionality of the button
Select the Normal Move button
then click on the "+" button
next to On Clicked to add an event
Then, it will switch to the Graph panel, and you'll see a new event created
Here, you'll first check if the player has been properly retrieved
Is Valid
First, check if the player exists
If they do
set the Use Teleport value
on this player
If it's Valid and it's Normal Move
then it should be False, correct?
So, you'll set it to False in the False state
Next, go back to the Designer panel
select Teleport Move
and then click the "+" button
next to On Clicked
Since it's the same
I'll duplicate the one above with Ctrl+D
After duplicating
connect it to Is Valid
and in this case, Use Teleport should be True
So, when clicking the Normal button
Teleport should be set to False
The name is set wrong
I'll go ahead and change the name as needed
I'll rename it to Button_TeleportMove
instead of
Normal Move_1
I'll change the name of the button
It looks like we didn't change the button name
After that, when you come back to the graph
you'll see that the name has been updated
In the Designer panel
click the "+" button
next to On Clicked for the updated Teleport Move variable
and then connect this event to Valid like this
So, pressing the Teleport Move button will set
Use Teleport to True
and pressing the Normal Move button will set
Teleport to False
and the functionality has been implemented correctly
Then, we can test it out by
simply clicking to see if everything works as intended
We'll use the right hand's Index Trigger
to simulate a click and test it out
And we'll implement it to click like a remote control
Let's give it a try
First
right-click in the Inputs section
and add
a new Input Action
I will create a button
named
IndexTrigger_Right
The value type for this
should be set to True/False
Pressed/Not Pressed
Then
open the Mapping Context
We need to add one more item to the Mapping Context
Add another mapping
and select
IA_IndexTrigger_Right Right for it
For the key
instead of Oculus Touch (R) Trigger Axis
assign the
Trigger True/False button
Save it again
Go back to the HunetVRPlayer pawn
and in the Event Graph
add another Event button Event
Right-click at the very bottom
and select IA_IndexTrigger_Right
IndexTrigger_Right
by right-clicking them
So here, when it starts
give it a click effect, and when it completes
give it a click to release effect, right?
So, we have a way to give a click effect
remotely like a remote control
Here in Components
if you search for "Widget"
you'll find
a component
related to Widget
and Interaction
Add this component
I'll name it
RightWidgetInteractionComp
Even though the name is long
I'll attach RightWidgetInteractionComp
as a child component
of the Right Hand Motion Controller
under the Motion Controller category
Then, if you look at the Viewport
currently there's no indication on the right hand
but if you check "Show Debug" in the Details panel of WidgetInteractionComp
a red line will suddenly appear
on the right hand
It seems you can't see it here right now
but let's compile and save first
Then, we can check it outside
Then
why is the right hand's motion controller
so far apart from the hand?
Why is the hand so far apart like this?
You might see it drawn like this in the upper direction
Right now, the position of RightWidgetInteractionComp
is probably not at 0, 0, 0
So, I'll adjust it to 0, 0, 0 like this
It should now match the position of the Motion Controller
So, placing it like this means
the hand's orientation doesn't quite match right now
So, I'll try moving it as a child of the hand
I'll move WidgetInteractionComp
as a child of the hand
and set all rotations to 0, 0, 0 to make sure there's no rotation
If you place it at 0, 0, 0 for now
WidgetInteractionComp will be facing upwards
with the red line
So, since it shouldn't be facing upwards
I'll bring it down a bit like this
The red line will point like this now, right?
I'll rotate it like this
so that WidgetInteractionComp's direction
matches the hand's direction
Since firing from the wrist seems odd
I'll move it forward to draw it
around the middle of the hand
near the middle finger
When you press Play
now it displays correctly aligned with the right hand
and when you get close to the button
there's an effect indicating you've touched the button
When you touch it
what we need to do
next is
set it up
so that the click effect in WidgetInteractionComp
triggers when you press the button
So
when we press
the IndexTrigger_Right button
during the Started event
we'll use
Press Pointer Key
in WidgetInteractionComp
to trigger
the click effect
So, during Press Started
we'll add the press effect
when the button is pressed
During this
the key effect will apply to the left mouse button
When we usually click a button
we typically use the Left Mouse Button
You can either find it directly here
or like before
once you select it here
click again with the mouse, and it will automatically change to Left Mouse Button
When releasing
we should also check that the button has been released
So, during Completed
we'll add the effect of releasing the pointer key
to indicate that the button has been released
So, I'll assign
Left Mouse Button here
and this function will be structured
to execute
when Completed
Alright, so the last thing we need to do is
to associate the Widget
with the Hunet Player pawn
that we created earlier
You've already created the variable for the player
We'll pass that variable here
First, we'll get
the Widget in MainWidget
then cast
this Widget to WBP_MainWidget
Then
in the Set Player variable here
we'll pass the Self value
to the Player variable
So, we need to pass the Player
to the Widget
so that MainWidget can receive the value
from the Player variable
and decide whether to teleport or change the value
So, let's go back
and try playing it again
Then
now we can point and interact with the button like this
So, for example
if I'm currently set to teleport
clicking Normal would allow me
to move forward and backward like this
If you select Normal, it switches to normal movement mode
and if you press Teleport
it switches to teleport mode, right?
If you press Normal again, it returns to this mode
If you press Teleport, it switches back to this mode
You can confirm that it's working well
In VR
using World Space UI allows us to effectively utilize widgets
enabling interaction with the environment through UI elements
In this world-space widget setup
unlike screen-space widgets
that can only be drawn on a plane
the monitor shape is
flat
But this HunetVRPlayer
has come to the MainWidget Componen
In world-space mode
there's a geometry mode
below
If it's labeled as Plane
it means it draws on a flat surface like the traditional screen
If you change this to Cylinder
let me take a look at it in the Viewport
How does it look now?
It looks slightly curved, doesn't it?
Here in Cylinder
there's something called Cylinder Arc Angle
If you decrease the Cylinder Arc Angle
to 1, it becomes completely flat
and if you set it to 180, it becomes fully curved
For example, what we've created is
long vertically, so it doesn't matter
but if it's too long horizontally
we might exceed the field of view when interacting with buttons
In VR, it's common to adjust widgets like this
slightly curved
to fit the field of view better
to avoid issues where text
or button images aren't clearly visible
Yes, in other VR UI designs
you may see panels stretched horizontally and curved
to better fit the user's field of view and make interactions more accessible
We can use these features
to easily curve widgets
and display them on the screen for better visibility and usability in VR
So, how does this look?
It looks slightly more curved, giving it a sense of depth
it remains visible
from my perspective
even if I turn my head a bit
Of course, this is only possible when placed in the world space
Then, one more thing to note is that
the size of the widget is
likely in pixels
The values here, like 600
in position and size
are in screen coordinates, which means they are in pixel units
If you draw this in world space like MainWidget earlier
those coordinates will be converted to centimeters
in the world space
So, 600cm x 600cm would translate to a panel
that's 6m x 6m in size
It's quite big, right? So, as you saw earlier
the widget looks really grand, doesn't it?
So when displaying the UI in a world method like this
the coordinate system changes to centimeters
instead of pixels because it's in centimeters
I think it would be good to keep in mind that
it needs to be scaled down a bit more
Then, let's wrap it up here for today
Thank you for your effort
in listening to the long lecture
Let's summarize what we learned in this session
First, due to the particular characteristics of the fisheye lens in VR
it is important to note that
traditional screen-based overlays are not possible
and thus the UI must be converted to a world-based method
Second, in VR, the user's gaze is free
providing a much wider field of view compared to a monitor
Because of this, as the lateral limits extend
the contents of the UI at the edges
may not be easily visible
so a curved method of bending the widget
into a curved shape is used
Third, by using the Press Pointer Key node provided by Unreal Engine
you can ensure that
the click effect of keyboard or mouse keys
is applied identically
when pressing buttons on VR controllers
Creating Widgets: Using World-Based UI in VR In VR, traditional screen-based overlays are not possible due to the unique nature of VR lenses. Therefore, it is necessary to use world-based UI approaches when creating widgets
Creating a UI Folder in Content Drawer Select [+Add] -> [User Interface] -> [Widget Blueprint] to create a Widget Blueprint
Open the Widget Blueprint Settings and add a Canvas Panel and a Text component While selecting the Text component, choose "Center" for Anchors in the Details panel
Set the Alignment of the Pivot to the center of the component (0.5, 0.5) Set the Position and Size, then enter Text, choose the Font, Size, and Alignment
Add a Button component named "Button_NormalMove" Add a Text component as a child component of the Button_NormalMove component
Add a Button component named "Button_TeleportMove" and add a Text component as a child component of it
Creating World-Type UI Rendering Methods in VR and Fisheye Lenses
Change the Canvas setting at the top of the Widget Designer panel of WBP_MainWidget from [Fill Screen] to [Custom]
Set Width and Height to 600 each
Setting up Widget components Creating a Widget Component by selecting [+Add] -> [Widget] in the Components panel
In the Details panel, set the Space item to World and assign the WBP_MainWidget file created earlier to the Widget Class item
Declare a boolean variable named "Use Teleport"
Connect the Triggered execution pin of the node: if the value of "Use Teleport" is True, execute the [Draw Teleport] node; if False, execute the [Normal Move] node
Connect the Completed execution pin of the node: if the value of "Use Teleport" is True, execute the [Teleport Action] node; if False, execute the [Normal Move] node
Implementing Widget Interaction Adding Widget Interaction Component
Add a WidgetInteraction component as a child component of the RightHand component by selecting [+Add] -> [WidgetInteraction], and then set its name
While selecting the Widget Interaction component, check (set to True) the Show Debug option in the right-side Details panel
Set the Rotation values to (-90, 0, 90) so that the debug lines align with the direction of the hand
Create a new Input Action file and set the Value Type to Digital (bool)
Double-click on the file to open the settings window, then add IA_IndexTrigger_Right key to the Mappings array
Setting up Curved UI Adjusting widget shapes in a curved manner to solve the issue of outer UI content not being clearly visible due to wide field of view
Change the Geometry Mode to [Cylinder] in the Details panel of the MainWidget component
The End