Realistic Immersion Advanced Edition VR project implementation 1
We'll now try to implement
grabbing and lifting objects
placed in the world space
by hand
Additionally, we will practice both lifting objects
while maintaining the grabbed position and using a snap feature
where objects attach to a specific position
on the hand when grabbed
We will also spend time implementing a feature
that applies animations of the hand opening and closing
based on the VR controller's input state
making the act of grabbing and releasing objects appear more natural
Applying Hand Animation
Let's continue from where we left off last time
When we grab or throw an object
we usually grasp it
as if we were clenching our hand to give it a more realistic feel
To create the effect of grasping
we will add a simple animation to the hand
And let's start by focusing on the right hand for now
Let's start by creating the input keys first
Right-click on Inputs and
add an Input Action to Input
and name it IA_HandTrigger_Right
Since we only need the Right Hand Trigger to detect
when it's pressed and released when grabbing
let's set it to digital input mode
We'll receive values as True or False for this input
Add another mapping
in the Input Mapping Context
and select IA_HandTrigger_Right
Let's select the Right Grip button
on the Oculus Touch for the key
This will now correspond to True or False values
Let's save it for now
Let's go ahead and add a few more keys while we're at it
To express animations
we need to remember how our index, middle, and other fingers moved, right?
So, let's add an Input Action to Input
and name it IA_IndexTrigger_Right
For the Index Trigger, we'll set the Value Type
to Axis1D (float)
so it can detect how much it's pressed
allowing animations to correspond to the degree of pressure applied
If we're going to receive it as a float anyway, let's change the name accordingly
Let's distinguish between True/False
and Float here by adding "Float" to the name
So it will be named IndexTriggerFloat_Right
Let's proceed to add an Input Action
named IA_HandTriggerFloat_Right in Input
I'll add three things
Index Trigger Float for the right hand
Hand Trigger Float for the right hand
and True/False Boolean values
Since Hand Trigger Float will also be received as a float
let's set it to Axis1D (float)
for one-dimensional coordinates
Then, since it's a Skeletal Mesh
for the right hand's Skeletal Mesh
there's an Anim Class where we can set up animations
Considering the time, we won't create a new one from scratch
We'll try using animations
that were already present in the VR template
The animations should be located under Characters here
In the Characters folder, inside MannequinsXR
there's a folder named Animations
So, here you'll find animations related to the hand movements
These animation files are used to blueprint
within the Meshes folder
where you can find
the MannequinsXR animation file
We'll use these to apply animations to our hands
So, we'll duplicate this by using Ctrl + D
Let's name this ABP_HunetHand then
I duplicated it with the name HunetHand
I created a new folder named Animations in Content
and placed the HunetHand file inside
I'll move the duplicated HunetHand file
to the Animations folder here
I'll place the ABP_HunetHand animation blueprint file
we just created
using this file into
the Animations folder
If you double-click to open the animation blueprint file
you'll see that it's already partially implemented
If you look at the values, in the preview section here
you'll see things like Pose Alpha Grasp, followed by Index Curl Point
Thumb Up, and so on
So just by looking at the name
Index Curl refers to when the index finger curls upwards
Alpha Grasp would refer to when making a fist
and Thumb Up would correspond to
when the thumb is raised
Go ahead and try changing some values here
If you increase the Pose Alpha Grasp value
what would happen?
The Pose Alpha Grasp value typically ranges from 0 to 1
influencing the positions of multiple fingers
like index, middle, and ring fingers
We can use the float value
from our right hand's HandTrigger to control this Pose Alpha Grasp value
allowing us to represent a range from 0 to 1
Below that, there's Pose Alpha Index Curl
Pose Alpha Index Curl also allows you to bend and straighten the index finger
using a value ranging from 0 to 1
So, to effectively use finger animations
we'll use Pose Alpha Grasp and Pose Alpha Index Curl values to apply animations
Let's give it a try
The rest, you can experiment with by adjusting them
based on when, how, and which inputs you want to use
to control the values
Let's focus on showing these two first
Then let's head over to the Hunet Player
where in BeginPlay
we'll store the initial animation blueprint values
So, we'll get the animation instance
from the right hand's Anim Instance
to directly pass values
to the animation blueprint
We'll cast the result of Get Anim Instance
We'll cast it to the ABP_HunetHand
that we created
So, we'll store this casted ABP_HunetHand
as a variable
Let's name this variable
RightHandAnim
We'll name the variable RightHandAnim without spaces
It's generally better to avoid spaces in variable names
so we'll name it accordingly
Now that we've stored the variables
we just need to pass values
to each of them
So, let's add an event below for this
We'll receive the IA_HandTriggerFloat_Right event
and continuously pass its values to RightHandAnim
while the event is pressed
So, we'll use Get RightHandAnim to retrieve RightHandAnim
to retrieve RightHandAnim
and then continuously pass this value
to Set Pose Alpha Grasp
We'll pass the Action Value
to Set Pose Alpha Grasp like this
Similarly, let's duplicate it with Ctrl + D
then ensure to also pass the Action Value to Set Pose Alpha Grasp
when releasing the hand at the end
To ensure it perfectly resets to 0
we should also target the RightHandAnim variable in this duplicated event
We'll do the same for the index finger as well
For IA_IndexTriggerFloat_Right
we'll also retrieve RightHandAnim
and use Set Pose Alpha Index Curl
to pass the Action Value like this
Of course, we'll duplicate it with Ctrl + D
connect it to the same target for when the action is completed
and pass the Action Value of 0
to ensure the hand fully extends
Let's compile and save the changes
Next, we need to link this event to the Input Mapping Context
as we've only created the event so far
We'll add two mappings
to the Input Mapping Context
The first one will be IA_HandTriggerFloat_Right
and we'll map it to the Right Grip Axis
on the Oculus Touch
Then, we'll add one more
and collapse any unnecessary sections for clarity
We'll add IA_IndexTriggerFloat_Right next
and we'll assign it to the Trigger Axis
on the Oculus Touch Right controller
Next, let's save the mappings
Let's test to see if everything is working correctly now
When you grab the right hand controller
and squeeze the Hand Trigger or the index trigger
you should see animations
that simulate gripping or making a fist
Now that we've simulated the gripping animation
let's create an object here
so that when we actually grasp it
the object will appear to be held in our hand
Let's place a table on the ground
so it's easier to pick up objects
then we'll place an object on top of it
Since we have StarterContent available
let's find a table from the Props section
in StarterContent
and place it here
If the table feels too small, feel free to scale it up
Let's place it nearby
so we can grab it easily
since we haven't implemented movement functionality yet
If you feel it's a bit small
locking the scale to about 1.2 times its current size
will adjust all dimensions simultaneously
Setting it to 1,2 will increase its size uniformly, making it slightly larger
Let's place an object on top of it
We'll create a blueprint for this object separately
First, let's create a new Blueprint Class
in Blueprints
Let's create it as an Actor type
Next, I'll create a blueprint file
named BP_GrabObject
Open up the file by double-clicking
Here, let me search for 'Box'
When you search for Box
there's something called Box Collision
So, you'll move this Box Collision to be the root
making it the Root Component
Next, I'll add a Static Mesh below it
And in the Static Mesh details panel
you can choose any object
that could serve as
the shape of the object
Shall we place something angular? If we place something too round
it might not fit well with the Box Collision
Let's place a statue
Since the color may not be very visible
let's compile it first
When you compile, the materials will appear, right?
Currently, it might be a glass material, so it may not be very visible
Shall we change it to a more opaque color
to make it more visible?
That should make it more noticeable, right?
After changing the color
let's adjust it to fit
somewhat within the box size like this
And compile, save. Since we need to handle collision later
for the Box - Static Mesh of this object
let's change the Collision Preset to No Collision
so that the Static Mesh itself doesn't have collision
We'll set it up so that only the box has collision applied
So, let's customize
the collision settings in the box
Let's customize it
so we can cut it as we please
For the Object Type, let's set it to World Dynamic
Next, let's change Collision Enabled
to Query and Physics
Let's set everything to block
so that it blocks everything like this
If you check the "Block" above
it will change everything uniformly to "Block"
So, we'll set it to "Block" for all channels
ensuring that
physically nothing can pass through
Next, for this object
we'll enable "Simulate Physics"
so that it can fall under gravity
and not float in the air
Turn on "Simulate Physics" for it
Then compile and save it
And now let's go back to the map
and place the Grab Object
on top of the table
The size seems quite large. Should we resize it smaller?
Reducing it to about half its size should be fine
Reducing it to 0.5 times its current size will still make it quite large
so let's press the End key on the keyboard
to snap it neatly to the floor
Just to be safe
let's place one more next to it like this
You know you can hold down the Alt key and drag to duplicate, right?
So, I'll place these two
on the table like this
And for the Table Actor
since its collision settings are probably default
I'll customize it to set the Object Type
to World Static
and leave it as it is
For the moving objects here
set them to World Dynamic
and for the fixed table, set it to World Static
So, to determine when
the player has grabbed an object
we'll add an event here
IA_HandTrigger will be for functionality, not animation
so we'll use True and False values
So, we'll create an Enhanced Input Action for IA_HandTrigger_Right
using True and False values
Next, we'll check
when the button is pressed
We'll check when the button is pressed (Started)
to see what objects are around our hand
at that moment
In this case, we'll use an Overlap check
Since we're checking around the hand, we can use a Sphere Overlap
We'll search
for Sphere Overlap Actors
This function searches
within a specified radius, measured in centimeters
from the position you define, within a single frame
So, what will the reference point be?
The reference point will be the position of the right hand
You can use either the Right Motion Controller
or the Right Hand for this
Most likely, they should be nearly identical in position
because the location value for the hand is
relative with coordinates set to 0, 0, 0
it aligns with the parent's position
We'll use the Right Motion Controller as the reference point for the parent
We're going to use
Get World Location
for the Right Motion Controller
to get its location in the world
So, w're going to use the World Location
to search around the hand
with a radius of about 15cm
15cm is quite a reasonable radius for detecting objects around the hand
Given the slight differences between VR and the real world
a bit of extra radius like 15cm should indeed help ensure
comfortable interaction without feeling too restrictive
I'll search around the hand
within a radius of about 15cm
Then we need to specify
which actors to search for by checking the Object Type
So, since it's in array format
let's use 'Make Array' to create an array
If you connect the array values to Object Type
you'll be able to choose the Object Type
Here, for the object like the Statue we intended to grab earlier
it was set to World Dynamic
So we'll check only for the World Dynamic type here
Next, for Ignore Actors
actors to ignore in collision checks
we shouldn't check against ourselves
So similarly, we will create an Array
and pass the Array values here
The actor that shouldn't collide is itself
So, we'll get a reference to self
put information about itself into the array
to not collide itself
I will register it as an Ignore Actor
The Dynamic actors might not be limited to
just the Grab Object we just created
So just in case, we'll use a class filter
to ensure only BP_GrabObject is checked
Even if they are dynamic
we'll only check those that are of the BP_GrabObject type
Then, anyway, the result will be multiple arrays
We will take the first one among them
So, we'll use Get to fetch the 0th element
which is the first one that was touched in the array
For the retrieved entity, physics are currently enabled, right?
So, gravity is applied
and when we grab manually
the object falls down due to gravity, making it hard to hold onto
We'll need to disable physics first
so casting should be done beforehand
First, cast to BP_GrabObject
We'll set the Simulate Physics of the Box Component
from the object to false
turning it off
If it's checked it means it's turned on (True)
and if it's unchecked, it means it's turned off (False)
We'll turn off physics first
then attach it to my hand's position
We'll attach it to the Right Hand
So, we'll use Attach To Component for this object
and our hand is the Component
Since it's a Skeletal Mesh Component
we'll attach it to the Component
To organize it neatly
I'll create a reroute node and double-click to make the connection look tidy like this
When attaching, there's a rule involved
This rule includes three options: Keep Relative
Keep World, and Snap to Target
The "Snap to Target" option means that when grabbed
it moves to exactly match the position of my hand
aligning precisely with it
"Keep World" or "Keep Relative" means that when grabbed, it maintains its position relative to the world
or relative to its parent
while being attached
at the moment it's grabbed
Since it's a bit tricky to explain verbally
I'll set all of these to
"Keep World" for now
Next, we'll attach it
to my hand
So, we'll fetch the Right Hand
and set it as the Parent for the attachment
So, I'll compile and save to attach it to the hand
Then let's play again to see how it works
It seems to be located below
Anyway, what happens when you grab it like this?
When you grab it now
it should attach precisely to your hand
at the position where you grabbed it
You've confirmed that when you grab it, it attaches to your hand like this
although you haven't made the release action yet
Implementing object grabbing functionality
But what happens to us when it attaches like this?
It might feel a bit awkward, right?
If we want to specify a precise grab point
and have it attach exactly there
we should change the Location Rule
from Keep
to Snap to Target
instead of Keep World
Rotation should also be set to Snap to Target
There's no need to change Scale
There's no need to adjust it to fit the size of the hand, just the statue
Let's ignore scaling
and just align the position and rotation according to the hand's direction
I'll go ahead and compile and save these changes
Let's test it again to see how it behaves now
I've grabbed it Now, when I grab it
does it attach exactly to the position of my hand like this?
But it seems like the position isn't quite what we wanted
So, now we can use a Socket
as the Grab Point
If you look at the viewport, I'll show you the Right Hand
If you look at the Right Hand
you'll see that currently the pivot is around the wrist area
So, when the statue snapped to attach
it ended up attaching at this wrist position where the pivot is currently located
So, earlier it looked a bit odd because of that
So, you would typically want it to attach
around the palm area instead
To align it around the palm area
you can specify the attachment point on the palm using something called a Socket
To go to the location
where the original file SKM_MannyXR_right for Right Hand is located
If you click on the folder icon below
you'll see the original location where it's located
It's located in the Characters - Meshes folder
Please double-click to open this file
Please click on the skeletal icon at the top
the skull icon
So
right-click
below the root and
add a Socket
to create a new socket there
In the Details panel
I'll name the socket "GrabPoint" and position it where we want it to be
Now, with the GrabPoint selected
I'll move it to align around the center of the hand, like this
This looks about right
If it's centered on the hand and facing upwards in the front direction
the statue should be grabbed like this
So, to match the shape of the hand
we need to rotate it so that the front direction
points towards the front of the hand
If you feel confused at any point
you can attach a preview here to help visualize it beforehand
You can add a preview asset here
by clicking on "Add Preview Asset"
and attaching the statue here as a mock-up
The size might be a bit large
but you can adjust it to match the shape and orientation of the hand
So, with this orientation
we should adjust the Grab Point slightly downward, right?
We should lower it slightly
so that it aligns like this when grabbed
considering that we actually reduced the size by half (0.5 scale)
and adjusted accordingly
I'll set the socket position
like this for attachment
save the rotation and position
and copy the name "Grab Point" with Ctrl + C for the socket
To make it easier
let's go back to the Blueprint settings
and in the Event Graph
use Attach Actor To Component again
but this time specify the socket name directly
So, we'll specify the socket name as "GrabPoint" when attaching it
directly telling it to attach there
Attach to the Grab Point socket on the Right Hand
using Snap for both location and rotation
We won't apply scaling
We'll maintain the scale
as it originally was for the object
I'll compile, save, and then let's test grabbing it again
How does it feel picking it up again now?
Now it has become like this, fitting to my hand
But it's a bit too high up, right?
Because we lowered the statue a bit
I think it would fit better if we raise it a bit
So earlier, we lowered it from the hand socket
but I'll try raising it a bit
Shall we raise it about this much?
Then, how about holding it like this again?
Now it feels like we're really holding the statue
Implementing the object placement function
Now that we've grasped it, let's try placing it down once
It's very easy to place it down
When is the right time to put it down?
When we've completed it, that's when we'll put it down, right?
Use "Detach From Actor" to
If you use "Detach From Actor,"
you'll be able to detach what's currently attached
When you detach something using "Detach From Actor,"
you can detach what's currently attached
However, if you haven't saved or stored what you're currently holding
you won't be able to detach it because the system doesn't remember what you were holding at that moment
So, when you attach something here
you'll save this attached actor to a variable
You'll create a variable
and name it something like "Current Grabbed Actor,"
of type "BP Grab Object,"
to store the actor you're currently holding
And you'll select the actual object reference
the Grab Object, for its Object Reference
Compile and save to ensure the variable is compiled properly
Then you'll set this variable
When you set it
you'll tidy up this Grabbed Actor
a bit beforehand
We're going to save
the currently attached actor
into the Current Grabbed Actor variable
Then, when you grab an object
the actual object you're holding will be stored in this variable, right?
So, when you release it
you'll retrieve it from Current Grabbed Actor and detach that actor
When releasing it
you'll detach it from its current position
keeping it in the world coordinate system (Keep World)
so it drops from its current position without changing its original world coordinates
If you just detach it without any additional actions
it will remain suspended in the air
Then you'll turn on physics again
You'll set "Simulate Physics" on the box
to turn physics simulation back on
Compile and then save
Should we test it out again?
Press play, grab the object
and now when you release it, it should drop like this
Now you can grab and throw objects like this
So, once you know how to grab
releasing is very easy
Handling exceptions
So, leaving it like this seems too complex
so I'll refactor this into a function
So, let's add a function
I'll create a function named GrabAction
Next, I'll create a function named GrabAction
and then I'll cut out all the actions we did
when we started from the Event Graph with Ctrl + X
Next, I'll paste everything we cut with Ctrl + X
into the GrabAction function
and connect it accordingly
Then, return to the Event Graph
and connect the GrabAction to the "Started" event
Similarly, I'll create a function for releasing as well
I'll create a function named ReleaseAction for releasing
We had actions for
when we completed the release in the Event Graph
I'll cut those actions from the Event Graph with Ctrl + X
paste them into the ReleaseAction function with Ctrl + V
and connect the execution pin accordingly
Then, return to the Event Graph
and connect ReleaseAction
to the "Completed" event
Now it should be much cleaner and organized
And now each functionality is neatly organized in its respective functions
making it easier to review
and understand each part
Let's wrap it up here for now then
Let's work on
adding movement functionality next time
Thank you for your hard work and patience throughout this long session
I'll summarize what we've learned in this session
First, to attach an object to a Hand Component
you need to use the "Attach Actor To Component" function node
It's important to note that
this can only be done
when the "Simulate Physics" feature of the object you're attaching is set to false
Secondly, when searching for objects that can be grabbed around the hand in VR
it's important to note that
the visual information in VR may not perfectly match the actual sense of distance for a person
Therefore
you should define
the detection area slightly wider than
the actual size of the hand
Otherwise, users might feel uncomfortable when trying to grab objects
as the virtual representation may not align perfectly with their real-world perception of reach and distance
Lastly, the third point we covered is
using the Axis values of buttons like the Index Trigger or Hand Trigger
to smoothly animate the fingers of the hand mesh in response to button inputs
This method simplifies handling
and was demonstrated
through
practical exercises
Applying Hand Animations Preparing Input Functionality
Create the IA_HandTrigger_Right file in the Inputs folder for handling object grabbing inputs Set the Value type to Digital (Bool) for the IA_HandTrigger_Right file
Inputs received for passing to the Hand Animation Blueprint Create two files: IA_HandTriggerFloat_Right and IA_IndexTriggerFloat_Right
Set the Value type to Axis1D (Float) to receive values corresponding to the degree of button press
Implement functionality to pass input values to the hand animation Double-click on the ABP_HunetHand file to open its settings window
In the Animation Settings window of ABP_HunetHand, navigate to the Anim Preview Editor tab located in the bottom left corner
Find the Default variables and adjust the Pose Alpha Grasp value to any random value between 0 and 1. Then, observe the animation to see it in motion
Adjust the Pose Alpha Index Curl value to any random value between 0 and 1 to observe the movement of the index finger in the animation
Implementing the object grabbing functionality Creating the Grab Object
In the Content Drawer, drag the SM_TableRound file from StarterContent - Props folder onto the viewport to place it
Adjust the Scale value to 1.2 times to make it a comfortable height for picking up objects
Apply gravity to the object so that it naturally falls downward when placed
Select the Box component in the left Component panel Check (True) the Simulate Physics property in the right Details panel
Implement the Grab function Keep World method, Snap to Target method
Implement the drop object functionality Implement the release function
Drag the Completed execution pin of the IA_HandTrigger_Right input event node
and connect it to the Detach From Actor node to detach the actor from its parent component
Specify the Current Grabbed Actor variable, which was stored when grabbing the object, to detach
Set Location Rule, Rotation Rule, and Scale Rule all to Keep World
Call the Simulate Physics node on the Box component of the detached actor and check the Simulate property
Since the object has been released, clear the value of the Current Grabbed Actor variable that was storing the grabbed object
Play the game and confirm that when releasing the right hand trigger, the object drops and flies away
Exception handling Organizing Blueprint Node Declare functions named 'GrabAction' and 'ReleaseAction'
Move the grab function nodes connected to the Started execution pin of the IA_HandTrigger_Right node into the implementation of the Grab Action function node
Similarly, move the release function nodes connected to the Completed execution pin into the implementation of the Release Action function node
Handle preventing duplicate grabs
To prevent grabbing overlapping objects, connect an Is Valid node in the Grab Action function node to execute detection
only when the Current Grabbed Actor variable is initially empty
Conversely, connect an Is Valid node at the beginning of the Release Action function implementation to execute the release function
only when there is a held object (Current Grabbed Actor)