erwan/phd-thesis
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Replace "immersive AR" with "AR headset"

Browse Source
This commit is contained in:
Erwan Normand
2025-04-11 22:51:10 +02:00
parent f1cf425e7c
commit f8ec931cd6
22 changed files with 94 additions and 101 deletions
Download Patch File Download Diff File Expand all files Collapse all files

19
5-conclusion/conclusion.tex
View File

@@ -7,17 +7,17 @@ We conclude this thesis manuscript by summarizing our contributions and the main
\section{Summary}
In this manuscript, we showed how wearable haptics can improve direct hand interaction in immersive \AR. % by augmenting the perception of the real and manipulation of the virtual.
In this manuscript, we showed how \OST-\AR headsets and wearable haptics can improve direct hand interaction with virtual and augmented objects. % by augmenting the perception of the real and manipulation of the virtual.
Wearable haptics can provide rich tactile feedback on virtual objects and augment the perception of real objects, both directly touched by the hand, while preserving freedom of movement and interaction with the \RE.
However, their integration with \AR is still in its infancy and presents many design, technical and human challenges.
We have structured this thesis around two research axes: \textbf{(I) modifying the visuo-haptic texture perception of real surfaces} and \textbf{(II) improving the manipulation of virtual objects}.
\noindentskip In \partref{perception}, we focused on the perception of wearable and immersive virtual textures that augment real surfaces.
\noindentskip In \partref{perception}, we focused on the perception of wearable virtual textures that augment real surfaces.
Texture is a fundamental property of an object, perceived equally by sight and touch.
It is also one of the most studied haptic augmentations, but has not yet been integrated into \AR or \VR.
We \textbf{(1) proposed a wearable visuo-haptic texture augmentation system}, \textbf{(2)} evaluated how the perception of haptic texture augmentations is \textbf{affected by the visual feedback of the virtual hand} and the environment (real, augmented, or virtual), and \textbf{(3)} investigated the \textbf{perception of co-localized visuo-haptic texture augmentations}.
In \chapref{vhar_system}, we presented a system for \textbf{augmenting any real surface} with virtual \textbf{roughness textures with visuo-haptic feedback} using an immersive \AR headset and a wearable vibrotactile device worn on the middle phalanx of the finger.
In \chapref{vhar_system}, we presented a system for \textbf{augmenting any real surface} with virtual \textbf{roughness textures with visuo-haptic feedback} using an \OST-\AR headset and a wearable vibrotactile device worn on the middle phalanx of the finger.
It allows \textbf{free visual and touch exploration} of the textures as if they were real, allowing the user to view them from different angles and touch them with the bare finger without constraints on hand movement.
The user studies in the next two chapters were based on this system.
@@ -30,7 +30,7 @@ In \chapref{vhar_textures}, we investigated the perception of co-localized visua
We transposed the \textbf{data-driven visuo-haptic textures} from the \HaTT database to the system presented in \chapref{vhar_system} and conducted a user study with 20 participants to rate the coherence, realism, and perceived roughness of the combination of nine visuo-haptic texture pairs.
Participants integrated roughness sensations from both visual and haptic modalities well, with \textbf{haptics dominating perception}, and consistently identified and matched \textbf{clusters of visual and haptic textures with similar perceived roughness}.
\noindentskip In \partref{manipulation}, we focused on improving the manipulation of virtual objects directly with the hand in immersive \OST-\AR.
\noindentskip In \partref{manipulation}, we focused on improving the manipulation of virtual objects directly with the hand using an \OST-\AR headset.
Our approach was to design visual augmentations of the hand and delocalized haptic feedback, based on the literature, and evaluate them in user studies.
We first considered \textbf{(1) the visual augmentation of the hand} and then the \textbf{(2)} combination of different \textbf{visuo-haptic feedback of the hand when manipulating virtual objects}.
@@ -102,7 +102,7 @@ As in the previous chapter, our aim was not to accurately reproduce real texture
However, the results also have some limitations, as they addressed a small set of visuo-haptic textures that augmented the perception of smooth and white real surfaces.
Visuo-haptic texture augmentation might be difficult on surfaces that already have strong visual or haptic patterns \cite{asano2012vibrotactile}, or on objects with complex shapes.
The role of visuo-haptic texture augmentation should also be evaluated in more complex tasks, such as object recognition and assembly, or in more concrete use cases, such as displaying and touching a museum object or a 3D printed object before it is manufactured.
Finally, the visual textures used were simple color images not intended for use in an immersive \VE, and enhancing their visual quality could improve the perception of visuo-haptic texture augmentation.
Finally, the visual textures used were simple color images not intended for use in an \ThreeD \VE, and enhancing their visual quality could improve the perception of visuo-haptic texture augmentation.
\comans{JG}{As future work, the effect of visual quality of the rendered textures on texture perception could also be of interest.}{A sentence along these lines has been added.}
\paragraph{Specificities of Direct Touch.}
@@ -119,7 +119,7 @@ Finally, the virtual texture models should also be adaptable to individual sensi
\paragraph{AR Displays.}
The visual hand augmentations we evaluated were displayed on the Microsoft HoloLens~2, which is a common \OST-\AR headset \cite{hertel2021taxonomy}.
We purposely chose this type of display because in \OST-\AR the lack of mutual occlusion between the hand and the virtual object is the most challenging to solve \cite{macedo2023occlusion}.
We purposely chose this type of display, because in \OST-\AR the lack of mutual occlusion between the hand and the virtual object is the most challenging to solve \cite{macedo2023occlusion}.
We therefore hypothesized that a visual hand augmentation would be more beneficial to users with this type of display.
However, the user's visual perception and experience are different with other types of displays, such as \VST-\AR, where the \RE view is seen through cameras and screens (\secref[related_work]{ar_displays}).
While the mutual occlusion problem and the hand pose estimation latency could be overcome with \VST-\AR, the visual hand augmentation could still be beneficial to users as it provides depth cues and feedback on the hand tracking, and should be evaluated as such.
@@ -150,7 +150,8 @@ It remains to be explored how to support rendering for different and larger area
\section{Perspectives}
Our goal was to improve direct hand interaction with virtual objects using wearable haptic devices in immersive \AR by providing more plausible and coherent perception and more natural and effective manipulation of the visuo-haptic augmentations.
Our goal was to improve direct hand interaction with virtual objects using wearable haptic devices and an \OST-\AR headset.
We aimed to provide more plausible and coherent perception and more natural and effective manipulation of the visuo-haptic augmentations.
Our contributions have enabled progress towards a seamless integration of the virtual into the real world.
They also allow us to outline longer-term research perspectives.
@@ -159,7 +160,7 @@ They also allow us to outline longer-term research perspectives.
We saw how complex the sense of touch is (\secref[related_work]{haptic_hand}).
Multiple sensory receptors all over the skin allow us to perceive different properties of objects, such as their texture, temperature, weight or shape.
Particularly concentrated in the hands, cutaneous sensory feedback, together with the muscles, is crucial for grasping and manipulating objects.
In this manuscript, we showed how wearable haptic devices can provide virtual tactile sensations to support direct hand interaction in immersive \AR.
In this manuscript, we showed how wearable haptic devices can provide virtual tactile sensations to support direct hand interaction with an \OST-\AR headset.
We investigated both the visuo-haptic perception of texture augmenting real surfaces (\partref{perception}) and the manipulation of virtual objects with visuo-haptic feedback of hand contact with virtual objects (\partref{manipulation}).
However, unlike the visual sense, which can be fully immersed in the virtual using an \AR/\VR headset, there is no universal wearable haptic device that can reproduce all the haptic properties perceived by the hand (\secref[related_work]{wearable_haptics}).
@@ -181,7 +182,7 @@ These results would enable the design of more universal wearable haptic devices
We reviewed the diversity of \AR and \VR reality displays and their respective characteristics in rendering (\secref[related_work]{ar_displays}) and the manipulation of virtual content with the hand (\chapref{visual_hand}).
The diversity of wearable haptic devices and the different sensations they can provide is even more important (\secref[related_work]{wearable_haptics}) and an active research topic \cite{pacchierotti2017wearable}.
Coupling wearable haptics with immersive \AR also requires the haptic actuator to be placed on the body other than at the hand contact points (\secref[related_work]{vhar_haptics}).
Coupling wearable haptics with \AR headsets also requires the haptic actuator to be placed on the body other than at the hand contact points (\secref[related_work]{vhar_haptics}).
In particular, in this thesis we investigated the perception of haptic texture augmentation using a vibrotactile device on the median phalanx (\chapref{vhar_system}) and also compared different positions of the haptics on the hand for manipulating virtual objects (\chapref{visuo_haptic_hand}).
Haptic feedback should be provided close to the point of contact of the hand with the virtual, to enhance the realism of texture augmentation (\chapref{vhar_textures}) and to render contact with virtual objects (\chapref{visuo_haptic_hand}), \eg rendering fingertip contact with a haptic ring worn on the middle or proximal phalanx.