erwan/phd-thesis
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

wip related work

Browse Source
This commit is contained in:
Erwan Normand
2024-08-12 15:25:21 +02:00
parent 749e4c862a
commit 611ff38503
3 changed files with 28 additions and 19 deletions
Download Patch File Download Diff File Expand all files Collapse all files

36
1-introduction/introduction/introduction.tex
View File

@@ -1,7 +1,7 @@
\chapter{Introduction}
\mainlabel{introduction}
This thesis presents research on the perception and interaction directly with the hand with real and virtual everyday objects, visually and haptically augmented using immersive \AR and wearable haptic devices.
This thesis presents research on direct hand interaction with real and virtual everyday objects, visually and haptically augmented using immersive \AR and wearable haptic devices.
\section{Visual and Tactile Object Augmentations}
@@ -170,7 +170,7 @@ The \RE and the user's hand are tracked in real time by sensors and reconstructe
%
The interactions between the virtual hand and objects are then simulated and rendered as visual and haptic feedback to the user using an \AR headset and a wearable haptic device.
%
Because the \vh-\VE is displayed in real time, colocalized and aligned with the real one, the user is given the illustion of directly perceiving and interacting with the virtual content as if it were part of the \RE.
Because the \vh-\VE is displayed in real time, colocalized and aligned with the real one, the user is given the illusion of directly perceiving and interacting with the virtual content as if it were part of the \RE.
\fig{interaction-loop}{%
The interaction loop between a user and a visuo-haptic augmented environment. %
@@ -185,7 +185,7 @@ In this context, we identify two main research challenges that we address in thi
%
\begin{enumerate*}[label=(\Roman*)]
\item providing plausible and coherent visuo-haptic augmentations, and
\item enabling effective interaction with the augmented environment.
\item enabling effective manipulation of the augmented environment.
\end{enumerate*}
%
Each of these challenges also raises numerous design, technical and human issues specific to each of the two types of feedback, wearable haptics and immersive \AR, as well as multimodal rendering and user experience issues in integrating these two sensorimotor feedbacks into a coherent and seamless \vh-\AE.
@@ -224,11 +224,11 @@ It is therefore unclear to what extent the real and virtual visuo-haptic sensati
% on voit sa propre main toucher, contrairement à la \VR, où la vision est particulièrement dominante (eg retargeting), difficile à dire si le cas en RA, surtout que si touche objets augmentés, difficile de modifier visuellement et haptiquement on peut ajouter des sensations pas vraiment en enlever. Lactuateur n'est pas là où on touche, à quel point les sensations seront réalistes ? En cohérence avec les sensations visuelles ? À quel point la perception différente de la \VR, en terme de rendu main env, et de latence ? Important car permettra d'utiliser efficacement, avwc correction si besoin par rapport à la \VR. Lq boucle d'interaction a forcément de la latence par rapport aux mouvements, à la proprioception, et pas les mêmes entre visuel et haptique, quel effet ?
\subsectionstarbookmark{Enable Effective Interaction with the Augmented Environment}
\subsectionstarbookmark{Enable Effective Manipulation of the Augmented Environment}
Touching, grasping and manipulating \VOs are fundamental interactions for \AR~\autocite{kim2018revisiting}, \VR~\autocite{bergstrom2021how} and VEs in general~\autocite{laviola20173d}.
%
As the hand is not occupied or covered with a haptic device to not impair interaction with the \RE, as described in the previous section, one can expect a a seamless and direct interaction of the hand with the virtual content as if it were real.
As the hand is not occupied or covered with a haptic device to not impair interaction with the \RE, as described in the previous section, one can expect a a seamless and direct manipulation of the hand with the virtual content as if it were real.
%
Thus, augmenting a tangible object has the advantage of physically constraining the hand, allowing for easy and natural interaction, but manipulating a purely \VO with the bare hand can be challenging without good haptic feedback~\autocite{maisto2017evaluation,meli2018combining}. %, and one will rely on visual and haptic feedback to guide the interaction.
@@ -253,7 +253,7 @@ Yet, it is unclear which type of visual and haptic feedback is the best suited t
\section{Approach and Contributions}
The aim of this thesis is to understand how immersive visual and wearable haptic feedback compare and complement each other in the context of direct hand perception and interaction with augmented and \VOs.
The aim of this thesis is to understand how immersive visual and wearable haptic feedback compare and complement each other in the context of direct hand perception and manipulation with augmented objects.
%
As described in the Research Challenges section above, providing a convincing, consistent and effective \vh-\AE to a user is complex and raises many issues.
%
@@ -268,7 +268,7 @@ We consider two main axes of research, each addressing one of the research chall
%
\begin{enumerate*}[label=(\Roman*)]
\item modifying the perception of tangible surfaces using visuo-haptic texture augmentations, and
\item improving the interaction with virtual objects using visuo-haptic augmentations of the hand.
\item improving the manipulation of virtual objects using visuo-haptic augmentations of the hand.
\end{enumerate*}
%
Our contributions in these two axes are summarized in \figref{contributions}.
@@ -322,14 +322,14 @@ However, the rendering of these textures in an immersive and natural \vh-\AR usi
Our third objective is to evaluate the perception of simultaneous and co-localized visuo-haptic texture augmentation of tangible surfaces in \AR, directly touched by the hand, and to understand to what extent each sensory modality contributes to the overall perception of the augmented texture.
\subsectionstarbookmark{Improving Virtual Object Interaction with Visuo-Haptic Augmentations of the Hand}
\subsectionstarbookmark{Improving Virtual Object Manipulation with Visuo-Haptic Augmentations of the Hand}
In immersive and wearable \vh-\AR, the hand is free to touch and interact seamlessly with real, augmented, and virtual objects, and one can expect natural and direct contact and manipulation of \VOs with the bare hand.
%
However, the intangibility of the vVE, the many display limitations of current \v-\AR systems and wearable haptic devices, and the potential discrepancies between these two types of feedback can make the interaction with \VOs particularly challenging.
However, the intangibility of the vVE, the many display limitations of current \v-\AR systems and wearable haptic devices, and the potential discrepancies between these two types of feedback can make the manipulation of \VOs particularly challenging.
%However, the intangibility of the virtual visual environment, the lack of kinesthetic feedback of wearable haptics, the visual rendering limitations of current \AR systems, as well as the spatial and temporal discrepancies between the real environment, the visual feedback, and the haptic feedback, can make the interaction with \VOs with bare hands particularly challenging.
%
Still two types of sensory feedback are known to improve such direct \VO interaction, but they have not been studied in combination in immersive \v-\AE: visual rendering of the hand~\autocite{piumsomboon2014graspshell,prachyabrued2014visual} and hand-object interaction rendering with wearable haptics~\autocite{lopes2018adding,teng2021touch}.
Still two types of sensory feedback are known to improve such direct \VO manipulation, but they have not been studied in combination in immersive \v-\AE: visual rendering of the hand~\autocite{piumsomboon2014graspshell,prachyabrued2014visual} and contact rendering with wearable haptics~\autocite{lopes2018adding,teng2021touch}.
%
For this second axis of research, we propose to design and evaluate the role of visuo-haptic augmentations of the hand as interaction feedback with \VOs.
%
@@ -345,7 +345,7 @@ Thus, our fourth objective is to evaluate and compare the effect of different vi
Finally, as described above, wearable haptics for \v-\AR rely on moving the haptic actuator away from the fingertips to not impair the hand movements, sensations, and interactions with the \RE.
%
Previous works have shown that wearable haptics that provide feedback on the hand interaction with \VOs in \AR can significantly improve the user performance and experience~\autocite{maisto2017evaluation,meli2018combining}.
Previous works have shown that wearable haptics that provide feedback on the hand manipulation with \VOs in \AR can significantly improve the user performance and experience~\autocite{maisto2017evaluation,meli2018combining}.
%
However, it is unclear which positioning of the actuator is the most beneficial nor how a haptic augmentation of the hand compares or complements with a visual augmentation of the hand.
%
@@ -360,15 +360,15 @@ This thesis is divided in four parts.
%
\partref{context} describes the context and background of our research, within which this first current \textit{Introduction} chapter presents the research challenges, and the objectives, approach, and contributions of this thesis.
%
\chapref{related_work} then provides an overview of related work on the perception of and interaction with visual and haptic augmentations of objects.
\chapref{related_work} then presents previous work on the perception of and interaction with visual and haptic augmentations.
%
Firstly, it gives an overview of existing wearable haptic devices and renderings, and how they have been used to enhance the touch perception with haptic augmentations and to improve the \VO interaction, with a focus on vibrotactile feedback and haptic textures.
Firstly, it gives an overview of how wearable haptics have been used to enhance the touch perception and interaction, with a focus on vibrotactile feedback and haptic textures.
%
Secondly, it introduces the principles and user perception of augmented reality, and describes the 3D interaction techniques used in \AR and \VR environments to interact with virtual and augmented objects, in particular using the visual rendering of the user's hand.
It then introduces \AR, and how users perceive and can interact with the augmented environments, in particular using the visual rendering of the user's hand.
%
Finally, it shows how multimodal visuo-haptic feedback has been used in \AR and \VR to alter the perception of tangible objects and to improve the interaction with \VOs.
Finally, it shows how multimodal visuo-haptic feedback has been used in \AR and \VR to alter the perception of tangible objects and to improve the manipulation of \VOs.
%
Then, we address each of our two research axes in a dedicated part.
Next, we address each of our two research axes in a dedicated part.
\bigskip
@@ -398,9 +398,9 @@ Our objective is to assess the perceived realism, plausibility and roughness of
\bigskip
\partref{manipulation} describes our contributions to the second axis of research, improving \VO interaction with visuo-haptic augmentations of the hand.
\partref{manipulation} describes our contributions to the second axis of research, improving direct hand manipulation of \VOs with visuo-haptic augmentations of the hand.
%
We evaluate how the visual and haptic augmentation of the hand can improve the interaction with \VOs directly with the hand.
We evaluate how the visual and haptic augmentation of the hand can such manipulations.
\chapref{visual_hand} explores in a first user study the effect of six visual hand augmentations that provide contact feedback with the \VO, as a set of the most popular hand renderings in the \AR literature.
%