erwan/phd-thesis
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add wearable haptic acronym

Browse Source
This commit is contained in:
Erwan Normand
2024-08-14 11:46:31 +02:00
parent fb09853fc0
commit b6e334de62
2 changed files with 28 additions and 27 deletions
Download Patch File Download Diff File Expand all files Collapse all files

54
1-introduction/introduction/introduction.tex
View File

@@ -1,7 +1,7 @@
\chapter{Introduction}
\mainlabel{introduction}
This thesis presents research on direct hand interaction with real and virtual everyday objects, visually and haptically augmented using immersive \AR and wearable haptic devices.
This thesis presents research on direct hand interaction with real and virtual everyday objects, visually and haptically augmented using immersive \AR and \WH devices.
\section{Visual and Tactile Object Augmentations}
@@ -66,20 +66,20 @@ Instead, wearable interfaces are directly mounted on the body to provide kinesth
\subfig[0.25]{culbertson2018haptics-wearable}
\end{subfigs}
A wide range of wearable haptic devices have been developed to provide the user with rich virtual haptic sensations, including normal force, skin stretch, vibration and thermal feedback.
A wide range of \WH devices have been developed to provide the user with rich virtual haptic sensations, including normal force, skin stretch, vibration and thermal feedback.
%
\figref{wearable-haptics} shows some examples of different wearable haptic devices with different form factors and rendering capabilities.
\figref{wearable-haptics} shows some examples of different \WH devices with different form factors and rendering capabilities.
%
Their portability, \ie their small form factor, light weight and unobtrusiveness, makes them particularly promising for everyday use in a variety of applications such as robotics, teleoperation, \VR, and social interactions~\autocite{pacchierotti2017wearable,culbertson2018haptics}.
%
But their use in combination with \AR has been little explored so far.
\begin{subfigs}{wearable-haptics}{
Wearable haptic devices can render sensations on the skin as feedback to real or virtual objects being touched.
\WH devices can render sensations on the skin as feedback to real or virtual objects being touched.
}[
\item Wolverine, a wearable exoskeleton that simulate contact and grasping of virtual objects with force feedback on the fingers~\autocite{choi2016wolverine}.
\item Touch\&Fold, a wearable haptic device mounted on the nail that fold on demand to render contact, normal force and vibrations to the fingertip~\autocite{teng2021touch}.
\item The hRing, a wearable haptic ring mounted on the proximal phalanx able to render normal and shear forces to the fingertip~\autocite{pacchierotti2016hring}.
\item Touch\&Fold, a \WH device mounted on the nail that fold on demand to render contact, normal force and vibrations to the fingertip~\autocite{teng2021touch}.
\item The hRing, a \WH ring mounted on the proximal phalanx able to render normal and shear forces to the fingertip~\autocite{pacchierotti2016hring}.
\item Tasbi, a haptic bracelet capable of rendering squeeze and vibrotactile feedback to the wrist~\autocite{pezent2019tasbi}.
]
\subfigsheight{28mm}
@@ -126,7 +126,7 @@ For example, a \v-\AE that uses a tangible (touchable) object as a proxy to mani
%
Haptic \AR is then the combination of real and virtual haptic stimuli~\autocite{bhatia2024augmenting} (middle row in \figref{visuo-haptic-rv-continuum3}).
%
In particular, it has been implemented by enhancing the haptic perception of tangible objects by providing timely tactile stimuli using wearable haptics.
In particular, it has been implemented by enhancing the haptic perception of tangible objects by providing timely tactile stimuli using \WHs.
%
\figref{salazar2020altering} shows an example of modifying the perceived stiffness of a tangible object in \VR using simultaneous pressure feedback on the finger (left middle cell in \figref{visuo-haptic-rv-continuum3}).
%
@@ -138,13 +138,13 @@ All \v-\VOs are inherently intangible and cannot physically constrain a user's h
%
It is therefore necessary to provide haptic feedback that is consistent with the \v-\AE and ensures the best possible user experience.
%
The integration of wearable haptics with \AR seems to be one of the most promising solutions, but it remains challenging due to their many respective characteristics and the additional constraints of combining them.
The integration of \WHs with \AR seems to be one of the most promising solutions, but it remains challenging due to their many respective characteristics and the additional constraints of combining them.
\begin{subfigs}{visuo-haptic-environments}{
Visuo-haptic environments with different degrees of reality-virtuality.
}[
\item Visual \AR environment with a real, tangible haptic object used as a proxy to manipulate a \VO~\autocite{kahl2023using}.
\item Visual \AR environment with a wearable haptic device that provides virtual, synthetic feedback from contact with a \VO~\autocite{meli2018combining}.
\item Visual \AR environment with a \WH device that provides virtual, synthetic feedback from contact with a \VO~\autocite{meli2018combining}.
\item A tangible object seen in a \v-\VR environment whose haptic perception of stiffness is augmented with the hRing haptic device~\autocite{salazar2020altering}.
\item Visuo-haptic rendering of texture on a touched tangible object with a \v-\AR display and haptic electrovibration feedback~\autocite{bau2012revel}.
]
@@ -159,17 +159,17 @@ The integration of wearable haptics with \AR seems to be one of the most promisi
\section{Research Challenges of Wearable Visuo-Haptic Augmented Reality}
\label{research_challenges}
The integration of wearable haptics with \AR to create a \vh \AE is complex and presents many perceptual and interaction challenges, \ie sensing the \AE and acting effectively upon it.
The integration of \WHs with \AR to create a \vh-\AE is complex and presents many perceptual and interaction challenges, \ie sensing the \AE and acting effectively upon it.
%
We are particularly interested in enabling direct contact and manipulation of virtual and augmented objects with the bare hand.
We are particularly interested in enabling direct contact of virtual and augmented objects with the bare hand.
%
Our goal is to enable congruent, intuitive and seamless perception of and interaction with the \vh-\AE.
Our goal is to enable congruent, intuitive and seamless perception and manipulation of the \vh-\AE.
The experience of such a \vh-\AE relies on an interaction loop with the user, as illustrated in \figref{interaction-loop}.
%
The \RE and the user's hand are tracked in real time by sensors and reconstructed in visual and haptic VEs.
%
The interactions between the virtual hand and objects are then simulated and rendered as visual and haptic feedback to the user using an \AR headset and a wearable haptic device.
The interactions between the virtual hand and objects are then simulated and rendered as visual and haptic feedback to the user using an \AR headset and a \WH device.
%
Because the \vh-\VE is displayed in real time, colocalized and aligned with the real one, the user is given the illusion of directly perceiving and interacting with the virtual content as if it were part of the \RE.
@@ -180,7 +180,7 @@ Because the \vh-\VE is displayed in real time, colocalized and aligned with the
The virtual environment is rendered back to the user colocalized with the real one (in gray) using a \v-\AR headset and a \WH device.
]
%This to ensure the best possible user experience, taking into account the current capabilities and limitations of wearable haptics and augmented reality technologies.
%This to ensure the best possible user experience, taking into account the current capabilities and limitations of \WHs and augmented reality technologies.
%
In this context, we identify two main research challenges that we address in this thesis:
%
@@ -189,7 +189,7 @@ In this context, we identify two main research challenges that we address in thi
\item enabling effective manipulation of the augmented environment.
\end{enumerate*}
%
Each of these challenges also raises numerous design, technical and human issues specific to each of the two types of feedback, wearable haptics and immersive \AR, as well as multimodal rendering and user experience issues in integrating these two sensorimotor feedbacks into a coherent and seamless \vh-\AE.
Each of these challenges also raises numerous design, technical and human issues specific to each of the two types of feedback, \WHs and immersive \AR, as well as multimodal rendering and user experience issues in integrating these two sensorimotor feedbacks into a coherent and seamless \vh-\AE.
%These challenges are illustrated in the visuo-haptic interaction loop in \figref{interaction-loop}.
@@ -205,7 +205,7 @@ Firstly, the user's hand and \RE are visible in \AR, unlike \VR where there is t
As such, in \VR, visual sensations are particularly dominant in perception, and conflicts with haptic sensations are also specifically created to influence the user's perception, for example to create pseudo-haptic~\autocite{ujitoko2021survey} or haptic retargeting~\autocite{azmandian2016haptic} effects.
%enabling techniques such as pseudo-haptic feedback that induce haptic feedback with visual stimuli~\autocite{ujitoko2021survey} or haptic retargeting that associate a single tangible object with multiple \VOs without the user noticing~\autocite{azmandian2016haptic}.
%
Moreover, many wearable haptic devices take the form of controllers, gloves or exoskeletons, all of which cover the fingertips and are therefore not suitable for \AR.
Moreover, many \WH devices take the form of controllers, gloves or exoskeletons, all of which cover the fingertips and are therefore not suitable for \AR.
%
The user's hand must be indeed free to touch and interact with the \RE.
%
@@ -221,7 +221,7 @@ These added virtual sensations can therefore be perceived as out of sync or even
%
It is therefore unclear to what extent the real and virtual visuo-haptic sensations will be perceived as realistic or plausible, and to what extent they will conflict or complement each other in the perception of the \AE.
%
%Therefore, it remains to be investigated how these three characteristics of using wearable haptics with \AR affect the perception, especially with visually and haptically augmented objects.
%Therefore, it remains to be investigated how these three characteristics of using \WHs with \AR affect the perception, especially with visually and haptically augmented objects.
% on voit sa propre main toucher, contrairement à la \VR, où la vision est particulièrement dominante (eg retargeting), difficile à dire si le cas en RA, surtout que si touche objets augmentés, difficile de modifier visuellement et haptiquement on peut ajouter des sensations pas vraiment en enlever. Lactuateur n'est pas là où on touche, à quel point les sensations seront réalistes ? En cohérence avec les sensations visuelles ? À quel point la perception différente de la \VR, en terme de rendu main env, et de latence ? Important car permettra d'utiliser efficacement, avwc correction si besoin par rapport à la \VR. Lq boucle d'interaction a forcément de la latence par rapport aux mouvements, à la proprioception, et pas les mêmes entre visuel et haptique, quel effet ?
@@ -255,7 +255,7 @@ Yet, it is unclear which type of visual and haptic feedback is the best suited t
\section{Approach and Contributions}
\label{contributions}
The aim of this thesis is to understand how immersive visual and wearable haptic feedback compare and complement each other in the context of direct hand perception and manipulation with augmented objects.
The aim of this thesis is to understand how immersive visual and \WH feedback compare and complement each other in the context of direct hand perception and manipulation with augmented objects.
%
As described in the Research Challenges section above, providing a convincing, consistent and effective \vh-\AE to a user is complex and raises many issues.
%
@@ -293,7 +293,7 @@ Our contributions in these two axes are summarized in \figref{contributions}.
% Very short abstract of contrib 2
Wearable haptic devices have proven to be effective in modifying the perception of a touched tangible surface, without modifying the tangible, nor covering the fingertip, forming a \h-\AE~\autocite{bau2012revel,detinguy2018enhancing,salazar2020altering}.
\WH devices have proven to be effective in modifying the perception of a touched tangible surface, without modifying the tangible, nor covering the fingertip, forming a \h-\AE~\autocite{bau2012revel,detinguy2018enhancing,salazar2020altering}.
%
%It is achieved by placing the haptic actuator close to the fingertip, to let it free to touch the surface, and rendering tactile stimuli timely synchronised with the finger movement.
%
@@ -321,7 +321,7 @@ Hence, our second objective is to understand how the perception of haptic textur
Finally, some visuo-haptic texture databases have been modelled from real texture captures~\autocite{culbertson2014penn,balasubramanian2024sens3}, to be rendered as virtual textures with graspable haptics that are perceived as similar to real textures~\autocite{culbertson2015should,friesen2024perceived}.
%
However, the rendering of these textures in an immersive and natural \vh-\AR using wearable haptics remains to be investigated.
However, the rendering of these textures in an immersive and natural \vh-\AR using \WHs remains to be investigated.
%
Our third objective is to evaluate the perception of simultaneous and co-localized visuo-haptic texture augmentation of tangible surfaces in \AR, directly touched by the hand, and to understand to what extent each sensory modality contributes to the overall perception of the augmented texture.
@@ -330,10 +330,10 @@ Our third objective is to evaluate the perception of simultaneous and co-localiz
In immersive and wearable \vh-\AR, the hand is free to touch and interact seamlessly with real, augmented, and virtual objects, and one can expect natural and direct contact and manipulation of \VOs with the bare hand.
%
However, the intangibility of the vVE, the many display limitations of current \v-\AR systems and wearable haptic devices, and the potential discrepancies between these two types of feedback can make the manipulation of \VOs particularly challenging.
%However, the intangibility of the virtual visual environment, the lack of kinesthetic feedback of wearable haptics, the visual rendering limitations of current \AR systems, as well as the spatial and temporal discrepancies between the real environment, the visual feedback, and the haptic feedback, can make the interaction with \VOs with bare hands particularly challenging.
However, the intangibility of the vVE, the many display limitations of current \v-\AR systems and \WH devices, and the potential discrepancies between these two types of feedback can make the manipulation of \VOs particularly challenging.
%However, the intangibility of the virtual visual environment, the lack of kinesthetic feedback of \WHs, the visual rendering limitations of current \AR systems, as well as the spatial and temporal discrepancies between the real environment, the visual feedback, and the haptic feedback, can make the interaction with \VOs with bare hands particularly challenging.
%
Still two types of sensory feedback are known to improve such direct \VO manipulation, but they have not been studied in combination in immersive \v-\AE: visual rendering of the hand~\autocite{piumsomboon2014graspshell,prachyabrued2014visual} and contact rendering with wearable haptics~\autocite{lopes2018adding,teng2021touch}.
Still two types of sensory feedback are known to improve such direct \VO manipulation, but they have not been studied in combination in immersive \v-\AE: visual rendering of the hand~\autocite{piumsomboon2014graspshell,prachyabrued2014visual} and contact rendering with \WHs~\autocite{lopes2018adding,teng2021touch}.
%
For this second axis of research, we propose to design and evaluate the role of visuo-haptic augmentations of the hand as interaction feedback with \VOs.
%
@@ -347,9 +347,9 @@ But \v-\AR has significant perceptual differences from \VR due to the visibility
%
Thus, our fourth objective is to evaluate and compare the effect of different visual hand augmentations on direct manipulation of \VOs in \AR.
Finally, as described above, wearable haptics for \v-\AR rely on moving the haptic actuator away from the fingertips to not impair the hand movements, sensations, and interactions with the \RE.
Finally, as described above, \WHs for \v-\AR rely on moving the haptic actuator away from the fingertips to not impair the hand movements, sensations, and interactions with the \RE.
%
Previous works have shown that wearable haptics that provide feedback on the hand manipulation with \VOs in \AR can significantly improve the user performance and experience~\autocite{maisto2017evaluation,meli2018combining}.
Previous works have shown that \WHs that provide feedback on the hand manipulation with \VOs in \AR can significantly improve the user performance and experience~\autocite{maisto2017evaluation,meli2018combining}.
%
However, it is unclear which positioning of the actuator is the most beneficial nor how a haptic augmentation of the hand compares or complements with a visual augmentation of the hand.
%
@@ -365,9 +365,9 @@ This thesis is divided in four parts.
%
\partref{context} describes the context and background of our research, within which this first current \textit{Introduction} chapter presents the research challenges, and the objectives, approach, and contributions of this thesis.
%
\chapref{related_work} then presents previous work on the perception of and interaction with visual and haptic augmentations.
\chapref{related_work} then presents previous work on the perception of and interaction with visual and haptic augmentations using \WHs and \AR, and how they have been combined in \vh-\AEs.
%
Firstly, it gives an overview of how wearable haptics have been used to enhance the touch perception and interaction, with a focus on vibrotactile feedback and haptic textures.
Firstly, it gives an overview of how \WHs have been used to enhance the touch perception and interaction, with a focus on vibrotactile feedback and haptic textures.
%
It then introduces \AR, and how users perceive and can interact with the augmented environments, in particular using the visual rendering of the user's hand.
%