Rename 1-introduction/ => 1-background/
2
1-background/background.tex
Normal file
@@ -0,0 +1,2 @@
|
|||||||
|
\part{Background}
|
||||||
|
\label{part:background}
|
||||||
|
Before Width: | Height: | Size: 59 KiB After Width: | Height: | Size: 59 KiB |
|
Before Width: | Height: | Size: 62 KiB After Width: | Height: | Size: 62 KiB |
|
Before Width: | Height: | Size: 46 KiB After Width: | Height: | Size: 46 KiB |
|
Before Width: | Height: | Size: 36 KiB After Width: | Height: | Size: 36 KiB |
|
Before Width: | Height: | Size: 30 KiB After Width: | Height: | Size: 30 KiB |
|
Before Width: | Height: | Size: 6.4 KiB After Width: | Height: | Size: 6.4 KiB |
|
Before Width: | Height: | Size: 5.2 KiB After Width: | Height: | Size: 5.2 KiB |
|
Before Width: | Height: | Size: 5.3 KiB After Width: | Height: | Size: 5.3 KiB |
|
Before Width: | Height: | Size: 9.9 KiB After Width: | Height: | Size: 9.9 KiB |
|
Before Width: | Height: | Size: 6.0 KiB After Width: | Height: | Size: 6.0 KiB |
|
Before Width: | Height: | Size: 5.9 KiB After Width: | Height: | Size: 5.9 KiB |
|
Before Width: | Height: | Size: 6.3 KiB After Width: | Height: | Size: 6.3 KiB |
|
Before Width: | Height: | Size: 8.2 KiB After Width: | Height: | Size: 8.2 KiB |
|
Before Width: | Height: | Size: 3.5 KiB After Width: | Height: | Size: 3.5 KiB |
|
Before Width: | Height: | Size: 50 KiB After Width: | Height: | Size: 50 KiB |
|
Before Width: | Height: | Size: 41 KiB After Width: | Height: | Size: 41 KiB |
|
Before Width: | Height: | Size: 48 KiB After Width: | Height: | Size: 48 KiB |
|
Before Width: | Height: | Size: 53 KiB After Width: | Height: | Size: 53 KiB |
|
Before Width: | Height: | Size: 53 KiB After Width: | Height: | Size: 53 KiB |
|
Before Width: | Height: | Size: 47 KiB After Width: | Height: | Size: 47 KiB |
@@ -167,7 +167,7 @@ Yet, it is unclear which type of visual and haptic feedback is the best suited t
|
|||||||
\section{Approach and Contributions}
|
\section{Approach and Contributions}
|
||||||
\label{contributions}
|
\label{contributions}
|
||||||
|
|
||||||
The aim of this thesis is to understand how immersive visual and wearable haptic augmentations compare and complement each other in the context of direct hand perception and manipulation with augmented objects.
|
The aim of this thesis is to understand how immersive visual and wearable haptic augmentations complement each other in the context of direct hand perception and manipulation with virtual and augmented objects.
|
||||||
As described in the Research Challenges section above, providing a convincing, consistent and effective visuo-haptic \AE to a user is complex and raises many issues.
|
As described in the Research Challenges section above, providing a convincing, consistent and effective visuo-haptic \AE to a user is complex and raises many issues.
|
||||||
Our approach is to
|
Our approach is to
|
||||||
\begin{enumerate*}[label=(\arabic*)]
|
\begin{enumerate*}[label=(\arabic*)]
|
||||||
@@ -178,7 +178,7 @@ Our approach is to
|
|||||||
We consider two main axes of research, each addressing one of the research challenges identified above:
|
We consider two main axes of research, each addressing one of the research challenges identified above:
|
||||||
\begin{enumerate*}[label=(\Roman*)]
|
\begin{enumerate*}[label=(\Roman*)]
|
||||||
\item modifying the perception of tangible surfaces using visuo-haptic texture augmentations, and
|
\item modifying the perception of tangible surfaces using visuo-haptic texture augmentations, and
|
||||||
\item improving the manipulation of virtual objects using visuo-haptic augmentations of the hand.
|
\item improving the manipulation of virtual objects using visuo-haptic augmentations of the hand-object interaction.
|
||||||
\end{enumerate*}
|
\end{enumerate*}
|
||||||
Our contributions in these two axes are summarized in \figref{contributions}.
|
Our contributions in these two axes are summarized in \figref{contributions}.
|
||||||
|
|
||||||
@@ -216,24 +216,24 @@ In immersive and wearable visuo-haptic \AR, the hand is free to touch and intera
|
|||||||
However, the intangibility of the visual \VE, the display limitations of current visual \OST-\AR systems and the inherent spatial and temporal discrepancies between the user's hand actions and the visual feedback in the \VE can make the interaction with \VOs particularly challenging.
|
However, the intangibility of the visual \VE, the display limitations of current visual \OST-\AR systems and the inherent spatial and temporal discrepancies between the user's hand actions and the visual feedback in the \VE can make the interaction with \VOs particularly challenging.
|
||||||
%However, the intangibility of the virtual visual environment, the lack of kinesthetic feedback of wearable haptics, the visual rendering limitations of current \AR systems, as well as the spatial and temporal discrepancies between the real environment, the visual feedback, and the haptic feedback, can make the interaction with \VOs with bare hands particularly challenging.
|
%However, the intangibility of the virtual visual environment, the lack of kinesthetic feedback of wearable haptics, the visual rendering limitations of current \AR systems, as well as the spatial and temporal discrepancies between the real environment, the visual feedback, and the haptic feedback, can make the interaction with \VOs with bare hands particularly challenging.
|
||||||
Two particular sensory feedbacks are known to improve such direct \VO manipulation, but they have not been properly investigated in immersive \AR: visual rendering of the hand \cite{piumsomboon2014graspshell,prachyabrued2014visual} and delocalized haptic rendering \cite{lopes2018adding,teng2021touch}.
|
Two particular sensory feedbacks are known to improve such direct \VO manipulation, but they have not been properly investigated in immersive \AR: visual rendering of the hand \cite{piumsomboon2014graspshell,prachyabrued2014visual} and delocalized haptic rendering \cite{lopes2018adding,teng2021touch}.
|
||||||
For this second axis of research, we propose to design and evaluate \textbf{the role of visuo-haptic augmentations of the hand as interaction feedback with \VOs in \OST-\AR}.
|
For this second axis of research, we propose to design and evaluate \textbf{the role of visuo-haptic augmentations of the hand as interaction feedback with \VOs in immersive \OST-\AR}.
|
||||||
We consider the effect of (1) the visual rendering as hand augmentation and (2) of combination of different visuo-haptic augmentations of the hand.
|
We consider the effect on the user performance an experience of (1) the visual rendering as hand augmentation and (2) combination of different visuo-haptic rendering of the hand manipulation with \VOs
|
||||||
|
|
||||||
First, the visual rendering of the virtual hand is a key element for interacting and manipulating \VOs in \VR \cite{prachyabrued2014visual,grubert2018effects}.
|
First, the visual rendering of the virtual hand is a key element for interacting and manipulating \VOs in \VR \cite{prachyabrued2014visual,grubert2018effects}.
|
||||||
A few works have also investigated the visual rendering of the virtual hand in \AR, from simulating mutual occlusions between the hand and \VOs \cite{piumsomboon2014graspshell,al-kalbani2016analysis} to displaying the virtual hand as an avatar overlay \cite{blaga2017usability,yoon2020evaluating}, augmenting the real hand.
|
A few works have also investigated the visual rendering of the virtual hand in \AR, from simulating mutual occlusions between the hand and \VOs \cite{piumsomboon2014graspshell,al-kalbani2016analysis} to displaying the virtual hand as an avatar overlay \cite{blaga2017usability,yoon2020evaluating}, augmenting the real hand.
|
||||||
But \OST-\AR has significant perceptual differences from \VR due to the visibility of the real hand and environment, and these visual hand augmentations have not been evaluated in the context of \VO manipulation with the bare hand.
|
But \OST-\AR has significant perceptual differences from \VR due to the visibility of the real hand and environment, and these visual hand augmentations have not been evaluated in the context of \VO manipulation with the bare hand.
|
||||||
Thus, our fourth objective is to \textbf{investigate the visual rendering as hand augmentation for direct manipulation of \VOs in \OST-\AR}.
|
Thus, our fourth objective is to \textbf{investigate the visual rendering as hand augmentation} for direct manipulation of \VOs in \OST-\AR.
|
||||||
|
|
||||||
Second, as described above, wearable haptics for visual \AR rely on moving the haptic actuator away from the fingertips to not impair the hand movements, sensations, and interactions with the \RE.
|
Second, as described above, wearable haptics for visual \AR rely on moving the haptic actuator away from the fingertips to not impair the hand movements, sensations, and interactions with the \RE.
|
||||||
Previous works have shown that wearable haptics that provide feedback on the hand manipulation with \VOs in \AR can significantly improve the user performance and experience \cite{maisto2017evaluation,meli2018combining}.
|
Previous works have shown that wearable haptics that provide feedback on the hand manipulation with \VOs in \AR can significantly improve the user performance and experience \cite{maisto2017evaluation,meli2018combining}.
|
||||||
However, it is unclear which positioning of the actuator is the most beneficial nor how a haptic augmentation of the hand compares or complements with a visual augmentation of the hand.
|
However, it is unclear which positioning of the actuator is the most beneficial nor how a haptic augmentation of the hand compares or complements with a visual augmentation of the hand.
|
||||||
Our last objective is to \textbf{investigate the role of visuo-haptic rendering of the hand manipulation with \VO in \OST-\AR}.
|
Our last objective is to \textbf{investigate the visuo-haptic rendering of the hand manipulation} with \VOs in \OST-\AR using wearable vibrotactile haptic.
|
||||||
|
|
||||||
\section{Thesis Overview}
|
\section{Thesis Overview}
|
||||||
\label{thesis_overview}
|
\label{thesis_overview}
|
||||||
|
|
||||||
This thesis is divided in four parts.
|
This thesis is divided in four parts.
|
||||||
In \textbf{\partref{context}}, we describe the context and background of our research, within which this first current \textit{Introduction} chapter we present the research challenges, and the objectives, approach, and contributions of this thesis.
|
In \textbf{\partref{background}}, we describe the context and background of our research, within which this first current \textit{Introduction} chapter we present the research challenges, and the objectives, approach, and contributions of this thesis.
|
||||||
|
|
||||||
In \textbf{\chapref{related_work}}, we then review previous work on the perception and manipulation with virtual and augmented objects, directly with the hand, using either wearable haptics, \AR, or their combination.
|
In \textbf{\chapref{related_work}}, we then review previous work on the perception and manipulation with virtual and augmented objects, directly with the hand, using either wearable haptics, \AR, or their combination.
|
||||||
First, we overview how the hand perceives and manipulate real everyday objects.
|
First, we overview how the hand perceives and manipulate real everyday objects.
|
||||||
@@ -243,7 +243,7 @@ Finally, we describe how multimodal visual and haptic feedback have been combine
|
|||||||
|
|
||||||
We then address each of our two research axes in a dedicated part.
|
We then address each of our two research axes in a dedicated part.
|
||||||
|
|
||||||
noindentskip
|
\noindentskip
|
||||||
In \textbf{\partref{perception}}, we describe our contributions to the first axis of research, augmenting the visuo-haptic texture perception of tangible surfaces.
|
In \textbf{\partref{perception}}, we describe our contributions to the first axis of research, augmenting the visuo-haptic texture perception of tangible surfaces.
|
||||||
We evaluate how the visual rendering of the hand (real or virtual), the environment (\AR or \VR) and the textures (displayed or hidden) affect the roughness perception of virtual vibrotactile textures rendered on real surfaces and touched directly with the index finger.
|
We evaluate how the visual rendering of the hand (real or virtual), the environment (\AR or \VR) and the textures (displayed or hidden) affect the roughness perception of virtual vibrotactile textures rendered on real surfaces and touched directly with the index finger.
|
||||||
|
|
||||||
@@ -258,15 +258,14 @@ In \textbf{\chapref{ar_textures}}, we evaluate the perception of visuo-haptic te
|
|||||||
The virtual textures are paired visual and tactile models of real surfaces \cite{culbertson2014one} that we render as visual and haptic overlays on the touched augmented surfaces.
|
The virtual textures are paired visual and tactile models of real surfaces \cite{culbertson2014one} that we render as visual and haptic overlays on the touched augmented surfaces.
|
||||||
Our objective is to assess the perceived realism, coherence and roughness of the combination of nine representative visuo-haptic texture pairs.
|
Our objective is to assess the perceived realism, coherence and roughness of the combination of nine representative visuo-haptic texture pairs.
|
||||||
|
|
||||||
noindentskip
|
\noindentskip
|
||||||
In \textbf{\partref{manipulation}}, we describe our contributions to the second axis of research, improving direct hand manipulation of \VOs with visuo-haptic augmentations of the hand.
|
In \textbf{\partref{manipulation}}, we describe our contributions to the second axis of research: improving the manipulation of \VOs using visuo-haptic augmentations of the hand as interaction feedback with \VOs in immersive \OST-\AR.
|
||||||
We explore how the visual and haptic augmentation of the hand, and their combination, as interaction feedback with \VOs in \OST-\AR can improve such manipulations.
|
|
||||||
|
|
||||||
In \textbf{\chapref{visual_hand}}, we investigate in a user study the effect of six visual renderings as hand augmentations for the direct manipulation of \VOs, as a set of the most popular hand renderings in the \AR literature.
|
In \textbf{\chapref{visual_hand}}, we investigate in a user study the effect of six visual renderings as hand augmentations for the direct manipulation of \VOs, as a set of the most popular hand renderings in the \AR literature.
|
||||||
Using the \OST-\AR headset Microsoft HoloLens~2, we evaluate the user performance and experience in two representative manipulation tasks: push-and-slide and grasp-and-place a \VO directly with the hand.
|
Using the \OST-\AR headset Microsoft HoloLens~2, we evaluate the user performance and experience in two representative manipulation tasks: push-and-slide and grasp-and-place a \VO directly with the hand.
|
||||||
|
|
||||||
In \textbf{\chapref{visuo_haptic_hand}}, we evaluate in a user study two vibrotactile contact techniques, provided at four different locations on the real hand, as haptic rendering of the hand-object interaction.
|
In \textbf{\chapref{visuo_haptic_hand}}, we evaluate in a user study two vibrotactile contact techniques, provided at four different positionings on the user's hand, as haptic rendering of the hand manipulation with \VOs.
|
||||||
They are compared to the two most representative visual hand augmentations from the previous study, and the user performance and experience are evaluated within the same \OST-\AR setup and manipulation tasks.
|
They are compared to the two most representative visual hand renderings from the previous chapter, and the user performance and experience are evaluated within the same \OST-\AR setup and manipulation tasks.
|
||||||
|
|
||||||
noindentskip
|
\noindentskip
|
||||||
In \textbf{\partref{part:conclusion}}, we conclude this thesis and discuss short-term future work and long-term perspectives for each of our contributions and research axes.
|
In \textbf{\partref{part:conclusion}}, we conclude this thesis and discuss short-term future work and long-term perspectives for each of our contributions and research axes.
|
||||||
@@ -133,7 +133,7 @@ They are small, lightweight and can be placed directly on any part of the hand.
|
|||||||
All vibrotactile actuators are based on the same principle: generating an oscillating motion from an electric current with a frequency and amplitude high enough to be perceived by cutaneous mechanoreceptors.
|
All vibrotactile actuators are based on the same principle: generating an oscillating motion from an electric current with a frequency and amplitude high enough to be perceived by cutaneous mechanoreceptors.
|
||||||
Several types of vibrotactile actuators are used in haptics, with different trade-offs between size, proposed \DoFs and application constraints.
|
Several types of vibrotactile actuators are used in haptics, with different trade-offs between size, proposed \DoFs and application constraints.
|
||||||
|
|
||||||
An \ERM is a \DC motor that rotates an off-center mass when a voltage or current is applied (\figref{precisionmicrodrives_erm}). \ERMs are easy to control, inexpensive and can be encapsulated in a few millimeters cylinder or coin form factor. However, they have only one \DoF because both the frequency and amplitude of the vibration are coupled to the speed of the rotation, \eg low (high) frequencies output at low (high) amplitudes, as shown on \figref{precisionmicrodrives_erm_performances}.
|
An \ERM is a direct current (DC) motor that rotates an off-center mass when a voltage or current is applied (\figref{precisionmicrodrives_erm}). \ERMs are easy to control, inexpensive and can be encapsulated in a few millimeters cylinder or coin form factor. However, they have only one \DoF because both the frequency and amplitude of the vibration are coupled to the speed of the rotation, \eg low (high) frequencies output at low (high) amplitudes, as shown on \figref{precisionmicrodrives_erm_performances}.
|
||||||
|
|
||||||
\begin{subfigs}{erm}{Diagram and performance of \ERMs. }[][
|
\begin{subfigs}{erm}{Diagram and performance of \ERMs. }[][
|
||||||
\item Diagram of a cylindrical encapsulated \ERM. From Precision Microdrives~\footnotemark.
|
\item Diagram of a cylindrical encapsulated \ERM. From Precision Microdrives~\footnotemark.
|
||||||
@@ -146,10 +146,10 @@ An \ERM is a \DC motor that rotates an off-center mass when a voltage or current
|
|||||||
|
|
||||||
\footnotetext{\url{https://www.precisionmicrodrives.com/}}
|
\footnotetext{\url{https://www.precisionmicrodrives.com/}}
|
||||||
|
|
||||||
A \LRA consists of a coil that creates a magnetic field from an \AC to oscillate a magnet attached to a spring, as an audio loudspeaker (\figref{precisionmicrodrives_lra}).
|
A \LRA consists of a coil that creates a magnetic field from an alternative current (AC) to oscillate a magnet attached to a spring, as an audio loudspeaker (\figref{precisionmicrodrives_lra}).
|
||||||
They are more complex to control and a bit larger than \ERMs.
|
They are more complex to control and a bit larger than \ERMs.
|
||||||
Each \LRA is designed to vibrate with maximum amplitude at a given resonant frequency, but won't vibrate efficiently at other frequencies, \ie their bandwidth is narrow, as shown on \figref{azadi2014vibrotactile}.
|
Each \LRA is designed to vibrate with maximum amplitude at a given resonant frequency, but won't vibrate efficiently at other frequencies, \ie their bandwidth is narrow, as shown on \figref{azadi2014vibrotactile}.
|
||||||
A \VCA is a \LRA but capable of generating vibration at two \DoF, with an independent control of the frequency and amplitude of the vibration on a wide bandwidth.
|
A voice-coil actuator is a \LRA but capable of generating vibration at two \DoF, with an independent control of the frequency and amplitude of the vibration on a wide bandwidth.
|
||||||
They are larger in size than \ERMs and \LRAs, but can generate more complex renderings.
|
They are larger in size than \ERMs and \LRAs, but can generate more complex renderings.
|
||||||
|
|
||||||
Piezoelectric actuators deform a solid material when a voltage is applied.
|
Piezoelectric actuators deform a solid material when a voltage is applied.
|
||||||
|
Before Width: | Height: | Size: 58 KiB After Width: | Height: | Size: 58 KiB |
|
Before Width: | Height: | Size: 22 KiB After Width: | Height: | Size: 22 KiB |
|
Before Width: | Height: | Size: 42 KiB After Width: | Height: | Size: 42 KiB |
|
Before Width: | Height: | Size: 64 KiB After Width: | Height: | Size: 64 KiB |
|
Before Width: | Height: | Size: 51 KiB After Width: | Height: | Size: 51 KiB |
|
Before Width: | Height: | Size: 56 KiB After Width: | Height: | Size: 56 KiB |
|
Before Width: | Height: | Size: 13 KiB After Width: | Height: | Size: 13 KiB |
|
Before Width: | Height: | Size: 212 KiB After Width: | Height: | Size: 212 KiB |
|
Before Width: | Height: | Size: 84 KiB After Width: | Height: | Size: 84 KiB |
|
Before Width: | Height: | Size: 29 KiB After Width: | Height: | Size: 29 KiB |
|
Before Width: | Height: | Size: 143 KiB After Width: | Height: | Size: 143 KiB |
|
Before Width: | Height: | Size: 27 KiB After Width: | Height: | Size: 27 KiB |
|
Before Width: | Height: | Size: 97 KiB After Width: | Height: | Size: 97 KiB |
|
Before Width: | Height: | Size: 61 KiB After Width: | Height: | Size: 61 KiB |
|
Before Width: | Height: | Size: 59 KiB After Width: | Height: | Size: 59 KiB |
|
Before Width: | Height: | Size: 130 KiB After Width: | Height: | Size: 130 KiB |
|
Before Width: | Height: | Size: 170 KiB After Width: | Height: | Size: 170 KiB |
|
Before Width: | Height: | Size: 62 KiB After Width: | Height: | Size: 62 KiB |
|
Before Width: | Height: | Size: 26 KiB After Width: | Height: | Size: 26 KiB |
|
Before Width: | Height: | Size: 21 KiB After Width: | Height: | Size: 21 KiB |
|
Before Width: | Height: | Size: 37 KiB After Width: | Height: | Size: 37 KiB |
|
Before Width: | Height: | Size: 107 KiB After Width: | Height: | Size: 107 KiB |
|
Before Width: | Height: | Size: 104 KiB After Width: | Height: | Size: 104 KiB |
|
Before Width: | Height: | Size: 103 KiB After Width: | Height: | Size: 103 KiB |
|
Before Width: | Height: | Size: 403 KiB After Width: | Height: | Size: 403 KiB |
|
Before Width: | Height: | Size: 54 KiB After Width: | Height: | Size: 54 KiB |
|
Before Width: | Height: | Size: 88 KiB After Width: | Height: | Size: 88 KiB |
|
Before Width: | Height: | Size: 42 KiB After Width: | Height: | Size: 42 KiB |
|
Before Width: | Height: | Size: 57 KiB After Width: | Height: | Size: 57 KiB |
|
Before Width: | Height: | Size: 41 KiB After Width: | Height: | Size: 41 KiB |
|
Before Width: | Height: | Size: 26 KiB After Width: | Height: | Size: 26 KiB |
|
Before Width: | Height: | Size: 183 KiB After Width: | Height: | Size: 183 KiB |
|
Before Width: | Height: | Size: 43 KiB After Width: | Height: | Size: 43 KiB |
|
Before Width: | Height: | Size: 18 KiB After Width: | Height: | Size: 18 KiB |
|
Before Width: | Height: | Size: 137 KiB After Width: | Height: | Size: 137 KiB |
|
Before Width: | Height: | Size: 35 KiB After Width: | Height: | Size: 35 KiB |