erwan/phd-thesis
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add back augmented environment (AE) acronym

Browse Source
This commit is contained in:
Erwan Normand
2025-04-11 23:06:10 +02:00
parent 1eb5ed0e42
commit 97e35ca282
5 changed files with 15 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files

17
1-introduction/introduction.tex
View File

@@ -70,7 +70,7 @@ However, the \textbf{integration of wearable haptics with \AR has been little ex
\subsectionstarbookmark{Augmented Reality Is Not Only Visual} \subsectionstarbookmark{Augmented Reality Is Not Only Visual}
\textbf{\emph{Augmented Reality (\AR)} integrates virtual content into the real world perception, creating the illusion of a unique \emph{augmented environment}} \cite{azuma1997survey,skarbez2021revisiting}. \textbf{\emph{Augmented Reality (\AR)} integrates virtual content into the real world perception, creating the illusion of a unique \emph{\AE}} \cite{azuma1997survey,skarbez2021revisiting}.
It thus promises natural and seamless interaction with physical and digital objects (and their combination) directly with our hands \cite{billinghurst2021grand}. It thus promises natural and seamless interaction with physical and digital objects (and their combination) directly with our hands \cite{billinghurst2021grand}.
It is technically and conceptually closely related to \emph{\VR}, which completely replaces \emph{\RE} perception with a \emph{\VE}. It is technically and conceptually closely related to \emph{\VR}, which completely replaces \emph{\RE} perception with a \emph{\VE}.
@@ -97,7 +97,7 @@ For example, (visual) \AR using a real object as a proxy to manipulate a virtual
In this thesis we call \AR/\VR \emph{systems} the computational set of hardware (input devices, sensors, displays and haptic devices) and software (tracking, simulation and rendering) that allows the user to interact with the \VE. % by implementing the interaction loop we proposed in \figref{interaction-loop}. In this thesis we call \AR/\VR \emph{systems} the computational set of hardware (input devices, sensors, displays and haptic devices) and software (tracking, simulation and rendering) that allows the user to interact with the \VE. % by implementing the interaction loop we proposed in \figref{interaction-loop}.
Many \AR displays have been explored, from projection systems to hand-held displays. Many \AR displays have been explored, from projection systems to hand-held displays.
\textbf{\AR headsets are the most promising display technology because they create a portable experience that allows the user to navigate the augmented environment and interact with it directly using their hands} \cite{hertel2021taxonomy}. \textbf{\AR headsets are the most promising display technology because they create a portable experience that allows the user to navigate the \AE and interact with it directly using their hands} \cite{hertel2021taxonomy}.
While \AR and \VR systems can address any of the human senses, most focus only on visual augmentation \cite[p.144]{billinghurst2015survey} and \cite{kim2018revisiting}. While \AR and \VR systems can address any of the human senses, most focus only on visual augmentation \cite[p.144]{billinghurst2015survey} and \cite{kim2018revisiting}.
\emph{Presence} is the illusion of \enquote{being there} when in \VR, or the illusion of the virtual content to \enquote{feel here} when in \AR \cite{slater2022separate,skarbez2021revisiting}. \emph{Presence} is the illusion of \enquote{being there} when in \VR, or the illusion of the virtual content to \enquote{feel here} when in \AR \cite{slater2022separate,skarbez2021revisiting}.
@@ -123,14 +123,14 @@ The \textbf{integration of wearable haptics with \AR headsets appears to be one
\section{Research Challenges of Wearable Visuo-Haptic Augmented Reality} \section{Research Challenges of Wearable Visuo-Haptic Augmented Reality}
\label{research_challenges} \label{research_challenges}
The integration of wearable haptics with \AR headsets to create a visuo-haptic augmented environment is complex and presents many perceptual and interaction challenges. The integration of wearable haptics with \AR headsets to create a visuo-haptic \AE is complex and presents many perceptual and interaction challenges.
In this thesis, we propose to \textbf{represent the user's experience with such a visuo-haptic augmented environment as an interaction loop}, shown in \figref{interaction-loop}. In this thesis, we propose to \textbf{represent the user's experience with such a visuo-haptic \AE as an interaction loop}, shown in \figref{interaction-loop}.
It is based on the interaction loops of users with \ThreeD systems \cite[p.84]{laviolajr20173d}. It is based on the interaction loops of users with \ThreeD systems \cite[p.84]{laviolajr20173d}.
The \RE and the user's hand are tracked in real time by sensors and reconstructed in visual and haptic \VEs. The \RE and the user's hand are tracked in real time by sensors and reconstructed in visual and haptic \VEs.
The interactions between the virtual hand and objects are then simulated, and rendered as feedback to the user using an \AR/\VR headset and wearable haptics. The interactions between the virtual hand and objects are then simulated, and rendered as feedback to the user using an \AR/\VR headset and wearable haptics.
Because the visuo-haptic \VE is displayed in real time and aligned with the \RE, the user is given the illusion of directly perceiving and interacting with the virtual content as if it were part of the \RE. Because the visuo-haptic \VE is displayed in real time and aligned with the \RE, the user is given the illusion of directly perceiving and interacting with the virtual content as if it were part of the \RE.
\fig{interaction-loop}{The interaction loop between a user and a visuo-haptic augmented environment as proposed in this thesis.}[ \fig{interaction-loop}{The interaction loop between a user and a visuo-haptic \AE as proposed in this thesis.}[
A user interacts with the visual (in blue) and haptic (in red) \VEs through a virtual hand (in purple) interaction technique that tracks real hand movements and simulates contact with virtual objects. A user interacts with the visual (in blue) and haptic (in red) \VEs through a virtual hand (in purple) interaction technique that tracks real hand movements and simulates contact with virtual objects.
The visual and haptic \VEs are rendered back using an \AR headset and wearable haptics, and are perceived by the user to be registered and co-localized with the \RE (in gray). The visual and haptic \VEs are rendered back using an \AR headset and wearable haptics, and are perceived by the user to be registered and co-localized with the \RE (in gray).
%\protect\footnotemark %\protect\footnotemark
@@ -138,9 +138,8 @@ Because the visuo-haptic \VE is displayed in real time and aligned with the \RE,
In this context, we focus on two main research challenges: In this context, we focus on two main research challenges:
\textbf{(I) providing plausible and coherent visuo-haptic augmentations}, and \textbf{(I) providing plausible and coherent visuo-haptic augmentations}, and
\textbf{(II) enabling effective manipulation of the augmented environment}. \textbf{(II) enabling effective manipulation of the \AE}.
Each of these challenges also raises numerous design, technical, perceptual and user experience issues specific to wearable haptics and \AR headsets. Each of these challenges also raises numerous design, technical, perceptual and user experience issues specific to wearable haptics and \AR headsets.
%, as well as virtual rendering and user experience issues.% in integrating these two sensorimotor feedbacks into a coherent and seamless visuo-haptic augmented environment.
%\footnotetext{% %\footnotetext{%
% The icons are \href{https://creativecommons.org/licenses/by/3.0/}{CC BY} licensed: % The icons are \href{https://creativecommons.org/licenses/by/3.0/}{CC BY} licensed:
@@ -166,7 +165,7 @@ It remains to be investigated how such potential discrepancies affect the overal
%Visual and haptic augmentations of the \RE add sensations to the user's overall perception. %Visual and haptic augmentations of the \RE add sensations to the user's overall perception.
The \textbf{added visual and haptic virtual sensations may also be perceived as incoherent} with the sensations of the real objects, for example with a lower rendering quality, a temporal latency, a spatial shift, or a combination of these. The \textbf{added visual and haptic virtual sensations may also be perceived as incoherent} with the sensations of the real objects, for example with a lower rendering quality, a temporal latency, a spatial shift, or a combination of these.
Moreover, with an \AR headset the user can still see the real world environment, including their hands, augmented real objects and worn haptic devices, unlike \VR where there is total control over the visual rendering. % of the hand and \VE. Moreover, with an \AR headset the user can still see the real world environment, including their hands, augmented real objects and worn haptic devices, unlike \VR where there is total control over the visual rendering. % of the hand and \VE.
It is therefore unclear to what extent the real and virtual visuo-haptic sensations will be perceived as a whole, and to what extent they will conflict or complement each other. % in the perception of the augmented environment. It is therefore unclear to what extent the real and virtual visuo-haptic sensations will be perceived as a whole, and to what extent they will conflict or complement each other.
With a better understanding of \textbf{how visual factors can influence the perception of haptic augmentations}, the many wearable haptic devices that already exist but have not yet been fully explored with \AR can be better applied, and new visuo-haptic augmentations adapted to \AR can be designed. With a better understanding of \textbf{how visual factors can influence the perception of haptic augmentations}, the many wearable haptic devices that already exist but have not yet been fully explored with \AR can be better applied, and new visuo-haptic augmentations adapted to \AR can be designed.
\subsectionstarbookmark{Challenge II: Enabling Effective Manipulation of the Augmented Environment} \subsectionstarbookmark{Challenge II: Enabling Effective Manipulation of the Augmented Environment}
@@ -193,7 +192,7 @@ Yet, it is unclear which type of visual and wearable haptic feedback, or their c
\section{Approach and Contributions} \section{Approach and Contributions}
\label{contributions} \label{contributions}
As we described in \secref{research_challenges}, providing a coherent and effective visuo-haptic augmented environment to a user is complex and raises many issues. As we described in \secref{research_challenges}, providing a coherent and effective visuo-haptic \AE to a user is complex and raises many issues.
Our approach is to: Our approach is to:
\begin{enumerate*}[label=(\arabic*)] \begin{enumerate*}[label=(\arabic*)]
\item design wearable visuo-haptic renderings that augment both the objects being interacted with and the hand interacting with them, and \item design wearable visuo-haptic renderings that augment both the objects being interacted with and the hand interacting with them, and

6
2-related-work/3-augmented-reality.tex
View File

@@ -102,7 +102,7 @@ As with \VR, virtual objects must be able to be seen from different angles by mo
The plausibility can be applied to \AR as is, but the virtual objects must additionally have knowledge of the \RE and react accordingly to it to be, again, perceived as coherently behaving with the real world \cite{skarbez2021revisiting}. The plausibility can be applied to \AR as is, but the virtual objects must additionally have knowledge of the \RE and react accordingly to it to be, again, perceived as coherently behaving with the real world \cite{skarbez2021revisiting}.
\begin{subfigs}{presence}{ \begin{subfigs}{presence}{
The sense of immersion in virtual and augmented environments. Adapted from \textcite{stevens2002putting}. The sense of immersion in virtual and \AEs. Adapted from \textcite{stevens2002putting}.
}[][ }[][
\item Place illusion is the sense of the user of \enquote{being there} in the \VE. \item Place illusion is the sense of the user of \enquote{being there} in the \VE.
\item Objet illusion is the sense of the virtual object to \enquote{feels here} in the \RE. \item Objet illusion is the sense of the virtual object to \enquote{feels here} in the \RE.
@@ -304,8 +304,8 @@ Taken together, these results suggest that a visual augmentation of the hand in
\AR systems integrate virtual content into the user's perception as if it were part of the \RE. \AR systems integrate virtual content into the user's perception as if it were part of the \RE.
\AR headsets now enable real-time pose estimation of the head and hands, and high-quality display of virtual content, while being portable and mobile. \AR headsets now enable real-time pose estimation of the head and hands, and high-quality display of virtual content, while being portable and mobile.
They create augmented environments that users can explore with a strong sense of the presence of the virtual content. They create \AEs that users can explore with a strong sense of the presence of the virtual content.
However, without direct and seamless interaction with the virtual objects using the hands, the coherence of the augmented environment experience is compromised. However, without direct and seamless interaction with the virtual objects using the hands, the coherence of the \AE experience is compromised.
In particular, when manipulating virtual objects in \OST-\AR, there is a lack of mutual occlusion and interaction cues between the hands and the virtual content, which could be mitigated by a visual augmentation of the hand. In particular, when manipulating virtual objects in \OST-\AR, there is a lack of mutual occlusion and interaction cues between the hands and the virtual content, which could be mitigated by a visual augmentation of the hand.
A common alternative approach is to use real objects as proxies for interaction with virtual objects, but this raises concerns about their coherence with visual augmentations. A common alternative approach is to use real objects as proxies for interaction with virtual objects, but this raises concerns about their coherence with visual augmentations.
In this context, the use of wearable haptic systems worn on the hand seems to be a promising solution both for improving direct hand manipulation of virtual objects and for coherent visuo-haptic augmentation of touched real objects. In this context, the use of wearable haptic systems worn on the hand seems to be a promising solution both for improving direct hand manipulation of virtual objects and for coherent visuo-haptic augmentation of touched real objects.

2
2-related-work/4-visuo-haptic-ar.tex
View File

@@ -106,7 +106,7 @@ Therefore, the perceived stiffness $\tilde{k}(t)$ increases with a haptic delay
\textcite{gaffary2017ar} compared perceived stiffness of virtual pistons in \OST-\AR and \VR. \textcite{gaffary2017ar} compared perceived stiffness of virtual pistons in \OST-\AR and \VR.
However, the force-feedback device and the participant's hand were not visible (\figref{gaffary2017ar}). However, the force-feedback device and the participant's hand were not visible (\figref{gaffary2017ar}).
The reference piston was judged to be stiffer when seen in \VR than in \AR, without participants noticing this difference, and more force was exerted on the piston overall in \VR. The reference piston was judged to be stiffer when seen in \VR than in \AR, without participants noticing this difference, and more force was exerted on the piston overall in \VR.
This suggests that the haptic stiffness of virtual objects feels \enquote{softer} in an augmented environment than in a full \VE. This suggests that the haptic stiffness of virtual objects feels \enquote{softer} in an \AE than in a full \VE.
Finally, \textcite{diluca2019perceptual} investigated the perceived simultaneity of visuo-haptic contact with a virtual object in \VR. Finally, \textcite{diluca2019perceptual} investigated the perceived simultaneity of visuo-haptic contact with a virtual object in \VR.
The contact was rendered both by a vibrotactile piezoelectric device on the fingertip and by a visual change in the color of the virtual object. The contact was rendered both by a vibrotactile piezoelectric device on the fingertip and by a visual change in the color of the virtual object.

1
2-related-work/related-work.tex
View File

@@ -6,7 +6,6 @@
\bigskip \bigskip
This chapter reviews previous work on the perception and manipulation of virtual and augmented objects directly with the hand, using either wearable haptics, \AR, or their combination. This chapter reviews previous work on the perception and manipulation of virtual and augmented objects directly with the hand, using either wearable haptics, \AR, or their combination.
%Experiencing a visual, haptic, or visuo-haptic augmented environment relies on one to many interaction loops between a user and the environment, as shown in \figref[introduction]{interaction-loop}, and each main step must be addressed and understood: the tracking and modelling of the \RE into a \VE, the interaction techniques to act on the \VE, the rendering of the \VE to the user through visual and haptic user interfaces, and, finally, the user's perception and actions on the overall augmented environment.
First, we review how the hand senses and interacts with its environment to perceive and manipulate the haptic properties of real everyday objects. First, we review how the hand senses and interacts with its environment to perceive and manipulate the haptic properties of real everyday objects.
Second, we present how wearable haptic devices and renderings have been used to augment the haptic perception of roughness and hardness of real objects. Second, we present how wearable haptic devices and renderings have been used to augment the haptic perception of roughness and hardness of real objects.
Third, we introduce the principles and user experience of \AR and review the main interaction techniques used to manipulate virtual objects directly with the hand. Third, we introduce the principles and user experience of \AR and review the main interaction techniques used to manipulate virtual objects directly with the hand.

3
config/acronyms.tex
View File

@@ -35,8 +35,11 @@
% Style the acronyms % Style the acronyms
\renewcommand*{\glstextformat}[1]{\textcolor{black}{#1}}% Hyperlink in black \renewcommand*{\glstextformat}[1]{\textcolor{black}{#1}}% Hyperlink in black
\let\AE\undefined % Command \AE is already defined
\acronym[TIFC]{2IFC}{two-interval forced choice} \acronym[TIFC]{2IFC}{two-interval forced choice}
\acronym[ThreeD]{3D}{three-dimensional} \acronym[ThreeD]{3D}{three-dimensional}
\acronym{AE}{augmented environment}
\acronym{ANOVA}{analysis of variance} \acronym{ANOVA}{analysis of variance}
\acronym{ART}{aligned rank transform} \acronym{ART}{aligned rank transform}
\acronym{AR}{augmented reality} \acronym{AR}{augmented reality}