erwan/phd-thesis
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

WIP xr-perception

Browse Source
This commit is contained in:
Erwan Normand
2024-09-27 22:11:28 +02:00
parent a9319210df
commit 344496cbef
19 changed files with 270 additions and 366 deletions
Download Patch File Download Diff File Expand all files Collapse all files

74
2-perception/xr-perception/1-introduction.tex
View File

@@ -1,69 +1,27 @@
% Delivers the motivation for your paper. It explains why you did the work you did.
% Insist on the advantage of wearable : augment any surface see bau2012revel
Most of the haptic augmentations of tangible surfaces using with wearable haptic devices, including roughness textures (\secref[related_work]{texture_rendering}), have been studied without a visual feedback, and none have considered the influence of the visual rendering on their perception or integrated them in \AR and \VR (\secref[related_work]{texture_rendering}).
Still, it is known that the visual rendering of a tangible can influence the perception of its haptic properties (\secref[related_work]{visual_haptic_influence}), and that the perception of same haptic force-feedback or vibrotactile rendering can differ between \AR and \VR, probably due to difference in perceived simultaneity between visual and haptic stimuli (\secref[related_work]{ar_vr_haptic}).
Indeed, while in \AR, the user can see their own hand touching, the haptic device worn and the \RE, in \VR they are hidden by the \VE while.
%Imagine you're an archaeologist or in a museum, and you want to examine an ancient object.
%
%But it is too fragile to touch directly.
%
%What if you could still grasp it and manipulate it through a tangible object in your hand, whose visual appearance has been modified using Augmented Reality (AR)?
%
%And what if you could also feel its shape or texture?
%
%Such tactile augmentation is made possible by wearable haptic devices, which are worn directly on the finger or hand and can provide a variety of sensations on the skin, while being small, light and discreet \cite{pacchierotti2017wearable}.
%
Wearable haptic devices, worn directly on the finger or hand, have been used to render a variety of tactile sensations to virtual objects in \VR \cite{detinguy2018enhancing,pezent2019tasbi} and \AR \cite{maisto2017evaluation,teng2021touch}.
%
They have also been used to alter the perception of roughness, stiffness, friction, and local shape perception of real tangible objects \cite{asano2015vibrotactile,detinguy2018enhancing,salazar2020altering}.
%
Such techniques place the actuator \emph{close} to the point of contact with the real environment, leaving the user free to directly touch the tangible.
%
This combined use of wearable haptics with tangible objects enables a haptic \emph{augmented} reality (HAR) \cite{bhatia2024augmenting} that can provide a rich and varied haptic feedback.
In this chapter, we investigate the role of the visual rendering of the hand (real or virtual) and its environment (\AR or \VR) on the perception of a \textbf{tangible surface whose haptic roughness is augmented with a wearable vibrotactile} device worn on the finger.
To do so, we employed the visuo-haptic system presented in \chapref{vhar_system} to render the texture augmentations of tangibles that can be directly touched with the bare finger.
We evaluated, in \textbf{user study with psychophysical methods and extensive questionnaire}, the perceived roughness augmentation in three visual rendering conditions: \textbf{(1) without visual augmentation}, in \textbf{(2) \OST-\AR with a realistic virtual hand} rendering, and in \textbf{(3) \VR with the same virtual hand}.
To control for the influence of the visual rendering, the tangible surface was not visually augmented.
The degree of reality/virtuality in both visual and haptic sensory modalities can be varied independently, but wearable haptic \AR has been little explored with \VR and (visual) \AR \cite{choi2021augmenting}.
%
Although \AR and \VR are closely related, they have significant differences that can affect the user experience \cite{genay2021virtual,macedo2023occlusion}.
%
%By integrating visual virtual content into the real environment, \AR keeps the hand of the user, the haptic devices worn and the tangibles touched visible, unlike \VR where they are hidden by immersing the user into a visual virtual environment.
%
%Current \AR systems also suffer from display and rendering limitations not present in \VR, affecting the user experience with virtual content that may be less realistic or inconsistent with the real augmented environment \cite{kim2018revisiting,macedo2023occlusion}.
%
Therefore, it seems necessary to investigate and understand the potential effect of these differences in visual rendering on the HAR perception.
%
For example, previous works have shown that the stiffness of a virtual piston rendered with a force feedback haptic system seen in \AR is perceived as less rigid than in \VR \cite{gaffary2017ar}, or when the visual rendering is ahead of the haptic rendering \cite{diluca2011effects,knorlein2009influence}.
%
%Taking our example from the beginning of this introduction, you now want to learn more about the context of the discovery of the ancient object or its use at the time of its creation by immersing yourself in a virtual environment in \VR.
%
%But how different is the perception of the haptic augmentation in \AR compared to \VR, with a virtual hand instead of the real hand?
\noindentskip The contributions of this chapter is: A psychophysical user study with 20 participants to evaluate the effect of visual hand rendering in \OST-\AR or \VR on the perception of haptic roughness texture augmentations, using wearable vibrotactile haptics.
The goal of this paper is to study the role of the visual rendering of the hand (real or virtual) and its environment (AR or \VR) on the perception of a tangible surface whose texture is augmented with a wearable vibrotactile device worn on the finger.
%
We focus on the perception of roughness, one of the main tactile sensations of materials \cite{baumgartner2013visual,hollins1993perceptual,okamoto2013psychophysical} and one of the most studied haptic augmentations \cite{asano2015vibrotactile,culbertson2014modeling,friesen2024perceived,strohmeier2017generating,ujitoko2019modulating}.
\noindentskip In the remainder of this chapter, we first describe the experimental design and apparatus of the user study.
We then present the results obtained and discuss them before concluding.
Our contributions are:
%
\begin{itemize}
\item A system for rendering virtual vibrotactile roughness textures in real time on a tangible surface touched directly with the finger, integrated with an immersive visual \AR/\VR headset to provide a coherent multimodal visuo-haptic augmentation of the real environment; and %It is presented in \secref{method}.
\item A psychophysical study with 20 participants to evaluate the perception of these virtual roughness textures in three visual rendering conditions: without visual augmentation, with a realistic virtual hand rendering in \AR, and with the same virtual hand in \VR. %It is described in \secref{experiment} and those results are detailed in \secref{discussion}.
\end{itemize}
%In the remainder of this paper, we first present related work on wearable haptic texture augmentations and the haptic perception in \AR and \VR in \secref{related_work}.
%
%We then describe the visuo-haptic texture rendering system in \secref{method}.
%
%We present the experimental protocol and apparatus of the user study in \secref{experiment}, and the results obtained in \secref{results}.
%
%We discuss these results in \secref{discussion}, and conclude in \secref{conclusion}.
%In the remainder of this paper, we first present related work on perception in \VR and \AR in Section 2. Then, in Section 3, we describe the protocol and apparatus of our experimental study. The results obtained are presented in Section 4, followed by a discussion in Section 5. The paper ends with a general conclusion in Section 6.
%First, we present a system for rendering virtual vibrotactile textures in real time without constraints on hand movements and integrated with an immersive visual \AR/\VR headset to provide a coherent multimodal visuo-haptic augmentation of the real environment.
%
%First, we present a system for rendering virtual vibrotactile textures in real time without constraints on hand movements and integrated with an immersive visual \AR/\VR headset to provide a coherent multimodal visuo-haptic augmentation of the \RE.
%An experimental setup is then presented to compare haptic roughness augmentation with an optical \AR headset (Microsoft HoloLens~2) that can be transformed into a \VR headset using a cardboard mask.
%
%We then conduct a psychophysical study with 20 participants, where various virtual haptic textures on a tangible surface directly touched with the finger are compared in a two-alternative forced choice (2AFC) task in three visual rendering conditions: (1) without visual augmentation, (2) with a realistic virtual hand rendering in \AR, and (3) with the same virtual hand in \VR.
\fig[1]{teaser/teaser2}{
\bigskip
\fig[0.9]{teaser/teaser2}{
Vibrotactile textures were rendered in real time on a real surface using a wearable vibrotactile device worn on the finger.
}[
Participants explored this haptic roughness augmentation with (Real) their real hand alone, (Mixed) a realistic virtual hand overlay in \AR, and (Virtual) the same virtual hand in \VR.
}[%
Participants explored this haptic roughness augmentation with (\level{Real}) their real hand alone, (\level{Mixed}) a realistic virtual hand overlay in \AR, and (\level{Virtual}) the same virtual hand in \VR.
]