WIP vhar_textures

2-perception/vhar-textures/1-introduction.tex

@@ -1,33 +1,24 @@
-When we look at the surface of an everyday object, we then touch it to confirm or contrast our initial visual impression and to estimate the properties of the object \cite{ernst2002humans}.
-%
-One of the main characteristics of a textured surface is its roughness, \ie the micro-geometry of the material \cite{klatzky2003feeling}, which is perceived equally well and similarly by both sight and touch \cite{bergmanntiest2007haptic,baumgartner2013visual,vardar2019fingertip}.
-%
-Many haptic devices and rendering methods have been used to generate realistic virtual rough textures \cite{culbertson2018haptics}.
-%
-One of the most common approaches is to reproduce the vibrations that occur when running across a surface, using a vibrotactile device attached to a hand-held tool \cite{culbertson2014modeling,culbertson2015should} or worn on the finger \cite{asano2015vibrotactile,friesen2024perceived}.
-%
-By providing timely vibrations synchronized with the movement of the tool or the finger moving on a real object, the perceived roughness of the surface can be augmented \cite{culbertson2015should,asano2015vibrotactile}.
-%
-In that sense, data-driven haptic textures have been developed as captures and models of real surfaces, resulting in the Penn Haptic Texture Toolkit (HaTT) database \cite{culbertson2014one}.
-%
-While these virtual haptic textures are perceived as similar to real textures \cite{culbertson2015should}, they have been evaluated using hand-held tools and not yet in a direct finger contact with the surface context, in particular combined with visual textures in an immersive \VE.
+\noindent When we look at the surface of an everyday object, we then touch it to confirm or contrast our initial visual impression and to estimate the properties of the object, particularly its texture \secref[related_work]{visual_haptic_influence}.
+Among the various haptic texture augmentations, data-driven methods allow to capture, model and reproduce the roughness perception of real surfaces when touched touched by a hand-held stylus \secref[related_work]{texture_rendering}.
+Databases of visuo-haptic textures have been developed in this way \cite{culbertson2014one,balasubramanian2024sens3}, but they have not yet been explored in an immersive and direct touch context with \AR and wearable haptics.
 
-Combined with virtual reality (VR), where the user is immersed in a visual \VE, wearable haptic devices have also proven to be effective in modifying the visuo-haptic perception of tangible objects touched with the finger, without needing to modify the object \cite{asano2012vibrotactile,asano2015vibrotactile,salazar2020altering}.
-%
-Worn on the finger, but not directly on the fingertip to keep it free to interact with tangible objects, they have been used to alter perceived stiffness, softness, friction and local deformations \cite{detinguy2018enhancing,salazar2020altering}.
-%
-However, the use of wearable haptic devices has been little explored in Augmented Reality (AR), where visual virtual content is integrated into the real-world environment, especially for augmenting texture sensations \cite{punpongsanon2015softar,maisto2017evaluation,meli2018combining,chan2021hasti,teng2021touch,fradin2023humans}.
-%
-A key difference in \AR compared to \VR is that the user can still see the real-world surroundings, including their hands, the augmented tangible objects and the worn haptic devices.
-%
-One additional issue of current \AR systems is their visual display limitations, or virtual content that may not be seen as consistent with the real world \cite{kim2018revisiting,macedo2023occlusion}.
-%
-These two factors have been shown to influence the perception of haptic stiffness rendering \cite{knorlein2009influence,gaffary2017ar}.
-%
-It remains to be investigated whether simultaneous and co-localized visual and haptic texture augmentation of tangible surfaces in \AR can be perceived in a coherent and realistic manner, and to what extent each sensory modality would contribute to the overall perception of the augmented texture.
-%
-Being able to coherently substitute the visuo-haptic texture of an everyday surface directly touched by a finger is an important step towards new \AR applications capable of visually and haptically augmenting the \RE of a user in a plausible way.
+In this chapter, we investigate whether simultaneous and \textbf{co-localized visual and wearable haptic texture augmentation of tangible surfaces} in \AR can be perceived in a coherent and realistic manner, and to what extent each sensory modality would contribute to the overall perception of the augmented texture.
+We used nine pairs of \textbf{data-driven visuo-haptic textures} from the \HaTT database \cite{culbertson2014one}, which we rendered using the visuo-haptic system presented in \chapref{vhar_system}, an \OST-\AR headset, and a wearable voice-coil device worn on the finger.
+In a \textbf{user study}, 20 participants freely explored the combination of the visuo-haptic texture pairs to rate their coherence, realism and perceived roughness.
+We aimed to assess \textbf{which haptic textures were matched with which visual textures}, how the roughness of the visual and haptic textures was perceived, and whether \textbf{the perceived roughness} could explain the matches made between them.
 
-In this paper, we investigate how users perceive a tangible surface touched with the index finger when it is augmented with a visuo-haptic roughness texture using immersive optical see-through \AR (OST-AR) and wearable vibrotactile stimuli provided on the index.
-%
-In a user study, twenty participants freely explored and evaluated the coherence, realism and roughness of the combination of nine representative pairs of visuo-haptic texture augmentations (\figref{setup}, left) from the HaTT database \cite{culbertson2014one}.
+\noindentskip The contributions of this chapter are:
+\begin{itemize}
+  \item Transposition of data-driven visuo-haptic textures to augment tangible objects in a direct touch context in immersive \AR.
+  \item A user study evaluating with 20 participants the coherence, realism and perceived roughness of nine pairs of these visuo-haptic texture augmentations.
+\end{itemize}
+
+\noindentskip In the next sections, we first describe the apparatus of the user study experimental design, including the two tasks performed. We then present the results obtained and discuss them before concluding.
+
+\bigskip
+
+\fig[0.7]{experiment/view}{First person view of the user study. }[
+  As seen through the immersive \AR headset Microsoft HoloLens~2.
+  The visual texture overlays were statically displayed on the surfaces, allowing the user to move around to view them from different angles.
+  The haptic texture augmentations were generated based on \HaTT data-driven texture models and finger speed, and were rendered on the middle index phalanx as it slides on the considered surface.
+][]