WIP vhar_system intro

1-background/related-work/4-visuo-haptic-ar.tex

@@ -152,7 +152,7 @@ We describe in the next section how wearable haptics have been integrated with i
 
 A few wearable haptic devices have been specifically designed or experimentally tested for direct hand interaction in immersive \AR.
 Since virtual or augmented objects are naturally touched, grasped, and manipulated directly with the fingertips (\secref{exploratory_procedures} and \secref{grasp_types}), the main challenge of wearable haptics for \AR is to provide haptic sensations of these interactions while keeping the fingertips free to interact with the \RE.
-Several approaches have been proposed to move the haptic actuator to a different location on the hand.
+Several approaches have been proposed to move the haptic actuator to a different location, on the outside of the finger or the hand, \eg the nail, the top of a phalanx, or the wrist.
 Yet, they differ greatly in the actuators used (\secref{wearable_haptic_devices}), thus the haptic feedback (\secref{tactile_rendering}), and the placement of the haptic rendering.
 
 Other wearable haptic actuators have been proposed for \AR, but are not discussed here.
@@ -162,7 +162,7 @@ Another category of actuators relies on systems that cannot be considered as por
 \subsubsection{Nail-Mounted Devices}
 \label{vhar_nails}
 
-\textcite{ando2007fingernailmounted} were the first to propose to moving the actuator from the fingertip to the nail, as described in \secref{texture_rendering}.
+\textcite{ando2007fingernailmounted} were the first to move the actuator from the fingertip to propose the nail, as described in \secref{texture_rendering}.
 This approach was later extended by \textcite{teng2021touch} with Touch\&Fold, a haptic device able to unfold its end-effector on demand to make contact with the fingertip when touching \VOs (\figref{teng2021touch_1}).
 This moving platform also contains a \LRA (\secref{moving_platforms}) and provides contact pressure and texture sensations.
 %The whole system is very compact (\qtyproduct{24 x 24 x 41}{\mm}), lightweight (\qty{9.5}{\g}), and fully portable by including a battery and Bluetooth wireless communication. \qty{20}{\ms} for the Bluetooth

2-perception/vhar-system/1-introduction.tex

@@ -1,64 +1,20 @@
-\noindent When we look at the surface of an everyday object, we then touch it to confirm or contrast our initial visual impression and to estimate the properties of the object \cite{ernst2002humans}.
-%
-One of the main characteristics of a textured surface is its roughness, \ie the micro-geometry of the material \cite{klatzky2003feeling}, which is perceived equally well and similarly by both sight and touch \cite{bergmanntiest2007haptic,baumgartner2013visual,vardar2019fingertip}.
-%
-Many haptic devices and rendering methods have been used to generate realistic virtual rough textures \cite{culbertson2018haptics}.
-%
-One of the most common approaches is to reproduce the vibrations that occur when running across a surface, using a vibrotactile device attached to a hand-held tool \cite{culbertson2014modeling,culbertson2015should} or worn on the finger \cite{asano2015vibrotactile,friesen2024perceived}.
-%
-By providing timely vibrations synchronized with the movement of the tool or the finger moving on a real object, the perceived roughness of the surface can be augmented \cite{culbertson2015should,asano2015vibrotactile}.
-%
-In that sense, data-driven haptic textures have been developed as captures and models of real surfaces, resulting in the \HaTT database \cite{culbertson2014one}.
-%
-While these virtual haptic textures are perceived as similar to real textures \cite{culbertson2015should}, they have been evaluated using hand-held tools and not yet in a direct finger contact with the surface context, in particular combined with visual textures in an immersive \VE.
+%\noindent La principale augmentation haptique de texture consiste à simuler la rugosité d'une surface périodique de grille
 
-Wearable haptic devices have proven to be effective in modifying the perception of a touched tangible surface, without modifying the tangible, nor covering the fingertip, forming a haptic \AE \cite{bau2012revel,detinguy2018enhancing,salazar2020altering}.
+%By providing timely vibrations synchronized with the movement of the tool or the finger moving on a real object, the perceived roughness of the surface can be augmented \cite{culbertson2015should,asano2015vibrotactile}.
+%While these virtual haptic textures are perceived as similar to real textures \cite{culbertson2015should}, they have been evaluated using hand-held tools and not yet in a direct finger contact with the surface context, in particular combined with visual textures in an immersive \VE.
+%Second, many works have investigated the haptic augmentation of textures, but few have integrated them with immersive \VEs or have considered the influence of the visual rendering on their perception.
+%Such techniques place the actuator \emph{close} to the point of contact with the \RE, leaving the user free to directly touch the tangible.
 
-Second, many works have investigated the haptic augmentation of textures, but few have integrated them with immersive \VEs or have considered the influence of the visual rendering on their perception.
-Still, it is known that the visual feedback can alter the perception of real and virtual haptic sensations \cite{schwind2018touch,choi2021augmenting} but also that the force feedback perception of grounded haptic devices is not the same in \AR and \VR \cite{diluca2011effects,gaffary2017ar}.
-
-% Insist on the advantage of wearable : augment any surface see bau2012revel
-
-Wearable haptic devices, worn directly on the finger or hand, have been used to render a variety of tactile sensations to \VOs seen in \VR \cite{choi2018claw,detinguy2018enhancing,pezent2019tasbi} or \AR \cite{maisto2017evaluation,meli2018combining,teng2021touch}.
-%
-They have also been used to alter the perception of roughness, stiffness, friction, and local shape perception of real tangible objects \cite{asano2015vibrotactile,detinguy2018enhancing,salazar2020altering}.
-%
-Such techniques place the actuator \emph{close} to the point of contact with the \RE, leaving the user free to directly touch the tangible.
-%
-This combined use of wearable haptics with tangible objects enables a haptic \emph{augmented} reality (HAR) \cite{bhatia2024augmenting} that can provide a rich and varied haptic feedback.
-
-The degree of reality/virtuality in both visual and haptic sensory modalities can be varied independently, but wearable haptic \AR has been little explored with \VR and (visual) \AR \cite{choi2021augmenting}.
-%
-Although \AR and \VR are closely related, they have significant differences that can affect the user experience \cite{genay2021virtual,macedo2023occlusion}.
-%
-%By integrating visual virtual content into the \RE, \AR keeps the hand of the user, the haptic devices worn and the tangibles touched visible, unlike \VR where they are hidden by immersing the user into a visual virtual environment.
-%
-%Current \AR systems also suffer from display and rendering limitations not present in \VR, affecting the user experience with virtual content that may be less realistic or inconsistent with the real augmented environment \cite{kim2018revisiting,macedo2023occlusion}.
-%
-It therefore seems necessary to investigate and understand the potential effect of these differences in visual rendering on the perception of haptically augmented tangible objects.
-%
-Previous works have shown, for example, that the stiffness of a virtual piston rendered with a force feedback haptic system seen in \AR is perceived as less rigid than in \VR \cite{gaffary2017ar} or when the visual rendering is ahead of the haptic rendering \cite{diluca2011effects,knorlein2009influence}.
-%
-%Taking our example from the beginning of this introduction, you now want to learn more about the context of the discovery of the ancient object or its use at the time of its creation by immersing yourself in a virtual environment in \VR.
-%
-%But how different is the perception of the haptic augmentation in \AR compared to \VR, with a virtual hand instead of the real hand?
+In this chapter, we propose a system for rendering coherent visual and haptic virtual textures that augment tangible surfaces.
+It is implemented with an immersive \OST-\AR headset Microsoft HoloLens~2 and a wearable vibrotactile (voice-coil) device worn on the outside of finger (not covering the fingertip, \secref[related_work]{vhar_haptics}).
+The visuo-haptic augmentations can be viewed freely from any angle and touched directly with the bare finger, as if they were real textures.
+To ensure real-time and realible renderings, the hand and the tangibles are tracked using a webcam and marker-based tracking.
+The haptic textures are rendered as a real-time vibrotactile signal representing a patterned grating texture that is synchronized with the finger movement on the augmented tangible surface.
 
 \noindentskip The contributions of this chapter are:
 \begin{itemize}
-  \item The rendering of virtual vibrotactile roughness textures in real time using webcam to track the finger touching.
-  \item A system to provide a coherent multimodal visuo-haptic texture augmentations of the \RE in direct touch context using an immersive visual AR/VR headset and wearable haptics.
+  \item The rendering of virtual vibrotactile roughness textures representing a patterned grating texture in real time from free finger movements.
+  \item A system to provide a coherent visuo-haptic texture augmentations of the \RE in a direct touch context using an immersive \AR headset and wearable haptics.
 \end{itemize}
 
 \noindentskip In the remainder of this chapter, we describe the principles of the system, how the real and virtual environments are registered, the generation of the vibrotactile textures, and measures of visual and haptic rendering latencies.
-
-%First, we present a system for rendering virtual vibrotactile textures in real time without constraints on hand movements and integrated with an immersive visual AR/VR headset to provide a coherent multimodal visuo-haptic augmentation of the \RE.
-%
-%An experimental setup is then presented to compare haptic roughness augmentation with an optical \AR headset (Microsoft HoloLens~2) that can be transformed into a \VR headset using a cardboard mask.
-%
-%We then conduct a psychophysical study with 20 participants, where various virtual haptic textures on a tangible surface directly touched with the finger are compared in a two-alternative forced choice (2AFC) task in three visual rendering conditions: (1) without visual augmentation, (2) with a realistic virtual hand rendering in \AR, and (3) with the same virtual hand in \VR.
-
-%\fig[1]{teaser/teaser2}{%
-%  Vibrotactile textures were rendered in real time on a real surface using a wearable vibrotactile device worn on the finger.
-%  %
-%  Participants explored this haptic roughness augmentation with (Real) their real hand alone, (Mixed) a realistic virtual hand overlay in \AR, and (Virtual) the same virtual hand in \VR.
-%}