This commit is contained in:
2024-09-22 15:28:36 +02:00
parent 363108d03e
commit ee2b739ddb
30 changed files with 502 additions and 473 deletions

View File

@@ -4,31 +4,31 @@
\subsection{Haptics in AR}
As in VR, the addition of haptic feedback in AR has been explored through numerous approaches, including %
grounded force feedback devices~\cite{jeon2009haptic,knorlein2009influence,hachisu2012augmentation,gaffary2017ar}, %
exoskeletons~\cite{lee2021wearable}, %
wearable haptic devices~\cite{maisto2017evaluation,detinguy2018enhancing,lopes2018adding,meli2018combining,pezent2022design,teng2021touch},
tangible objects~\cite{punpongsanon2015softar,hettiarachchi2016annexing,kahl2021investigation}, and %
mid-air haptics~\cite{ochiai2016crossfield}. %
grounded force feedback devices \cite{jeon2009haptic,knorlein2009influence,hachisu2012augmentation,gaffary2017ar}, %
exoskeletons \cite{lee2021wearable}, %
wearable haptic devices \cite{maisto2017evaluation,detinguy2018enhancing,lopes2018adding,meli2018combining,pezent2022design,teng2021touch},
tangible objects \cite{punpongsanon2015softar,hettiarachchi2016annexing,kahl2021investigation}, and %
mid-air haptics \cite{ochiai2016crossfield}. %
%
Most have been used to provide haptic feedback to virtual objects.
%
While this may seem similar to haptic feedback in VR, there are significant differences in terms of perception, as in AR the real world and the hand of the user remain visible, but also because the virtual content may be less realistic or inconsistent with the real world~\cite{kim2018revisiting,macedo2023occlusion}.
While this may seem similar to haptic feedback in VR, there are significant differences in terms of perception, as in AR the real world and the hand of the user remain visible, but also because the virtual content may be less realistic or inconsistent with the real world \cite{kim2018revisiting,macedo2023occlusion}.
%
Indeed, the same haptic stimuli can be perceived differently in AR and VR, \eg the perceived stiffness of a piston seemed higher in AR than in VR~\cite{gaffary2017ar} or was altered in the presence of a delay between the haptic and visual feedback~\cite{knorlein2009influence}.
Indeed, the same haptic stimuli can be perceived differently in AR and VR, \eg the perceived stiffness of a piston seemed higher in AR than in VR \cite{gaffary2017ar} or was altered in the presence of a delay between the haptic and visual feedback \cite{knorlein2009influence}.
%
It might be therefore interesting to study how haptic and visual augmentations of textures of tangible surfaces are perceived in AR.
An additional challenge in AR is to let the hand of the user free to touch, feel, and interact with the real objects~\cite{maisto2017evaluation,detinguy2018enhancing,teng2021touch}.
An additional challenge in AR is to let the hand of the user free to touch, feel, and interact with the real objects \cite{maisto2017evaluation,detinguy2018enhancing,teng2021touch}.
%
For example, mounted on the nail, the haptic device of \textcite{teng2021touch} can be quickly unfolded on demand to the fingertip to render haptic feedback of virtual objects.
%
It is however not suitable for rendering haptic feedback when touching real objects.
%
In this respect, some wearable haptic devices were specifically designed to provide haptic feedback about fingertip interactions with the virtual content, but delocalized elsewhere on the body: on the proximal finger phalanx with the hRing haptic ring device~\cite{pacchierotti2016hring,ferro2023deconstructing}, on the wrist with the Tasbi bracelet~\cite{pezent2022design}, or on the arm~\cite{lopes2018adding}.
In this respect, some wearable haptic devices were specifically designed to provide haptic feedback about fingertip interactions with the virtual content, but delocalized elsewhere on the body: on the proximal finger phalanx with the hRing haptic ring device \cite{pacchierotti2016hring,ferro2023deconstructing}, on the wrist with the Tasbi bracelet \cite{pezent2022design}, or on the arm \cite{lopes2018adding}.
%
Compared to a fingertip worn device, the hRing was even preferred by participants and perceived as more effective in virtual object manipulation task in AR~\cite{maisto2017evaluation,meli2018combining}.
Compared to a fingertip worn device, the hRing was even preferred by participants and perceived as more effective in virtual object manipulation task in AR \cite{maisto2017evaluation,meli2018combining}.
%
This device has been then taken further to alter cutaneous perception of touched tangible objects in VR and AR~\cite{detinguy2018enhancing,salazar2020altering}: by providing normal and shear forces to the proximal phalanx skin in a timely manner, the perceived stiffness, softness, slipperiness, and local deformations (bumps and holes) of the touched tangible object were augmented.
This device has been then taken further to alter cutaneous perception of touched tangible objects in VR and AR \cite{detinguy2018enhancing,salazar2020altering}: by providing normal and shear forces to the proximal phalanx skin in a timely manner, the perceived stiffness, softness, slipperiness, and local deformations (bumps and holes) of the touched tangible object were augmented.
%
However, wearable haptic devices have not yet been used in AR to modify the texture perception of a tangible surface.
@@ -71,15 +71,15 @@ However, wearable haptic devices have not yet been used in AR to modify the text
Many approaches have been used to generate realistic haptic virtual textures.
%
Ultrasonic vibrating screens are capable of modulating their friction~\cite{rekik2017localized,ito2019tactile}, but their use in AR is limited.
Ultrasonic vibrating screens are capable of modulating their friction \cite{rekik2017localized,ito2019tactile}, but their use in AR is limited.
%
By simulating the roughness of a surface instead, force feedback devices can reproduce perceptions of patterned textures identical to those of real textures~\cite{unger2011roughness}, but they are expensive and have a limited workspace.
By simulating the roughness of a surface instead, force feedback devices can reproduce perceptions of patterned textures identical to those of real textures \cite{unger2011roughness}, but they are expensive and have a limited workspace.
%
An alternative is to reproduce the vibrations that occur when a tool or the finger is moved across a surface using a vibrotactile device attached to a hand-held tool~\cite{culbertson2018haptics}.
An alternative is to reproduce the vibrations that occur when a tool or the finger is moved across a surface using a vibrotactile device attached to a hand-held tool \cite{culbertson2018haptics}.
%
Several physical models have been proposed to represent such vibrations~\cite{okamura1998vibration,guruswamy2011iir,chan2021hasti}.
Several physical models have been proposed to represent such vibrations \cite{okamura1998vibration,guruswamy2011iir,chan2021hasti}.
%
However, as they can be difficult to tune, measurement-based models have been developed to record, model, and render these vibrations~\cite{culbertson2014modeling,culbertson2017ungrounded}.
However, as they can be difficult to tune, measurement-based models have been developed to record, model, and render these vibrations \cite{culbertson2014modeling,culbertson2017ungrounded}.
%
In this work, we employed such data-driven haptic models to augment and studied the visuo-haptic texture perception of tangible surfaces in AR.%\CP{Here the original sentence was: ``We use these data-driven haptic models to augment [...].''. It was not clear what ``we use'' meant. Check that the new sentence is correct.}
@@ -91,9 +91,9 @@ Similarly, \textcite{culbertson2014modeling} compared the similarity of all poss
%
Virtual data-driven textures were perceived as similar to real textures, except for friction, which was not rendered properly.
%
For grating textures, an arbitrary roughness rating is used to determine a psycho-physical curve as a function of pattern spacing~\cite{unger2011roughness,asano2015vibrotactile,degraen2019enhancing}.
For grating textures, an arbitrary roughness rating is used to determine a psycho-physical curve as a function of pattern spacing \cite{unger2011roughness,asano2015vibrotactile,degraen2019enhancing}.
%
Another common method is to identify a given haptic texture among visual representations of all haptic textures~\cite{ando2007fingernailmounted,rekik2017localized,degraen2019enhancing,chan2021hasti}.
Another common method is to identify a given haptic texture among visual representations of all haptic textures \cite{ando2007fingernailmounted,rekik2017localized,degraen2019enhancing,chan2021hasti}.
%
In this user study, participants matched the pairs of visual and haptic textures they find most coherent and ranked the textures according to their perceived roughness.
%\CP{Do you refer to the one in our paper? Not super clear.}
@@ -102,7 +102,7 @@ A few studies have explored vibrotactile haptic devices worn directly on the fin
%
\textcite{ando2007fingernailmounted} mounted a vibrotactile actuator on the index nail, which generated impulse vibrations to render virtual edges and gaps on a real surface.
%
%This rendering method was compared later to providing the vibrations with pressure directly on the fingertip in AR and was found more realistic to render virtual objects and textures~\cite{teng2021touch}.
%This rendering method was compared later to providing the vibrations with pressure directly on the fingertip in AR and was found more realistic to render virtual objects and textures \cite{teng2021touch}.
%
%Covering the fingertip is however not suitable for rendering haptic feedback when touching real objects.
%
@@ -123,13 +123,13 @@ The well-known phycho-physical model of \textcite{ernst2002humans} established t
%
This effect has been used to alter the texture perception in AR and VR.
%
For example, superimposed virtual visual opaque textures on real surfaces in AR can be perceived as coherent together even though they have very different roughnesses~\cite{kitahara2010sensory}.
For example, superimposed virtual visual opaque textures on real surfaces in AR can be perceived as coherent together even though they have very different roughnesses \cite{kitahara2010sensory}.
%
\textcite{fradin2023humans} explored this effect further, finding that a superimposed AR visual texture slightly different from a colocalized haptic texture affected the ability to recognize the haptic texture.
%
Similarly, \textcite{punpongsanon2015softar} altered the softness perception of a tangible surface using AR-projected visual textures whereas \textcite{chan2021hasti} evaluated audio-haptic texture perception in VR.
%
Conversely, colocalized 3D-printed real hair structures were able to correctly render several virtual visual textures seen in VR in terms of haptic hardness and roughness~\cite{degraen2019enhancing}.
Conversely, colocalized 3D-printed real hair structures were able to correctly render several virtual visual textures seen in VR in terms of haptic hardness and roughness \cite{degraen2019enhancing}.
%
This study investigated how virtual roughness haptic texture can be used to enhance touched real surfaces augmented with visual AR textures.
%Dans cet article, les textures haptiques sont senties co-localisées avec des textures visuelles