erwan/phd-thesis
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Corrections related work

Browse Source
This commit is contained in:
Erwan Normand
2024-10-18 14:46:42 +02:00
parent 3563ba3cb1
commit 625892eb9c
13 changed files with 235 additions and 182 deletions
Download Patch File Download Diff File Expand all files Collapse all files

52
1-background/related-work/4-visuo-haptic-ar.tex
View File

@@ -66,7 +66,7 @@ The \MLE model implies that when seeing and touching a \VO in \AR, the combinati
\subsubsection{Influence of Visual Rendering on Haptic Perception}
\label{visual_haptic_influence}
Thus, a visuo-haptic perception of an object's property is robust to some difference between the two sensory modalities, as long as one can match their respective sensations to the same property.
Thus, a visuo-haptic perception of an object's property is robust to some differences between the two sensory modalities, as long as one can match their respective sensations to the same property.
In particular, the texture perception of objects is known to be constructed from both vision and touch \cite{klatzky2010multisensory}.
More precisely, when surfaces are evaluated by vision or touch alone, both senses discriminate their materials mainly by the same properties of roughness, hardness, and friction, and with similar performance \cite{bergmanntiest2007haptic,baumgartner2013visual,vardar2019fingertip}.
@@ -88,7 +88,7 @@ For example, in a fixed \VST-\AR screen (\secref{ar_displays}), by visually defo
\item A virtual soft texture projected on a table and that deforms when pressed by the hand \cite{punpongsanon2015softar}.
\item Modifying visually a real object and the hand touching it in \VST-\AR to modify its perceived shape \cite{ban2014displaying}.
]
\subfigsheight{42mm}
\subfigsheight{50mm}
\subfig{punpongsanon2015softar}
\subfig{ban2014displaying}
\end{subfigs}
@@ -112,18 +112,18 @@ Adding a visual delay increased the perceived stiffness of the reference piston,
\item Participant pressing a virtual piston rendered by a force-feedback device with their hand.
\item Proportion of comparison piston perceived as stiffer than reference piston (vertical axis) as a function of the comparison stiffness (horizontal axis) and visual and haptic delays of the reference (colors).
]
\subfig[.44]{knorlein2009influence_1}
\subfigbox[.44]{knorlein2009influence_1}
\subfig[.55]{knorlein2009influence_2}
\end{subfigs}
%explained how these delays affected the integration of the visual and haptic perceptual cues of stiffness.
The stiffness $\tilde{k}(t)$ of the piston is indeed estimated at time $t$ by both sight and proprioception as the ratio of the exerted force $F(t)$ and the displacement $D(t)$ of the piston, following \eqref{stiffness}, but with potential visual $\Delta t_v$ or haptic $\Delta t_h$ delays:
\begin{equation}{stiffness_delay}
\tilde{k}(t) = \frac{F(t + \Delta t_h)}{D(t + \Delta t_v)}
\end{equation}
The stiffness $\tilde{k}(t)$ of the piston is indeed estimated at time $t$ by both sight and proprioception as the ratio of the exerted force $F(t)$ and the displacement $\Delta L(t)$ of the piston, following \eqref{stiffness}, but with potential visual $\Delta t_v$ or haptic $\Delta t_h$ delays.
%\begin{equation}{stiffness_delay}
% \tilde{k}(t) = \frac{F(t + \Delta t_h)}{D(t + \Delta t_v)}
%\end{equation}
Therefore, the perceived stiffness $\tilde{k}(t)$ increases with a haptic delay in force and decreases with a visual delay in displacement \cite{diluca2011effects}.
In a similar \TIFC user study, participants compared perceived stiffness of virtual pistons in \OST-\AR and \VR \cite{gaffary2017ar}.
\textcite{gaffary2017ar} compared perceived stiffness of virtual pistons in \OST-\AR and \VR.
However, the force-feedback device and the participant's hand were not visible (\figref{gaffary2017ar}).
The reference piston was judged to be stiffer when seen in \VR than in \AR, without participants noticing this difference, and more force was exerted on the piston overall in \VR.
This suggests that the haptic stiffness of \VOs feels \enquote{softer} in an \AE than in a full \VE.
@@ -135,8 +135,8 @@ This suggests that the haptic stiffness of \VOs feels \enquote{softer} in an \AE
\item in \VR.
]
\subfig[0.35]{gaffary2017ar_1}
\subfig[0.3]{gaffary2017ar_3}
\subfig[0.3]{gaffary2017ar_4}
\subfigbox[0.31]{gaffary2017ar_3}
\subfigbox[0.31]{gaffary2017ar_4}
\end{subfigs}
Finally, \textcite{diluca2019perceptual} investigated the perceived simultaneity of visuo-haptic contact with a \VO in \VR.
@@ -183,16 +183,14 @@ Finally, \textcite{preechayasomboon2021haplets} (\figref{preechayasomboon2021hap
However, no proper user study has been conducted to evaluate these devices in \AR.
\begin{subfigs}{ar_wearable}{Nail-mounted wearable haptic devices designed for \AR. }[][
%\item A voice-coil rendering a virtual haptic texture on a real sheet of paper \cite{ando2007fingernailmounted}.
\item Touch\&Fold provide contact pressure and vibrations on demand to the fingertip \cite{teng2021touch}.
\item Fingeret is a finger-side wearable haptic device that pulls and pushs the fingertip skin \cite{maeda2022fingeret}.
\item Haplets is a compact nail device with integrated sensing and vibrotactile feedback \cite{preechayasomboon2021haplets}.
]
\subfigsheight{33mm}
%\subfig{ando2007fingernailmounted}
\subfig{teng2021touch_1}
\subfig{maeda2022fingeret}
\subfig{preechayasomboon2021haplets}
\subfigbox{teng2021touch_1}
\subfigbox{maeda2022fingeret}
\subfigbox{preechayasomboon2021haplets}
\end{subfigs}
\subsubsection{Belt Devices}
@@ -206,22 +204,21 @@ The middle phalanx of each of these fingers was equipped with a haptic ring of \
%However, no proper user study was conducted to evaluate this feedback.% on the manipulation of the cube.
%that simulated the weight of the cube.
%A virtual cube that could push on the cube was manipulated with the other hand through a force-feedback device.
\textcite{scheggi2010shape} report that 12 out of 15 participants found the weight haptic feedback essential to feeling the presence of the virtual cube.
\textcite{scheggi2010shape} reported that 12 out of 15 participants found the weight haptic feedback essential to feeling the presence of the virtual cube.
In a pick-and-place task in non-immersive \VST-\AR involving direct hand manipulation of both virtual and real objects (\figref{maisto2017evaluation}), \textcite{maisto2017evaluation} and \textcite{meli2018combining} compared the effects of providing haptic or visual feedback about fingertip-object contacts.
They compared the haptic ring of \textcite{pacchierotti2016hring} on the proximal phalanx, the moving platform of \textcite{chinello2020modular} on the fingertip, and a visual rendering of the tracked fingertips as virtual points.
They compared the haptic ring of \textcite{pacchierotti2016hring} on the proximal phalanx, the moving platform of \textcite{chinello2020modular} on the fingertip, and a visual feedback of the tracked fingertips as virtual points.
They showed that the haptic feedback improved the completion time, reduced the force exerted on the cubes compared to the visual feedback (\figref{visual-hands}).
The haptic ring was also perceived as more effective than the moving platform.
However, the measured difference in performance could be due to either the device or the device position (proximal vs fingertip), or both.
These two studies were also conducted in non-immersive setups, where users viewed a screen displaying the visual interactions, and only compared the haptic and visual rendering of the hand-object contacts, but did not examine them together.
These two studies were also conducted in non-immersive setups, where users viewed a screen displaying the visual interactions, and only compared the haptic and visual feedback of the hand-object contacts, but did not examine them together.
\begin{subfigs}{ar_rings}{Wearable haptic ring devices for \AR. }[][
\item Rendering weight of a virtual cube placed on a real surface \cite{scheggi2010shape}.
\item Rendering the contact force exerted by the fingers on a virtual cube \cite{maisto2017evaluation,meli2018combining}.
]
\subfigsheight{57mm}
\subfig{scheggi2010shape}
\subfig{maisto2017evaluation}
\subfigbox[.48][m]{scheggi2010shape}
\subfig[.48][m]{maisto2017evaluation}
\end{subfigs}
%\subsubsection{Wrist Bracelet Devices}
@@ -250,5 +247,14 @@ A user study was conducted in \VR to compare the perception of visuo-haptic stif
% \cite{sarac2022perceived,palmer2022haptic} not in \AR but studies on relocating to the wrist the haptic feedback of the fingertip-object contacts.
%\subsection{Conclusion}
%\label{visuo_haptic_conclusion}
\subsection{Conclusion}
\label{visuo_haptic_conclusion}
Providing coherent visuo-haptic feedback to enhance direct hand perception and manipulation with \VOs in immersive \AR is challenging.
While many wearable haptic devices have been developed and are capable of providing varied tactile feedback, few have be integrated or experimentally evaluated for direct hand interaction in \AR.
Their haptic end-effector must be moved away from the inside of the hand so as not to interfere with the user interaction with the \RE.
Different relocation strategies have been proposed for different parts of the hand, such as the nail, the index phalanges, or the wrist, but it remains unclear whether any of them are best suited for direct hand interaction in \AR.
In all cases, the real and virtual visual sensations are considered co-localized, but the virtual haptic feedback is not.
Such a discrepancy may affect the user's perception and experience and should be further investigated.
When integrating different sensory feedback, haptic and visual, real and virtual, into a single object property, perception is robust to variations in reliability and to spatial and temporal differences.
Conversely, the same haptic feedback or augmentation can be influenced by the user's visual expectation or the visual rendering of the \VO.