erwan/phd-thesis
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

WIP related work

Browse Source
This commit is contained in:
Erwan Normand
2024-09-20 16:50:31 +02:00
parent 5eba5f75d9
commit 6daf361654
7 changed files with 83 additions and 34 deletions
Download Patch File Download Diff File Expand all files Collapse all files

81
1-introduction/related-work/2-wearable-haptics.tex
View File

@@ -2,7 +2,7 @@
\label{wearable_haptics}
One of the roles of haptic systems is to render virtual interactions and sensations that are \emph{similar and comparable} to those experienced by the haptic sense with real objects, particularly in \v-\VE~\cite{maclean2008it,culbertson2018haptics}.
Moreover, a haptic \AR system should \enquote{modulating the feel of a real object by virtual [haptic] feedback}~\cite{jeon2009haptic}, \ie a touch interaction with a real object whose perception is modified by the addition of virtual haptic feedback.
Moreover, a haptic augmentation system should \enquote{modulating the feel of a real object by virtual [haptic] feedback}~\cite{jeon2009haptic}, \ie a touch interaction with a real object whose perception is modified by the addition of virtual haptic feedback.
The haptic system should be hand-held or worn, \eg on the hand, and \enquote{not permanently attached to or integrated in the object}~\cite{bhatia2024augmenting}.
@@ -164,13 +164,16 @@ Several types of vibrotactile actuators are used in haptics, with different trad
\label{tactile_rendering}
Tactile rendering of haptic properties consists in modelling and reproducing virtual tactile sensations comparable to those perceived when interacting with real objects~\cite{klatzky2013haptic}.
By adding such tactile rendering as feedback to the touch actions of the hand on a real object~\cite{bhatia2024augmenting}, both the real and virtual haptic sensations are integrated into a single property perception, as presented in \secref{sensations_perception}.
Therefore, the visual rendering of a touched object can also greatly influence the perception of its haptic properties, \eg by modifying its visual texture in \AR or \VR, as discussed in the \secref{visuo_haptic}.
By adding such tactile rendering as feedback to the touch actions of the hand on a real object~\cite{bhatia2024augmenting}, the perception of the object's haptic property is modified.
The integration of the real and virtual haptic sensations into a single property perception is discussed in more details in \secref{sensations_perception}.
%, both the real and virtual haptic sensations are integrated into a single property perception, as presented in \secref{sensations_perception}, \ie the perceived haptic property is modulated by the added virtual feedback.
In particular, the visual rendering of a touched object can also greatly influence the perception of its haptic properties, \eg by modifying its visual texture in \AR or \VR, as discussed in the \secref{visuo_haptic}.
\textcite{bhatia2024augmenting} categorize the tactile augmentations of real objects into three types: direct touch, touch-through, and tool mediated.
\textcite{bhatia2024augmenting} categorize the haptic augmentations into three types: direct touch, touch-through, and tool mediated.
Also called direct feel-through~\cite{jeon2015haptic}, in \emph{direct touch}, the haptic device does not cover the interior of the hand to not impair the user to interact with the \RE.
In touch-through and tool-mediated, or \emph{indirect feel-through}, the haptic device is interposed between the hand and the \RE or worn on the hand, respectively.
We are interested in direct touch augmentations with wearable haptic devices (\secref{wearable_haptic_devices}), as their integration with \AR is particularly promising for direct hand interaction with visuo-haptic augmentations.
%We are interested in direct touch augmentations with wearable haptics (\secref{wearable_haptic_devices}), as their integration with \AR is particularly promising for free hand interaction with visuo-haptic augmentations.
Many haptic augmentations were first developed with grounded haptic devices and later transposed to wearable haptic devices.
%We also focus on tactile augmentations stimulating the mechanoreceptors of the skin (\secref{haptic_sense}), thus excluding temperature perception, as they are the most common existing haptic interfaces.
% \cite{klatzky2003feeling} : rendering roughness, friction, deformation, temperatures
@@ -180,43 +183,75 @@ We are interested in direct touch augmentations with wearable haptic devices (\s
\label{texture_rendering}
Several approaches have been proposed to render virtual haptic texture~\cite{culbertson2018haptics}.
High-fidelity force feedback devices can reproduce patterned textures with great precision and provide similar perceptions to real textures, but they are expensive, have a limited workspace, and impose to hold a probe to explore the texture~\cite{unger2011roughness}.
As more traditional force feedback systems are unable to accurately render such micro-details on a simulated surface, vibrotactile devices attached to the end effector instead generate vibrations to simulate interaction with the virtual texture~\cite{culbertson2018haptics}.
As more traditional force feedback systems are unable to accurately render such micro-details on a simulated surface, vibrotactile devices attached to the end effector instead generate vibrations to simulate interaction with the virtual texture~\cite{campion2005fundamental,culbertson2018haptics}.
In this way, physics-based models~\cite{chan2021hasti,okamura1998vibration,guruswamy2011iir} and data-based models~\cite{culbertson2015should,romano2010automatic} have been developed and evaluated, the former being simpler but more approximate to real textures, and the latter being more realistic but limited to the captured textures.
\paragraph{Physics-based Models}
High-fidelity force feedback devices can reproduce patterned textures with great precision and provide similar perceptions to real textures, but they are expensive, have a limited workspace, and impose to hold a probe to explore the texture~\cite{unger2011roughness}.
Notably, \textcite{okamura1998vibration} rendered grating textures with exponentially decaying sinudoids that simulated the strokes of the grooves and ridges of the surface, while \textcite{culbertson2014modeling} captured and modelled the roughness of real surfaces to render them using the speed and force of the user.
\paragraph{Data-driven Models}
An effective approach to rendering virtual roughness is to generate vibrations to simulate interaction with the virtual texture~\cite{culbertson2018haptics}, relying on the user's real-time measurements of position, velocity and force to modulate the frequencies and amplitudes of the vibrations, with position and velocity being the most important parameters~\cite{culbertson2015should}.
For example, when comparing the same virtual texture pairwise, but with different parameters, \textcite{culbertson2015should} showed that the roughness vibrations generated should vary with user speed, but not necessarily with user force.
Virtual data-driven textures were perceived as similar to real textures, except for friction, which was not rendered properly.
The perceived roughness of real surfaces can be then modified when touched by a tool with a vibrotactile actuator attached~\cite{culbertson2014modeling,ujitoko2019modulating} or directly with the finger wearing the vibrotactile actuator~\cite{asano2015vibrotactile}, creating a haptic texture augmentation.
The objective is not just to render a virtual texture, but to alter the perception of a real, tangible surface, usually with wearable haptic devices, in what is known as haptic augmented reality (HAR)~\cite{bhatia2024augmenting,jeon2009haptic}.
The perceived roughness of real surfaces can be then modified when touched by a tool with a vibrotactile actuator attached~\cite{culbertson2014modeling,ujitoko2019modulating} or directly with the finger wearing the vibrotactile actuator~\cite{asano2015vibrotactile}, creating a haptic texture augmentation.
The objective is not just to render a virtual texture, but to alter the perception of a real, tangible surface, usually with wearable haptic devices, in what is known as haptic augmented reality (HAR)~\cite{bhatia2024augmenting,jeon2009haptic}.
One additional challenge of augmenting the finger touch is to keep the fingertip free to touch the real environment, thus delocalizing the actuator elsewhere on the hand~\cite{ando2007fingernailmounted,friesen2024perceived,teng2021touch}.
Of course, the fingertip skin is not deformed by the virtual texture and only vibrations are felt, but it has been shown that the vibrations produced on the fingertip skin running over a real surface are texture specific and similar between individuals~\cite{manfredi2014natural}.
Of course, the fingertip skin is not deformed by the virtual texture and only vibrations are felt, but it has been shown that the vibrations produced on the fingertip skin running over a real surface are texture specific and similar between individuals~\cite{delhaye2012textureinduced,manfredi2014natural}.
A common method vibrotactile rendering of texture is to use a sinusoidal signal whose frequency is modulated by the finger position or velocity~\cite{asano2015vibrotactile,friesen2024perceived,strohmeier2017generating,ujitoko2019modulating}.
\subsubsection{Hardness}
\label{hardness_rendering}
Modulating the perceived stiffness $k$ of a real surface with a force-feedback device
\cite{jeon2008modulating,jeon2010stiffness,jeon2012extending}
The two main approaches to modulate the perceived hardness of a real surface with wearable haptics are to render forces or vibrations.
\paragraph{Modulating Forces}
When tapping or pressing a real object with a tool, the perceived stiffness $\tilde{k}$ (\secref{hardness}) of its surface can be modulated with force feedback~\cite{jeon2015haptic}.
This was first proposed by \textcite{jeon2008modulating} who augmented a real surface tapped in 1 \DoF with a grounded force-feedback device held in hand (\figref{jeon2009haptic_1}).
When the haptic end-effector contacts the object at time $t$, the object's surface deforms by displacement $x_r(t)$ and opposes a real reaction force $f_r(t)$.
The virtual force of the device $\tilde{f_r}(t)$ is then controlled to:
\begin{equation}{stiffness_augmentation}
\tilde{f_r}(t) = f_r(t) - \tilde{k} x_r(t)
\end{equation}
A force sensor embedded in the device measures the reaction force $f_r(t)$.
The displacement $x_r(t)$ is estimated with the reaction force and tapping velocity using a pre-defined model of various materials, as described by \textcite{jeon2011extensions}.
As shown in \figref{jeon2009haptic_2}, the force $\tilde{f_r}(t)$ perceived by the user being modulated, but not the displacement $x_r(t)$, the perceived stiffness is $\tilde{k}(t)$.
This stiffness augmentation technique was then extended to enable tapping and pressing with 3 \DoFs~\cite{jeon2010stiffness}, to render friction and weight augmentations~\cite{jeon2011extensions}, and to grasping and squeezing the real object with two contact points~\cite{jeon2012extending}.
\begin{subfigs}{stiffness_rendering}{Augmenting perceived stiffness of a real surface. }[%
\item Diagram of a user tapping a real surface with a hand-held force-feedback device~\cite{jeon2009haptic}.
\item Displacement-force curves of a real rubber ball (dashed line) and when its perceived stiffness $\tilde{k}$ is modulated~\cite{jeon2009haptic}.
]
\subfigsheight{35mm}
\subfig[0.2]{jeon2009haptic_1}
\subfig[0.4]{jeon2009haptic_2}
\end{subfigs}
\cite{detinguy2018enhancing}
\cite{salazar2020altering}
\cite{kildal20103dpress}
\cite{tao2021altering} % wearable softness
\paragraph{Vibrations Augmentations}
The second main approach is to modulate the vibrations felt when tapping a real surface with a tool~\cite{okamura1998vibration}.
Tapping with a tool on a real surface augmented with a vibrotactile actuator generating exponential decaying sinusoids
\begin{equation}{contact_transient}
\tilde{f}_c(t) = a \, |v_{in}| \, e^{- \tau t} sin(2 \pi f t)
\end{equation}
\cite{kuchenbecker2006improving}
\cite{hachisu2012augmentation}
\cite{park2019realistic}
\cite{park2023perceptual}
Comparing the two previous methods
\cite{choi2021perceived}
With wearable haptics
\cite{kildal20103dpress}
\cite{detinguy2018enhancing}
\cite{salazar2020altering}
\cite{park2017compensation}
\cite{tao2021altering} % wearable softness
%\textcite{choi2021perceived} combined and compared these two rendering approaches (spring-damper and exponential decaying sinusoids) but to render purely virtual surfaces.
%They found that the perceived intensity of the virtual hardness $\tilde{h}$ followed a power law, similarly to \eqref{hardness_intensity}, with the amplitude $a$, the %frequency $f$ and the damping $b$ of the vibration, but not the decay time $\tau$.
%\subsubsection{Friction}
%\label{friction_rendering}