erwan/phd-thesis
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Clean related work

Browse Source
This commit is contained in:
Erwan Normand
2024-09-24 11:49:30 +02:00
parent 6d8203fee0
commit 9ba5d344a5
3 changed files with 144 additions and 160 deletions
Download Patch File Download Diff File Expand all files Collapse all files

183
1-introduction/related-work/2-wearable-haptics.tex
View File

@@ -2,10 +2,12 @@
\label{wearable_haptics}
One of the roles of haptic systems is to render virtual interactions and sensations that are \emph{similar and comparable} to those experienced by the haptic sense with real objects, particularly in a visual \VE \cite{maclean2008it,culbertson2018haptics}.
Due to the high complexity of the haptic sense and the variety of sensations it can feel, haptic actuators and renderings are designed to only address a subset of these sensations.
While it is challenging to create a realistic haptic experience, it is more important to provide the right sensory stimulus \enquote{at the right moment and at the right place} \cite{hayward2007it}.
Moreover, a haptic augmentation system should \enquote{modulating the feel of a real object by virtual [haptic] feedback} \cite{jeon2009haptic}, \ie a touch interaction with a real object whose perception is modified by the addition of virtual haptic feedback.
The haptic system should be hand-held or worn, \eg on the hand, and \enquote{not permanently attached to or integrated in the object} \cite{bhatia2024augmenting}.
\subsection{Level of Wearability}
\label{wearability_level}
@@ -14,16 +16,16 @@ Different types of haptic devices can be worn on the hand, but only some of them
An increasing wearability resulting in the loss of the system's kinesthetic feedback capability.
\begin{subfigs}{pacchierotti2017wearable}{
Schematic wearability level of haptic devices for the hand \cite{pacchierotti2017wearable}.
}[
\item World-grounded haptic devices are fixed on the environment to provide kinesthetic feedback to the user.
\item Exoskeletons are body-grounded kinesthetic devices.
\item Wearable haptic devices are grounded on the point of application of the tactile stimulus.
]
\subfigsheight{34mm}
\subfig{pacchierotti2017wearable_1}
\subfig{pacchierotti2017wearable_2}
\subfig{pacchierotti2017wearable_3}
Schematic wearability level of haptic devices for the hand \cite{pacchierotti2017wearable}.
}[
\item World-grounded haptic devices are fixed on the environment to provide kinesthetic feedback to the user.
\item Exoskeletons are body-grounded kinesthetic devices.
\item Wearable haptic devices are grounded on the point of application of the tactile stimulus.
]
\subfigsheight{34mm}
\subfig{pacchierotti2017wearable_1}
\subfig{pacchierotti2017wearable_2}
\subfig{pacchierotti2017wearable_3}
\end{subfigs}
Haptic research comes from robotics and teleoperation, and historically led to the design of haptic systems that are \emph{world-grounded} to an external support in the environment, such as a table (\figref{pacchierotti2017wearable_1}).
@@ -40,21 +42,20 @@ An approach is then to move the grounding point very close to the end-effector (
Moreover, as detailed in \secref{object_properties}, cutaneous sensations are necessary and often sufficient for the perception of the haptic properties of an object explored with the hand, as also argued by \textcite{pacchierotti2017wearable}.
\begin{subfigs}{grounded_to_wearable}{
Haptic devices for the hand with different wearability levels.
}[
\item Teleoperation of a virtual cube grasped with the thumb and index fingers each attached to a grounded haptic device \cite{pacchierotti2015cutaneous}.
\item A passive exoskeleton for fingers simulating stiffness of a trumpet's pistons \cite{achibet2017flexifingers}.
\item Manipulation of a virtual cube with the thumb and index fingers each attached with the 3-RSR wearable haptic device \cite{leonardis20173rsr}.
]
\subfigsheight{38mm}
\subfig{pacchierotti2015cutaneous}
\subfig{achibet2017flexifingers}
\subfig{leonardis20173rsr}
Haptic devices for the hand with different wearability levels.
}[
\item Teleoperation of a virtual cube grasped with the thumb and index fingers each attached to a grounded haptic device \cite{pacchierotti2015cutaneous}.
\item A passive exoskeleton for fingers simulating stiffness of a trumpet's pistons \cite{achibet2017flexifingers}.
\item Manipulation of a virtual cube with the thumb and index fingers each attached with the 3-RSR wearable haptic device \cite{leonardis20173rsr}.
]
\subfigsheight{38mm}
\subfig{pacchierotti2015cutaneous}
\subfig{achibet2017flexifingers}
\subfig{leonardis20173rsr}
\end{subfigs}
% Tradeoff realistic and cost + analogy with sound, Hi-Fi costs a lot and is realistic, but 40$ BT headphone is more practical and enough, as cutaneous feedback without kinesthesic could be enough for wearable haptics and far more affordable and comfortable than world- or body-grounded haptics + cutaneous even better than kine for rendering surface curvature and fine manipulation
\subsection{Wearable Haptic Devices for the Hand}
\label{wearable_haptic_devices}
@@ -82,18 +83,18 @@ Multiple membranes are often used in a grid to simulate edges and textures, as i
Although these two types of effector can be considered wearable, their actuation requires a high level of mechanical and electronic complexity that makes the system as a whole not portable.
\begin{subfigs}{normal_actuators}{
Normal indentation actuators for the fingertip.
}[
\item A moving platform actuated with cables \cite{gabardi2016new}.
\item A moving platform actuated by articulated limbs \cite{perez2017optimizationbased}.
\item Diagram of a pin-array of tactors \cite{sarakoglou2012high}.
\item A pneumatic system composed of a \numproduct{12 x 10} array of air cylinders \cite{ujitoko2020development}.
]
\subfigsheight{37mm}
\subfig{gabardi2016new}
\subfig{perez2017optimizationbased}
\subfig{sarakoglou2012high}
\subfig{ujitoko2020development}
Normal indentation actuators for the fingertip.
}[
\item A moving platform actuated with cables \cite{gabardi2016new}.
\item A moving platform actuated by articulated limbs \cite{perez2017optimizationbased}.
\item Diagram of a pin-array of tactors \cite{sarakoglou2012high}.
\item A pneumatic system composed of a \numproduct{12 x 10} array of air cylinders \cite{ujitoko2020development}.
]
\subfigsheight{37mm}
\subfig{gabardi2016new}
\subfig{perez2017optimizationbased}
\subfig{sarakoglou2012high}
\subfig{ujitoko2020development}
\end{subfigs}
\subsubsection{Tangential Motion Actuators}
@@ -112,17 +113,17 @@ Conversely, by turning simultaneously in the same direction, the belt pulls on t
The simplicity of this approach allows the belt to be placed anywhere on the hand, leaving the fingertip free to interact with the \RE, \eg the hRing on the proximal phalanx in \figref{pacchierotti2016hring} \cite{pacchierotti2016hring} or Tasbi on the wrist in \figref{pezent2022design} \cite{pezent2022design}.
\begin{subfigs}{tangential_belts}{Tangential motion actuators and compression belts. }[
\item A skin strech actuator for the fingertip \cite{leonardis2015wearable}.
\item A 3 \DoF actuator capable of normal and tangential motion on the fingertip \cite{schorr2017fingertip}.
%\item A shearing belt actuator for the fingertip \cite{minamizawa2007gravity}.
\item The hRing, a shearing belt actuator for the proximal phalanx of the finger \cite{pacchierotti2016hring}.
\item Tasbi, a wristband capable of pressure and vibrotactile feedback \cite{pezent2022design}.
]
\subfigsheight{33.5mm}
\subfig{leonardis2015wearable}
\subfig{schorr2017fingertip}
\subfig{pacchierotti2016hring}
\subfig{pezent2022design}
\item A skin strech actuator for the fingertip \cite{leonardis2015wearable}.
\item A 3 \DoF actuator capable of normal and tangential motion on the fingertip \cite{schorr2017fingertip}.
%\item A shearing belt actuator for the fingertip \cite{minamizawa2007gravity}.
\item The hRing, a shearing belt actuator for the proximal phalanx of the finger \cite{pacchierotti2016hring}.
\item Tasbi, a wristband capable of pressure and vibrotactile feedback \cite{pezent2022design}.
]
\subfigsheight{33.5mm}
\subfig{leonardis2015wearable}
\subfig{schorr2017fingertip}
\subfig{pacchierotti2016hring}
\subfig{pezent2022design}
\end{subfigs}
\subsubsection{Vibrotactile Actuators}
@@ -137,12 +138,12 @@ Several types of vibrotactile actuators are used in haptics, with different trad
An \ERM is a \DC motor that rotates an off-center mass when a voltage or current is applied (\figref{precisionmicrodrives_erm}). \ERMs are easy to control, inexpensive and can be encapsulated in a few millimeters cylinder or coin form factor. However, they have only one \DoF because both the frequency and amplitude of the vibration are coupled to the speed of the rotation, \eg low (high) frequencies output at low (high) amplitudes, as shown on \figref{precisionmicrodrives_erm_performances}.
\begin{subfigs}{erm}{Diagram and performance of \ERMs. }[
\item Diagram of a cylindrical encapsulated \ERM. From Precision Microdrives~\footnotemark.
\item Amplitude and frequency output of an \ERM as a function of the input voltage.
]
\subfigsheight{45mm}
\subfig{precisionmicrodrives_erm}
\subfig{precisionmicrodrives_erm_performances}
\item Diagram of a cylindrical encapsulated \ERM. From Precision Microdrives~\footnotemark.
\item Amplitude and frequency output of an \ERM as a function of the input voltage.
]
\subfigsheight{45mm}
\subfig{precisionmicrodrives_erm}
\subfig{precisionmicrodrives_erm_performances}
\end{subfigs}
\footnotetext{\url{https://www.precisionmicrodrives.com/}}
@@ -158,15 +159,14 @@ They are very small and thin and provide two \DoFs of amplitude and frequency co
However, they require high voltages to operate, limiting their use in wearable devices.
\begin{subfigs}{lra}{Diagram and performance of \LRAs. }[
\item Diagram. From Precision Microdrives~\footnotemarkrepeat.
\item Force generated by two \LRAs as a function of sinusoidal wave input with different frequencies: both their maximum force and resonant frequency are different \cite{azadi2014vibrotactile}.
]
\subfigsheight{50mm}
\subfig{precisionmicrodrives_lra}
\subfig{azadi2014vibrotactile}
\item Diagram. From Precision Microdrives~\footnotemarkrepeat.
\item Force generated by two \LRAs as a function of sinusoidal wave input with different frequencies: both their maximum force and resonant frequency are different \cite{azadi2014vibrotactile}.
]
\subfigsheight{50mm}
\subfig{precisionmicrodrives_lra}
\subfig{azadi2014vibrotactile}
\end{subfigs}
\subsection{Modifying Perceived Haptic Roughness and Hardness}
\label{tactile_rendering}
@@ -202,7 +202,7 @@ There are two main approaches to modify virtual textures perception: \emph{simul
The simplest texture simulation model is a 1D sinusoidal grating $v(t)$ with spatial period $\lambda$ and amplitude $A$ that is scanned by the user at velocity $\dot{x}(t)$:
\begin{equation}{grating_rendering}
v(t) = A \sin(\frac{2 \pi \dot{x}(t)}{\lambda})
v(t) = A \sin(\frac{2 \pi \dot{x}(t)}{\lambda})
\end{equation}
That is, this model generates a periodic signal whose frequency is modulated and proportional to the user's velocity, implementing the speed-frequency ratio observed with real patterned textures (\eqref{grating_vibrations}).
It gives the user the illusion of a texture with a \emph{fixed spatial period} that approximate the real manufactured grating textures (\secref{roughness}).
@@ -239,19 +239,18 @@ When comparing real textures felt through a stylus with their virtual models ren
\textcite{culbertson2015should} further showed that the perceived realism of the virtual textures, and similarity to the real textures, depended mostly on the user's speed but not on the user's force as inputs to the model, \ie responding to speed is sufficient to render isotropic virtual textures.
\begin{subfigs}{textures_rendering_data}{Augmentating haptic texture perception with voice-coil actuators. }[
\item Increasing and decreasing the perceived roughness of a real patterned texture in direct touch \cite{asano2015vibrotactile}.
\item Comparing real patterned texture with virtual texture augmentation in direct touch \cite{friesen2024perceived}.
\item Rendering virtual contacts in direct touch with the virtual texture \cite{ando2007fingernailmounted}.
\item Rendering an isotropic virtual texture over a real surface while sliding a hand-held stylus across it \cite{culbertson2012refined}.
]
\subfigsheight{36mm}
\subfig{asano2015vibrotactile_2}
\subfig{friesen2024perceived}
\subfig{ando2007fingernailmounted}
\subfig{culbertson2012refined}
\item Increasing and decreasing the perceived roughness of a real patterned texture in direct touch \cite{asano2015vibrotactile}.
\item Comparing real patterned texture with virtual texture augmentation in direct touch \cite{friesen2024perceived}.
\item Rendering virtual contacts in direct touch with the virtual texture \cite{ando2007fingernailmounted}.
\item Rendering an isotropic virtual texture over a real surface while sliding a hand-held stylus across it \cite{culbertson2012refined}.
]
\subfigsheight{36mm}
\subfig{asano2015vibrotactile_2}
\subfig{friesen2024perceived}
\subfig{ando2007fingernailmounted}
\subfig{culbertson2012refined}
\end{subfigs}
\subsubsection{Hardness}
\label{hardness_rendering}
@@ -264,7 +263,7 @@ This was first proposed by \textcite{jeon2008modulating} who augmented a real su
When the haptic end-effector contacts the object at time $t$, the object's surface deforms by displacement $x_r(t)$ and opposes a real reaction force $f_r(t)$.
The virtual force of the device $\tilde{f_r}(t)$ is then controlled to:
\begin{equation}{stiffness_augmentation}
\tilde{f_r}(t) = f_r(t) - \tilde{k} x_r(t)
\tilde{f_r}(t) = f_r(t) - \tilde{k} x_r(t)
\end{equation}
A force sensor embedded in the device measures the reaction force $f_r(t)$.
The displacement $x_r(t)$ is estimated with the reaction force and the tapping velocity using a predefined model of different materials as described in \textcite{jeon2011extensions}.
@@ -272,11 +271,11 @@ As shown in \figref{jeon2009haptic_2}, the force $\tilde{f_r}(t)$ perceived by t
This stiffness augmentation technique was then extended to allow tapping and pressing with 3 \DoFs \cite{jeon2010stiffness}, to render friction and weight augmentations \cite{jeon2011extensions}, and to grasp and squeez the real object with two contact points \cite{jeon2012extending}.
\begin{subfigs}{stiffness_rendering_grounded}{Augmenting the perceived stiffness of a real surface with a hand-held force-feedback device. }[%
\item Diagram of a user tapping the surface \cite{jeon2009haptic}.
\item Displacement-force curves of a real rubber ball (dashed line) and when its perceived stiffness $\tilde{k}$ is modulated \cite{jeon2009haptic}.
]
\subfig[0.38]{jeon2009haptic_1}
\subfig[0.42]{jeon2009haptic_2}
\item Diagram of a user tapping the surface \cite{jeon2009haptic}.
\item Displacement-force curves of a real rubber ball (dashed line) and when its perceived stiffness $\tilde{k}$ is modulated \cite{jeon2009haptic}.
]
\subfig[0.38]{jeon2009haptic_1}
\subfig[0.42]{jeon2009haptic_2}
\end{subfigs}
\textcite{detinguy2018enhancing} transposed this stiffness augmentation technique with the hRing device (\secref{belt_actuators}): While pressing a real piston with the fingertip by displacement $x_r(t)$, the belt compressed the finger with a virtual force $\tilde{k}\,x_r(t)$ where $\tilde{k}$ is the added stiffness (\eqref{stiffness_augmentation}), increasing the perceived stiffness of the piston (\figref{detinguy2018enhancing}).
@@ -285,20 +284,19 @@ Conversely, the technique allowed to \emph{decrease} the perceived stiffness by
\textcite{tao2021altering} proposed instead to restrict the deformation of the fingerpad by pulling a hollow frame around it to decrease perceived stiffness (\figref{tao2021altering}): it augments the finger contact area and thus the perceived Young's modulus of the object (\secref{hardness}).
\begin{subfigs}{stiffness_rendering_wearable}{Modifying the perceived stiffness with wearable pressure devices. }[%
\item Modify the perceived stiffness of a piston by pressing the finger during or prior the contact \cite{detinguy2018enhancing,salazar2020altering}.
\item Decrease perceived stiffness of hard object by restricting the fingerpad deformation \cite{tao2021altering}.
]
\subfigsheight{35mm}
\subfig{detinguy2018enhancing}
\subfig{tao2021altering}
\item Modify the perceived stiffness of a piston by pressing the finger during or prior the contact \cite{detinguy2018enhancing,salazar2020altering}.
\item Decrease perceived stiffness of hard object by restricting the fingerpad deformation \cite{tao2021altering}.
]
\subfigsheight{35mm}
\subfig{detinguy2018enhancing}
\subfig{tao2021altering}
\end{subfigs}
\paragraph{Vibrations Augmentations}
\textcite{okamura2001realitybased} measured impact vibrations $v(t)$ when tapping on real objects and found they can be modeled as exponential decaying sinusoid:
\begin{equation}{contact_transient}
v(t) = A \, |v_{in}| \, e^{- \tau t} sin(2 \pi f t)
v(t) = A \, |v_{in}| \, e^{- \tau t} sin(2 \pi f t)
\end{equation}
With $A$ the amplitude slope, $\tau$ the decay rate and $f$ the frequency, which are measured material properties, and $v_{in}$ the impact velocity.
It has been shown that these material properties perceptually express the stiffness (\secref{hardness}) of real \cite{higashi2019hardness} and virtual surface \cite{choi2021perceived}.
@@ -306,14 +304,14 @@ Therefore, when contacting or tapping a real object through an indirect feel-thr
A challenge with this technique is to provide the vibration feedback at the right time to be felt simultaneously with the real contact \cite{park2023perceptual}.
\begin{subfigs}{contact_vibrations}{Augmenting perceived stiffness using vibrations when touching a real surface \cite{choi2021augmenting}. }[%
%\item Experimental setup with a voice-coil actuator attached to a touch-through interface.
\item Voltage inputs (top) to the voice-coil for soft, medium, and hard vibrations, with the corresponding displacement (middle) and force (bottom) outputs of the actuator.
\item Perceived stiffness intensity of real sponge ("Sp") and wood ("Wd") surfaces without added vibrations ("N") and modified by soft ("S"), medium ("M") and hard ("H") vibrations.
]
%\subfig[.15]{choi2021augmenting_demo}
\subfigsheight{50mm}
\subfig{choi2021augmenting_control}
\subfig{choi2021augmenting_results}
%\item Experimental setup with a voice-coil actuator attached to a touch-through interface.
\item Voltage inputs (top) to the voice-coil for soft, medium, and hard vibrations, with the corresponding displacement (middle) and force (bottom) outputs of the actuator.
\item Perceived stiffness intensity of real sponge ("Sp") and wood ("Wd") surfaces without added vibrations ("N") and modified by soft ("S"), medium ("M") and hard ("H") vibrations.
]
%\subfig[.15]{choi2021augmenting_demo}
\subfigsheight{50mm}
\subfig{choi2021augmenting_control}
\subfig{choi2021augmenting_results}
\end{subfigs}
Vibrations on contact have been employed with wearable haptics, but to the best of our knowledge only to render \VOs \cite{pezent2019tasbi,teng2021touch,sabnis2023haptic}.
@@ -351,7 +349,6 @@ We describe them in the \secref{vhar_haptics}.
%\cite{choi2017grabity}
%\cite{culbertson2017waves}
\subsection{Conclusion}
\label{wearable_haptics_conclusion}