phd-thesis/1-introduction/related-work/2-wearable-haptics.tex

\section{Rendering Objects with Wearable Haptics}
\label{wearable_haptics}

One of the roles of haptic systems is to render virtual interactions and sensations that are \emph{similar and comparable} to those experienced by the haptic sense with real objects, particularly in \v-\VE~\cite{maclean2008it,culbertson2018haptics}.


\subsection{Level of Wearability}
\label{wearability_level}

Different types of haptic devices can be worn on the hand, but only some of them can be considered \emph{wearable}.
\textcite{pacchierotti2017wearable} classify them into three levels of wearability, as illustrated in the \figref{pacchierotti2017wearable}.
An increasing \emph{wearability} resulting in the loss of the system's kinesthetic feedback capability.

\begin{subfigs}{pacchierotti2017wearable}{
        Schematic wearability level of haptic devices for the hand~\cite{pacchierotti2017wearable}.
    }[
        \item World-grounded haptic devices are fixed on the environment to provide kinesthetic feedback to the user.
        \item Exoskeletons are body-grounded kinesthetic devices.
        \item Wearable haptic devices are grounded on the point of application of the tactile stimulus.
    ]
    \subfigsheight{38mm}
    \subfig{pacchierotti2017wearable_1}
    \subfig{pacchierotti2017wearable_2}
    \subfig{pacchierotti2017wearable_3}
\end{subfigs}

Haptic research comes from robotics and teleoperation, and historically led to the design of haptic systems that are \emph{grounded} to an external support in the environment, such as a table (see \figref{pacchierotti2017wearable_1}).
These are robotic arms whose end-effector is either held in the hand or worn on a finger and which simulate interactions with a \VE by providing kinesthetic forces and torques feedback (see \figref{pacchierotti2015cutaneous}).
They provide high fidelity haptic feedback but are heavy, bulky and limited to small workspaces~\cite{culbertson2018haptics}.

More portable designs have been developed by moving the grounded part to the user's body.
The entire robotic system is thus mounted on the user, forming an exoskeleton capable of providing kinesthetic feedback to the finger, \eg in \figref{achibet2017flexifingers}.
However, it cannot constrain the movements of the wrist and the reaction force is transmitted to the user where the device is grounded (see \figref{pacchierotti2017wearable_2}).
They are often heavy and bulky and cannot be considered wearable.

\textcite{pacchierotti2017wearable} defined that : \enquote{A wearable haptic interface should also be small, easy to carry, comfortable, and it should not impair the motion of the wearer}.
An approach is then to move the grounding point very close to the end-effector (see \figref{pacchierotti2017wearable_3}): the interface is limited to cutaneous haptic feedback, but its design is more compact, lightweight and comfortable, \eg in \figref{leonardis20173rsr}, and the system is wearable.
Moreover, as detailed in \secref{object_properties}, cutaneous sensations are necessary and often sufficient for the perception of the haptic properties of an object explored with the hand, as also argued by \textcite{pacchierotti2017wearable}.

\begin{subfigs}{grounded_to_wearable}{
        Haptic devices for the hand with different wearability levels.
    }[
        \item Teleoperation of a virtual cube grasped with the thumb and index fingers each attached to a grounded haptic device~\cite{pacchierotti2015cutaneous}.
        \item A passive exoskeleton for fingers simulating stiffness of a trumpet's pistons~\cite{achibet2017flexifingers}.
        \item Manipulation of a virtual cube with the thumb and index fingers each attached with the 3-RSR wearable haptic device~\cite{leonardis20173rsr}.
    ]
    \subfigsheight{38mm}
    \subfig{pacchierotti2015cutaneous}
    \subfig{achibet2017flexifingers}
    \subfig{leonardis20173rsr}
\end{subfigs}

% Tradeoff realistic and cost + analogy with sound, Hi-Fi costs a lot and is realistic, but 40$ BT headphone is more practical and enough, as cutaneous feedback without kinesthesic could be enough for wearable haptics and far more affordable and comfortable than world- or body-grounded haptics + cutaneous even better than kine for rendering surface curvature and fine manipulation


\subsection{Wearable Haptic Devices for the Hand}
\label{wearable_haptic_devices}

We present an overview of wearable haptic devices for the hand, following the categories of \textcite{pacchierotti2017wearable}.
The rendering of a haptic device is indeed determined by the nature of the actuators employed, which form the interface between the haptic system and the user's skin, and therefore the types of mechanical stimuli they can supply.
Several actuators are often combined in a haptic device to obtain richer haptic feedback.

\subsubsection{Moving Platforms}
\label{normal_actuators}

The moving platforms translate perpendicularly on the skin to create sensations of contact, pressure and edges~\cite{pacchierotti2017wearable}.
Placed under the fingertips, they can come into contact with the skin with different forces, speeds and orientations.
The platform is moved by means of cables, \eg in \figref{gabardi2016new}, or articulated arms, \eg in \figref{perez2017optimizationbased}, activated by motors grounded to the nail~\cite{gabardi2016new,perez2017optimizationbased}.
The motors lengthen and shorten the cables or orient the arms to move the platform over 3 \DoFs: two for orientation and one for normal force relative to the finger.
However, these platforms are specifically designed to provide haptic feedback to the fingertip in \VEs, preventing interaction with a \RE.

\subsubsection{Pin and Pneumatic Arrays}
\label{array_actuators}

A pin-array is a surface made up of small, rigid pins arranged very close together in a grid and that can be moved individually.
When placed in contact with the fingertip, it can create sensations of edge, pressure and texture.
The \figref{sarakoglou2012high} shows an example of a pin-array consisting of \numproduct{4 x 4} pins of \qty{1.5}{\mm} diameter and \qty{2}{\mm} height, spaced at \qty{2}{\mm}~\cite{sarakoglou2012high}.
Pneumatic systems use a fluid such as air or water to inflate membranes under the skin, creating sensations of contact and pressure~\cite{raza2024pneumatically}.
Multiple membranes are often used in a grid to simulate edges and textures, as in the \figref{ujitoko2020development}~\cite{ujitoko2020development}.
Although these two types of effector can be considered wearable, their actuation requires a high level of mechanical and electronic complexity that makes the system as a whole not portable.

\begin{subfigs}{normal_actuators}{
        Normal indentation actuators for the fingertip.
    }[
        \item A moving platform actuated with cables~\cite{gabardi2016new}.
        \item A moving platform actuated by articulated limbs~\cite{perez2017optimizationbased}.
        \item Diagram of a pin-array of tactors~\cite{sarakoglou2012high}.
        \item A pneumatic system composed of a \numproduct{12 x 10} array of air cylinders~\cite{ujitoko2020development}.
    ]
    \subfigsheight{37mm}
    \subfig{gabardi2016new}
    \subfig{perez2017optimizationbased}
    \subfig{sarakoglou2012high}
    \subfig{ujitoko2020development}
\end{subfigs}

\subsubsection{Tangential Motion Actuators}
\label{tangential_actuators}

Similar in design to the mobile platforms, the tangential motion actuators activate a rigid pin or surface in contact with the fingertip under the finger to create shearing sensation on the skin.
An articulated and motorized arm structure moves the effector in multiple directions over 2 \DoFs parallel to the skin, \eg in \figref{leonardis2015wearable}~\cite{leonardis2015wearable}.
Some actuators are capable of both normal and tangential motion over 3 \DoFs on the skin and can also make or break contact with the finger, \eg in \figref{schorr2017fingertip}~\cite{schorr2017fingertip}.

\subsubsection{Compression Belts}
\label{belt_actuators}

A simpler alternative approach is to place a belt under the finger, and to actuate it over 2 \DoFs by two motors placed on top of the finger~\cite{minamizawa2007gravity}.
By turning in opposite directions, the motors shorten the belt and create a sensation of pressure.
Conversely, by turning simultaneously in the same direction, the belt pulls on the skin, creating a shearing sensation.
The simplicity of this approach allows the belt to be placed anywhere on the hand, leaving the fingertip free to interact with the \RE, \eg the hRing on the proximal phalanx in \figref{pezent2019tasbi}~\cite{pacchierotti2016hring} or Tasbi on the wrist in \figref{pezent2019tasbi}~\cite{pezent2019tasbi}.

\begin{subfigs}{tangential_belts}{Tangential motion actuators and compression belts. }[
        \item A skin strech actuator for the fingertip~\cite{leonardis2015wearable}.
        \item A 3 \DoF actuator capable of normal and tangential motion on the fingertip~\cite{schorr2017fingertip}.
        %\item A shearing belt actuator for the fingertip~\cite{minamizawa2007gravity}.
        \item The hRing, a shearing belt actuator for the proximal phalanx of the finger~\cite{pacchierotti2016hring}.
        \item Tasbi, a wristband capable of pressure and vibrotactile feedback~\cite{pezent2019tasbi}.
    ]
    \subfigsheight{33.5mm}
    \subfig{leonardis2015wearable}
    \subfig{schorr2017fingertip}
    \subfig{pacchierotti2016hring}
    \subfig{pezent2019tasbi}
\end{subfigs}

\subsubsection{Vibrotactile Actuators}
\label{vibrotactile_actuators}

Vibrotactile actuators are the most common and simplest wearable haptic interfaces, and are available as consumer products.
They are small, lightweight and can be placed directly on any part of the hand.
\textcite{choi2013vibrotactile} provide a detailed review.
All vibrotactile actuators are based on the same principle: generating an oscillating motion from an electric current with a frequency and amplitude high enough to be perceived by cutaneous mechanoreceptors.
Several types of vibrotactile actuators are used in haptics, with different trade-offs between size, proposed \DoFs and application constraints:
\begin{itemize}
    \item An \ERM is a \DC motor that rotates an off-center mass when a voltage or current is applied (see \figref{precisionmicrodrives_erm}). \ERMs are easy to control, inexpensive and can be encapsulated in a few millimeters cylinder or coin form factor. However, they have only one \DoF because both the frequency and amplitude of the vibration are coupled to the speed of the rotation, \eg low (high) frequencies output at low (high) amplitudes, as shown on \figref{precisionmicrodrives_erm_performances}.
    \item A \LRA consists of a coil that creates a magnetic field from an \AC to oscillate a magnet attached to a spring, as an audio loudspeaker (see \figref{precisionmicrodrives_lra}). They are more complex to control and a bit larger than \ERMs. Each \LRA is designed to vibrate with maximum amplitude at a given frequency, but won't vibrate efficiently at other frequencies, \ie their bandwidth is narrow, as shown on \figref{azadi2014vibrotactile}.
    \item A \VCA is a \LRA but capable of generating vibration at two \DoF, with an independent control of the frequency and amplitude of the vibration on a wide bandwidth. They are larger in size than \ERMs and \LRAs, but can generate more complex renderings.
    \item Piezoelectric actuators deform a solid material when a voltage is applied. They are very small and thin, and allow two \DoFs of amplitude and frequency control. However, they require high voltages to operate thus limiting their use in wearable devices.
\end{itemize}

\begin{subfigs}{vibrotactile_actuators}{Diagrams of vibrotactile acuators. }[
        \item Diagram of a cylindrical encapsulated \ERM. From Precision Microdrives.~\footnotemark
        \item Diagram of a \LRA. From Precision Microdrives.~\footnotemarkrepeat
    ]
    \subfigsheight{50mm}
    \subfig{precisionmicrodrives_erm}
    \subfig{precisionmicrodrives_lra}
\end{subfigs}

\footnotetext{\url{https://www.precisionmicrodrives.com/}}

\begin{subfigs}{vibrotactile_performances}{Performances of vibrotactile acuators. }[
        \item Amplitude and frequency output of an \ERM as a function of the input voltage. From Precision Microdrives.~\footnotemarkrepeat
        \item Force generated by two \LRAs as a function of sine wave input with different frequencies: both their maximum force and frequency are different~\cite{azadi2014vibrotactile}.
    ]
    \subfig[.58]{precisionmicrodrives_erm_performances}
    \subfig[.38]{azadi2014vibrotactile}
\end{subfigs}


\subsection{Tactile Renderings of Object Properties}
\label{tactile_rendering}

Le rendu tactile des propriétés haptiques consiste à modéliser et reproduire des sensations cutanées virtuelles comparables à celles perçues lors de l'interaction avec des objets réels.

En particulier, nous nous intéressons aux actuateurs portables stimulant les méchano-récepteurs de la peau (voir \secref{haptic_sense}) et n'empêchant pas de toucher et interagir avec l'environnement réel et aux rendus de propriétés haptiques d'objets virtuels ou augmentés.

%, unlike most previous actuators that are designed specifically for fingertips and would require mechanical adaptation to be placed on other parts of the hand.
%thanks to the vibration propagation and the sensory capabilities distributed throughout the skin, they can be placed without adaption and on any part of the hand

\subsubsection{Contact}

\subsubsection{Hardness}
\label{contact_rendering}

\subsubsection{Texture}
\label{texture_rendering}

Several approaches have been proposed to render virtual haptic texture~\cite{culbertson2018haptics}.
%
High-fidelity force feedback devices can reproduce patterned textures with great precision and provide similar perceptions to real textures, but they are expensive, have a limited workspace, and impose to hold a probe to explore the texture~\cite{unger2011roughness}.
%
As more traditional force feedback systems are unable to accurately render such micro-details on a simulated surface, vibrotactile devices attached to the end effector instead generate vibrations to simulate interaction with the virtual texture~\cite{culbertson2018haptics}.
%
In this way, physics-based models~\cite{chan2021hasti,okamura1998vibration,guruswamy2011iir} and data-based models~\cite{culbertson2015should,romano2010automatic} have been developed and evaluated, the former being simpler but more approximate to real textures, and the latter being more realistic but limited to the captured textures.
%
Notably, \textcite{okamura1998vibration} rendered grating textures with exponentially decaying sinudoids that simulated the strokes of the grooves and ridges of the surface, while \textcite{culbertson2014modeling} captured and modelled the roughness of real surfaces to render them using the speed and force of the user.
%
An effective approach to rendering virtual roughness is to generate vibrations to simulate interaction with the virtual texture~\cite{culbertson2018haptics}, relying on the user's real-time measurements of position, velocity and force to modulate the frequencies and amplitudes of the vibrations, with position and velocity being the most important parameters~\cite{culbertson2015should}.
% For example, when comparing the same virtual texture pairwise, but with different parameters, \textcite{culbertson2015should} showed that the roughness vibrations generated should vary with user speed, but not necessarily with user force.
% Virtual data-driven textures were perceived as similar to real textures, except for friction, which was not rendered properly.
%
The perceived roughness of real surfaces can be then modified when touched by a tool with a vibrotactile actuator attached~\cite{culbertson2014modeling,ujitoko2019modulating} or directly with the finger wearing the vibrotactile actuator~\cite{asano2015vibrotactile}, creating a haptic texture augmentation.
%
The objective is not just to render a virtual texture, but to alter the perception of a real, tangible surface, usually with wearable haptic devices, in what is known as haptic augmented reality (HAR)~\cite{bhatia2024augmenting,jeon2009haptic}.

One additional challenge of augmenting the finger touch is to keep the fingertip free to touch the real environment, thus delocalizing the actuator elsewhere on the hand~\cite{ando2007fingernailmounted,friesen2024perceived,normand2024visuohaptic,teng2021touch}.
%
Of course, the fingertip skin is not deformed by the virtual texture and only vibrations are felt, but it has been shown that the vibrations produced on the fingertip skin running over a real surface are texture specific and similar between individuals~\cite{manfredi2014natural}.
%
A common method vibrotactile rendering of texture is to use a sinusoidal signal whose frequency is modulated by the finger position or velocity~\cite{asano2015vibrotactile,friesen2024perceived,strohmeier2017generating,ujitoko2019modulating}.
%
It remains unclear whether such vibrotactile texture augmentation is perceived the same when integrated into visual AR or VR environments or touched with a virtual hand instead of the real hand.
%
%We also add a phase adjustment to this sinusoidal signal to allow free exploration movements of the finger with a simple camera-based tracking system.


\subsection{Conclusion}
\label{wearable_haptics_conclusion}