378 lines
32 KiB
TeX
378 lines
32 KiB
TeX
\section{Augmenting Object Perception with Wearable Haptics}
|
|
\label{wearable_haptics}
|
|
|
|
Haptic systems aim to render virtual interactions and sensations that are \emph{similar and comparable} to those experienced by the haptic sense with real objects \cite{maclean2008it,culbertson2018haptics}.
|
|
Due to the high complexity of the haptic sense and the variety of sensations it can feel, haptic actuators and renderings are designed to address only a subset of these sensations.
|
|
While it is challenging to create a realistic haptic experience, \ie that reproduce the real object interaction with high fidelity \cite{unger2011roughness,culbertson2015should}, it is more important to provide the right sensory stimulus \enquote{at the right moment and at the right place} \cite{hayward2007it}.
|
|
|
|
A haptic augmentation system \enquote{modulates the feel of a real object by virtual [haptic] feedback} \cite{jeon2009haptic}, \ie a touch interaction with a real object whose perception is modified by the addition of virtual haptic feedback.
|
|
The haptic system should be hand-held or worn, \eg on the hand, and \enquote{not permanently attached to or integrated in the object} \cite{bhatia2024augmenting}.
|
|
|
|
\subsection{Level of Wearability}
|
|
\label{wearability_level}
|
|
|
|
Different types of haptic devices can be worn on the hand, but only some of them can be considered wearable.
|
|
\textcite{pacchierotti2017wearable} classify them into three levels of wearability, as illustrated in \figref{pacchierotti2017wearable}.
|
|
An increasing wearability resulting in the loss of the system's kinesthetic feedback capability.
|
|
|
|
\begin{subfigs}{pacchierotti2017wearable}{
|
|
Schematic wearability level of haptic devices for the hand \cite{pacchierotti2017wearable}.
|
|
}[][
|
|
\item World-grounded haptic devices are fixed on the environment to provide kinesthetic feedback to the user.
|
|
\item Exoskeletons are body-grounded kinesthetic devices.
|
|
\item Wearable haptic devices are grounded on the point of application of the tactile stimulus.
|
|
]
|
|
\subfigsheight{34mm}
|
|
\subfig{pacchierotti2017wearable_1}
|
|
\subfig{pacchierotti2017wearable_2}
|
|
\subfig{pacchierotti2017wearable_3}
|
|
\end{subfigs}
|
|
|
|
Haptic research comes from robotics and teleoperation, and historically led to the design of haptic systems that are \emph{world-grounded} to an external support in the environment, such as a table (\figref{pacchierotti2017wearable_1}).
|
|
These are robotic arms whose end-effector is either held in the hand or worn on a finger and which simulate interactions with a \VE by providing kinesthetic forces and torques feedback (\figref{pacchierotti2015cutaneous}).
|
|
They provide high fidelity haptic feedback but are heavy, bulky and limited to small workspaces \cite{culbertson2018haptics}.
|
|
|
|
More portable designs have been developed by moving the grounded part to the user's body.
|
|
The entire robotic system is thus mounted on the user, forming an exoskeleton capable of providing kinesthetic feedback to the finger, \eg in \figref{achibet2017flexifingers}.
|
|
However, it cannot constrain the movements of the wrist and the reaction force is transmitted to the user where the device is grounded (\figref{pacchierotti2017wearable_2}).
|
|
Such \emph{body-grounded} devices are often heavy and bulky and cannot be considered wearable.
|
|
|
|
\textcite{pacchierotti2017wearable} defined that : \enquote{A wearable haptic interface should also be small, easy to carry, comfortable, and it should not impair the motion of the wearer}.
|
|
An approach is then to move the grounding point close to the end-effector (\figref{pacchierotti2017wearable_3}): the interface is limited to cutaneous haptic feedback, but its design is more compact, lightweight, comfortable and portable, \eg in \figref{grounded_to_wearable}.
|
|
Moreover, as detailed in \secref{object_properties}, cutaneous sensations are necessary and often sufficient for the perception of the haptic properties of an object explored with the hand \cite{pacchierotti2017wearable}.
|
|
|
|
\begin{subfigs}{grounded_to_wearable}{Haptic devices for the hand with different wearability levels. }[][
|
|
\item Teleoperation of a virtual cube grasped with the thumb and index fingers each attached to a grounded haptic device \cite{pacchierotti2015cutaneous}.
|
|
\item A passive exoskeleton for fingers simulating stiffness of a trumpet's pistons \cite{achibet2017flexifingers}.
|
|
\item Manipulation of a virtual cube with the thumb and index fingers each attached with the 3-RSR wearable haptic device \cite{leonardis20173rsr}.
|
|
]
|
|
\subfigsheight{38mm}
|
|
\subfig{pacchierotti2015cutaneous}
|
|
\subfig{achibet2017flexifingers}
|
|
\subfig{leonardis20173rsr}
|
|
\end{subfigs}
|
|
|
|
% Tradeoff realistic and cost + analogy with sound, Hi-Fi costs a lot and is realistic, but 40$ BT headphone is more practical and enough, as cutaneous feedback without kinesthesic could be enough for wearable haptics and far more affordable and comfortable than world- or body-grounded haptics + cutaneous even better than kine for rendering surface curvature and fine manipulation
|
|
|
|
\subsection{Wearable Haptic Devices for the Hand}
|
|
\label{wearable_haptic_devices}
|
|
|
|
We present an overview of wearable haptic devices for the hand, following the categories of \textcite{pacchierotti2017wearable}.
|
|
It should be noted that the rendering capabilities of a haptic device is determined by the type of actuators employed.
|
|
The actuator forms the interface between the haptic device and the user, and provides the haptic rendering as mechanical stimuli to the user's skin.
|
|
Multiple actuators are often combined in a haptic device to provide richer feedback.
|
|
|
|
\subsubsection{Moving Platforms}
|
|
\label{moving_platforms}
|
|
|
|
The moving platforms translate perpendicularly on the skin to create sensations of contact, pressure and edges \cite{pacchierotti2017wearable}.
|
|
Placed under the fingertips, they can come into contact with the skin with different forces, speeds and orientations.
|
|
The platform is moved by means of cables, \eg in \figref{gabardi2016new}, or articulated arms, \eg in \figref{chinello2017three}, activated by motors grounded to the nail \cite{gabardi2016new,chinello2017three}.
|
|
The motors lengthen and shorten the cables or orient the arms to move the platform over 3 \DoFs: two for orientation and one for normal force relative to the finger.
|
|
However, these platforms are specifically designed to provide haptic feedback to the fingertip in \VEs, preventing interaction with a \RE.
|
|
|
|
\subsubsection{Pin and Pneumatic Arrays}
|
|
\label{array_actuators}
|
|
|
|
A pin-array is a surface made up of small, rigid pins arranged close together in a grid and that can be moved individually.
|
|
When placed in contact with the fingertip, it can create sensations of edge, pressure and texture.
|
|
\figref{sarakoglou2012high} shows an example of a pin-array consisting of \numproduct{4 x 4} pins of \qty{1.5}{\mm} diameter and \qty{2}{\mm} height, spaced at \qty{2}{\mm} \cite{sarakoglou2012high}.
|
|
Pneumatic systems use a fluid such as air or water to inflate membranes under the skin, creating sensations of contact and pressure \cite{raza2024pneumatically}.
|
|
Multiple membranes are often used in a grid to simulate edges and textures, as in \figref{ujitoko2020development} \cite{ujitoko2020development}.
|
|
Although these two types of effector can be considered wearable, their actuation requires a high level of mechanical and electronic complexity that makes the system as a whole not portable.
|
|
|
|
\begin{subfigs}{normal_actuators}{
|
|
Normal indentation actuators for the fingertip.
|
|
}[][
|
|
\item A moving platform actuated with cables \cite{gabardi2016new}.
|
|
\item A moving platform actuated by articulated limbs \cite{chinello2017three}.
|
|
\item Diagram of a pin-array of tactors \cite{sarakoglou2012high}.
|
|
\item A pneumatic system composed of a \numproduct{12 x 10} array of air cylinders \cite{ujitoko2020development}.
|
|
]
|
|
\subfigsheight{37mm}
|
|
\subfig{gabardi2016new}
|
|
\subfig{chinello2017three}
|
|
\subfig{sarakoglou2012high}
|
|
\subfig{ujitoko2020development}
|
|
\end{subfigs}
|
|
|
|
\subsubsection{Tangential Motion Actuators}
|
|
\label{tangential_actuators}
|
|
|
|
Similar in design to the mobile platforms, the tangential motion actuators activate a rigid pin or surface in contact with the fingertip under the finger to create shearing sensation on the skin.
|
|
An articulated and motorized arm structure moves the effector in multiple directions over 2 \DoFs parallel to the skin, \eg in \figref{leonardis2015wearable} \cite{leonardis2015wearable}.
|
|
Some actuators are capable of both normal and tangential motion over 3 \DoFs on the skin and can also make or break contact with the finger, \eg in \figref{schorr2017fingertip} \cite{schorr2017fingertip}.
|
|
|
|
\subsubsection{Compression Belts}
|
|
\label{belt_actuators}
|
|
|
|
A simpler alternative approach is to place a belt under the finger, and to actuate it over 2 \DoFs by two motors placed on top of the finger \cite{minamizawa2007gravity}.
|
|
By turning in opposite directions, the motors shorten the belt and create a sensation of pressure.
|
|
Conversely, by turning simultaneously in the same direction, the belt pulls on the skin, creating a shearing sensation.
|
|
The simplicity of this approach allows the belt to be placed anywhere on the hand, leaving the fingertip free to interact with the \RE, \eg the hRing on the proximal phalanx in \figref{pacchierotti2016hring_1} \cite{pacchierotti2016hring} or Tasbi on the wrist in \figref{pezent2022design} \cite{pezent2022design}.
|
|
|
|
\begin{subfigs}{tangential_belts}{Tangential motion actuators and compression belts. }[][
|
|
\item A skin strech actuator for the fingertip \cite{leonardis2015wearable}.
|
|
\item A 3 \DoF actuator capable of normal and tangential motion on the fingertip \cite{schorr2017fingertip}.
|
|
%\item A shearing belt actuator for the fingertip \cite{minamizawa2007gravity}.
|
|
\item The hRing, a shearing belt actuator for the proximal phalanx of the finger \cite{pacchierotti2016hring}.
|
|
\item Tasbi, a wristband capable of pressure and vibrotactile feedback \cite{pezent2022design}.
|
|
]
|
|
\subfigsheight{33.5mm}
|
|
\subfig{leonardis2015wearable}
|
|
\subfig{schorr2017fingertip}
|
|
\subfig{pacchierotti2016hring_1}
|
|
\subfig{pezent2022design}
|
|
\end{subfigs}
|
|
|
|
\subsubsection{Vibrotactile Actuators}
|
|
\label{vibrotactile_actuators}
|
|
|
|
Vibrotactile actuators are the most common and simplest wearable haptic interfaces, and are available as consumer products.
|
|
They are small, lightweight and can be placed directly on any part of the hand.
|
|
\textcite{choi2013vibrotactile} provided a detailed review.
|
|
All vibrotactile actuators are based on the same principle: generating an oscillating motion from an electric current with a frequency and amplitude high enough to be perceived by cutaneous mechanoreceptors.
|
|
Several types of vibrotactile actuators are used in haptics, with different trade-offs between size, proposed \DoFs and application constraints.
|
|
|
|
An \ERM is a direct current (DC) motor that rotates an off-center mass when a voltage or current is applied (\figref{precisionmicrodrives_erm}). \ERMs are easy to control, inexpensive and can be encapsulated in a few millimeters cylinder or coin form factor. However, they have only one \DoF because both the frequency and amplitude of the vibration are coupled to the speed of the rotation, \eg low (high) frequencies output at low (high) amplitudes, as shown in \figref{precisionmicrodrives_erm_performances}.
|
|
|
|
\begin{subfigs}{erm}{Diagram and performance of an \ERM. }[][
|
|
\item Diagram of a cylindrical encapsulated \ERM. From Precision Microdrives~\footnotemark.
|
|
\item Amplitude and frequency output of an \ERM as a function of the input voltage.
|
|
]
|
|
\subfigsheight{45mm}
|
|
\subfig{precisionmicrodrives_erm}
|
|
\subfig{precisionmicrodrives_erm_performances}
|
|
\end{subfigs}
|
|
|
|
\footnotetext{\url{https://www.precisionmicrodrives.com/}}
|
|
|
|
A \LRA consists of a coil that creates a magnetic field from an alternative current (AC) to oscillate a magnet attached to a spring, as an audio loudspeaker (\figref{precisionmicrodrives_lra}).
|
|
They are more complex to control and a bit larger than \ERMs.
|
|
Each \LRA is designed to vibrate with maximum amplitude at a given resonant frequency, but won't vibrate efficiently at other frequencies, \ie their bandwidth is narrow, as shown in \figref{azadi2014vibrotactile}.
|
|
A voice-coil actuator is a \LRA but capable of generating vibration at two \DoF, with an independent control of the frequency and amplitude of the vibration on a wide bandwidth.
|
|
They are larger than \ERMs and \LRAs, but can generate more complex renderings.
|
|
|
|
Piezoelectric actuators deform a solid material when a voltage is applied.
|
|
They are small and thin and provide two \DoFs of amplitude and frequency control.
|
|
However, they require high voltages to operate, limiting their use in wearable devices.
|
|
|
|
\begin{subfigs}{lra}{Diagram and performance of \LRAs. }[][
|
|
\item Diagram. From Precision Microdrives~\footnotemarkrepeat.
|
|
\item Force generated by two \LRAs as a function of sinusoidal wave input with different frequencies: both their maximum force and resonant frequency are different \cite{azadi2014vibrotactile}.
|
|
]
|
|
\subfigsheight{50mm}
|
|
\subfig{precisionmicrodrives_lra}
|
|
\subfig{azadi2014vibrotactile}
|
|
\end{subfigs}
|
|
|
|
\subsection{Modifying Perceived Haptic Roughness and Hardness}
|
|
\label{tactile_rendering}
|
|
|
|
Rendering a haptic property consists in modeling and reproducing virtual sensations comparable to those perceived when interacting with real objects \cite{klatzky2013haptic}.
|
|
As we have just seen, the haptic sense is rich and complex (\secref{haptic_hand}).
|
|
Thus, a wide variety of wearable haptic actuators have been developed (\secref{wearable_haptic_devices}).
|
|
However, each actuator is only able to provide a subset of the haptic sensations felt by the hand.
|
|
We review in this section the rendering methods with wearable haptics to modify perceived roughness and hardness of real objects.
|
|
|
|
\subsubsection{Haptic Augmentations}
|
|
|
|
By adding haptic rendering as feedback timely synchronized with the touch actions of the hand on a real object \cite{bhatia2024augmenting}, the perception of the object's haptic property can be modified.
|
|
That is both the real and virtual haptic sensations are integrated into a single property perception, \ie the perceived haptic property is modulated by the added virtual feedback.
|
|
The integration of the real and virtual sensations into a single property perception is discussed in more details in \secref{sensations_perception}.
|
|
In particular, the visual rendering of a touched object can also influence the perception of its haptic properties, \eg by modifying its visual texture in \AR or \VR, as discussed in the \secref{visuo_haptic}.
|
|
|
|
\textcite{bhatia2024augmenting} categorize the haptic augmentations into three types: direct touch, touch-through, and tool-mediated.
|
|
In \emph{direct touch}, the haptic device does not cover the inside of the hand so as not to impair the user's interaction with the \RE, and is typically achieved with wearable haptics.
|
|
In touch-through and tool-mediated, or \emph{indirect feel-through} \cite{jeon2015haptic}, the haptic device is placed between the hand and the \RE.
|
|
%We are interested in direct touch augmentations with wearable haptics (\secref{wearable_haptic_devices}), as their integration with \AR is particularly promising for free hand interaction with visuo-haptic augmentations.
|
|
Many haptic augmentations were first developed with touch-through devices, and some (but not all) were later transposed to direct touch augmentation with wearable haptic devices.
|
|
%We also focus on tactile augmentations stimulating the mechanoreceptors of the skin (\secref{haptic_sense}), thus excluding temperature perception, as they are the most common existing haptic interfaces.
|
|
|
|
Since we have chosen to focus in \secref{object_properties} on the haptic perception of roughness and hardness of objects, we review below the methods to modify the perception of these properties.
|
|
Of course, wearable haptics can also be used in a direct touch context to modify the perceived friction \cite{konyo2008alternative,salazar2020altering}, weight \cite{minamizawa2007gravity}, or local deformation \cite{salazar2020altering} of real objects, but they are rare \cite{bhatia2024augmenting} and will not be detailed here.
|
|
|
|
% \cite{klatzky2003feeling} : rendering roughness, friction, deformation, temperatures
|
|
% \cite{girard2016haptip} : renderings with a tangential motion actuator
|
|
|
|
\subsubsection{Roughness Augmentation}
|
|
\label{texture_rendering}
|
|
|
|
To modify the perception of the haptic roughness (or texture, see \secref{roughness}) of a real object, vibrations are typically applied to the skin by the haptic device as the user moves over the surface.
|
|
%This is because running the finger or a tool on a textured surface generates pressures and vibrations (\secref{roughness}) at frequencies that are too high for rendering capabilities of most haptic devices \cite{campion2005fundamental,culbertson2018haptics}.
|
|
There are two main approaches to modify virtual textures perception: \emph{simulation models} and \emph{data-driven models} \cite{klatzky2013haptic,culbertson2018haptics}.
|
|
|
|
\paragraph{Simulation Models}
|
|
|
|
%Simulations of virtual textures are based on the physics of the interaction between the finger and the surface, and are used to generate the vibrations that the user feels when running the finger over the surface.
|
|
|
|
The simplest texture simulation model is a 1D sinusoidal grating $v(t)$ with spatial period $\lambda$ and amplitude $A$ that is scanned by the user at velocity $\dot{x}(t)$:
|
|
\begin{equation}{grating_rendering}
|
|
v(t) = A \sin(\frac{2 \pi \dot{x}(t)}{\lambda})
|
|
\end{equation}
|
|
That is, this model generates a periodic signal whose frequency is modulated and proportional to the user's velocity, implementing the speed-frequency ratio observed with real patterned textures (\eqref{grating_vibrations}).
|
|
It gives the user the illusion of a texture with a \emph{fixed spatial period} that approximate the real manufactured grating textures (\secref{roughness}).
|
|
The user's position could have been used instead of the velocity, but it requires measuring the position and generating the signal at frequencies too high (\qty{10}{\kHz}) for most sensors and haptic actuators \cite{campion2005fundamental}.
|
|
|
|
With a voice-coil actuator attached to the middle phalanx of the finger, \textcite{asano2015vibrotactile} used this model to increase the perceived roughness (\figref{asano2015vibrotactile_2})
|
|
Participants moved their finger over real grating textures (\qtyrange{0.15}{.29}{\mm} groove and ridge width) with a virtual sine grating (\qty{1}{\mm} spatial period) superimposed, rendered after \eqref{grating_rendering}.
|
|
The perceived roughness increased proportionally to the virtual texture amplitude, but a high amplitude decreased it instead.
|
|
\textcite{ujitoko2019modulating} instead used a square wave signal and a hand-held stylus with an embedded voice-coil.
|
|
\textcite{friesen2024perceived} compared the frequency modulation of \eqref{grating_rendering} with amplitude modulation (\figref{friesen2024perceived}), and found that the frequency modulation was perceived as more similar to real sinusoidal gratings for lower spatial periods (\qty{0.5}{\mm}) but both modulations were effective for higher spatial periods (\qty{1.5}{\mm}).
|
|
|
|
%\textcite{friesen2024perceived} proposed
|
|
|
|
The model in \eqref{grating_rendering} can be extended to 2D textures by adding a second sinusoidal grating with an orthogonal orientation as \textcite{girard2016haptip}.
|
|
More complex models have also been developed to be physically accurate and reproduce with high fidelity the roughness perception of real patterned surfaces \cite{unger2011roughness}, but they require high-fidelity force feedback devices that are expensive and have a limited workspace.
|
|
|
|
\paragraph{Data-driven Models}
|
|
|
|
Because simulations of realistic virtual textures can be complex to design and to render in real-time, direct capture and models of real textures have been developed \cite{culbertson2018haptics}.
|
|
|
|
\textcite{okamura1998vibration} were the first to measure the vibrations produced by the interaction of a stylus dragged over sandpaper and patterned surfaces.
|
|
They found that the contact with patterns when sliding on the texture generates vibrations that can be modelled as exponentially decaying sinusoids (\eqref{contact_transient}) that depend on the normal force and the scanning velocity of the stylus on the surface.
|
|
This technique was used by \textcite{ando2007fingernailmounted} to augment a smooth sheet of paper with a virtual patterned texture: With a \LRA mounted on the nail, they rendered the virtual finger contacts with \qty{20}{\ms} vibration impulses at \qty{130}{\Hz} (\figref{ando2007fingernailmounted}).
|
|
Participants were able to match the virtual textures to the real ones (\qty{0.25}{\mm} height and \qtyrange{1}{10}{\mm} widths) but systematically overestimated the virtual width to be \qty{4}{\mm} longer.
|
|
This model was refined to capture everyday, unpatterned textures as well \cite{guruswamy2011iir}
|
|
|
|
More complex models were then created to more systematically capture everyday textures from many stylus scan measures \cite{romano2012creating,culbertson2014modeling}.
|
|
This led to the release of the \HaTT database, a public set of stylus recordings and models of 100 haptic textures \cite{culbertson2014one}.
|
|
A similar database, but captured from a direct touch context with the fingertip, has also recently been released \cite{balasubramanian2024sens3}.
|
|
A common limitation of these data-driven models is that they can only render \emph{isotropic} textures: their record does not depend on the position of the measure, and the rendering is the same regardless of the direction of the movement.
|
|
This was eventually addressed to include the user's velocity direction into the capture, modelling and rendering of the textures \cite{abdulali2016datadriven,abdulali2018datadriven}.
|
|
|
|
%A third approach is to model
|
|
%Alternative models have been proposed to both render both isotropic and patterned textures \cite{chan2021hasti}., or to simulate the vibrations from the (visual) texture maps used to visually render a \ThreeD object \cite{chan2021hasti}.
|
|
|
|
Using the user's velocity magnitude and normal force as input, these data-driven models are able to interpolate from the scan measures to generate a virtual texture in real time as vibrations with a high realism.
|
|
When comparing real textures felt through a stylus with their virtual models rendered with a voice-coil actuator attached to the stylus (\figref{culbertson2012refined}), the virtual textures were found to accurately reproduce the perception of roughness, but hardness and friction were not rendered properly \cite{culbertson2014modeling}.
|
|
\textcite{culbertson2015should} further showed that the perceived realism of the virtual textures, and similarity to the real textures, depended mostly on the user's velocity magnitude but not on the user's force as inputs to the model, \ie responding to velocity magnitude is sufficient to render isotropic virtual textures.
|
|
|
|
\begin{subfigs}{textures_rendering_data}{Augmentating haptic texture perception with voice-coil actuators. }[][
|
|
\item Increasing and decreasing the perceived roughness of a real patterned texture in direct touch \cite{asano2015vibrotactile}.
|
|
\item Comparing real patterned texture with virtual texture augmentation in direct touch \cite{friesen2024perceived}.
|
|
\item Rendering virtual contacts in direct touch with the virtual texture \cite{ando2007fingernailmounted}.
|
|
\item Rendering an isotropic virtual texture over a real surface while sliding a hand-held stylus across it \cite{culbertson2012refined}.
|
|
]
|
|
\subfigsheight{36mm}
|
|
\subfig{asano2015vibrotactile_2}
|
|
\subfig{friesen2024perceived}
|
|
\subfig{ando2007fingernailmounted}
|
|
\subfig{culbertson2012refined}
|
|
\end{subfigs}
|
|
|
|
\subsubsection{Hardness Augmentation}
|
|
\label{hardness_rendering}
|
|
|
|
The perceived hardness (\secref{hardness}) of a real surface can be modified by rendering forces or vibrations.
|
|
|
|
\paragraph{Modulating Forces}
|
|
|
|
When tapping or pressing a real object, the perceived stiffness $\tilde{k}$ of its surface can be modulated with force feedback \cite{jeon2015haptic}.
|
|
This was first proposed by \textcite{jeon2008modulating} who augmented a real surface tapped in 1 \DoF with a grounded force-feedback device held in the hand (\figref{jeon2009haptic_1}).
|
|
When the haptic end-effector contacts the object at time $t$, the object's surface deforms by displacement $x_r(t)$ and opposes a real reaction force $f_r(t)$.
|
|
The virtual force of the device $\tilde{f_r}(t)$ is then controlled to:
|
|
\begin{equation}{stiffness_augmentation}
|
|
\tilde{f_r}(t) = f_r(t) - \tilde{k} x_r(t)
|
|
\end{equation}
|
|
A force sensor embedded in the device measures the reaction force $f_r(t)$.
|
|
The displacement $x_r(t)$ is estimated with the reaction force and the tapping velocity using a predefined model of different materials as described in \textcite{jeon2011extensions}.
|
|
As shown in \figref{jeon2009haptic_2}, the force $\tilde{f_r}(t)$ perceived by the user is modulated, but not the displacement $x_r(t)$, hence the perceived stiffness is $\tilde{k}(t)$.
|
|
This stiffness augmentation technique was then extended to allow tapping and pressing with 3 \DoFs \cite{jeon2010stiffness}, to render friction and weight augmentations \cite{jeon2011extensions}, and to grasp and squeeze the real object with two contact points \cite{jeon2012extending}.
|
|
|
|
\begin{subfigs}{stiffness_rendering_grounded}{
|
|
Augmenting the perceived stiffness of a real surface with a hand-held force-feedback device.
|
|
}[][
|
|
\item Diagram of a user tapping the surface \cite{jeon2009haptic}.
|
|
\item Displacement-force curves of a real rubber ball (dashed line) and when its perceived stiffness $\tilde{k}$ is modulated \cite{jeon2009haptic}.
|
|
]
|
|
\subfig[0.45]{jeon2009haptic_1}
|
|
\subfig[0.45]{jeon2009haptic_2}
|
|
\end{subfigs}
|
|
|
|
\textcite{detinguy2018enhancing} transposed this stiffness augmentation technique with the hRing device (\secref{belt_actuators}): While pressing a real piston with the fingertip by displacement $x_r(t)$, the belt compressed the finger with a virtual force $\tilde{k}\,x_r(t)$ where $\tilde{k}$ is the added stiffness (\eqref{stiffness_augmentation}), increasing the perceived stiffness of the piston (\figref{detinguy2018enhancing}).
|
|
More importantly, the augmentation proved to be robust to the placement of the device, as the increased stiffness was perceived the same on the fingertip, middle phalanx, and proximal.
|
|
Conversely, the technique allowed to \emph{decrease} the perceived stiffness by compressing the phalanx before the contact and reducing the pressure when the user pressed the piston \cite{salazar2020altering}.
|
|
\textcite{tao2021altering} proposed instead to restrict the deformation of the fingerpad by pulling a hollow frame around it to decrease perceived stiffness (\figref{tao2021altering}): it augments the finger contact area and thus the perceived Young's modulus of the object (\secref{hardness}).
|
|
|
|
\begin{subfigs}{stiffness_rendering_wearable}{
|
|
Modifying the perceived stiffness with wearable pressure devices.
|
|
}[][
|
|
\item Modify the perceived stiffness of a piston by pressing the finger during or prior the contact \cite{detinguy2018enhancing,salazar2020altering}.
|
|
\item Decrease perceived stiffness of hard object by restricting the fingerpad deformation \cite{tao2021altering}.
|
|
]
|
|
\subfigsheight{35mm}
|
|
\subfigbox{detinguy2018enhancing}
|
|
\subfigbox{tao2021altering}
|
|
\end{subfigs}
|
|
|
|
\paragraph{Vibrations Augmentations}
|
|
|
|
\textcite{okamura2001realitybased} measured impact vibrations $v(t)$ when tapping on real objects and found they can be modeled as exponential decaying sinusoid:
|
|
\begin{equation}{contact_transient}
|
|
v(t) = A \, |v_{in}| \, e^{- \tau t} sin(2 \pi f t)
|
|
\end{equation}
|
|
With $A$ the amplitude slope, $\tau$ the decay rate and $f$ the frequency, which are measured material properties, and $v_{in}$ the impact velocity.
|
|
It has been shown that these material properties perceptually express the stiffness (\secref{hardness}) of real \cite{higashi2019hardness} and virtual surface \cite{choi2021perceived}.
|
|
Therefore, when contacting or tapping a real object through an indirect feel-through interface that provides such vibrations (\figref{choi2021augmenting_control}) using a voice-coil (\secref{vibrotactile_actuators}), the perceived stiffness can be increased or decreased \cite{kuchenbecker2006improving,hachisu2012augmentation,choi2021augmenting}, \eg sponge feels stiffer or wood feels softer (\figref{choi2021augmenting_results}).
|
|
A challenge with this technique is to provide the vibration feedback at the right time to be felt simultaneously with the real contact \cite{park2023perceptual}.
|
|
|
|
\begin{subfigs}{contact_vibrations}{
|
|
Augmenting perceived stiffness using vibrations when touching a real surface \cite{choi2021augmenting}.
|
|
}[][
|
|
\item Voltage inputs (top) to the voice-coil for soft, medium, and hard vibrations, with the corresponding displacement (middle) and force (bottom) outputs of the actuator.
|
|
\item Perceived stiffness intensity of real sponge ("Sp") and wood ("Wd") surfaces without added vibrations ("N") and modified by soft ("S"), medium ("M") and hard ("H") vibrations.
|
|
]
|
|
\subfigsheight{49mm}
|
|
\subfig{choi2021augmenting_control}
|
|
\subfig{choi2021augmenting_results}
|
|
\end{subfigs}
|
|
|
|
Vibrations on contact have been employed with wearable haptics, but to the best of our knowledge only to render virtual objects \cite{pezent2019tasbi,teng2021touch,sabnis2023haptic}.
|
|
We describe them in the \secref{vhar_haptics}.
|
|
|
|
%A promising alternative approach
|
|
%\cite{kildal20103dpress}
|
|
|
|
%\begin{subfigs}{vibtration_grains}{Augmenting perceived stiffness of a real surface with vibrations. }
|
|
% \subfigsheight{35mm}
|
|
% \subfig{sabnis2023haptic_device}
|
|
% \subfig{sabnis2023haptic_control}
|
|
%\end{subfigs}
|
|
|
|
%\textcite{choi2021perceived} combined and compared these two rendering approaches (spring-damper and exponential decaying sinusoids) but to render purely virtual surfaces.
|
|
%They found that the perceived intensity of the virtual hardness $\tilde{h}$ followed a power law, similarly to \eqref{hardness_intensity}, with the amplitude $a$, the %frequency $f$ and the damping $b$ of the vibration, but not the decay time $\tau$.
|
|
%\cite{park2023perceptual}
|
|
|
|
%\subsubsection{Friction}
|
|
%\label{friction_rendering}
|
|
|
|
%\cite{konyo2008alternative}
|
|
%\cite{provancher2009fingerpad}
|
|
%\cite{smith2010roughness}
|
|
%\cite{jeon2011extensions}
|
|
%\cite{salazar2020altering}
|
|
%\cite{yim2021multicontact}
|
|
|
|
%\subsubsection{Weight}
|
|
%\label{weight_rendering}
|
|
|
|
%\cite{minamizawa2007gravity}
|
|
%\cite{minamizawa2008interactive}
|
|
%\cite{jeon2011extensions}
|
|
%\cite{choi2017grabity}
|
|
%\cite{culbertson2017waves}
|
|
|
|
\subsection{Conclusion}
|
|
\label{wearable_haptics_conclusion}
|
|
|
|
Haptic systems aim to provide virtual interactions and sensations similar to those with real objects.
|
|
The complexity of the haptic sense has led to the design of numerous haptic devices and renderings.
|
|
While many haptic devices can be worn on the hand, only a few can be considered wearable as they are compact and portable, but they are limited to cutaneous feedback.
|
|
If the haptic rendering of the device is timely associated with the user's touch actions on a real object, the perceived haptic properties of the object can be modified.
|
|
Several haptic augmentation methods have been developed to modify the perceived roughness and hardness, mostly using vibrotactile feedback and, to a lesser extent, pressure feedback.
|
|
However, not all of these haptic augmentations have yet been already transposed to wearable haptics, and the use of wearable haptic augmentations has not yet been investigated in the context of \AR.
|
|
|
|
%, unlike most previous actuators that are designed specifically for fingertips and would require mechanical adaptation to be placed on other parts of the hand.
|
|
% thanks to the vibration propagation and the sensory capabilities distributed throughout the skin, they can be placed without adaption and on any part of the hand
|