erwan/phd-thesis
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Remove comments

Browse Source
This commit is contained in:
Erwan Normand
2025-05-01 22:31:51 +02:00
parent 0202efeb06
commit e01e63d4cf
22 changed files with 19 additions and 244 deletions
Download Patch File Download Diff File Expand all files Collapse all files

97
2-related-work/1-haptic-hand.tex
View File

@@ -1,12 +1,9 @@
\section{Perception and Interaction with the Hand}
\label{haptic_hand}
% describe how the hand senses and acts on its environment to perceive the haptic properties of real everyday objects.
The haptic sense has specific characteristics that make it unique in regard to other senses.
It enables us to perceive a large diversity of properties of everyday objects, up to a complex combination of sensations produced by numerous sensory receptors distributed throughout the body, but especially in the hand \cite{johansson2009coding}.
It also allows us to act on these objects, to come into contact with them, to grasp them and to actively explore them. % , and to manipulate them.
%This implies that the haptic perception is localized at the points of contact between the hand and the environment, \ie we cannot haptically perceive an object without actively touching it.
It also allows us to act on these objects, to come into contact with them, to grasp them and to actively explore them.
These two mechanisms, \emph{action} and \emph{perception}, are closely associated and both are essential to form a complete haptic experience of interacting with the environment using the hand \cite{lederman2009haptic}.
\subsection{The Haptic Sense}
@@ -188,8 +185,7 @@ But when running the finger over the surface with a lateral movement (\secref{ex
In particular, when the asperities are smaller than \qty{0.1}{mm}, such as paper fibers, the pressure cues are no longer captured and only the movement, \ie the vibrations, can be used to detect the roughness \cite{hollins2000evidence}.
This limit distinguishes \emph{macro-roughness} from \emph{micro-roughness}.
%The physical properties of the surface determine the haptic perception of roughness.
The perception of roughness can be characterized by the density of the surface elements: the perceived (subjective) intensity of roughness increases with the spacing between the elements. %, for macro-roughness \cite{klatzky2003feeling,lawrence2007haptic} and micro-roughness \cite{bensmaia2003vibrations}.
The perception of roughness can be characterized by the density of the surface elements: the perceived (subjective) intensity of roughness increases with the spacing between the elements.
For macro-textures, the size of the elements, the force applied and the speed of exploration have limited effects on the intensity perceived \cite{klatzky2010multisensory}: macro-roughness is a \emph{spatial perception}.
This allows us to read Braille \cite{lederman2009haptic}.
However, the speed of exploration affects the perceived intensity of micro-roughness \cite{bensmaia2003vibrations}.
@@ -279,99 +275,10 @@ Stiffness depends on the structure of the object: a thick object can be more com
\subfig[.45]{bergmanntiest2009cues}
\end{subfigs}
%\textcite{bergmanntiest2009cues} showed how of these two physical measures in the perception of hardness.
An object with low stiffness, but high Young's modulus can be perceived as hard, and vice versa, as shown in \figref{bergmanntiest2009cues}.
With finger pressure, a relative difference (the \emph{Weber fraction}) of \percent{\sim 15} is required to discriminate between two objects of different stiffness or elasticity.
However, in the absence of pressure sensations (by placing a thin disc between the finger and the object), the necessary relative difference becomes much larger (Weber fraction of \percent{\sim 50}).
That is, \textcite{bergmanntiest2009cues} showed the perception of hardness relies on \percent{90} on surface deformation cues and \percent{10} on displacement cues.
%Finally, when pressing with the finger, the perceived hardness intensity $h$ follows a power law with the stiffness $k$ \cite{harper1964subjective}:
%\begin{equation}{hardness_intensity}
% h = k^{0.8}
%\end{equation}
%En pressant du doigt, l'intensité perçue (subjective) de dureté suit avec la raideur une relation selon une loi de puissance avec un exposant de \num{0.8} \cite{harper1964subjective}, \ie quand la raideur double, la dureté perçue augmente de \num{1.7}.
%\textcite{bergmanntiest2009cues} ont ainsi observé une relation quadratique d'égale intensité perçue de dureté, comme illustré sur la \figref{bergmanntiest2009cues}.
%\subsubsection{Friction}
%\label{friction}
%
%Friction (or slipperiness) is the perception of \emph{resistance to movement} on a surface \cite{bergmanntiest2010tactual}.
%Sandpaper is typically perceived as sticky because it has a strong resistance to sliding on its surface, while glass is perceived as more slippery.
%This perceptual property is closely related to the perception of roughness \cite{hollins1993perceptual,baumgartner2013visual}.
%
%When running the finger on a surface with a lateral movement (\secref{exploratory_procedures}), the skin-surface contacts generate frictional forces in the opposite direction to the finger movement, giving kinesthetic cues, and also stretch the skin, giving cutaneous cues.
%As illustrated in \figref{smith1996subjective_1}, a stick-slip phenomenon can also occur, where the finger is intermittently slowed by friction before continuing to move, on both rough and smooth surfaces \cite{derler2013stick}.
%The amplitude of the frictional force $F_s$ is proportional to the normal force of the finger $F_n$, \ie the force perpendicular to the surface, according to a coefficient of friction $\mu$:
%\begin{equation}{friction}
% F_s = \mu \, F_n
%\end{equation}
%The perceived intensity of friction is thus roughly related to the friction coefficient $\mu$ \cite{smith1996subjective}.
%However, it is a complex perception because it is more determined by the micro-scale interactions between the surface and the skin: It depends on many factors such as the normal force applied, the speed of movement, the contact area and the moisture of the skin and the surface \cite{adams2013finger,messaoud2016relation}.
%In this sense, the perception of friction is still poorly understood \cite{okamoto2013psychophysical}.
%
%\begin{subfigs}{smith1996subjective}{Perceived intensity of friction of different materials by active exploration with the finger \cite{smith1996subjective}. }[
% \item Measurements of normal $F_n$ and tangential $F_t$ forces when exploring two surfaces: one smooth (glass) and one rough (nyloprint). The fluctuations in the tangential force are due to the stick-slip phenomenon. The coefficient of friction $\mu$ can be estimated as the slope of the relationship between the normal and tangential forces.
% \item Perceived friction intensity (vertical axis) as a function of the estimated friction coefficient $\mu$ of the exploration (horizontal axis) for four materials (shapes and colors).
% ]
% \subfigsheight{55mm}
% \subfig{smith1996subjective_1}
% \subfig{smith1996subjective_2}
%\end{subfigs}
%
%Yet, it is a fundamental perception for grasping and manipulating objects.
%The forces of friction make it indeed possible to hold the object firmly in the hand and prevent it from slipping
%The perception of friction also allows us to automatically and very quickly adjust the force we apply to the object in order to grasp it \cite{johansson1984roles}.
%If the finger is anaesthetized, the lack of cutaneous sensation prevents effective adjustment of the gripping force: the forces of the object on the finger are no longer correctly perceived, and the fingers then press harder on the object in compensation, but without achieving good opposition of the fingers \cite{witney2004cutaneous}.
%\subsubsection{Temperature}
%\label{temperature}
%
%Temperature (or coldness/warmness) is the perception of the \emph{transfer of heat} between the touched surface and the skin \cite{bergmanntiest2010tactual}:
%When heat is removed from (added to) the skin, the surface is perceived as cold (hot).
%Metal will be perceived as colder than wood at the same room temperature: This perception is different from the physical temperature of the material and is therefore an important property for distinguishing between materials \cite{ho2006contribution}.
%This perception depends on the thermal conductivity and heat capacity of the material, the volume of the object, the initial temperature difference and the area of contact between the surface and the skin \cite{kappers2013haptic}.
%For example, a larger object or a smoother surface, which increases the contact area, causes more heat circulation and a more intense temperature sensation (hot or cold) \cite{bergmanntiest2008thermosensory}.
%Parce qu'elle est basée sur la circulation de la chaleur, la perception de la température est plus lente que les autres propriétés matérielles et demande un toucher statique (voir \figref{exploratory_procedures}) de plusieurs secondes pour que la température de la peau s'équilibre avec celle de l'objet.
%La température $T(t)$ du doigt à l'instant $t$ et au contact avec une surface suit une loi décroissante exponentielle, où $T_s$ est la température initiale de la peau, $T_e$ est la température de la surface, $t$ est le temps et $\tau$ est la constante de temps:
%\begin{equation}{temperature}
% T(t) = (T_s - T_e) \, e^{-t / \tau} + T_e
%\end{equation}
%Le taux de transfert de chaleur, décrit par $\tau$, et l'écart de température $T_s - T_e$, sont les deux indices essentiels pour la perception de la température.
%Dans des conditions de la vie de tous les jours, avec une température de la pièce de \qty{20}{\celsius}, une différence relative du taux de transfert de chaleur de \percent{43} ou un écart de \qty{2}{\celsius} est nécessaire pour percevoir une différence de température \cite{bergmanntiest2009tactile}.
%\subsubsection{Spatial Properties}
%\label{spatial_properties}
%Weight, size and shape are haptic spatial properties that are independent of the material properties described above.
%Weight (or heaviness/lightness) is the perceived \emph{mass} of the object \cite{bergmanntiest2010haptic}.
%It is typically estimated by holding the object statically in the palm of the hand to feel the gravitational force (\secref{exploratory_procedures}).
%A relative weight difference of \percent{8} is then required to be perceptible \cite{brodie1985jiggling}.
%By lifting the object, it is also possible to feel the object's force of inertia, \ie its resistance to velocity.
%This provides an additional perceptual cue to its mass and slightly improves weight discrimination.
%For both gravity and inertia, kinesthetic cues to force are much more important than cutaneous cues to pressure \cite{bergmanntiest2012investigating}.
%Le lien entre le poids physique et l'intensité perçue est variable selon les individus \cite{kappers2013haptic}.
%Size can be perceived as the object's \emph{length} (in one dimension) or its \emph{volume} (in three dimensions) \cite{kappers2013haptic}.
%In both cases, and if the object is small enough, a precision grip (\figref{gonzalez2014analysis}) between the thumb and index finger can discriminate between sizes with an accuracy of \qty{1}{\mm}, but with an overestimation of length (power law with exponent \qty{1.3}).
%Alternatively, it is necessary to follow the contours of the object with the fingers to estimate its length (\secref{exploratory_procedures}), but with ten times less accuracy and an underestimation of length (power law with an exponent of \qty{0.9}) \cite{bergmanntiest2011cutaneous}.
%The perception of the volume of an object that is not small is typically done by hand enclosure, but the estimate is strongly influenced by the size, shape and mass of the object, for an identical volume \cite{kahrimanovic2010haptic}.
%The shape of an object can be defined as the perception of its \emph{global geometry}, \ie its shape and contours.
%This is the case, for example, when looking for a key in a pocket.
%The exploration of contours and enclosure are then employed, as for the estimation of length and volume.
%If the object is not known in advance, object identification is rather slow, taking several seconds \cite{norman2004visual}.
%Therefore, the exploration of other properties is favoured to recognize the object more quickly, in particular marked edges \cite{klatzky1987there}, \eg a screw among nails (\figref{plaisier2009salient_2}), or certain material properties \cite{lakatos1999haptic,plaisier2009salient}, \eg a metal object among plastic objects.
%\begin{subfigs}{plaisier2009salient}{Identifcation of a sphere among cubes \cite{plaisier2009salient}. }[
% \item The shape has a significant effect on the perception of the volume of an object, \eg a sphere is perceived smaller than a cube of the same volume.
% \item The absence of a marked edge on the sphere makes it easy to identify among cubes.
% ]
% \subfigsheight{40mm}
% \subfig{plaisier2009salient_1}
% \subfig{plaisier2009salient_2}
%\end{subfigs}
\subsection{Conclusion}
\label{haptic_sense_conclusion}

51
2-related-work/2-wearable-haptics.tex
View File

@@ -52,8 +52,6 @@ Moreover, as detailed in \secref{object_properties}, cutaneous sensations are ne
\subfig{leonardis20173rsr}
\end{subfigs}
% Tradeoff realistic and cost + analogy with sound, Hi-Fi costs a lot and is realistic, but 40$ BT headphone is more practical and enough, as cutaneous feedback without kinesthesic could be enough for wearable haptics and far more affordable and comfortable than world- or body-grounded haptics + cutaneous even better than kine for rendering surface curvature and fine manipulation
\subsection{Wearable Haptic Devices for the Hand}
\label{wearable_haptic_devices}
@@ -114,7 +112,6 @@ The simplicity of this approach allows the belt to be placed anywhere on the han
\begin{subfigs}{tangential_belts}{Tangential motion actuators and compression belts. }[][
\item A skin strech actuator for the fingertip \cite{leonardis2015wearable}.
\item A 3 \DoF actuator capable of normal and tangential motion on the fingertip \cite{schorr2017fingertip}.
%\item A shearing belt actuator for the fingertip \cite{minamizawa2007gravity}.
\item The hRing, a shearing belt actuator for the proximal phalanx of the finger \cite{pacchierotti2016hring}.
\item Tasbi, a wristband capable of pressure and vibrotactile feedback \cite{pezent2022design}.
]
@@ -185,27 +182,19 @@ In particular, the visual rendering of a touched object can also influence the p
\textcite{bhatia2024augmenting} categorize the haptic augmentations into three types: direct touch, touch-through, and tool-mediated.
In \emph{direct touch}, the haptic device does not cover the inside of the hand so as not to impair the interaction of the user with the \RE, and is typically achieved with wearable haptics.
In touch-through and tool-mediated, or \emph{indirect feel-through} \cite{jeon2015haptic}, the haptic device is placed between the hand and the \RE.
%We are interested in direct touch augmentations with wearable haptics (\secref{wearable_haptic_devices}), as their integration with \AR is particularly promising for free hand interaction with visuo-haptic augmentations.
Many haptic augmentations were first developed with touch-through devices, and some (but not all) were later transposed to direct touch augmentation with wearable haptic devices.
%We also focus on tactile augmentations stimulating the mechanoreceptors of the skin (\secref{haptic_sense}), thus excluding temperature perception, as they are the most common existing haptic interfaces.
Since we have chosen to focus in \secref{object_properties} on the haptic perception of roughness and hardness of objects, we review below the methods to modify the perception of these properties.
Of course, wearable haptics can also be used in a direct touch context to modify the perceived friction \cite{konyo2008alternative,salazar2020altering}, weight \cite{minamizawa2007gravity}, or local deformation \cite{salazar2020altering} of real objects, but they are rare \cite{bhatia2024augmenting} and will not be detailed here.
% \cite{klatzky2003feeling} : rendering roughness, friction, deformation, temperatures
% \cite{girard2016haptip} : renderings with a tangential motion actuator
\subsubsection{Roughness Augmentation}
\label{texture_rendering}
To modify the perception of the haptic roughness (or texture, see \secref{roughness}) of a real object, vibrations are typically applied to the skin by the haptic device as the user moves over the surface.
%This is because running the finger or a tool on a textured surface generates pressures and vibrations (\secref{roughness}) at frequencies that are too high for rendering capabilities of most haptic devices \cite{campion2005fundamental,culbertson2018haptics}.
There are two main approaches to modify virtual textures perception: \emph{simulation models} and \emph{data-driven models} \cite{klatzky2013haptic,culbertson2018haptics}.
\paragraph{Simulation Models}
%Simulations of virtual textures are based on the physics of the interaction between the finger and the surface, and are used to generate the vibrations that the user feels when running the finger over the surface.
The simplest texture simulation model is a 1D sinusoidal grating $v(t)$ with spatial period $\lambda$ and amplitude $A$ that is scanned by the user at velocity $\dot{x}(t)$:
\begin{equation}{grating_rendering}
v(t) = A \sin(\frac{2 \pi \dot{x}(t)}{\lambda})
@@ -220,8 +209,6 @@ The perceived roughness increased proportionally to the virtual texture amplitud
\textcite{ujitoko2019modulating} instead used a square wave signal and a hand-held stylus with an embedded voice-coil.
\textcite{friesen2024perceived} compared the frequency modulation of \eqref{grating_rendering} with amplitude modulation (\figref{friesen2024perceived}), and found that the frequency modulation was perceived as more similar to real sinusoidal gratings for lower spatial periods (\qty{0.5}{\mm}) but both modulations were effective for higher spatial periods (\qty{1.5}{\mm}).
%\textcite{friesen2024perceived} proposed
The model in \eqref{grating_rendering} can be extended to 2D textures by adding a second sinusoidal grating with an orthogonal orientation as \textcite{girard2016haptip}.
More complex models have also been developed to be physically accurate and reproduce with high fidelity the roughness perception of real patterned surfaces \cite{unger2011roughness}, but they require high-fidelity force feedback devices that are expensive and have a limited workspace.
@@ -241,9 +228,6 @@ A similar database, but captured from a direct touch context with the fingertip,
A common limitation of these data-driven models is that they can only render \emph{isotropic} textures: their record does not depend on the position of the measure, and the rendering is the same regardless of the direction of the movement.
This was eventually addressed to include the user's velocity direction into the capture, modelling and rendering of the textures \cite{abdulali2016datadriven,abdulali2016datadrivena}.
%A third approach is to model
%Alternative models have been proposed to both render both isotropic and patterned textures \cite{chan2021hasti}., or to simulate the vibrations from the (visual) texture maps used to visually render a \ThreeD object \cite{chan2021hasti}.
Using the user's velocity magnitude and normal force as input, these data-driven models are able to interpolate from the scan measures to generate a virtual texture in real time as vibrations with a high realism.
When comparing real textures felt through a stylus with their virtual models rendered with a voice-coil actuator attached to the stylus (\figref{culbertson2012refined}), the virtual textures were found to accurately reproduce the perception of roughness, but hardness and friction were not rendered properly \cite{culbertson2014modeling}.
\textcite{culbertson2015should} further showed that the perceived realism of the virtual textures, and similarity to the real textures, depended mostly on the user's velocity magnitude but not on the user's force as inputs to the model, \ie responding to velocity magnitude is sufficient to render isotropic virtual textures.
@@ -331,38 +315,6 @@ A challenge with this technique is to provide the vibration feedback at the righ
Vibrations on contact have been employed with wearable haptics, but to the best of our knowledge only to render virtual objects \cite{pezent2019tasbi,teng2021touch,sabnis2023haptic}.
We describe them in the \secref{vhar_haptics}.
%A promising alternative approach
%\cite{kildal20103dpress}
%\begin{subfigs}{vibtration_grains}{Augmenting perceived stiffness of a real surface with vibrations. }
% \subfigsheight{35mm}
% \subfig{sabnis2023haptic_device}
% \subfig{sabnis2023haptic_control}
%\end{subfigs}
%\textcite{choi2021perceived} combined and compared these two rendering approaches (spring-damper and exponential decaying sinusoids) but to render purely virtual surfaces.
%They found that the perceived intensity of the virtual hardness $\tilde{h}$ followed a power law, similarly to \eqref{hardness_intensity}, with the amplitude $a$, the %frequency $f$ and the damping $b$ of the vibration, but not the decay time $\tau$.
%\cite{park2023perceptual}
%\subsubsection{Friction}
%\label{friction_rendering}
%\cite{konyo2008alternative}
%\cite{provancher2009fingerpad}
%\cite{smith2010roughness}
%\cite{jeon2011extensions}
%\cite{salazar2020altering}
%\cite{yim2021multicontact}
%\subsubsection{Weight}
%\label{weight_rendering}
%\cite{minamizawa2007gravity}
%\cite{minamizawa2008interactive}
%\cite{jeon2011extensions}
%\cite{choi2017grabity}
%\cite{culbertson2017waves}
\subsection{Conclusion}
\label{wearable_haptics_conclusion}
@@ -372,6 +324,3 @@ While many haptic devices can be worn on the hand, only a few can be considered
If the haptic rendering of the device is timely associated with the user's touch actions on a real object, the perceived haptic properties of the object can be modified.
Several haptic augmentation methods have been developed to modify the perceived roughness and hardness, mostly using vibrotactile feedback and, to a lesser extent, pressure feedback.
However, not all of these haptic augmentations have yet been already transposed to wearable haptics, and the use of wearable haptic augmentations has not yet been investigated in the context of \AR.
%, unlike most previous actuators that are designed specifically for fingertips and would require mechanical adaptation to be placed on other parts of the hand.
% thanks to the vibration propagation and the sensory capabilities distributed throughout the skin, they can be placed without adaption and on any part of the hand

8
2-related-work/5-conclusion.tex
View File

@@ -12,7 +12,7 @@ Wearable haptic augmentation of roughness and hardness is mostly achieved with v
\AR headsets integrate virtual content into the user's perception as if it were part of the \RE, with real-time pose estimation of the head and hands (\secref{what_is_ar}).
Direct interaction with the hand of virtual content is often implemented using virtual hand interaction technique, which reconstructs the user's hand in the \VE and simulates its interactions with the virtual.
However, the perception and manipulation of the virtual is difficult due to the lack of haptic feedback and the mutual occlusion of the hand with the virtual content (\secref{ar_interaction}). %, which could be addressed by a visual augmentation of the hand (\secref{ar_visual_hands}).
However, the perception and manipulation of the virtual is difficult due to the lack of haptic feedback and the mutual occlusion of the hand with the virtual content (\secref{ar_interaction}).
Real surrounding objects can also be used as proxies to interact with the virtual, but they may be incoherent with their visual augmentation because they are haptically passive (\secref{ar_interaction}).
Wearable haptics on the hand seems to be a promising solution to enable coherent and effective visuo-haptic augmentation of both virtual and real objects.
@@ -31,13 +31,7 @@ Their haptic end-effector can't cover the fingertips or the inside of the palm,
Many strategies have been explored to move the actuator to other parts of the hand.
However, it is still unclear which delocalized positioning is most beneficial, and how it compares or complements the visual feedback of the virtual hand (\secref{vhar_haptics}).
%The lack of mutual occlusion during virtual object manipulation could be addressed by visual feedback of the virtual hand (\secref{ar_visual_hands}).
In \chapref{visual_hand}, we will investigate the most common visual feedback of the virtual hand as an augmentation of the real hand in \AR.
We will compare these visual hand augmentation in virtual object manipulation tasks, directly with the bare hand.
Then, in \chapref{visuo_haptic_hand}, we will evaluate five common delocalized positioning of vibrotactile feedback, combined with two contact rendering techniques and visual hand augmentations.
We will compare these visuo-haptic augmentations of the hand with the same direct hand manipulation tasks of virtual objects in \AR.
%Wearable haptics on the hand is thus another solution to improve direct hand manipulation of virtual objects in \AR.
%for rendering contact with the hand when manipulating virtual objects
%We will also investigate two contact rendering techniques and compare them with two visual hand augmentations.