WIP

1-introduction/introduction/introduction.tex

@@ -70,7 +70,7 @@ But their use in combination with \AR has been little explored so far.
         \item Wolverine, a wearable exoskeleton that simulate contact and grasping of virtual objects with force feedback on the fingers~\cite{choi2016wolverine}.
         \item Touch\&Fold, a \WH device mounted on the nail that fold on demand to render contact, normal force and vibrations to the fingertip~\cite{teng2021touch}.
         \item The hRing, a \WH ring mounted on the proximal phalanx able to render normal and shear forces to the finger~\cite{pacchierotti2016hring}.
-        \item Tasbi, a haptic bracelet capable of rendering squeeze and vibrotactile feedback to the wrist~\cite{pezent2019tasbi}.
+        \item Tasbi, a haptic bracelet capable of rendering squeeze and vibrotactile feedback to the wrist~\cite{pezent2022design}.
     ]
     \subfigsheight{28mm}
     \subfig{choi2016wolverine}

1-introduction/related-work/2-wearable-haptics.tex

@@ -109,20 +109,20 @@ Some actuators are capable of both normal and tangential motion over 3 \DoFs on
 A simpler alternative approach is to place a belt under the finger, and to actuate it over 2 \DoFs by two motors placed on top of the finger~\cite{minamizawa2007gravity}.
 By turning in opposite directions, the motors shorten the belt and create a sensation of pressure.
 Conversely, by turning simultaneously in the same direction, the belt pulls on the skin, creating a shearing sensation.
-The simplicity of this approach allows the belt to be placed anywhere on the hand, leaving the fingertip free to interact with the \RE, \eg the hRing on the proximal phalanx in \figref{pezent2019tasbi}~\cite{pacchierotti2016hring} or Tasbi on the wrist in \figref{pezent2019tasbi}~\cite{pezent2019tasbi}.
+The simplicity of this approach allows the belt to be placed anywhere on the hand, leaving the fingertip free to interact with the \RE, \eg the hRing on the proximal phalanx in \figref{pacchierotti2016hring}~\cite{pacchierotti2016hring} or Tasbi on the wrist in \figref{pezent2022design}~\cite{pezent2022design}.
 
 \begin{subfigs}{tangential_belts}{Tangential motion actuators and compression belts. }[
         \item A skin strech actuator for the fingertip~\cite{leonardis2015wearable}.
         \item A 3 \DoF actuator capable of normal and tangential motion on the fingertip~\cite{schorr2017fingertip}.
         %\item A shearing belt actuator for the fingertip~\cite{minamizawa2007gravity}.
         \item The hRing, a shearing belt actuator for the proximal phalanx of the finger~\cite{pacchierotti2016hring}.
-        \item Tasbi, a wristband capable of pressure and vibrotactile feedback~\cite{pezent2019tasbi}.
+        \item Tasbi, a wristband capable of pressure and vibrotactile feedback~\cite{pezent2022design}.
     ]
     \subfigsheight{33.5mm}
     \subfig{leonardis2015wearable}
     \subfig{schorr2017fingertip}
     \subfig{pacchierotti2016hring}
-    \subfig{pezent2019tasbi}
+    \subfig{pezent2022design}
 \end{subfigs}
 
 \subsubsection{Vibrotactile Actuators}
@@ -165,7 +165,7 @@ Several types of vibrotactile actuators are used in haptics, with different trad
 
 Tactile rendering of haptic properties consists in modelling and reproducing virtual tactile sensations comparable to those perceived when interacting with real objects.
 By adding such tactile rendering as feedback to the touch actions of the hand on a real object~\cite{bhatia2024augmenting}, both the real and virtual haptic sensations are integrated into a single property perception~\cite{ernst2004merging}.
-Therefore, the visual rendering of a touched object can also greatly influence the perception of its haptic properties, \eg by modifying its visual texture in \AR or \VR, as discussed in the \secref{visuo_haptic_ar}.
+Therefore, the visual rendering of a touched object can also greatly influence the perception of its haptic properties, \eg by modifying its visual texture in \AR or \VR, as discussed in the \secref{visuo_haptic}.
 
 \textcite{bhatia2024augmenting} categorize the tactile augmentations of real objects into three types: direct touch, touch-through, and tool mediated.
 In direct touch, the haptic device does not cover the interior of the hand to not impair the user to interact with the \RE.

1-introduction/related-work/4-visuo-haptic-ar.tex

@@ -1,5 +1,5 @@
 \section{Visuo-Haptic Augmentations of Hand-Object Interactions}
-\label{visuo_haptic_ar}
+\label{visuo_haptic}
 
 % Answer the following four questions: “Who else has done work with relevance to this work of yours? What did they do? What did they find? And how is your work here different?”
 
@@ -51,7 +51,7 @@ Conversely, as discussed by \textcite{ujitoko2021survey} in their review, a co-l
 
 Even before manipulating a visual representation to induce a haptic sensation, shifts and latencies between user input and co-localised visuo-haptic feedback can be experienced differently in AR and VR, which we aim to investigate in this work.
 
-\subsubsection{Comparing Haptics in AR \vs VR}
+\subsubsection{Comparing Haptic Perception in AR \vs VR}
 \label{AR_vs_VR}
 
 A few studies specifically compared visuo-haptic perception in AR \vs VR.
@@ -64,21 +64,16 @@ While a large literature has investigated these differences in visual perception
 \subsection{Wearable Haptics for AR}
 \label{vhar_haptics}
 
-Some wearable haptic devices have been specifically designed for direct hand interaction in immersive \AR.
+A few wearable haptic devices have been specifically designed or experimentally tested for direct hand interaction in immersive \AR.
 The main challenge of wearable haptics for \AR is to provide haptic sensations of virtual or augmented objects that are touched and manipulated directly with the fingers while keeping the fingertips free to interact with the \RE.
-%In this context of integrating \WHs with \AR to create a \vh-\AE (see \chapref{introduction}), the definition of \textcite{pacchierotti2017wearable} can be extended to an additional criterion: The wearable haptic interface should not impair the interaction with the \RE, \ie the user should be able to touch and manipulate objects in the real world while wearing the haptic device.
-%Many approaches have been proposed and they differ greatly in the actuators used (see \secref{wearable_haptic_devices}), the type of rendered object (real or virtual), the rendered haptic property (contact, hardness, texture, see \secref{tactile_rendering}), and the placement of the haptic rendering.
+Several approaches have been proposed to move the actuator away to another location on the hand.
+Yet, they differ greatly in the actuators used (see \secref{wearable_haptic_devices}) thus the haptic feedback (see \secref{tactile_rendering}), and the placement of the haptic rendering.
 
-As with the general overview of wearable haptic devices for the hand in \secref{wearable_haptics}, the type of the actuator used strongly determines the haptic feedback.
-For \AR, we distinguish between devices that rely on mechanical actuators, similar to those described in \secref{wearable_haptic_devices}, and devices that use electrical phenomena to generate haptic sensations.
-A few other wearable haptic devices have been proposed and tested in immersive \AR, such as thin-skin tactile interfaces~\cite{withana2018tacttoo,teng2024haptic} or fluid-based interfaces~\cite{han2018hydroring}, but as they permanently cover the fingertip and affect the interaction with the \RE, they are not detailed here.
+Other wearable haptic actuators have been proposed for \AR but are not detailed here.
+A first reason is that they permanently cover the fingertip and affect the interaction with the \RE, such as thin-skin tactile interfaces~\cite{withana2018tacttoo,teng2024haptic} or fluid-based interfaces~\cite{han2018hydroring}.
+Another category of actuators relies on systems that cannot be considered as portable, such as REVEL~\cite{bau2012revel} that provide friction sensations with reverse electrovibration that need to modify the real objects to augment, or Electrical Muscle Stimulation (EMS) devices~\cite{lopes2018adding} that provide kinesthetic feedback by contracting the muscles.
 
-\subsubsection{Mechanical Rendering}
-
-All wearable haptics for \AR using mechanical actuators (as in \secref{wearable_haptic_devices}) move the actuator away from the fingertip to another location on the hand.
-The haptic feedback is thus rendered de-localized from the point of contact of the finger on the rendered object.
-
-\paragraph{Nail-Mounted Devices}
+\subsubsection{Nail-Mounted Devices}
 
 \textcite{ando2007fingernailmounted} were the first to propose this approach that they experimented with a voice-coil mounted on the index nail (see \figref{ando2007fingernailmounted}).
 The sensation of crossing edges of a virtual patterned texture (see \secref{texture_rendering}) on a real sheet of paper were rendered with \qty{20}{\ms} vibration impulses at \qty{130}{\Hz}.
@@ -98,45 +93,72 @@ Mounted on the nail, the device actuates two rollers, one on each side of the fi
 By doing quick rotations, the rollers can also simulate a texture sensation.
 %The device is also very compact (\qty{60 x 25 x 36}{\mm}), lightweight (\qty{18}{\g}), and portable with a battery and Bluetooth wireless communication with \qty{83}{\ms} latency.
 In a user study not in \AR, but involving touching different images on a tablet, Fingeret was found to be more realistic (4/7) than a \LRA at \qty{100}{\Hz} on the nail (3/7) for rendering buttons and a patterned texture (see \secref{texture_rendering}), but not different from vibrations for rendering high-frequency textures (3.5/7 for both).
-However, as for \textcite{teng2021touch}, finger speed was not taken into account for rendering vibrations, which may have been detrimental to texture perception.
+However, as for \textcite{teng2021touch}, finger speed was not taken into account for rendering vibrations, which may have been detrimental to texture perception (see \secref{texture_rendering}).
 
 \begin{subfigs}{ar_wearable}{Nail-mounted wearable haptic devices designed for \AR. }[
         \item A voice-coil rendering a virtual haptic texture on a real sheet of paper~\cite{ando2007fingernailmounted}.
         \item Touch\&Fold provide contact pressure and vibrations on demand to the fingertip~\cite{teng2021touch}.
         \item Fingeret is a finger-side wearable haptic device that pulls and pushs the fingertip skin~\cite{maeda2022fingeret}.
     ]
-    \subfigsheight{34mm}
+    \subfigsheight{33mm}
     \subfig{ando2007fingernailmounted}
     \subfig{teng2021touch}
     \subfig{maeda2022fingeret}
 \end{subfigs}
 
-\paragraph{Bracelet Devices}
+\subsubsection{Ring Belt Devices}
 
-With their \enquote{Tactile And Squeeze Bracelet Interface} (Tasbi), already mentioned in \secref{belt_actuators}, \textcite{pezent2019tasbi} explored the use of a wrist-worn bracelet actuator.
-It is capable of providing a uniform pressure sensation (up to \qty{15}{\N} and \qty{10}{\Hz}) and vibration with six \LRAs (\qtyrange{150}{200}{\Hz} bandwidth).
-Although the device has not been tested in \AR, a user study conducted in \VR to compare the perception of haptic and visual stiffness rendering of a virtual button is detailed in the \secref{vhar_interaction}.
+The haptic ring belt devices of \textcite{minamizawa2007gravity} and \textcite{pacchierotti2016hring}, presented in \secref{belt_actuators}, have been employed to improve the manipulation of real and virtual objects in \AR.
 
-\subsubsection{Electrical Based Rendering}
+In a \VST-\AR setup, \textcite{scheggi2010shape} explored the effect of rendering the weight (see \secref{weight_rendering}) of a virtual cube placed on a real surface hold with the thumb, index, and middle fingers (see \figref{scheggi2010shape}).
+The middle phalanx of each of these fingers was equipped with a haptic ring of \textcite{minamizawa2007gravity}.
+However, no proper user study was conducted to evaluate this feedback.% on the manipulation of the cube.
+%that simulated the weight of the cube.
+%A virtual cube that could push on the cube was manipulated with the other hand through a force-feedback device.
+%\textcite{scheggi2010shape} report that \percent{80} of the participants appreciated the weight feedback.
 
-\cite{bau2012revel} alterer the texture of touched real objects using reverse electrovibration. They call this kind of haptic devices that can alter the touch perception of any object without any setup as *intrinsic haptic displays*.
-
-\cite{lopes2018adding}
-
-
-\subsection{Improving the Interactions with Virtual Objects}
-\label{vhar_interaction}
-
-\cite{scheggi2010shape}
-\cite{pacchierotti2015cutaneous}
-
-In pick-and-place tasks in AR involving both virtual and real objects, \textcite{maisto2017evaluation} and \textcite{meli2018combining} showed that having a haptic {rendering of the} fingertip interactions with the virtual objects led to better performance and perceived effectiveness than having only a visual rendering of the hand.
-Moreover, employing the haptic ring of~\cite{pacchierotti2016hring} on the proximal finger phalanx led to an improved performance with respect to more standard fingertip haptic devices~\cite{chinello2020modular}.
+In pick-and-place tasks in non-immersive \VST-\AR involving both virtual and real objects (see \figref{maisto2017evaluation}), \textcite{maisto2017evaluation} and \textcite{meli2018combining} compared the effects of providing haptic feedback about contacts at the fingertips using either the haptic ring of \textcite{pacchierotti2016hring}, or on the proximal phalanx, the moving platform of \textcite{chinello2020modular} on the fingertip.
+They showed that the haptic feedback improved the performance (completion time), reduced the exerted force on the cubes over a visual feedback alone.
+The haptic ring was also perceived by users to be more effective than the moving platform.
 However, the measured difference in performance could be attributed to either the device or the device position (proximal vs fingertip), or both.
+These two studies were also conducted in non-immersive setups, where users looked at a screen displaying the visual interactions, and only compared haptic and visual feedback, but did not examine them together.
 
-Conjointly, a few studies have explored and compared the effects of visual and haptic feedback in tasks involving the manipulation of virtual objects with the hand.
-\textcite{sarac2022perceived} and \textcite{palmer2022haptic} studied the effects of providing haptic feedback about contacts at the fingertips using haptic devices worn at the wrist, testing different mappings.
-Results proved that moving the haptic feedback away from the point(s) of contact is possible and effective, and that its impact is more significant when the visual feedback is limited.
+\begin{subfigs}{ar_rings}{Wearable haptic ring devices for \AR. }[
+        \item Rendering weight of a virtual cube placed on a real surface~\cite{scheggi2010shape}.
+        \item Rendering the contact force exerted by the fingers on a virtual cube~\cite{maisto2017evaluation,meli2018combining}.
+    ]
+    \subfigsheight{53mm}
+    \subfig{scheggi2010shape}
+    \subfig{maisto2017evaluation}
+\end{subfigs}
 
-Furthermore, all of these studies were conducted in non-immersive setups, where users looked at a screen displaying the visual interactions, and only compared haptic and visual feedback, but did not examine them together.
-The improved performance and perceived effectiveness of a delocalized haptic feedback over a visual feedback alone, or their multimodal combination, remains to be verified in an immersive OST-AR setup.
+
+\subsubsection{Wrist Bracelet Devices}
+
+With their \enquote{Tactile And Squeeze Bracelet Interface} (Tasbi), already mentioned in \secref{belt_actuators}, \textcite{pezent2019tasbi} and \textcite{pezent2022design} explored the use of a wrist-worn bracelet actuator.
+It is capable of providing a uniform pressure sensation (up to \qty{15}{\N} and \qty{10}{\Hz}) and vibration with six \LRAs (\qtyrange{150}{200}{\Hz} bandwidth).
+Although the device has not been tested in \AR, a user study was conducted in \VR to compare the perception of visuo-haptic stiffness rendering~\cite{pezent2019tasbi}.
+Participants pressed a virtual button with different levels of stiffness using a virtual hand, constrained by the \VE (see \figref{pezent2019tasbi_2}).
+A higher visual stiffness required a larger physical displacement to press the button (C/D ratio, see \secref{pseudo_haptic}), while the haptic stiffness control the rate of the pressure feedback when pressing.
+When the visual and haptic stiffness were coherent or when only the haptic stiffness changed, participants easily discriminated two buttons with different stiffness levels  (see \figref{pezent2019tasbi_3}).
+However, if only the visual stiffness changed, participants were not able to discriminate the different stiffness levels (see \figref{pezent2019tasbi_4}).
+This suggests that in \VR, the haptic pressure is more important perceptual cue than the visual displacement to render stiffness.
+A short vibration (\qty{25}{\ms} \qty{175}{\Hz} square-wave) was also rendered when contacting the button, but kept constant across all conditions: It may have affected the overall perception when only the visual stiffness changed.
+
+\begin{subfigs}{pezent2019tasbi}{Visuo-haptic stiffness rendering of a virtual button in \VR with the Tasbi bracelet. }[
+        \item The \VE seen by the user: the virtual hand (in beige) is constrained by the virtual button.  The displacement is proportional to the visual stiffness. The real hand (in green) is hidden by the \VE.
+        \item When the rendered visuo-haptic stiffness are coherents (in purple) or only the haptic stiffness change (in blue), participants easily discrimated the different levels.
+        \item When varying only the visual stiffness (in red) but keeping the haptic stiffness constant, participants were not able to discriminate the different stiffness levels.
+    ]
+    \subfigsheight{45mm}
+    \subfig{pezent2019tasbi_2}
+    \subfig{pezent2019tasbi_3}
+    \subfig{pezent2019tasbi_4}
+\end{subfigs}
+
+\subsection{Conclusion}
+\label{visuo_haptic_conclusion}
+
+% the type of rendered object (real or virtual), the rendered haptic property (contact, hardness, texture, see \secref{tactile_rendering}), and .
+%In this context of integrating \WHs with \AR to create a \vh-\AE (see \chapref{introduction}), the definition of \textcite{pacchierotti2017wearable} can be extended to an additional criterion: The wearable haptic interface should not impair the interaction with the \RE, \ie the user should be able to touch and manipulate objects in the real world while wearing the haptic device.
+% The haptic feedback is thus rendered de-localized from the point of contact of the finger on the rendered object.

1-introduction/related-work/articles/1-hands.tex

@@ -58,13 +58,13 @@ Different haptic feedback systems have been explored to improve interactions in
 grounded force feedback devices~\cite{bianchi2006high, jeon2009haptic, knorlein2009influence}, %
 exoskeletons~\cite{lee2021wearable}, %
 tangible objects~\cite{hettiarachchi2016annexing, detinguy2018enhancing, salazar2020altering, normand2018enlarging, xiao2018mrtouch}, and %
-wearable haptic devices~\cite{pacchierotti2016hring, lopes2018adding, pezent2019tasbi, teng2021touch}.
+wearable haptic devices~\cite{pacchierotti2016hring, lopes2018adding, pezent2022design, teng2021touch}.
 
-Wearable haptics seems particularly suited for this context, as it takes into account many of the AR constraints, \eg limited impact on hand tracking performance and reduced impairment of the senses and ability of the users to interact with real content~\cite{pacchierotti2016hring, maisto2017evaluation, lopes2018adding, meli2018combining, pezent2019tasbi, teng2021touch, kourtesis2022electrotactile, marchal2022virtual}.
+Wearable haptics seems particularly suited for this context, as it takes into account many of the AR constraints, \eg limited impact on hand tracking performance and reduced impairment of the senses and ability of the users to interact with real content~\cite{pacchierotti2016hring, maisto2017evaluation, lopes2018adding, meli2018combining, pezent2022design, teng2021touch, kourtesis2022electrotactile, marchal2022virtual}.
 %
 For example, \textcite{pacchierotti2016hring} designed a haptic ring providing pressure and skin stretch sensations to be worn at the proximal finger phalanx, so as to improve the hand tracking during a pick-and-place task.
 %
-\textcite{pezent2019tasbi} proposed Tasbi: a wristband haptic device capable of rendering vibrations and pressures.
+\textcite{pezent2022design} proposed Tasbi: a wristband haptic device capable of rendering vibrations and pressures.
 %
 \textcite{teng2021touch} presented Touch\&Fold, a haptic device attached to the nail that provides pressure and texture sensations when interacting with virtual content, but also folds away when the user interacts with real objects, leaving the fingertip free.
 %

1-introduction/related-work/articles/3-ar-textures.tex

@@ -6,7 +6,7 @@
 As in VR, the addition of haptic feedback in AR has been explored through numerous approaches, including %
 grounded force feedback devices~\cite{jeon2009haptic,knorlein2009influence,hachisu2012augmentation,gaffary2017ar}, %
 exoskeletons~\cite{lee2021wearable}, %
-wearable haptic devices~\cite{maisto2017evaluation,detinguy2018enhancing,lopes2018adding,meli2018combining,pezent2019tasbi,teng2021touch},
+wearable haptic devices~\cite{maisto2017evaluation,detinguy2018enhancing,lopes2018adding,meli2018combining,pezent2022design,teng2021touch},
 tangible objects~\cite{punpongsanon2015softar,hettiarachchi2016annexing,kahl2021investigation}, and %
 mid-air haptics~\cite{ochiai2016crossfield}. %
 %
@@ -24,7 +24,7 @@ For example, mounted on the nail, the haptic device of \textcite{teng2021touch}
 %
 It is however not suitable for rendering haptic feedback when touching real objects.
 %
-In this respect, some wearable haptic devices were specifically designed to provide haptic feedback about fingertip interactions with the virtual content, but delocalized elsewhere on the body: on the proximal finger phalanx with the hRing haptic ring device~\cite{pacchierotti2016hring,ferro2023deconstructing}, on the wrist with the Tasbi bracelet~\cite{pezent2019tasbi}, or on the arm~\cite{lopes2018adding}.
+In this respect, some wearable haptic devices were specifically designed to provide haptic feedback about fingertip interactions with the virtual content, but delocalized elsewhere on the body: on the proximal finger phalanx with the hRing haptic ring device~\cite{pacchierotti2016hring,ferro2023deconstructing}, on the wrist with the Tasbi bracelet~\cite{pezent2022design}, or on the arm~\cite{lopes2018adding}.
 %
 Compared to a fingertip worn device, the hRing was even preferred by participants and perceived as more effective in virtual object manipulation task in AR~\cite{maisto2017evaluation,meli2018combining}.
 %

3-manipulation/visuo-haptic-hand/1-introduction.tex

@@ -1,2 +1,6 @@
 \section{Introduction}
 \label{introduction}
+
+%  Conjointly, a few studies have explored and compared the effects of visual and haptic feedback in tasks involving the manipulation of virtual objects with the hand.
+% \textcite{sarac2022perceived} and \textcite{palmer2022haptic} studied the effects of providing haptic feedback about contacts at the fingertips using haptic devices worn at the wrist, testing different mappings.
+% Results proved that moving the haptic feedback away from the point(s) of contact is possible and effective, and that its impact is more significant when the visual feedback is limited.