Add visuo-haptic-hand chapter

3-manipulation/visuo-haptic-hand/1-introduction.tex

@@ -0,0 +1,2 @@
+\section{Introduction}
+\label{sec:introduction}

3-manipulation/visuo-haptic-hand/2-method.tex

@@ -0,0 +1,203 @@
+\section{User Study}
+\label{sec:method}
+
+Providing haptic feedback during free-hand manipulation in AR is not trivial, as wearing haptic devices on the hand might affect the tracking capabilities of the system.
+%
+Moreover, it is important to leave the user capable of interacting with both virtual and real objects, avoiding the use of haptic interfaces that cover the fingertips or palm.
+%
+For this reason, it is often considered beneficial to move the point of application of the haptic rendering elsewhere on the hand.% (see \secref{haptics}).
+
+This second experiment aims to evaluate whether a visuo-haptic hand rendering affects the performance and user experience of manipulation of virtual objects with bare hands in AR.
+%
+The chosen visuo-haptic hand renderings are the combination of the two most representative visual hand renderings established in the first experiment, \ie Skeleton and None, described in \secref{visual_hand:sec:hands}, with two contact vibration techniques provided at four delocalized positions on the hand.
+
+
+\subsection{Vibrotactile Renderings}
+\label{sec:vibration}
+
+The vibrotactile hand rendering provided information about the contacts between the virtual object and the thumb and index fingers of the user, as they were the two fingers most used for grasping in our first experiment.
+%
+We evaluated both the delocalized positioning and the contact vibration technique of the vibrotactile hand rendering.
+
+
+\subsubsection{Vibrotactile Positionings}
+\label{sec:positioning}
+
+\fig[0.30]{method/locations}{%
+    Experiment \#2: setup of the vibrotactile devices.
+    %
+    To ensure minimal encumbrance, we used the same two motors throughout the experiment, moving them to the considered positioning before each new experimental block (in this case, on the co-located proximal phalanges, \emph{Prox}).
+    %
+    Thin self-gripping straps were placed on the five considered positionings during the entirety of the experiment.
+}
+
+\begin{itemize}
+    \item \textit{Fingertips (Tips):} Vibrating actuators were placed right above the nails, similarly to~\cite{ando2007fingernailmounted}. This is the positioning closest to the fingertips.
+          %
+    \item \textit{Proximal Phalanges (Prox):} Vibrating actuators were placed on the dorsal side of the proximal phalanges, similarly to~\cite{maisto2017evaluation, meli2018combining, chinello2020modular}.
+          %
+    \item \textit{Wrist (Wris):} Vibrating actuators providing contacts rendering for the index and thumb were placed on ulnar and radial sides of the wrist, similarly to~\cite{pezent2019tasbi, palmer2022haptic, sarac2022perceived}.
+          %
+    \item \textit{Opposite fingertips (Oppo):} Vibrating actuators were placed on the fingertips of contralateral hand, also above the nails, similarly to~\cite{prattichizzo2012cutaneous, detinguy2018enhancing}.
+          %
+    \item \textit{Nowhere (Nowh):} As a reference, we also considered the case where we provided no vibrotactile rendering.
+\end{itemize}
+
+
+\subsubsection{Contact Vibration Techniques}
+\label{sec:technique}
+
+When a fingertip contacts the virtual cube, we activate the corresponding vibrating actuator.
+%
+We considered two representative contact vibration techniques, \ie two ways of rendering such contacts through vibrations:
+%
+\begin{itemize}
+    \item \textit{Impact (Impa):} a \qty{200}{\ms}--long vibration burst is applied when the fingertip makes contact with the object; the amplitude of the vibration is proportional to the speed of the fingertip at the moment of the contact.
+    \item \textit{Distance (Dist):} a continuous vibration is applied whenever the fingertip is in contact with the object; the amplitude of the vibration is proportional to the interpenetration between the fingertip and the virtual cube surface.
+\end{itemize}
+%
+The implementation of these two techniques have been tuned according to the results of a preliminary experiment.
+%
+Three participants were asked to carry out a series of push and grasp tasks similar to those used in the actual experiment.
+%
+Results showed that 95~\% of the contacts between the fingertip and the virtual cube happened at speeds below \qty{1.5}{\m\per\s}.
+%
+We also measured the perceived minimum amplitude to be 15~\% (\qty{0.6}{\g}) of the maximum amplitude of the motors we used.
+%
+For this reason, we designed the Impact vibration technique (Impa) so that contact speeds from \qtyrange{0}{1.5}{\m\per\s} are linearly mapped into \qtyrange{15}{100}{\%} amplitude commands for the motors.
+%
+Similarly, we designed the distance vibration technique (Dist) so that interpenetrations from \qtyrange{0}{2.5}{\cm} are linearly mapped into \qtyrange{15}{100}{\%} amplitude commands for the motors, recalling that the virtual cube has an edge of \qty{5}{\cm}.
+
+
+\subsection{Experimental Design}
+\label{sec:design}
+
+\begin{subfigs}{tasks}{%
+        Experiment \#2. The two manipulation tasks: %
+        (a) pushing a virtual cube along a table toward a target placed on the same surface; %
+        (b) grasping and lifting a virtual cube toward a target placed on a \qty{20}{\cm} higher plane. %
+        Both pictures show the cube to manipulate in the middle (\qty{5}{\cm} and opaque) and the eight possible targets to reach (\qty{7}{\cm} cube and semi-transparent). %
+        Only one target at a time was shown during the experiments.%
+    }
+    \subfig[0.23]{method/task-push}[Push task]
+    \subfig[0.23]{method/task-grasp}[Grasp task]
+\end{subfigs}
+
+\begin{subfigswide}{push_results}{%
+        Experiment \#2: Push task.
+        %
+        Geometric means with bootstrap 95~\% confidence interval for each vibrotactile positioning (a, b, and c) or visual hand rendering (d)
+        %
+        and Tukey's HSD pairwise comparisons: *** is \pinf{0.001}, ** is \pinf{0.01}, and * is \pinf{0.05}.
+    }
+    \subfig[0.24]{results/Push-CompletionTime-Location-Overall-Means}[Time to complete a trial.]
+    \subfig[0.24]{results/Push-Contacts-Location-Overall-Means}[Number of contacts with the cube.]
+    \subfig[0.24]{results/Push-TimePerContact-Location-Overall-Means}[Mean time spent on each contact.]
+    \subfig[0.24]{results/Push-TimePerContact-Hand-Overall-Means}[Mean time spent on each contact.]
+\end{subfigswide}
+
+We considered the same two tasks as in Experiment \#1, described in \secref{tasks}, that we analyzed separately, considering four independent, within-subject variables:
+
+\begin{itemize}
+    \item \emph{{Vibrotactile Positioning}:} the five positionings for providing vibrotactile hand rendering of the virtual contacts, as described in \secref{positioning}.
+    \item \emph{Contact Vibration Technique}: the two contact vibration techniques, as described in \secref{technique}.
+    \item \emph{visual Hand rendering}: two visual hand renderings from the first experiment, Skeleton (Skel) and None, as described in \secref{hands}; we considered Skeleton as it performed the best in terms of performance and perceived effectiveness and None as reference.
+    \item \emph{Target}: we considered target volumes located at NW and SW during the Push task, and at NE, NW, SW, and SE during the Grasp task (see \figref{tasks}); we considered these targets because they presented different difficulties.
+\end{itemize}
+
+To account for learning and fatigue effects, the positioning of the vibrotactile hand rendering (positioning) was counter-balanced using a balanced \numproduct{10 x 10} Latin square.
+%
+In these ten blocks, all possible Technique \x Hand \x Target combination conditions were repeated three times in a random order.
+%
+As we did not find any relevant effect of the order in which the tasks were performed in the first experiment, we fixed the order of the tasks: first, the Push task and then the Grasp task.
+
+This design led to a total of 5 vibrotactile positionings \x 2 vibration contact techniques \x 2 visual hand rendering \x (2 targets on the Push task + 4 targets on the Grasp task) \x 3 repetitions $=$ 420 trials per participant.
+
+
+\subsection{Apparatus and Protocol}
+\label{sec:apparatus}
+
+Apparatus and protocol were very similar to the first experiment, as described in \secref{apparatus} and \secref{protocol}, respectively.
+%
+We report here only the differences.
+
+We employed the same vibrotactile device used by~\cite{devigne2020power}.
+%
+It is composed of two encapsulated Eccentric Rotating Mass (ERM) vibration motors (Pico-Vibe 304-116, Precision Microdrive, UK).
+%
+They are small and very light (\qty{5}{\mm} \x \qty{20}{\mm}, \qty{1.2}{\g}) actuators capable of vibration frequencies from \qtyrange{120}{285}{\Hz} and
+amplitudes from \qtyrange{0.2}{1.15}{\g}.
+%
+They have a latency of \qty{20}{\ms} that we partially compensated for at the software level with slightly larger colliders to trigger the vibrations very close the moment the finger touched the cube.
+%
+These two outputs vary linearly together, based on the tension applied.
+%
+They were controlled by an Arduino Pro Mini (3.3 V) and a custom board that delivered the tension independently to each motor.
+%
+A small \qty{400}{mAh} Li-ion battery allowed for 4 hours of constant vibration at maximum intensity.
+%
+A Bluetooth module (RN42XV module, Microchip Technology Inc., USA) mounted on the Arduino ensured wireless communication with the HoloLens~2.
+
+To ensure minimal encumbrance, we used the same two motors throughout the experiment, moving them to the considered positioning before each new block.
+%
+Thin self-gripping straps were placed on the five positionings, with an elastic strap stitched on top to place the motor, as shown in \figref{method/locations}.
+%
+The straps were fixed during the entirety of the experiment to ensure similar hand tracking conditions.
+%
+We confirmed that this setup ensured a good transmission of the rendering and guaranteed a good hand tracking performance, that was measured to be constant (\qty{15}{\ms}) with and without motors, regardless their positioning.
+%
+The control board was fastened to the arm with an elastic strap.
+%
+Finally, participants wore headphones diffusing brown noise to mask the sound of the vibrotactile motors.
+
+We improved the hand tracking performance of the system by placing on the table a black sheet that absorbs the infrared light as well as placing the participants in front of a wall to ensure a more constant exposure to the light.
+%
+We also made grasping easier by adding a grasping helper, similar to UltraLeap's Physics Hands.\footnoteurl{https://docs.ultraleap.com/unity-api/Preview/physics-hands.html}.
+%
+When a phalanx collider of the tracked hand contacts the virtual cube,
+%
+a spring with a low stiffness is created and attached between the cube and the collider.
+%
+The spring pulls gently the cube toward the phalanxes in contact with the object so as to help maintain a natural and stable grasp.
+%
+When the contact is lost, the spring is destroyed.
+%
+Preliminary tests confirmed this approach.
+
+
+\subsection{Collected Data}
+\label{sec:metrics}
+
+During the experiment, we collected the same data as in the first experiment, see \secref{visual_hand:sec:metrics}.
+%
+At the end of the experiment, participants were asked if they recognized the different contact vibration techniques.
+%
+They then rated the ten combinations of Positioning \x Technique using a 7-item Likert scale (1=Not at all, 7=Extremely):
+%
+\emph{(Vibration Rating)} How much do you like each vibrotactile rendering? %
+\emph{(Workload)} How demanding or frustrating was each vibrotactile rendering? %
+\emph{(Usefulness)} How useful was each vibrotactile rendering? %
+\emph{(Realism)} How realistic was each vibrotactile rendering?
+%
+Finally, they rated the ten combinations of Positioning \x Hand on a 7-item Likert scale (1=Not at all, 7=Extremely): %
+\emph{(positioning \x Hand Rating)} How much do you like each combination of vibrotactile location for each visual hand rendering?
+
+
+\subsection{Participants}
+\label{sec:participants}
+
+Twenty subjects participated in the study (mean age = 26.8, SD = 4.1; 19~males, 1~female).
+%
+One was left-handed, while the other nineteen were right-handed. They all used their dominant hand during the trials.
+%
+They all had a normal or corrected-to-normal vision.
+%
+Thirteen subjects participated also in the previous experiment.
+
+Participants rated their expertise (\enquote{I use it more than once a year}) with VR, AR, and haptics in a pre-experiment questionnaire.
+%
+There were twelve experienced with VR, eight experienced with AR, and ten experienced with haptics.
+%
+VR and haptics expertise were highly correlated (\pearson{0.9}), as well as AR and haptics expertise (\pearson{0.6}).
+%
+Other expertise correlations were low ($r<0.35$).

3-manipulation/visuo-haptic-hand/3-1-push.tex

@@ -0,0 +1,47 @@
+\subsection{Push Task}
+\label{sec:push}
+
+\subsubsection{Completion Time}
+\label{sec:push_tct}
+
+On the time to complete a trial, there were two statistically significant effects: %
+Positioning (\anova{4}{1990}{3.8}, \p{0.004}, see \figref{results/Push-CompletionTime-Location-Overall-Means}) %
+and Target (\anova{1}{1990}{3.9}, \p{0.05}).
+%
+Fingertips was slower than Proximal (\qty{+11}{\%}, \p{0.01}) or Opposite (\qty{+12}{\%}, \p{0.03}).
+%
+There was no evidence of an advantage of Proximal or Opposite on No Vibrations, nor a disadvantage of Fingertips on No Vibrations.
+%
+Yet, there was a tendency of faster trials with Proximal and Opposite.
+%
+The NW target volume was also faster than the SW (\p{0.05}).
+
+
+\subsubsection{Contacts}
+\label{sec:push_contacts_count}
+
+On the number of contacts, there was one statistically significant effect of %
+Positioning (\anova{4}{1990}{2.4}, \p{0.05}, see \figref{results/Push-Contacts-Location-Overall-Means}).
+%
+More contacts were made with Fingertips than with Opposite (\qty{+12}{\%}, \p{0.03}).
+%
+This could indicate more difficulties to adjust the virtual cube inside the target volume.
+
+
+\subsubsection{Time per Contact}
+\label{sec:push_time_per_contact}
+
+On the mean time spent on each contact, there were two statistically significant effects of %
+Positioning (\anova{4}{1990}{11.5}, \pinf{0.001}, see \figref{results/Push-TimePerContact-Location-Overall-Means}) %
+and of Hand (\anova{1}{1990}{16.1}, \pinf{0.001}, see \figref{results/Push-TimePerContact-Hand-Overall-Means})%
+but not of the Positioning \x Hand interaction.
+%
+It was shorter with Fingertips than with Wrist (\qty{-15}{\%}, \pinf{0.001}), Opposite (\qty{-11}{\%}, \p{0.01}), or NoVi (\qty{-15}{\%}, \pinf{0.001});
+%
+and shorter with Proximal than with Wrist (\qty{-16}{\%}, \pinf{0.001}), Opposite (\qty{-12}{\%}, \p{0.005}), or No Vibrations (\qty{-16}{\%}, \pinf{0.001}).
+%
+This showed different strategies to adjust the cube inside the target volume, with faster repeated pushes with the Fingertips and Proximal positionings.
+%
+It was also shorter with None than with Skeleton (\qty{-9}{\%}, \pinf{0.001}).
+%
+This indicates, as for the first experiment, more confidence with a visual hand rendering.

3-manipulation/visuo-haptic-hand/3-2-grasp.tex

@@ -0,0 +1,62 @@
+\subsection{Grasp Task}
+\label{sec:grasp}
+
+\subsubsection{Completion Time}
+\label{sec:grasp_tct}
+
+On the time to complete a trial, there were two statistically significant effects: %
+Positioning (\anova{4}{3990}{13.6}, \pinf{0.001}, see \figref{results/Grasp-CompletionTime-Location-Overall-Means}) %
+and Target (\anova{3}{3990}{18.8}, \pinf{0.001}).
+%
+Opposite was faster than Fingertips (\qty{+19}{\%}, \pinf{0.001}), Proximal (\qty{+13}{\%}, \pinf{0.001}), Wrist (\qty{+14}{\%}, \pinf{0.001}), and No Vibrations (\qty{+8}{\%}, \p{0.03}).
+%
+No Vibrations was faster than Fingertips (\qty{+11}{\%}, \pinf{0.001}).
+%
+SE was faster than NE (\pinf{0.001}), NW (\pinf{0.001}), and SW (\pinf{0.001});
+%
+and SW was faster than NE (\p{0.03}).
+
+
+\subsubsection{Contacts}
+\label{sec:grasp_contacts_count}
+
+On the number of contacts, there were two statistically significant effects: %
+Positioning (\anova{4}{3990}{15.1}, \pinf{0.001}, see \figref{results/Grasp-Contacts-Location-Overall-Means}) %
+and Target (\anova{3}{3990}{7.6}, \pinf{0.001}).
+%
+Fewer contacts were made with Opposite than with Fingertips (\qty{-26}{\%}, \pinf{0.001}), Proximal (\qty{-17}{\%}, \pinf{0.001}), or Wrist (\qty{-12}{\%}, \p{0.002});
+%
+but more with Fingertips than with Wrist (\qty{+13}{\%}, \p{0.002}) or No Vibrations (\qty{+17}{\%}, \pinf{0.001}).
+%
+It was also easier on SW than on NE (\pinf{0.001}), NW (\p{0.006}), or SE (\p{0.03}).
+
+
+\subsubsection{Time per Contact}
+\label{sec:grasp_time_per_contact}
+
+On the mean time spent on each contact, there were two statistically significant effects: %
+Positioning (\anova{4}{3990}{2.9}, \p{0.02}, see \figref{results/Grasp-TimePerContact-Location-Overall-Means}) %
+and Target (\anova{3}{3990}{62.6}, \pinf{0.001}).
+%
+It was shorter with Fingertips than with Opposite (\qty{+7}{\%}, \p{0.01}).
+%
+It was also shorter on SE than on NE, NW or SW (\pinf{0.001});
+%
+but longer on SW than on NE or NW (\pinf{0.001}).
+
+
+\subsubsection{Grip Aperture}
+\label{sec:grasp_grip_aperture}
+
+On the average distance between the thumb's fingertip and the other fingertips during grasping, there were two
+statistically significant effects: %
+Positioning (\anova{4}{3990}{30.1}, \pinf{0.001}, see \figref{results/Grasp-GripAperture-Location-Overall-Means}) %
+and Target (\anova{3}{3990}{19.9}, \pinf{0.001}).
+%
+It was longer with Fingertips than with Proximal (\pinf{0.001}), Wrist (\pinf{0.001}), Opposite (\pinf{0.001}), or No Vibrations (\pinf{0.001});
+%
+and longer with Proximal than with Wrist (\pinf{0.001}) or No Vibrations (\pinf{0.001}).
+%
+But, it was shorter with NE than with NW or SW (\pinf{0.001});
+%
+and shorter with SE than with NW or SW (\pinf{0.001}).

3-manipulation/visuo-haptic-hand/3-3-questions.tex

@@ -0,0 +1,103 @@
+\subsection{Discrimination of Vibration Techniques}
+\label{sec:technique_results}
+
+Seven participants were able to correctly discriminate between the two vibration techniques, which they described as the contact vibration (being the Impact technique) and the continuous vibration (being the Distance technique) respectively.
+%
+Seven participants said they only felt differences of intensity with a weak one (being the Impact technique) and a strong one (being the Distance technique).
+%
+Six participants did not notice the difference between the two vibration techniques.
+%
+There was no evidence that the ability to discriminate the vibration techniques was correlated with the participants' haptic or AR/VR expertise (\pearson{0.4}), nor that it had a statistically significant effect on the performance in the tasks.
+%
+As the tasks had to be completed as quickly as possible, we hypothesize that little attention was devoted to the different vibration techniques.
+%
+Indeed, some participants explained that the contact cues were sufficient to indicate whether the cube was being properly pushed or grasped.
+%
+Although the Distance technique provided additional feedback on the interpenetration of the finger with the cube, it was not strictly necessary to manipulate the cube quickly.
+
+
+\subsection{Questionnaire}
+\label{sec:questions}
+
+\begin{subfigswide}{questions}{%
+        Experiment \#2. Boxplots of the questionnaire results of each vibrotactile positioning
+        %
+        and pairwise Wilcoxon signed-rank tests with Holm-Bonferroni adjustment: *** is \pinf{0.001}, ** is \pinf{0.01}, and * is \pinf{0.05}.
+    }
+    \subfig[0.24]{results/Question-Vibration Rating-Positioning-Overall}
+    \subfig[0.24]{results/Question-Workload-Positioning-Overall}
+    \subfig[0.24]{results/Question-Usefulness-Positioning-Overall}
+    \subfig[0.24]{results/Question-Realism-Positioning-Overall}
+\end{subfigswide}
+
+\figref{questions} shows the questionnaire results for each vibrotactile positioning.
+%
+Questionnaire results were analyzed using Aligned Rank Transform (ART) non-parametric analysis of variance (see \secref{metrics}).
+%
+Statistically significant effects were further analyzed with post-hoc pairwise comparisons with Holm-Bonferroni adjustment.
+%
+Wilcoxon signed-rank tests were used for main effects and ART contrasts procedure for interaction effects.
+%
+Only significant results are reported.
+
+
+\subsubsection{Vibrotactile Rendering Rating}
+\label{sec:vibration_ratings}
+
+There was a main effect of Positioning (\anova{4}{171}{27.0}, \pinf{0.001}).
+%
+Participants preferred Fingertips more than Wrist (\p{0.01}), Opposite (\pinf{0.001}), and No Vibration (\pinf{0.001});
+%
+Proximal more than Wrist (\p{0.007}), Opposite (\pinf{0.001}), and No Vibration (\pinf{0.001});
+%
+And Wrist more than Opposite (\p{0.01}) and No Vibration (\pinf{0.001}).
+
+
+\subsubsection{Positioning \x Hand Rating}
+\label{sec:positioning_hand}
+
+There were two main effects of Positioning (\anova{4}{171}{20.6}, \pinf{0.001}) and of Hand (\anova{1}{171}{12.2}, \pinf{0.001}).
+%
+Participants preferred Fingertips more than Wrist (\p{0.03}), Opposite (\pinf{0.001}), and No Vibration (\pinf{0.001});
+%
+Proximal more than Wrist (\p{0.003}), Opposite (\pinf{0.001}), and No Vibration (\pinf{0.001});
+%
+Wrist more than Opposite (\p{0.03}) and No Vibration (\pinf{0.001});
+%
+And Skeleton more than No Hand (\pinf{0.001}).
+
+
+\subsubsection{Workload}
+\label{sec:workload}
+
+There was a main of Positioning (\anova{4}{171}{3.9}, \p{0.004}).
+%
+Participants found Opposite more fatiguing than Fingertips (\p{0.01}), Proximal (\p{0.003}), and Wrist (\p{0.02}).
+
+
+\subsubsection{Usefulness}
+\label{sec:usefulness}
+
+There was a main effect of Positioning (\anova{4}{171}{38.0}, \p{0.041}).
+%
+Participants found Fingertips the most useful, more than Proximal (\p{0.02}), Wrist (\pinf{0.001}), Opposite (\pinf{0.001}), and No Vibrations (\pinf{0.001});
+%
+Proximal more than Wrist (\p{0.008}), Opposite (\pinf{0.001}), and No Vibrations (\pinf{0.001});
+%
+Wrist more than Opposite (\p{0.008}) and No Vibrations (\pinf{0.001});
+%
+And Opposite more than No Vibrations (\p{0.004}).
+
+
+\subsubsection{Realism}
+\label{sec:realism}
+
+There was a main effect of Positioning (\anova{4}{171}{28.8}, \pinf{0.001}).
+%
+Participants found Fingertips the most realistic, more than Proximal (\p{0.05}), Wrist (\p{0.004}), Opposite (\pinf{0.001}), and No Vibrations (\pinf{0.001});
+%
+Proximal more than Wrist (\p{0.03}), Opposite (\pinf{0.001}), and No Vibrations (\pinf{0.001});
+%
+Wrist more than Opposite (\p{0.03}) and No Vibrations (\pinf{0.001});
+%
+And Opposite more than No Vibrations (\p{0.03}).

3-manipulation/visuo-haptic-hand/3-results.tex

@@ -0,0 +1,20 @@
+\section{Results}
+\label{sec:results}
+
+\begin{subfigswide}{grasp_results}{%
+    Experiment \#{2}: Grasp task.
+    %
+    Geometric means with bootstrap 95~\% confidence interval for each {vibrotactile positioning}
+    %
+    and Tukey's HSD pairwise comparisons: *** is \pinf{0.001}, ** is \pinf{0.01}, and * is \pinf{0.05}.
+    }
+    \subfig[0.24]{results/Grasp-CompletionTime-Location-Overall-Means}[Time to complete a trial.]
+    \subfig[0.24]{results/Grasp-Contacts-Location-Overall-Means}[Number of contacts with the cube.]
+    \subfig[0.24]{results/Grasp-TimePerContact-Location-Overall-Means}[Mean time spent on each contact.]
+    \subfig[0.24]{results/Grasp-GripAperture-Location-Overall-Means}[\centering Distance between thumb and the other fingertips when grasping.]
+\end{subfigswide}
+
+Results were analyzed similarly as for the first experiment (see \secref{results}).
+%
+The LMM were fitted with the order of the five vibrotactile positionings (Order), the vibrotactile positionings (Positioning), the visual hand rendering (Hand), the {contact vibration techniques} (Technique), and the target volume position (Target), and their interactions as fixed effects and Participant as random intercept.
+

3-manipulation/visuo-haptic-hand/4-discussion.tex

@@ -0,0 +1,96 @@
+\section{Discussion}
+\label{sec:discussion}
+
+We evaluated sixteen visuo-haptic renderings of the hand, in the same two virtual object manipulation tasks in AR as in the first experiment, as the combination of two vibrotactile contact techniques provided at four delocalized positions on the hand with the two most representative visual hand renderings established in the first experiment.
+
+In the Push task, vibrotactile haptic hand rendering has been proven beneficial with the Proximal positioning, which registered a low completion time, but detrimental with the Fingertips positioning, which performed worse (see \figref{results/Push-CompletionTime-Location-Overall-Means}) than the Proximal and Opposite (on the contralateral hand) positionings.
+%
+The cause might be the intensity of vibrations, which many participants found rather strong and possibly distracting when provided at the fingertips.
+%
+This result was also observed by \citeauthorcite{bermejo2021exploring}, who provided vibrotactile cues when pressing a virtual keypad.
+%
+Another reason could be the visual impairment caused by the vibrotactile motors when worn on the fingertips, which could have disturbed the visualization of the virtual cube.
+
+We observed different strategies than in the first experiment for the two tasks.
+%
+During the Push task, participants made more and shorter contacts to adjust the cube inside the target volume (see \figref{results/Push-Contacts-Location-Overall-Means} and \figref{results/Push-TimePerContact-Location-Overall-Means}).
+%
+During the Grasp task, participants pressed the cube 25~\% harder on average (see \figref{results/Grasp-GripAperture-Location-Overall-Means}).
+%
+The Fingertips and Proximal positionings led to a slightly larger grip aperture than the others.
+%
+We think that the proximity of the vibrotactile rendering to the point of contact made users to take more time to adjust their grip in a more realistic manner, \ie closer to the surface of the cube.
+%
+This could also be the cause of the higher number of failed grasps or cube drops: indeed, we observed that the larger the grip aperture, the higher the number of contacts.
+%
+Consequently, the Fingertips positioning was slower (see \figref{results/Grasp-CompletionTime-Location-Overall-Means}) and more prone to error (see \figref{results/Grasp-Contacts-Location-Overall-Means}) than the Opposite and Nowhere positionings.
+
+In both tasks, the Opposite positioning also seemed to be faster (see \figref{results/Push-CompletionTime-Location-Overall-Means}) than having no vibrotactile hand rendering (Nowhere positioning).
+%
+However, participants also felt more workload (see \figref{results/questions}) with this positioning opposite to the site of the interaction.
+%
+This result might mean that participants focused more on learning to interpret these sensations, which led to better performance in the long run.
+
+Overall, many participants appreciated the vibrotactile hand renderings, commenting that they made the tasks more realistic and easier.
+%
+However, the closer to the contact point, the better the vibrotactile rendering was perceived (see \figref{results/questions}).
+%
+This seemed inversely correlated with the performance, except for the Nowhere positioning, \eg both the Fingertips and Proximal positionings were perceived as more effective, useful, and realistic than the other positionings despite lower performance.
+
+Considering the two tasks, no clear difference in performance or appreciation was found between the two contact vibration techniques.
+%
+While the majority of participants discriminated the two different techniques, only a minority identified them correctly (see \secref{results/technique_results}).
+%
+It seemed that the Impact technique was sufficient to provide contact information compared to the Distance technique, which provided additional feedback on interpenetration, as reported by participants.
+
+No difference in performance was found between the two visual hand renderings, except for the Push task, where the Skeleton hand rendering resulted again in longer contacts.
+%
+Additionally, the Skeleton rendering was appreciated and perceived as more effective than having no visual hand rendering, confirming the results of our first experiment.
+%
+Participants reported that this visual hand rendering provided good feedback on the status of the hand tracking while being constrained to the cube, and helped with rotation adjustment in both tasks.
+%
+However, many also felt that it was a bit redundant with the vibrotactile hand rendering.
+%
+Indeed, receiving a vibrotactile hand rendering was found by participants as a more accurate and reliable information regarding the contact with the cube than simply seeing the cube and the visual hand reacting to the manipulation.
+%
+This result suggests that providing a visual hand rendering may not be useful during the grasping phase, but may be beneficial prior to contact with the virtual object and during position and rotation adjustment, providing valuable information about the hand pose.
+%
+It is also worth noting that the improved hand tracking and grasp helper improved the manipulation of the cube with respect to the first experiment, as shown by the shorter completion time during the Grasp task.
+%
+This improvement could also be the reason for the smaller differences between the Skeleton and the None visual hand renderings in this second experiment.
+
+In summary, the positioning of the vibrotactile haptic rendering of the hand affected on the performance and experience of users manipulating virtual objects with their bare hands in AR.
+%
+The closer the vibrotactile hand rendering was to the point of contact, the better it was perceived in terms of effectiveness, usefulness, and realism.
+%
+These subjective appreciations of wearable haptic hand rendering for manipulating virtual objects in AR were also observed by \citeauthorcite{maisto2017evaluation} and \citeauthorcite{meli2018combining}.
+%
+However, the best performance was obtained with the farthest positioning on the contralateral hand, which is somewhat surprising.
+%
+This apparent paradox could be explained in two ways.
+%
+On the one hand, participants behave differently when the haptic rendering was given on the fingers, close to the contact point, with shorter pushes and larger grip apertures.
+%
+This behavior has likely given them a better experience of the tasks and more confidence in their actions, as well as leading to a lower interpenetration/force applied to the cube~\cite{pacchierotti2015cutaneous}.
+%
+On the other hand, the unfamiliarity of the contralateral hand positioning caused participants to spend more time understanding the haptic stimuli, which might have made them more focused on performing the task.
+%
+In terms of the contact vibration technique, the continuous vibration technique on the finger interpenetration (Distance technique) did not make a difference to performance, although it provided more information.
+%
+Participants felt that vibration bursts were sufficient (Impact technique) to confirm contact with the virtual object.
+%
+Finally, it was interesting to note that the visual hand renderings was appreciated but felt less necessary when provided together with vibrotactile hand rendering, as the latter was deemed sufficient for acknowledging the contact.
+
+As we already said in \secref{visual_hand:sec:discussion}, these results have some limitations as they address limited types of visuo-haptic renderings and manipulations were restricted to the thumb and index fingertips.
+%
+While the simpler vibration technique (Impact technique) was sufficient to confirm contacts with the cube, richer vibrotactile renderings may be required for more complex interactions, such as collision or friction renderings between objects~\cite{kuchenbecker2006improving, pacchierotti2015cutaneous} or texture rendering~\cite{culbertson2014one, asano2015vibrotactile}.
+%
+More generally, a broader range of haptic sensations should be considered, such as pressure or stretching of the skin~\cite{maisto2017evaluation, teng2021touch}.
+%
+However, moving the point of application of the sensation away may be challenging for some types of haptic rendering.
+%
+Similarly, as the interactions were limited to the thumb and index fingertips, positioning a delocalized haptic rendering over a larger area of the hand could be challenging and remains to be explored.
+%
+Also, given that some users found the vibration rendering too strong, adapting/personalizing the haptic feedback to one's preference (and body positioning) might also be a promising approach.
+%
+Indeed, personalized haptics is recently gaining interest in the community~\cite{malvezzi2021design, umair2021exploring}.

3-manipulation/visuo-haptic-hand/5-conclusion.tex

@@ -0,0 +1,22 @@
+\section{Conclusion}
+\label{sec:conclusion}
+
+This paper presented two human subject studies aimed at better understanding the role of visuo-haptic rendering of the hand during virtual object manipulation in OST-AR.
+%
+The first experiment compared six visual hand renderings in two representative manipulation tasks in AR, \ie push-and-slide and grasp-and-place of a virtual object.
+%
+Results show that a visual hand rendering improved the performance, perceived effectiveness, and user confidence.
+%
+A skeleton rendering, providing a detailed view of the tracked joints and phalanges while not hiding the real hand, was the most performant and effective.
+%
+The second experiment compared, in the same two manipulation tasks as before, sixteen visuo-haptic renderings of the hand as the combination of two vibrotactile contact techniques, provided at four different delocalized positions on the hand, and with the two most representative visual hand renderings established in the first experiment, \ie the skeleton hand rendering and no hand rendering.
+%
+Results show that delocalized vibrotactile haptic hand rendering improved the perceived effectiveness, realism, and usefulness when it is provided close to the contact point.
+%
+However, the farthest positioning on the contralateral hand gave the best performance even though it was disliked: the unfamiliarity of the positioning probably caused the participants to take more effort to consider the haptic stimuli and to focus more on the task.
+%
+The visual hand rendering was perceived less necessary than the vibrotactile haptic hand rendering, but still provided a useful feedback on the hand tracking.
+
+Future work will focus on including richer types of haptic feedback, such as pressure and skin stretch, analyzing the best compromise between well-round haptic feedback and wearability of the system with respect to AR constraints.
+%
+As delocalizing haptic feedback seems to be a simple but very promising approach for haptic-enabled AR, we will keep including this dimension in our future study, even when considering other types of haptic sensations.

3-manipulation/visuo-haptic-hand/visuo-haptic-hand.tex

@@ -1,4 +1,13 @@
 \chapter{Visuo-Haptic Rendering of the Hand in Augmented Reality}
-\mainlabel{visuo-haptic-hand}
+\mainlabel{visuo_haptic_hand}
 
 \chaptertoc
+
+\input{1-introduction}
+\input{2-method}
+\input{3-results}
+\input{3-1-push}
+\input{3-2-grasp}
+\input{3-3-questions}
+\input{4-discussion}
+\input{5-conclusion}