erwan/phd-thesis
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Visual hand {rendering => augmentation}

Browse Source
This commit is contained in:
Erwan Normand
2024-11-04 14:37:23 +01:00
parent 613e683902
commit 5dc3e33a15
15 changed files with 93 additions and 79 deletions
Download Patch File Download Diff File Expand all files Collapse all files

26
4-manipulation/visual-hand/2-method.tex
View File

@@ -1,7 +1,7 @@
\section{Visual Hand Renderings}
\section{Visual Hand Augmentations}
\label{hands}
We compared a set of the most popular visual hand renderings, as found in the literature \secref[related_work]{ar_visual_hands}.
We compared a set of the most popular visual hand augmentations, as found in the literature \secref[related_work]{ar_visual_hands}.
Since we address hand-centered manipulation tasks, we only considered renderings including the fingertips (\secref[related_work]{grasp_types}).
Moreover, as to keep the focus on the hand rendering itself, we used neutral semi-transparent grey meshes, consistent with the choices made in \cite{yoon2020evaluating,vanveldhuizen2021effect}.
All considered hand renderings are drawn following the tracked pose of the user's real hand.
@@ -11,7 +11,7 @@ They are shown in \figref{hands} and described below, with an abbreviation in br
\paragraph{None}
As a reference, we considered no visual hand rendering (\figref{method/hands-none}), as is common in \AR \cite{hettiarachchi2016annexing,blaga2017usability,xiao2018mrtouch,teng2021touch}.
As a reference, we considered no visual hand augmentation (\figref{method/hands-none}), as is common in \AR \cite{hettiarachchi2016annexing,blaga2017usability,xiao2018mrtouch,teng2021touch}.
Users have no information about hand tracking and no feedback about contact with the virtual objects, other than their movement when touched.
As virtual content is rendered on top of the \RE, the hand of the user can be hidden by the virtual objects when manipulating them (\secref[related_work]{ar_displays}).
@@ -45,7 +45,7 @@ It can be seen as a filled version of the Contour hand rendering, thus partially
\section{User Study}
\label{method}
We aim to investigate whether the chosen visual hand rendering affects the performance and user experience of manipulating virtual objects with free hands in \AR.
We aim to investigate whether the chosen visual feedback of the virtual hand affects the performance and user experience of manipulating virtual objects with free hands in \AR.
\subsection{Manipulation Tasks and Virtual Scene}
\label{tasks}
@@ -89,13 +89,13 @@ As before, the task is considered completed when the cube is \emph{fully} inside
We analyzed the two tasks separately.
For each of them, we considered two independent, within-subject, variables:
\begin{itemize}
\item \factor{Hand}, consisting of the six possible visual hand renderings discussed in \secref{hands}: \level{None}, \level{Occlusion} (Occl), \level{Tips}, \level{Contour} (Cont), \level{Skeleton} (Skel), and \level{Mesh}.
\item \factor{Hand}, consisting of the six possible visual hand augmentations discussed in \secref{hands}: \level{None}, \level{Occlusion} (Occl), \level{Tips}, \level{Contour} (Cont), \level{Skeleton} (Skel), and \level{Mesh}.
\item \factor{Target}, consisting of the eight possible locations of the target volume, named from the participant's point of view and as shown in \figref{tasks}: right (\level{R}), right-back (\level{RB}), back (\level{B}), left-back (\level{LB}), left (\level{L}), left-front (\level{LF}), front (\level{F}) and right-front (\level{RF}).
\end{itemize}
Each condition was repeated three times.
To control learning effects, we counter-balanced the orders of the two manipulation tasks and visual hand renderings following a 6 \x 6 Latin square, leading to six blocks where the position of the target volume was in turn randomized.
This design led to a total of 2 manipulation tasks \x 6 visual hand renderings \x 8 targets \x 3 repetitions $=$ 288 trials per participant.
To control learning effects, we counter-balanced the orders of the two manipulation tasks and visual hand augmentations following a 6 \x 6 Latin square, leading to six blocks where the position of the target volume was in turn randomized.
This design led to a total of 2 manipulation tasks \x 6 visual hand augmentations \x 8 targets \x 3 repetitions $=$ 288 trials per participant.
\subsection{Apparatus}
\label{apparatus}
@@ -108,9 +108,9 @@ We measured the latency of the hand tracking at \qty{15}{\ms}, independent of th
The implementation of our experiment was done using Unity 2022.1, PhysX 4.1, and the Mixed Reality Toolkit (MRTK) 2.8.
The compiled application ran directly on the HoloLens~2 at \qty{60}{FPS}.
The default \ThreeD hand model from MRTK was used for all visual hand renderings.
The default \ThreeD hand model from MRTK was used for all visual hand augmentations.
By changing the material properties of this hand model, we were able to achieve the six renderings shown in \figref{hands}.
A calibration was performed for every participant, to best adapt the size of the visual hand rendering to their real hand.
A calibration was performed for every participant, to best adapt the size of the visual hand augmentation to their real hand.
A set of empirical tests enabled us to choose the best rendering characteristics in terms of transparency and brightness for the virtual objects and hand renderings, which were applied throughout the experiment.
The hand tracking information provided by MRTK was used to construct a virtual articulated physics-enabled hand (\secref[related_work]{ar_virtual_hands}) using PhysX.
@@ -149,12 +149,12 @@ Inspired by \textcite[p.674]{laviolajr20173d}, we collected the following metric
\item \response{Completion Time}, defined as the time elapsed between the first contact with the virtual cube and its correct placement inside the target volume; as subjects were asked to complete the tasks as fast as possible, lower completion times mean better performance.
\item \response{Contacts}, defined as the number of separate times the user's hand makes contact with the virtual cube; in both tasks, a lower number of contacts means a smoother continuous interaction with the object.
\item \response{Time per Contact}, defined as the total time any part of the user's hand contacted the cube divided by the number of contacts; higher values mean that the user interacted with the object for longer non-interrupted periods of time.
\item \response{Grip Aperture} (solely for the grasp-and-place task), defined as the average distance between the thumb's fingertip and the other fingertips during the grasping of the cube; lower values indicate a greater finger interpenetration with the cube, resulting in a greater discrepancy between the real hand and the visual hand rendering constrained to the cube surfaces and showing how confident users are in their grasp \cite{prachyabrued2014visual, al-kalbani2016analysis, blaga2017usability, chessa2019grasping}.
\item \response{Grip Aperture} (solely for the grasp-and-place task), defined as the average distance between the thumb's fingertip and the other fingertips during the grasping of the cube; lower values indicate a greater finger interpenetration with the cube, resulting in a greater discrepancy between the real hand and the visual hand augmentation constrained to the cube surfaces and showing how confident users are in their grasp \cite{prachyabrued2014visual, al-kalbani2016analysis, blaga2017usability, chessa2019grasping}.
\end{itemize}
Taken together, these measures provide an overview of the performance and usability of each of the visual hand renderings tested, as we hypothesized that they should influence the behavior and effectiveness of the participants.
Taken together, these measures provide an overview of the performance and usability of each visual hand augmentation, as we hypothesized that they should influence the behavior and effectiveness of the participants.
At the end of each task, participants were asked to rank the visual hand renderings according to their preference with respect to the considered task.
Participants also rated each visual hand rendering individually on six questions using a 7-item Likert scale (1=Not at all, 7=Extremely):
At the end of each task, participants were asked to rank the visual hand augmentations according to their preference with respect to the considered task.
Participants also rated each visual hand augmentation individually on six questions using a 7-item Likert scale (1=Not at all, 7=Extremely):
\begin{itemize}
\item \response{Difficulty}: How difficult were the tasks?
\item \response{Fatigue}: How fatiguing (mentally and physically) were the tasks?