erwan/phd-thesis
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Visual hand {rendering => augmentation}

Browse Source
This commit is contained in:
Erwan Normand
2024-11-04 14:37:23 +01:00
parent 613e683902
commit 5dc3e33a15
15 changed files with 93 additions and 79 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
4-manipulation/visuo-haptic-hand/2-method.tex
View File

@@ -45,7 +45,7 @@ Similarly, we designed the distance vibration technique (Dist) so that interpene
\label{method}
This user study aims to evaluate whether a visuo-haptic rendering of the hand affects the user performance and experience of manipulation of virtual objects with bare hands in \OST-\AR.
The chosen visuo-haptic hand renderings are the combination of the two most representative visual hand renderings established in the \chapref{visual_hand}, \ie \level{Skeleton} and \level{No Hand}, described in \secref[visual_hand]{hands}, with the two contact vibration techniques provided at the four delocalized positions on the hand described in \secref{vibration}.
The chosen visuo-haptic hand renderings are the combination of the two most representative visual hand augmentations established in the \chapref{visual_hand}, \ie \level{Skeleton} and \level{No Hand}, described in \secref[visual_hand]{hands}, with the two contact vibration techniques provided at the four delocalized positions on the hand described in \secref{vibration}.
\subsection{Experimental Design}
\label{design}
@@ -55,7 +55,7 @@ We considered the same two \level{Push} and \level{Grasp} tasks as described in
\begin{itemize}
\item \factor{Positioning}: the five positionings for providing vibrotactile hand rendering of the virtual contacts, as described in \secref{positioning}.
\item \factor{Vibration Technique}: the two contact vibration techniques, as described in \secref{technique}.
\item \factor{Hand}: two visual hand renderings from the \chapref{visual_hand}, \level{Skeleton} (Skel) and \level{No Hand}, as described in \secref[visual_hand]{hands}; we considered \level{Skeleton} as it performed the best in terms of performance and perceived effectiveness and \level{No Hand} as reference.
\item \factor{Hand}: two visual hand augmentations from the \chapref{visual_hand}, \level{Skeleton} (Skel) and \level{No Hand}, as described in \secref[visual_hand]{hands}; we considered \level{Skeleton} as it performed the best in terms of performance and perceived effectiveness and \level{No Hand} as reference.
\item \factor{Target}: we considered the target volumes (\figref{tasks}), from the participant's point of view, located at:
\begin{itemize}
\item left-bottom (\level{LB}) and left-right (\level{LF}) during the \level{Push} task; and
@@ -78,7 +78,7 @@ To account for learning and fatigue effects, the order of the \factor{Positionin
In these ten blocks, all possible \factor{Technique} \x \factor{Hand} \x \factor{Target} combination conditions were repeated three times in a random order.
As we did not find any relevant effect of the order in which the tasks were performed in the \chapref{visual_hand}, we fixed the order of the tasks: first, the \level{Push} task and then the \level{Grasp} task.
This design led to a total of 5 vibrotactile positionings \x 2 vibration contact techniques \x 2 visual hand rendering \x (2 targets on the Push task + 4 targets on the Grasp task) \x 3 repetitions $=$ 420 trials per participant.
This design led to a total of 5 vibrotactile positionings \x 2 vibration contact techniques \x 2 visual hand augmentation \x (2 targets on the Push task + 4 targets on the Grasp task) \x 3 repetitions $=$ 420 trials per participant.
\subsection{Apparatus and Procedure}
\label{apparatus}

7
4-manipulation/visuo-haptic-hand/3-0-results.tex
View File

@@ -1,6 +1,7 @@
\section{Results}
\label{results}
Results were analyzed similarly as in the user study of the visual hand renderings (\secref[visual_hand]{results}).
The \LMM were fitted with the order of the five vibrotactile positionings (\factor{Order}), the vibrotactile positionings (\factor{Positioning}), the visual hand rendering (\factor{Hand}), the {contact vibration techniques} (\factor{Vibration Technique}), and the target volume position (\factor{Target}), and their interactions as fixed effects and Participant as random intercept.
Results were analyzed similarly as in the user study of the visual hand augmentations (\secref[visual_hand]{results}).
The \LMM were fitted with the vibrotactile positionings (\factor{Positioning}), the visual hand augmentation (\factor{Hand}), the {contact vibration techniques} (\factor{Vibration Technique}), the target volume position (\factor{Target}) and their interactions as fixed effects, as well as by-participant random intercepts.
Depending on the data, different random effect structures were tested.
Only the best converging models are reported, with the lowest Akaike Information Criterion (AIC) values.

19
4-manipulation/visuo-haptic-hand/3-1-push.tex
View File

@@ -3,8 +3,9 @@
\paragraph{Completion Time}
On the time to complete a trial, there were two statistically significant effects:
\factor{Positioning} (\anova{4}{1990}{3.8}, \p{0.004}, see \figref{results/Push-CompletionTime-Location-Overall-Means}) %
On the time to complete a trial,
a \LMM \ANOVA with by-participant random intercepts indicated two statistically significant effects:
\factor{Positioning} (\anova{4}{2341}{3.6}, \p{0.007}, see \figref{results/Push-CompletionTime-Location-Overall-Means}) %
and \factor{Target} (\anova{1}{1990}{3.9}, \p{0.05}).
\level{Fingertips} was slower than \level{Proximal} (\percent{+11}, \p{0.01}) or \level{Opposite} (\percent{+12}, \p{0.03}).
There was no evidence of an advantage of \level{Proximal} or \level{Opposite} on \level{Nowhere}, nor a disadvantage of \level{Fingertips} on \level{Nowhere}.
@@ -13,25 +14,27 @@ The \level{LB} target volume was also faster than the \level{LF} (\p{0.05}).
\paragraph{Contacts}
On the number of contacts, there was one statistically significant effect of
On the number of contacts,
a \LMM \ANOVA with by-participant random intercepts indicated one statistically significant effect of
\factor{Positioning} (\anova{4}{1990}{2.4}, \p{0.05}, see \figref{results/Push-Contacts-Location-Overall-Means}).
More contacts were made with \level{Fingertips} than with \level{Opposite} (\percent{+12}, \p{0.03}).
This could indicate more difficulties to adjust the virtual cube inside the target volume.
\paragraph{Time per Contact}
On the mean time spent on each contact, there were two statistically significant effects of
On the mean time spent on each contact,
a \LMM \ANOVA with by-participant random intercepts indicated two statistically significant effects of
\factor{Positioning} (\anova{4}{1990}{11.5}, \pinf{0.001}, see \figref{results/Push-TimePerContact-Location-Overall-Means}) %
and of \factor{Hand} (\anova{1}{1990}{16.1}, \pinf{0.001}, see \figref{results/Push-TimePerContact-Hand-Overall-Means})%
and of \factor{Hand} (\anova{1}{1990}{16.1}, \pinf{0.001}, see \figref{results/Push-TimePerContact-Hand-Overall-Means}) %
but not of the \factor{Positioning} \x \factor{Hand} interaction.
It was shorter with \level{Fingertips} than with \level{Wrist} (\percent{-15}, \pinf{0.001}), \level{Opposite} (\percent{-11}, \p{0.01}), or NoVi (\percent{-15}, \pinf{0.001});
It was shorter with \level{Fingertips} than with \level{Wrist} (\percent{-15}, \pinf{0.001}), \level{Opposite} (\percent{-11}, \p{0.01}), or \level{Nowhere} (\percent{-15}, \pinf{0.001});
and shorter with \level{Proximal} than with \level{Wrist} (\percent{-16}, \pinf{0.001}), \level{Opposite} (\percent{-12}, \p{0.005}), or \level{Nowhere} (\percent{-16}, \pinf{0.001}).
This showed different strategies to adjust the cube inside the target volume, with faster repeated pushes with the \level{Fingertips} and \level{Proximal} positionings.
It was also shorter with \level{None} than with \level{Skeleton} (\percent{-9}, \pinf{0.001}).
This indicates, as for the \chapref{visual_hand}, more confidence with a visual hand rendering.
This indicates, as for the \chapref{visual_hand}, more confidence with a visual hand augmentation.
\begin{subfigs}{push_results}{Results of the grasp task performance metrics.}[
Geometric means with bootstrap \percent{95} \CI for each vibrotactile positioning (a, b and c) or visual hand rendering (d)
Geometric means with bootstrap \percent{95} \CI for each vibrotactile positioning (a, b and c) or visual hand augmentation (d)
and Tukey's \HSD pairwise comparisons: *** is \pinf{0.001}, ** is \pinf{0.01}, and * is \pinf{0.05}.
][
\item Time to complete a trial.

13
4-manipulation/visuo-haptic-hand/3-2-grasp.tex
View File

@@ -3,7 +3,8 @@
\paragraph{Completion Time}
On the time to complete a trial, there were two statistically significant effects:
On the time to complete a trial,
a \LMM \ANOVA with by-participant random intercepts indicated two statistically significant effects:
\factor{Positioning} (\anova{4}{3990}{13.6}, \pinf{0.001}, see \figref{results/Grasp-CompletionTime-Location-Overall-Means})
and \factor{Target} (\anova{3}{3990}{18.8}, \pinf{0.001}).
\level{\level{Opposite}} was faster than \level{Fingertips} (\percent{+19}, \pinf{0.001}), \level{Proximal} (\percent{+13}, \pinf{0.001}), \level{Wrist} (\percent{+14}, \pinf{0.001}), and \level{Nowhere} (\percent{+8}, \p{0.03}).
@@ -13,7 +14,8 @@ and \level{LF} was faster than \level{RB} (\p{0.03}).
\paragraph{Contacts}
On the number of contacts, there were two statistically significant effects:
On the number of contacts,
a \LMM \ANOVA with by-participant random intercepts indicated two statistically significant effects:
\factor{Positioning} (\anova{4}{3990}{15.1}, \pinf{0.001}, see \figref{results/Grasp-Contacts-Location-Overall-Means}) %
and \factor{Target} (\anova{3}{3990}{7.6}, \pinf{0.001}).
Fewer contacts were made with \level{Opposite} than with \level{Fingertips} (\percent{-26}, \pinf{0.001}), \level{Proximal} (\percent{-17}, \pinf{0.001}), or \level{Wrist} (\percent{-12}, \p{0.002});
@@ -22,7 +24,8 @@ It was also easier on \level{LF} than on \level{RB} (\pinf{0.001}), \level{LB} (
\paragraph{Time per Contact}
On the mean time spent on each contact, there were two statistically significant effects:
On the mean time spent on each contact,
a \LMM \ANOVA with by-participant random intercepts indicated two statistically significant effects:
\factor{Positioning} (\anova{4}{3990}{2.9}, \p{0.02}, see \figref{results/Grasp-TimePerContact-Location-Overall-Means})
and \factor{Target} (\anova{3}{3990}{62.6}, \pinf{0.001}).
It was shorter with \level{Fingertips} than with \level{Opposite} (\percent{+7}, \p{0.01}).
@@ -31,8 +34,8 @@ but longer on \level{LF} than on \level{RB} or \level{LB} (\pinf{0.001}).
\paragraph{Grip Aperture}
On the average distance between the thumb's fingertip and the other fingertips during grasping, there were two
statistically significant effects:
On the average distance between the thumb's fingertip and the other fingertips during grasping,
a \LMM \ANOVA with by-participant random intercepts indicated two statistically significant effects:
\factor{Positioning} (\anova{4}{3990}{30.1}, \pinf{0.001}, see \figref{results/Grasp-GripAperture-Location-Overall-Means})
and \factor{Target} (\anova{3}{3990}{19.9}, \pinf{0.001}).
It was longer with \level{Fingertips} than with \level{Proximal} (\pinf{0.001}), \level{Wrist} (\pinf{0.001}), \level{Opposite} (\pinf{0.001}), or \level{Nowhere} (\pinf{0.001});

2
4-manipulation/visuo-haptic-hand/3-3-questions.tex
View File

@@ -13,7 +13,7 @@ Although the \level{Distance} technique provided additional feedback on the inte
\label{questions}
\figref{results_questions} shows the questionnaire results for each vibrotactile positioning.
Questionnaire results were analyzed using \ART non-parametric \ANOVA (\secref{metrics}).
The results of each question were analyzed using non-parametric \ANOVA on an \ART model.
Statistically significant effects were further analyzed with post-hoc pairwise comparisons with Holm-Bonferroni adjustment.
Wilcoxon signed-rank tests were used for main effects and \ART contrasts procedure for interaction effects.
Only significant results are reported.

14
4-manipulation/visuo-haptic-hand/4-discussion.tex
View File

@@ -1,7 +1,7 @@
\section{Discussion}
\label{discussion}
We evaluated twenty visuo-haptic renderings of the hand, in the same two virtual object manipulation tasks in \AR as in the \chapref{visual_hand}, as the combination of two vibrotactile contact techniques provided at five delocalized positions on the hand with the two most representative visual hand renderings established in the \chapref{visual_hand}.
We evaluated twenty visuo-haptic renderings of the hand, in the same two virtual object manipulation tasks in \AR as in the \chapref{visual_hand}, as the combination of two vibrotactile contact techniques provided at five delocalized positions on the hand with the two most representative visual hand augmentations established in the \chapref{visual_hand}.
In the \level{Push} task, vibrotactile haptic hand rendering has been proven beneficial with the \level{Proximal} positioning, which registered a low completion time, but detrimental with the \level{Fingertips} positioning, which performed worse (\figref{results/Push-CompletionTime-Location-Overall-Means}) than the \level{Proximal} and \level{Opposite} (on the contralateral hand) positionings.
The cause might be the intensity of vibrations, which many participants found rather strong and possibly distracting when provided at the fingertips.
@@ -28,14 +28,14 @@ Considering the two tasks, no clear difference in performance or appreciation wa
While the majority of participants discriminated the two different techniques, only a minority identified them correctly (\secref{technique_results}).
It seemed that the Impact technique was sufficient to provide contact information compared to the \level{Distance} technique, which provided additional feedback on interpenetration, as reported by participants.
No difference in performance was found between the two visual hand renderings, except for the \level{Push} task, where the \level{Skeleton} hand rendering resulted again in longer contacts.
Additionally, the \level{Skeleton} rendering was appreciated and perceived as more effective than having no visual hand rendering, confirming the results of our \chapref{visual_hand}.
Participants reported that this visual hand rendering provided good feedback on the status of the hand tracking while being constrained to the cube, and helped with rotation adjustment in both tasks.
No difference in performance was found between the two visual hand augmentations, except for the \level{Push} task, where the \level{Skeleton} hand rendering resulted again in longer contacts.
Additionally, the \level{Skeleton} rendering was appreciated and perceived as more effective than having no visual hand augmentation, confirming the results of our \chapref{visual_hand}.
Participants reported that this visual hand augmentation provided good feedback on the status of the hand tracking while being constrained to the cube, and helped with rotation adjustment in both tasks.
However, many also felt that it was a bit redundant with the vibrotactile hand rendering.
Indeed, receiving a vibrotactile hand rendering was found by participants as a more accurate and reliable information regarding the contact with the cube than simply seeing the cube and the visual hand reacting to the manipulation.
This result suggests that providing a visual hand rendering may not be useful during the grasping phase, but may be beneficial prior to contact with the virtual object and during position and rotation adjustment, providing valuable information about the hand pose.
This result suggests that providing a visual hand augmentation may not be useful during the grasping phase, but may be beneficial prior to contact with the virtual object and during position and rotation adjustment, providing valuable information about the hand pose.
It is also worth noting that the improved hand tracking and grasp helper improved the manipulation of the cube with respect to the \chapref{visual_hand}, as shown by the shorter completion time during the \level{Grasp} task.
This improvement could also be the reason for the smaller differences between the \level{Skeleton} and the \level{None} visual hand renderings in this second experiment.
This improvement could also be the reason for the smaller differences between the \level{Skeleton} and the \level{None} visual hand augmentations in this second experiment.
In summary, the positioning of the vibrotactile haptic rendering of the hand affected on the performance and experience of users manipulating virtual objects with their bare hands in \AR.
The closer the vibrotactile hand rendering was to the point of contact, the better it was perceived in terms of effectiveness, usefulness, and realism.
@@ -47,4 +47,4 @@ This behavior has likely given them a better experience of the tasks and more co
On the other hand, the unfamiliarity of the contralateral hand positioning (\level{Opposite}) caused participants to spend more time understanding the haptic stimuli, which might have made them more focused on performing the task.
In terms of the contact vibration technique, the continuous vibration technique on the finger interpenetration (\level{Distance}) did not make a difference to performance, although it provided more information.
Participants felt that vibration bursts were sufficient (\level{Distance}) to confirm contact with the virtual object.
Finally, it was interesting to note that the visual hand rendering was appreciated but felt less necessary when provided together with vibrotactile hand rendering, as the latter was deemed sufficient for acknowledging the contact.
Finally, it was interesting to note that the visual hand augmentation was appreciated but felt less necessary when provided together with vibrotactile hand rendering, as the latter was deemed sufficient for acknowledging the contact.