erwan/phd-thesis
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

WIP visuo-haptic hand

Browse Source
This commit is contained in:
Erwan Normand
2024-09-25 22:09:12 +02:00
parent e7f732bf3d
commit 08c57b6941
25 changed files with 169 additions and 289 deletions
Download Patch File Download Diff File Expand all files Collapse all files

129
3-manipulation/visuo-haptic-hand/2-method.tex
View File

@@ -36,7 +36,7 @@ We considered two representative contact vibration techniques, \ie two ways of r
The implementation of these two techniques have been tuned according to the results of a preliminary experiment.
Three participants were asked to carry out a series of push and grasp tasks similar to those used in the actual experiment.
Results showed that 95~\% of the contacts between the fingertip and the virtual cube happened at speeds below \qty{1.5}{\m\per\s}.
Results showed that \percent{95} of the contacts between the fingertip and the virtual cube happened at speeds below \qty{1.5}{\m\per\s}.
We also measured the perceived minimum amplitude to be 15~\% (\qty{0.6}{\g}) of the maximum amplitude of the motors we used.
For this reason, we designed the Impact vibration technique (Impa) so that contact speeds from \qtyrange{0}{1.5}{\m\per\s} are linearly mapped into \qtyrange{15}{100}{\%} amplitude commands for the motors.
Similarly, we designed the distance vibration technique (Dist) so that interpenetrations from \qtyrange{0}{2.5}{\cm} are linearly mapped into \qtyrange{15}{100}{\%} amplitude commands for the motors, recalling that the virtual cube has an edge of \qty{5}{\cm}.
@@ -44,50 +44,39 @@ Similarly, we designed the distance vibration technique (Dist) so that interpene
\section{User Study}
\label{method}
This user study aims to evaluate whether a visuo-haptic hand rendering affects the user performance and experience of manipulation of \VOs with bare hands in \OST-\AR.
The chosen visuo-haptic hand renderings are the combination of the two most representative visual hand renderings established in the first experiment, \ie \level{Skeleton} and \level{None}, described in \secref[visual_hand]{hands}, with the two contact vibration techniques provided at the four delocalized positions on the hand described in \secref{vibration}.
This user study aims to evaluate whether a visuo-haptic rendering of the hand affects the user performance and experience of manipulation of \VOs with bare hands in \OST-\AR.
The chosen visuo-haptic hand renderings are the combination of the two most representative visual hand renderings established in the \chapref{visual_hand}, \ie \level{Skeleton} and \level{None}, described in \secref[visual_hand]{hands}, with the two contact vibration techniques provided at the four delocalized positions on the hand described in \secref{vibration}.
\subsection{Experimental Design}
\label{design}
We considered the same two \level{Push} and \level{Grasp} tasks as described in \secref[visual_hand]{tasks}, that we analyzed separately, considering four independent, within-subject variables:
\begin{itemize}
\item \factor{Positioning}: the five positionings for providing vibrotactile hand rendering of the virtual contacts, as described in \secref{positioning}.
\item \factor{Vibration Technique}: the two contact vibration techniques, as described in \secref{technique}.
\item \factor{Hand}: two visual hand renderings from the first experiment, \level{Skeleton} (Skel) and \level{None}, as described in \secref[visual_hand]{hands}; we considered \level{Skeleton} as it performed the best in terms of performance and perceived effectiveness and \level{None} as reference.
\item \factor{Target}: we considered the target volumes (\figref{tasks}), from the participant's point of view, located at:
\begin{itemize}
\item left-bottom (\level{LB}) and left-right (\level{LF}) during the \level{Push} task; and
\item right-bottom (\level{RB}), left-bottom (\level{LB}), left-right (\level{LF}) and right-front (\level{RF}) during the \level{Grasp} task.
\end{itemize}. We considered these targets because they presented different difficulties.
\end{itemize}
\begin{subfigs}{tasks}{The two manipulation tasks of the user study.}[
Both pictures show the cube to manipulate in the middle (\qty{5}{\cm} and opaque) and the eight possible targets to reach (\qty{7}{\cm} cube and semi-transparent).
Only one target at a time was shown during the experiments.
][
\item Pushing a virtual cube along a table toward a target placed on the same surface.
\item Grasping and lifting a virtual cube toward a target placed on a \qty{20}{\cm} higher plane.
\item Push task: pushing the virtual cube along a table towards a target placed on the same surface.
\item Grasp task: grasping and lifting the virtual cube towards a target placed on a \qty{20}{\cm} higher plane.
]
\subfig[0.23]{method/task-push}
\subfig[0.23]{method/task-grasp}
\subfig[0.45]{method/task-push}
\subfig[0.45]{method/task-grasp}
\end{subfigs}
\begin{subfigs}{push_results}{Results of the grasp task performance metrics.}[
Geometric means with bootstrap 95~\% \CI for each vibrotactile positioning (a, b and c) or visual hand rendering (d)
and Tukey's \HSD pairwise comparisons: *** is \pinf{0.001}, ** is \pinf{0.01}, and * is \pinf{0.05}.
][
\item Time to complete a trial.
\item Number of contacts with the cube.
\item Mean time spent on each contact.
\item Mean time spent on each contact.
]
\subfig[0.24]{results/Push-CompletionTime-Location-Overall-Means}
\subfig[0.24]{results/Push-Contacts-Location-Overall-Means}
\subfig[0.24]{results/Push-TimePerContact-Location-Overall-Means}
\subfig[0.24]{results/Push-TimePerContact-Hand-Overall-Means}
\end{subfigs}
We considered the same two tasks as described in \secref[visual_hand]{tasks}, that we analyzed separately, considering four independent, within-subject variables:
\begin{itemize}
\item \emph{{Vibrotactile Positioning}:} the five positionings for providing vibrotactile hand rendering of the virtual contacts, as described in \secref{positioning}.
\item \emph{Contact Vibration Technique}: the two contact vibration techniques, as described in \secref{technique}.
\item \emph{visual Hand rendering}: two visual hand renderings from the first experiment, \level{Skeleton} (Skel) and \level{None}, as described in \secref[visual_hand]{hands}; we considered \level{Skeleton} as it performed the best in terms of performance and perceived effectiveness and \level{None} as reference.
\item \emph{Target}: we considered target volumes located at \level{LB} and \level{LF} during the \factor{Push} task, and at \level{RB}, \level{LB}, \level{LF}, and \level{RF} during the \factor{Grasp} task (\figref{tasks}); we considered these targets because they presented different difficulties.
\end{itemize}
To account for learning and fatigue effects, the positioning of the vibrotactile hand rendering (positioning) was counter-balanced using a balanced \numproduct{10 x 10} Latin square.
In these ten blocks, all possible Technique \x Hand \x Target combination conditions were repeated three times in a random order.
As we did not find any relevant effect of the order in which the tasks were performed in the first experiment, we fixed the order of the tasks: first, the \factor{Push} task and then the \factor{Grasp} task.
To account for learning and fatigue effects, the order of the \factor{Positioning} conditions were counter-balanced using a balanced \numproduct{10 x 10} Latin square.
In these ten blocks, all possible \factor{Technique} \x \factor{Hand} \x \factor{Target} combination conditions were repeated three times in a random order.
As we did not find any relevant effect of the order in which the tasks were performed in the first experiment, we fixed the order of the tasks: first, the \level{Push} task and then the \level{Grasp} task.
This design led to a total of 5 vibrotactile positionings \x 2 vibration contact techniques \x 2 visual hand rendering \x (2 targets on the Push task + 4 targets on the Grasp task) \x 3 repetitions $=$ 420 trials per participant.
@@ -95,84 +84,58 @@ This design led to a total of 5 vibrotactile positionings \x 2 vibration contact
\label{apparatus}
Apparatus and protocol were very similar to the first experiment, as described in \secref[visual_hand]{apparatus} and \secref[visual_hand]{protocol}, respectively.
%
We report here only the differences.
We employed the same vibrotactile device used by \cite{devigne2020power}.
%
It is composed of two encapsulated Eccentric Rotating Mass (ERM) vibration motors (Pico-Vibe 304-116, Precision Microdrive, UK).
%
They are small and very light (\qty{5}{\mm} \x \qty{20}{\mm}, \qty{1.2}{\g}) actuators capable of vibration frequencies from \qtyrange{120}{285}{\Hz} and
amplitudes from \qtyrange{0.2}{1.15}{\g}.
%
They have a latency of \qty{20}{\ms} that we partially compensated for at the software level with slightly larger colliders to trigger the vibrations very close the moment the finger touched the cube.
%
These two outputs vary linearly together, based on the tension applied.
%
They were controlled by an Arduino Pro Mini (3.3 V) and a custom board that delivered the tension independently to each motor.
%
They were controlled by an Arduino Pro Mini (\qty{3.3}{\V}) and a custom board that delivered the tension independently to each motor.
A small \qty{400}{mAh} Li-ion battery allowed for 4 hours of constant vibration at maximum intensity.
%
A Bluetooth module (RN42XV module, Microchip Technology Inc., USA) mounted on the Arduino ensured wireless communication with the HoloLens~2.
To ensure minimal encumbrance, we used the same two motors throughout the experiment, moving them to the considered positioning before each new block.
%
Thin self-gripping straps were placed on the five positionings, with an elastic strap stitched on top to place the motor, as shown in \figref{method/locations}.
%
The straps were fixed during the entirety of the experiment to ensure similar hand tracking conditions.
%
We confirmed that this setup ensured a good transmission of the rendering and guaranteed a good hand tracking performance, that was measured to be constant (\qty{15}{\ms}) with and without motors, regardless their positioning.
%
The control board was fastened to the arm with an elastic strap.
%
Finally, participants wore headphones diffusing brown noise to mask the sound of the vibrotactile motors.
We improved the hand tracking performance of the system by placing on the table a black sheet that absorbs the infrared light as well as placing the participants in front of a wall to ensure a more constant exposure to the light.
%
We also made grasping easier by adding a grasping helper, similar to UltraLeap's Physics Hands.\footnoteurl{https://docs.ultraleap.com/unity-api/Preview/physics-hands.html}.
%
When a phalanx collider of the tracked hand contacts the virtual cube,
%
a spring with a low stiffness is created and attached between the cube and the collider.
%
The spring pulls gently the cube toward the phalanxes in contact with the object to help maintain a natural and stable grasp.
%
When the contact is lost, the spring is destroyed.
%
Preliminary tests confirmed this approach.
\subsection{Participants}
\label{participants}
Twenty subjects participated in the study (mean age = 26.8, \sd{4.1}; 19~males, 1~female).
One was left-handed, while the other nineteen were right-handed. They all used their dominant hand during the trials.
They all had a normal or corrected-to-normal vision.
Thirteen subjects participated also in the previous experiment.
Participants rated their expertise (\enquote{I use it more than once a year}) with \VR, \AR, and haptics in a pre-experiment questionnaire.
There were twelve experienced with \VR, eight experienced with \AR, and ten experienced with haptics.
VR and haptics expertise were highly correlated (\pearson{0.9}), as well as \AR and haptics expertise (\pearson{0.6}).
Other expertise correlations were low ($r<0.35$).
\subsection{Collected Data}
\label{metrics}
During the experiment, we collected the same data as in the first experiment, see \secref[visual_hand]{metrics}.
%
At the end of the experiment, participants were asked if they recognized the different contact vibration techniques.
%
They then rated the ten combinations of Positioning \x Technique using a 7-item Likert scale (1=Not at all, 7=Extremely):
%
\emph{(Vibration Rating)} How much do you like each vibrotactile rendering? %
\emph{(Workload)} How demanding or frustrating was each vibrotactile rendering? %
\emph{(Usefulness)} How useful was each vibrotactile rendering? %
\emph{(Realism)} How realistic was each vibrotactile rendering?
%
Finally, they rated the ten combinations of Positioning \x Hand on a 7-item Likert scale (1=Not at all, 7=Extremely): %
\emph{(positioning \x Hand Rating)} How much do you like each combination of vibrotactile location for each visual hand rendering?
They then rated the ten combinations of \factor{Positioning} \x \factor{Vibration Technique} using a 7-item Likert scale (1=Not at all, 7=Extremely):
\begin{itemize}
\item \response{Vibration Rating}: How much do you like each vibrotactile rendering?
\item \response{Workload}: How demanding or frustrating was each vibrotactile rendering?
\item \response{Usefulness}: How useful was each vibrotactile rendering?
\item \response{Realism}: How realistic was each vibrotactile rendering?
\end{itemize}
\subsection{Participants}
\label{participants}
Twenty subjects participated in the study (mean age = 26.8, SD = 4.1; 19~males, 1~female).
%
One was left-handed, while the other nineteen were right-handed. They all used their dominant hand during the trials.
%
They all had a normal or corrected-to-normal vision.
%
Thirteen subjects participated also in the previous experiment.
Participants rated their expertise (\enquote{I use it more than once a year}) with \VR, \AR, and haptics in a pre-experiment questionnaire.
%
There were twelve experienced with \VR, eight experienced with \AR, and ten experienced with haptics.
%
VR and haptics expertise were highly correlated (\pearson{0.9}), as well as \AR and haptics expertise (\pearson{0.6}).
%
Other expertise correlations were low ($r<0.35$).
Finally, they rated the ten combinations of \factor{Positioning} \x factor{Hand} on a 7-item Likert scale (1=Not at all, 7=Extremely):
\response{Positioning \x Hand Rating}: How much do you like each combination of vibrotactile location for each visual hand rendering?