110 lines
10 KiB
TeX
110 lines
10 KiB
TeX
\section{User Study}
|
|
\label{experiment}
|
|
|
|
\subsection{The textures}
|
|
\label{textures}
|
|
|
|
The 100 visuo-haptic texture pairs of the \HaTT database \cite{culbertson2014one} were preliminary tested and compared using the apparatus described in \secref{apparatus} to select the most representative textures for the user study.
|
|
These texture models were chosen as they are visuo-haptic representations of a wide range of real textures that are publicly available online.
|
|
Nine texture pairs were selected (\figref{experiment/textures}) to cover various perceived roughness, from rough to smooth, as named on the database: \level{Metal Mesh}, \level{Sandpaper~100}, \level{Brick~2}, \level{Cork}, \level{Sandpaper~320}, \level{Velcro Hooks}, \level{Plastic Mesh~1}, \level{Terra Cotta}, \level{Coffee Filter}.
|
|
All these visual and haptic textures are isotropic: their rendering (appearance or roughness) is the same whatever the direction of the movement on the surface, \ie there are no local deformations (holes, bumps, or breaks).
|
|
|
|
\subsection{Apparatus}
|
|
\label{apparatus}
|
|
|
|
\figref{experiment/setup} shows the experimental setup, and \figref{experiment/view} the first person view of participants during the user study.
|
|
The user study was held in a quiet room with no windows, with one light source of \qty{800}{\lumen} placed \qty{70}{\cm} above the table.
|
|
|
|
Nine \qty{5}{\cm} square cardboards with smooth, white melamine surface, arranged in a \numproduct{3 x 3} grid, were used as real surfaces to augment.
|
|
Their poses were estimated with three \qty{2}{\cm} AprilTag fiducial markers glued on the surfaces grid.
|
|
Similarly, a \qty{2}{\cm} fiducial marker was glued on top of the vibrotactile actuator to detect the finger pose.
|
|
Positioned \qty{20}{\cm} above the surfaces, a webcam (StreamCam, Logitech) filmed the markers to track finger movements relative to the surfaces, as described in \secref[vhar_system]{virtual_real_registration}.
|
|
The visual textures were displayed on the real surfaces using the \OST-\AR headset Microsoft HoloLens~2 running a custom application at \qty{60}{FPS} made with Unity (v2021.1) and Mixed Reality Toolkit (v2.7).
|
|
A set of empirical tests enabled us to choose the best rendering characteristics in terms of transparency and brightness for the visual textures, that were used throughout the user study.
|
|
|
|
When a virtual haptic texture was touched, a \qty{48}{kHz} audio signal was generated using the rendering procedure described in \cite{culbertson2014modeling} from the corresponding \HaTT haptic texture model and the measured tangential speed of the finger (\secref[vhar_system]{texture_generation}).
|
|
The normal force on the texture was assumed to be constant at \qty{1.2}{\N} to generate the audio signal from the model, as \textcite{culbertson2015should}, who found that the \HaTT textures can be rendered using only the speed as input without decreasing their perceived realism.
|
|
The rendering of the virtual texture is described in \secref[vhar_system]{texture_generation}.
|
|
The vibrotactile voice-coil actuator (HapCoil-One, Actronika) was firmly attached to the middle index phalanx of the participant's dominant hand using a Velcro strap, similarly to previous studies \cite{asano2015vibrotactile,friesen2024perceived}.
|
|
|
|
\begin{subfigs}{setup}{Textures used and experimental setup of the user study. }[][
|
|
\item The nine visuo-haptic textures used in the user study, selected from the \HaTT database \cite{culbertson2014one}.
|
|
The texture names were never shown, to prevent the use of the user's visual or haptic memory of the textures.
|
|
\item Experimental setup.
|
|
Participant sat in front of the real surfaces, which were augmented with visual textures displayed by the Microsoft HoloLens~2 \AR headset and haptic roughness textures rendered by the vibrotactile haptic device placed on the middle index phalanx.
|
|
A webcam above the surfaces tracked the finger movements.
|
|
]
|
|
\subfig[0.49]{experiment/textures}
|
|
\subfig[0.49]{experiment/setup}
|
|
\end{subfigs}
|
|
|
|
\subsection{Procedure and Collected Data}
|
|
\label{procedure}
|
|
|
|
Participants were first given written instructions about the experimental setup, the tasks, and the procedure of the user study.
|
|
Then, after having signed an informed consent form, they were asked to seat in front of the table with the experimental setup and to wear the \AR headset.
|
|
As the haptic textures generated no audible noise, participants did not wear any noise reduction headphones.
|
|
A calibration of both the HoloLens~2 and the finger pose estimation was performed to ensure the correct registration of the visuo-haptic textures and the real finger with the real surfaces, as described in \secref[vhar_system]{virtual_real_registration}.
|
|
Finally, participants familiarized with the augmented surface in a \qty{2}{min} training session with textures different from the ones used in the user study.
|
|
|
|
Participants started with the \level{Matching} task.
|
|
They were informed that the user study involved nine pairs of corresponding visual and haptic textures that were separated and shuffled.
|
|
On each trial, the same visual texture was displayed on the nine real surfaces, while the nine haptic textures were rendered on only one of the surfaces at a time, \ie all surfaces were augmented by the same visual texture, but each surface was augmented by a different haptic texture.
|
|
The placement of the haptic textures was randomized before each trial.
|
|
Participants were instructed to look closely at the details of the visual textures and explore the haptic textures with a constant pressure and various speeds to find the haptic texture that best matched the visual texture, \ie choose the surface with the most coherent visual-haptic texture pair.
|
|
The texture names were never given or shown to prevent the use of visual or haptic memory of the textures, nor a definition of what roughness is was given, to let participants complete the task as naturally as possible, similarly to \textcite{bergmanntiest2007haptic}.
|
|
|
|
Then, participants performed the \level{Ranking} task, employing the same setup as the matching task and the same 9 textures.
|
|
In this case, participants were asked to rank the textures according to their perceived roughness.
|
|
First, they ranked all the haptic textures (without any visual augmentation given), then all the visual textures (without any haptic augmentation given), and finally all the visuo-haptic texture pairs together, being informed that they were the correct matches as per the original \HaTT database.
|
|
The placement of the textures was also randomized before each trial.
|
|
|
|
The user study took on average 1 hour to complete.
|
|
|
|
\subsection{Experimental Design}
|
|
\label{design}
|
|
|
|
The user study was a within-subjects design with two tasks: \level{Matching} and \level{Ranking}.
|
|
|
|
In the \level{Matching} task, participants had to find the haptic texture that best matched a given visual texture.
|
|
It had one within-subjects factor, \factor{Visual Texture} with the following levels:
|
|
\level{Metal Mesh}, \level{Sandpaper~100}, \level{Brick~2}, \level{Cork}, \level{Sandpaper~320}, \level{Velcro Hooks}, \level{Plastic Mesh~1}, \level{Terra Cotta}, \level{Coffee Filter}.
|
|
To account for learning and fatigue effects, the order of \factor{Visual Texture} was counterbalanced using a balanced \numproduct{18 x 18} Latin square design.
|
|
A total of 9 textures \x 3 repetitions = 18 matching trials were performed per participant.
|
|
|
|
In the \level{Ranking} task, participants had to rank the haptic textures, the visual textures, and the visuo-haptic texture pairs according to their perceived roughness.
|
|
It had one within-subjects factor, \factor{Modality} with the following levels: \level{Visual}, \level{Haptic}, \level{Visuo-Haptic}.
|
|
Each modality level was ranked once per participant following the fixed order listed above (\secref{procedure}).
|
|
|
|
\subsection{Participants}
|
|
\label{participants}
|
|
|
|
Twenty participants took part in the user study (12 males, 7 females, 1 preferred not to say), aged between 20 and 60 years (\median{26.5}, \iqr{9}{}).
|
|
One participant was left-handed, all others were right-handed; they all performed the user study with their dominant hand.
|
|
All participants had normal or corrected-to-normal vision and none of them had a known hand or finger impairment.
|
|
They rated their experience with haptics, \AR, and \VR (\enquote{I use it every month or more}); 10 were experienced with haptics, 2 with \AR, and 10 with \VR.
|
|
Experiences were correlated between haptics and \AR (\spearman{0.53}), haptics and \VR (\spearman{0.61}), and \AR and \VR (\spearman{0.74}); but not with age (\spearman{-0.06} to \spearman{-0.05}) or gender (\spearman{0.10} to \spearman{0.27}).
|
|
Participants were recruited at the university on a voluntary basis.
|
|
They all signed an informed consent form before the user study.
|
|
|
|
\subsection{Collected Data}
|
|
\label{collected_data}
|
|
|
|
For each trial of the \level{Matching} task, the chosen \response{Haptic Texture} for the given displayed \factor{Visual Texture} was recorded.
|
|
The \response{Completion Time} was also measured as the time between the visual texture display and the haptic texture selection.
|
|
For each modality of the \level{Ranking} task, the \response{Rank} of each of the visual, haptic, or visuo-haptic pairs of the textures presented was recorded.
|
|
|
|
After each of the two tasks, participants answered to the following 7-item Likert scale questions (1=Not at all, 7=Extremely):
|
|
\begin{itemize}
|
|
\item \response{Haptic Difficulty}: How difficult was it to differentiate the tactile textures?
|
|
\item \response{Visual Difficulty}: How difficult was it to differentiate the visual textures?
|
|
\item \response{Textures Match}: For the visual-tactile pairs you have chosen, how coherent were the tactile textures with the corresponding visual textures?
|
|
\item \response{Haptic Realism}: How realistic were the tactile textures?
|
|
\item \response{Visual Realism}: How realistic were the visual textures?
|
|
\item \response{Uncomfort}: How uncomfortable was to use the haptic device?
|
|
\end{itemize}
|
|
|
|
In an open question, participants also commented on their strategy for completing the \level{Matching} task (\enquote{How did you associate the tactile textures with the visual textures?}) and the \level{Ranking} task (\enquote{How did you rank the textures?}).
|
|
|
|
The results were analyzed using R (v4.4) and the packages \textit{afex} (v1.4), \textit{ARTool} (v0.11), \textit{corrr} (v0.4), \textit{FactoMineR} (v2.11), \textit{lme4} (v1.1), and \textit{performance} (v0.13).
|