74 lines
8.1 KiB
TeX
74 lines
8.1 KiB
TeX
\section{Discussion}
|
|
\label{sec:discussion}
|
|
|
|
In this study, we investigated the perception of visuo-haptic texture augmentation of tangible surfaces touched directly with the index fingertip, using visual texture overlays in AR and haptic roughness textures generated by a vibrotactile device worn on the middle index phalanx.
|
|
%
|
|
The nine evaluated pairs of visuo-haptic textures, taken from the HaTT database~\autocite{culbertson2014one}, are models of real texture captures.
|
|
%
|
|
Their perception was evaluated in a two-task user study in which participants chose the most coherent combinations of visual and haptic textures (matching task), and ranked all textures according to their perceived roughness (ranking task).
|
|
%
|
|
The visual textures were displayed statically on the tangible surface, while the haptic textures adapted in real time to the speed of the finger on the surface, giving the impression that the visuo-haptic textures were integrated into the tangible surface.
|
|
%
|
|
In addition, the interaction with the textures was designed to be as natural as possible, without imposing a specific speed of finger movement, as in similar studies~\autocite{asano2015vibrotactile,friesen2024perceived}.
|
|
|
|
In the matching task, participants were not able to effectively match the original visual and haptic texture pairs (see \figref{results_matching_ranking}, left), except for the Coffee Filter texture, which was the smoothest both visually and haptically.
|
|
%
|
|
However, almost all visual textures, except Sandpaper~100, were matched with at least one haptic texture at a level above chance.
|
|
%
|
|
Similarly, five haptic textures were favored over the others to be matched with the visual textures.
|
|
%
|
|
Thus, it seems that not all participants perceived visual textures in the same way and that they also hesitated between several haptic textures for a given visual texture.
|
|
%
|
|
Indeed, the majority of users explained that, based on the roughness, granularity, or imperfections of the visual texture, they matched the haptic texture that seemed most similar or natural to what they imagined.
|
|
%
|
|
Several strategies were used, as some participants reported using vibration frequency and/or amplitude to match a haptic texture.
|
|
%
|
|
It should be noted that the task was rather difficult (see \figref{results_questions}), as participants had no prior knowledge of the textures, there were no additional visual cues such as the shape of an object, and the term \enquote{roughness} had not been used by the experimenter prior to the ranking task.
|
|
|
|
The correspondence analysis (see \figref{results_similarity}, left) highlighted that participants did indeed match visual and haptic textures primarily on the basis of their perceived roughness (60\% of variance), which is in line with previous perception studies on real~\autocite{baumgartner2013visual} and virtual~\autocite{culbertson2014modeling} textures.
|
|
%
|
|
The rankings (see \figref{results_matching_ranking}, right) confirmed that the participants all perceived the roughness of haptic textures very similarly, but that there was less consensus for visual textures, which is also in line with roughness rankings for real haptic and visual textures~\autocite{bergmanntiest2007haptic}.
|
|
%
|
|
These results made it possible to identify and name groups of textures in the form of clusters, and to construct confusion matrices between these clusters and between visual texture ranks with haptic clusters, showing that participants consistently identified and matched haptic and visual textures (see \figref{results_clusters}).
|
|
%
|
|
Interestingly, 30\% of the matching variance was captured with a second dimension, opposing the roughest textures (Metal Mesh, Sandpaper~100), and to a lesser extent the smoothest (Coffee Filter, Sandpaper~320), with all other textures.
|
|
%
|
|
One hypothesis is that this dimension could be the perceived stiffness of the textures, with Metal Mesh and smooth textures appearing stiffer than the other textures, whose granularity could have been perceived as bumps on the surface that could deform under finger pressure.
|
|
%
|
|
Stiffness is, with roughness, one of the main characteristics perceived by the vision and touch of real materials~\autocite{baumgartner2013visual,vardar2019fingertip}, but also on virtual haptic textures~\autocite{culbertson2014modeling,degraen2019enhancing}.
|
|
%
|
|
The last visuo-haptic roughness ranking (see \figref{results_matching_ranking}, right) showed that both haptic and visual sensory information were well integrated as the resulting roughness ranking was being in between the two individual haptic and visual rankings.
|
|
%
|
|
Several strategies were reported: some participants first classified visually and then corrected with haptics, others classified haptically and then integrated visuals.
|
|
%
|
|
While visual sensation did influence perception, as observed in previous haptic AR studies~\autocite{punpongsanon2015softar,gaffary2017ar,fradin2023humans}, haptic sensation dominated here.
|
|
%
|
|
This indicates that participants were more confident and relied more on the haptic roughness perception than on the visual roughness perception when integrating both in one coherent perception.
|
|
%
|
|
Several participants also described attempting to identify visual and haptic textures using spatial breaks, edges or patterns, that were not observed when these textures were displayed in non-immersive virtual environments with a screen~\autocite{culbertson2014modeling,culbertson2015should}.
|
|
%
|
|
A few participants even reported that they clearly sensed patterns on haptic textures.
|
|
%
|
|
However, the visual and haptic textures used were isotropic and homogeneous models of real texture captures, \ie their rendered roughness was constant and did not depend on the direction of movement but only on the speed of the finger.
|
|
%
|
|
Overall, the haptic device was judged to be comfortable, and the visual and haptic textures were judged to be fairly realistic and to work well together (see \figref{results_questions}).
|
|
|
|
These results have of course some limitations as they addressed a small set of visuo-haptic textures augmenting the perception of smooth white tangible surfaces.
|
|
%
|
|
Indeed, the increase in visuo-haptic texture perception may be limited on surfaces that already have strong visual or haptic patterns~\autocite{asano2012vibrotactile}, or on objects with complex shapes.
|
|
%
|
|
In addition, the haptic textures used were modelled from the vibrations of a probe sliding over the captured surfaces.
|
|
%
|
|
The perception of surface roughness with the finger is actually more complex because it involves both the perception of vibrations and the spatial deformation of the skin~\autocite{klatzky2003feeling}, but also because the sensations generated when exploring a surface depend on factors other than the speed of the finger alone, such as the force of contact, the angle, the posture or the surface of the contact~\autocite{schafer2017transfer}, and the integration of these sensory information into one unified perception is not yet fully understood~\autocite{richardson2022learning}.
|
|
%
|
|
Another limitation that may have affected the perception of haptic textures is the lack of compensation for the frequency response of the actuator and amplifier~\autocite{asano2012vibrotactile,culbertson2014modeling,friesen2024perceived}.
|
|
%
|
|
Finally, the visual textures used were also simple color captures not meant to be used in an immersive virtual environment.
|
|
%
|
|
However, our objective was not to accurately reproduce real textures, but to alter the perception of simultaneous visual and haptic roughness augmentation of a real surface directly touched by the finger in AR.
|
|
%
|
|
In addition of these limitations, both visual and haptic texture models should be improved by integrating the rendering of spatially localized breaks, edges or patterns, like real textures~\autocite{richardson2022learning}, and by being adaptable to individual sensitivities, as personalized haptics is a promising approach~\autocite{malvezzi2021design,young2020compensating}.
|
|
%
|
|
More generally, a wide range of haptic feedbacks should be integrated to form rich and complete haptic augmentations in AR~\autocite{maisto2017evaluation,detinguy2018enhancing,salazar2020altering,normand2024visuohaptic,pacchierotti2024haptics}.
|
|
|