diff --git a/2-perception/vhar-system/6-conclusion.tex b/2-perception/vhar-system/6-conclusion.tex index e754b86..f81f0e4 100644 --- a/2-perception/vhar-system/6-conclusion.tex +++ b/2-perception/vhar-system/6-conclusion.tex @@ -5,3 +5,12 @@ We designed and implemented a system for rendering virtual haptic grating textures on a real tangible surface touched directly with the fingertip, using a wearable vibrotactile voice-coil device mounted on the middle phalanx of the finger. %, and allowing free explorative movements of the hand on the surface. This tactile feedback was integrated with an immersive visual virtual environment, using an OST-AR headset, to provide users with a coherent multimodal visuo-haptic augmentation of the \RE, that can be switched between an \AR and a \VR view. + +We investigated virtual textures that modify the roughness perception of real, tangible surfaces, using a wearable vibrotactile device worn on the finger. +% +%We studied how different such wearable haptic augmented textures are perceived when touched with a virtual hand instead of one's own hand, and when the hand and its environment are visually rendered in AR or VR. +% +To this end, we first designed and implemented a visuo-haptic texture rendering system that allows free exploration of the augmented surface using a visual AR/VR headset. +%to render virtual vibrotactile textures on any tangible surface, allowing free exploration of the surface, and integrated them with an immersive visual OST-AR headset, that could be switched to a VR view. +% +%This provided a coherent and synchronised multimodal visuo-haptic augmentation of the \RE, which could also be switched between an AR and a VR view. diff --git a/2-perception/xr-perception/1-introduction.tex b/2-perception/xr-perception/1-introduction.tex index b0f4357..1b97cac 100644 --- a/2-perception/xr-perception/1-introduction.tex +++ b/2-perception/xr-perception/1-introduction.tex @@ -4,8 +4,8 @@ Most of the haptic augmentations of tangible surfaces using with wearable haptic Still, it is known that the visual rendering of a tangible can influence the perception of its haptic properties (\secref[related_work]{visual_haptic_influence}), and that the perception of same haptic force-feedback or vibrotactile rendering can differ between \AR and \VR, probably due to difference in perceived simultaneity between visual and haptic stimuli (\secref[related_work]{ar_vr_haptic}). Indeed, while in \AR, the user can see their own hand touching, the haptic device worn and the \RE, in \VR they are hidden by the \VE while. -In this chapter, we investigate the role of the visual rendering of the hand (real or virtual) and its environment (\AR or \VR) on the perception of a \textbf{tangible surface whose haptic roughness is augmented with a wearable vibrotactile} device worn on the finger. -To do so, we employed the visuo-haptic system presented in \chapref{vhar_system} to render the texture augmentations of tangibles that can be directly touched with the bare finger. +In this chapter, we investigate the role of the visual virtuality of the hand (real or virtual) and its environment (\AR or \VR) on the perception of a \textbf{tangible surface whose haptic roughness is augmented with a wearable vibrotactile} device worn on the finger. +To do so, we used the visuo-haptic system presented in \chapref{vhar_system} to render virtual vibrotactile patterned textures (\secref[related_work]{texture_rendering}) to augment the tangible surface being touched.% touched by the finger.% that can be directly touched with the bare finger. We evaluated, in \textbf{user study with psychophysical methods and extensive questionnaire}, the perceived roughness augmentation in three visual rendering conditions: \textbf{(1) without visual augmentation}, in \textbf{(2) \OST-\AR with a realistic virtual hand} rendering, and in \textbf{(3) \VR with the same virtual hand}. To control for the influence of the visual rendering, the tangible surface was not visually augmented. diff --git a/2-perception/xr-perception/5-discussion.tex b/2-perception/xr-perception/5-discussion.tex index a9f18a0..b4e7f06 100644 --- a/2-perception/xr-perception/5-discussion.tex +++ b/2-perception/xr-perception/5-discussion.tex @@ -21,7 +21,7 @@ Very interestingly, the evaluation of the vibrotactile device and the textures w Conversely, the perceived latency of the virtual hand (\response{Hand Latency} question) seemed to be related to the perceived roughness of the textures (with the \PSEs). The \level{Mixed} rendering had the lowest \PSE and highest perceived latency, the \level{Virtual} rendering had a higher \PSE and lower perceived latency, and the \level{Real} rendering had the highest \PSE and no virtual hand latency (as it was not displayed). -Our visuo-haptic augmentation system, described in \chapref{vhar_system}, aimed to provide a coherent visuo-haptic augmentation integrated with the \RE. +Our wearable visuo-haptic texture augmentation system, described in \chapref{vhar_system}, aimed to provide a coherent visuo-haptic renderings registered with the \RE. Yet, it involves different sensory interaction loops between the user's movements and the visuo-haptic feedback (\figref{method/diagram} and \figref[introduction]{interaction_loop}), which may not feel to be in synchronized with each other or with proprioception. %When a user runs their finger over a vibrotactile virtual texture, the haptic sensations and eventual display of the virtual hand lag behind the visual displacement and proprioceptive sensations of the real hand. % diff --git a/2-perception/xr-perception/6-conclusion.tex b/2-perception/xr-perception/6-conclusion.tex index b6be91d..8af80c5 100644 --- a/2-perception/xr-perception/6-conclusion.tex +++ b/2-perception/xr-perception/6-conclusion.tex @@ -1,45 +1,21 @@ \section{Conclusion} \label{conclusion} -In this chapter, we studied how the perception of wearable haptic augmented textures is affected by the visual virtuality of the hand and the environment. -Using the visuo-haptic augmentation system presented in \chapref{vhar_system}, we rendered virtual vibrotactile patterned textures +In this chapter, we studied how the perception of wearable haptic augmented textures is affected by the visual virtuality of the hand and the environment, being either real, augmented or virtual. +Using the wearable visuo-haptic augmentation system presented in \chapref{vhar_system}, we rendered virtual vibrotactile patterned textures on the voice-coil worn on the middle-phalanx of the finger to augment the roughness perception of the tangible surface being touched. +With an immersive \AR headset, that could be switched to a \VR only view, we considered three visual rendering conditions: (1) without visual augmentation, (2) with a realistic virtual hand rendering in \AR, and (3) with the same virtual hand in \VR. +We then evaluated the perceived roughness augmentation in these three visual conditions with a psychophysical user study involving 20 participants and extensive questionnaires. -investigated virtual textures that modify the roughness perception of real, tangible surfaces, using a wearable vibrotactile device worn on the finger. +Our results showed that the visual virtuality of the hand and the environment had a significant effect on the perception of haptic textures and the exploration behaviour of the participants. +The textures were on average perceived as \enquote{rougher} and with a higher sensitivity when touched with the real hand alone than with a virtual hand either in \AR, with \VR in between. +Exploration behaviour was also slower in \VR than with real hand alone, although subjective evaluation of the texture was not affected. +We hypothesised that this difference in perception was due to the \emph{perceived latency} between the finger movements and the different visual, haptic and proprioceptive feedbacks, which were the same in all visual renderings, but were more noticeable in \AR and \VR than without visual augmentation. -evaluated the perception of haptic textures on a tangible surface augmented with a wearable vibrotactile device worn on the finger. - -different such wearable haptic augmented textures are perceived when touched with a virtual hand instead of one's own hand, and when the hand and its environment are visually rendered in AR or VR. - -We investigated virtual textures that modify the roughness perception of real, tangible surfaces, using a wearable vibrotactile device worn on the finger. -% -%We studied how different such wearable haptic augmented textures are perceived when touched with a virtual hand instead of one's own hand, and when the hand and its environment are visually rendered in AR or VR. -% -To this end, we first designed and implemented a visuo-haptic texture rendering system that allows free exploration of the augmented surface using a visual AR/VR headset. -%to render virtual vibrotactile textures on any tangible surface, allowing free exploration of the surface, and integrated them with an immersive visual OST-AR headset, that could be switched to a VR view. -% -%This provided a coherent and synchronised multimodal visuo-haptic augmentation of the \RE, which could also be switched between an AR and a VR view. -% -We then conducted a psychophysical user study with 20 participants to assess the roughness perception of these virtual texture augmentations directly touched with the finger (1) without visual augmentation, (2) with a realistic virtual hand rendering in AR, and (3) with the same virtual hand in VR. -% -%The results showed that the visual rendering of the hand and environment had a significant effect on the perception of haptic textures and the exploration behaviour of the participants. -% -The textures were on average perceived as \enquote{rougher} and with a higher sensitivity when touched with the real hand alone than with a virtual hand either in AR or VR. -% -We hypothesised that this difference in perception was due to the \emph{perceived latency} between the finger movements and the different visual, haptic and proprioceptive feedbacks, which were the same in all visual renderings, but were more noticeable in AR and VR. % than without visual augmentation. -% -With a better understanding of how visual factors influence the perception of haptically augmented tangible objects, the many wearable haptic systems that already exist but have not yet been fully explored with AR can be better applied and new visuo-haptic renderings adapted to AR can be designed. - -We can outline recommendations for future \AR/\VR studies or applications using wearable haptics. -Attention should be paid to the respective latencies of the visual and haptic sensory feedbacks inherent in such systems and, more importantly, to \emph{the perception of their possible asynchrony}. -% -%This is in line with embodiment studies in \VR that compared realism, latency and control \cite{waltemate2016impact,fribourg2020avatar}. -% -Latencies should be measured \cite{friston2014measuring}, minimised to an acceptable level for users and kept synchronised with each other \cite{diluca2019perceptual}. -% -It seems that the visual aspect of the hand or the environment on itself has little effect on the perception of haptic feedback, but the degree of visual reality-virtuality can affect the asynchrony sensation of the latencies, even though they remain identical. -% -%As we have shown, the visual representation of the hand or the environment can affect the experience of the unchanged latencies and thus the perception of haptic feedback. -% -Therefore, when designing for wearable haptics or integrating it into \AR/\VR, it seems important to test its perception in real, augmented and virtual environments. -%Finally, a visual hand representation in OST-\AR together with wearable haptics should be avoided until acceptable tracking latencies are achieved, as was also observed for \VO interaction with the bare hand \cite{normand2024visuohaptic}. +%We can outline recommendations for future \AR/\VR studies or applications using wearable haptics. +This study suggests that attention should be paid to the respective latencies of the visual and haptic sensory feedbacks inherent in such systems and, more importantly, to \emph{the perception of their possible asynchrony}. +Latencies should be measured \cite{friston2014measuring}, minimized to an acceptable level for users and kept synchronised with each other \cite{diluca2019perceptual}. +It seems also that the visual aspect of the hand or the environment on itself has little effect on the perception of haptic feedback, but the degree of visual reality-virtuality can affect the asynchrony sensation of the latencies, even though they remain identical. +When designing for wearable haptics or integrating it into \AR/\VR, it seems important to test its perception in real, augmented and virtual environments. +%With a better understanding of how visual factors influence the perception of haptically augmented tangible objects, the many wearable haptic systems that already exist but have not yet been fully explored with \AR can be better applied and new visuo-haptic renderings adapted to \AR can be designed. +%Finally, a visual hand representation in OST-\AR together with wearable haptics should be avoided until acceptable tracking latencies \are achieved, as was also observed for \VO interaction with the bare hand \cite{normand2024visuohaptic}.