tangible -> real

This commit is contained in:
2024-10-12 15:24:56 +02:00
parent 000a0a0fc5
commit f624ed5d44
16 changed files with 91 additions and 84 deletions

View File

@@ -5,26 +5,26 @@
\section{Summary}
In this thesis, entitled \enquote{\textbf{\ThesisTitle}}, we have shown how wearable haptics, worn on the outside of the hand, can improve direct hand interaction in immersive \AR by augmenting the perception and manipulation of the virtual.
In this manuscript, we have shown how wearable haptics, worn on the outside of the hand, can improve direct hand interaction in immersive \AR by augmenting the perception and manipulation of the virtual.
Wearable haptics can provide a rich tactile feedback on \VOs and augment the perception of real objects, both directly touched by the hand, while preserving the freedom of movement and interaction with the \RE.
However, their integration with \AR is still in its infancy, and presents many design, technical and human challenges.
We have structured our research around two axes: \textbf{(I) modifying the texture perception of tangible surfaces}, and \textbf{(II) improving the manipulation of \VOs}.
We have structured our research around two axes: \textbf{(I) modifying the texture perception of real surfaces}, and \textbf{(II) improving the manipulation of \VOs}.
\noindentskip In \partref{perception} we focused on modifying the perception of wearable and immersive virtual visuo-haptic textures that augment tangible surfaces.
\noindentskip In \partref{perception} we focused on modifying the perception of wearable and immersive virtual visuo-haptic textures that augment real surfaces.
Texture is a fundamental property of an object, perceived equally by sight and touch.
It is also one of the most studied haptic augmentations, but it had not yet been integrated into \AR or \VR.
We \textbf{(1)} proposed a \textbf{wearable visuo-haptic texture augmentation system}, \textbf{(2)} evaluated how the perception of haptic texture augmentations is \textbf{affected by the visual virtuality of the hand} and the environment (real, augmented, or virtual), and \textbf{(3)} investigated the \textbf{perception of co-localized visuo-haptic texture augmentations}.
In \chapref{vhar_system}, we presented a system for \textbf{augmenting any tangible surface} with virtual \textbf{visuo-haptic roughness textures} using an immersive \AR headset and a wearable vibrotactile device worn on the middle phalanx of the finger.
In \chapref{vhar_system}, we presented a system for \textbf{augmenting any real surface} with virtual \textbf{visuo-haptic roughness textures} using an immersive \AR headset and a wearable vibrotactile device worn on the middle phalanx of the finger.
It allows a \textbf{free visual and touch exploration} of the textures, as if they were real, allowing the user to view them from different angles and touch them with the bare finger without constraints on hand movements.
The user studies in the next two chapters are based on this system.
In \chapref{xr_perception} we explored how the perception of wearable haptic augmented textures is affected by the visual virtuality of the hand and the environment, whether it is real, augmented or virtual.
We augmented the perceived roughness of the tangible surface with virtual vibrotactile patterned textures, and rendered the visual conditions by switching the \OST-\AR headset to a \VR-only view.
We augmented the perceived roughness of the real surface with virtual vibrotactile patterned textures, and rendered the visual conditions by switching the \OST-\AR headset to a \VR-only view.
We then conducted a psychophysical user study with 20 participants and extensive questionnaires to evaluate the perceived roughness augmentation in these three visual conditions.
The textures were perceived as \textbf{rougher when touched with the real hand alone compared to a virtual hand} in either \AR or \VR, possibly due to the \textbf{perceived latency} between finger movements and different visual, haptic, and proprioceptive feedbacks.
In \chapref{vhar_textures}, we investigated the perception of co-localized visual and wearable haptic texture augmentations on tangible surfaces.
In \chapref{vhar_textures}, we investigated the perception of co-localized visual and wearable haptic texture augmentations on real surfaces.
We transposed the \textbf{data-driven visuo-haptic textures} from the \HaTT database to the system presented in \chapref{vhar_system} and conducted a user study with 20 participants to rate the coherence, realism, and perceived roughness of the combination of nine visuo-haptic texture pairs.
Participants integrated roughness sensations from both visual and haptic modalities well, with \textbf{haptics predominating the perception}, and consistently identified and matched \textbf{clusters of visual and haptic textures with similar perceived roughness}.
@@ -47,13 +47,13 @@ The results showed that providing vibrotactile feedback \textbf{improved the per
The wearable visuo-haptic augmentations of perception and manipulation we presented, and the user studies we conducted for this thesis have of course some limitations.
In this section, we present some future work for each chapter that could address these issues.
\subsection*{Augmenting the Visuo-haptic Texture Perception of Tangible Surfaces}
\subsection*{Augmenting the Visuo-haptic Texture Perception of Real Surfaces}
\paragraph{Other Augmented Object Properties}
We focused on the visuo-haptic augmentation of roughness using vibrotactile feedback, because it is one of the most salient properties of surfaces (\secref[related_work]{object_properties}), one of the most studied in haptic perception (\secref[related_work]{texture_rendering}), and equally perceived by sight and touch (\secref[related_work]{visual_haptic_influence}).
However, many other wearable augmentation of object properties should be considered, such as hardness, friction, temperature, or local deformations.
Such integration of haptic augmentation of a tangible surface has almost been achieved with the hand-held devices of \citeauthor{culbertson2017ungrounded} \cite{culbertson2017importance,culbertson2017ungrounded}, but will be more challenging with wearable haptic devices.
Such integration of haptic augmentation of a real surface has almost been achieved with the hand-held devices of \citeauthor{culbertson2017ungrounded} \cite{culbertson2017importance,culbertson2017ungrounded}, but will be more challenging with wearable haptic devices.
In addition, combination with pseudo-haptic rendering techniques \cite{ujitoko2021survey} should be systematically investigated to expand the range of possible wearable haptic augmentations.
\paragraph{Fully Integrated Tracking}
@@ -75,18 +75,19 @@ In particular, it remains to be investigated how the vibrotactile patterned text
\paragraph{Broader Visuo-Haptic Conditions}
Our study was conducted with an \OST-\AR headset, but the results may be different with a \VST-\AR headset, where the \RE is seen through cameras and screens, and the perceived simultaneity between visual and haptic stimuli, real or virtual, is different.
We also focused on the perception of roughness augmentation using wearable vibrotactile haptics and a square wave signal to simulate a patterned texture: Our objective was not to accurately reproduce real textures, but to induce different percevied roughness on the same tangible surface with a well controlled parameters.
Our study was conducted with an \OST-\AR headset, but the results may be different with a \VST-\AR headset, where the \RE is seen through cameras and screens (\secref[related_work]{ar_displays}), and the perceived simultaneity between visual and haptic stimuli, real or virtual, is different.
The effect of perceived visuo-haptic simultaneity on the augmented haptic perception and its size should also be systematically investigated, for example by inducing various delays between the visual and haptic feedback.
We also focused on the perception of roughness augmentation using wearable vibrotactile haptics and a square wave signal to simulate a patterned texture: Our objective was not to accurately reproduce real textures, but to induce different perceived roughness on the same real surface with a well controlled parameters.
However, more accurate models for simulating interaction with virtual textures should be applied to wearable haptic augmentations, such as in \textcite{unger2011roughness}.
Another limitation that may have affected the perception of the haptic texture augmentations is the lack of compensation for the frequency response of the actuator and amplifier \cite{asano2012vibrotactile,culbertson2014modeling,friesen2024perceived}.
The dynamic response of the finger should also be considered, and may vary between individuals.
%The dynamic response of the finger should also be considered, and may vary between individuals.
\subsection*{Perception of Visual and Haptic Texture Augmentations in Augmented Reality}
\paragraph{Assess the Applicability of the Method}
As in the previous chapter, our aim was not to accurately reproduce real textures, but to alter the perception of a tangible surface being touched with simultaneous visual and haptic texture augmentations.
However, the results also have some limitations, as they addressed a small set of visuo-haptic textures that augmented the perception of smooth and white tangible surfaces.
As in the previous chapter, our aim was not to accurately reproduce real textures, but to alter the perception of a real surface being touched with simultaneous visual and haptic texture augmentations.
However, the results also have some limitations, as they addressed a small set of visuo-haptic textures that augmented the perception of smooth and white real surfaces.
Visuo-haptic texture augmentations may be difficult on surfaces that already have strong visual or haptic patterns \cite{asano2012vibrotactile}, or on objects with complex shapes.
The role of visuo-haptic texture augmentation should also be evaluated in more complex tasks, such as object recognition and assembly, or in more concrete use cases, such as displaying and touching a museum object or a 3D print before it is manufactured.
@@ -112,7 +113,7 @@ While the mutual occlusion problem and the hand tracking latency can be overcome
\paragraph{More Ecological Conditions}
We conducted the user study with two manipulation tasks that involved placing a virtual cube in a target volume, either by pushing it on a table or by grasping and lifting it.
While these tasks are fundamental building blocks for more complex manipulation tasks \cite{laviolajr20173d}, such as stacking or assembly, more ecological uses should be considered.
While these tasks are fundamental building blocks for more complex manipulation tasks \cite[p.390]{laviolajr20173d}, such as stacking or assembly, more ecological uses should be considered.
Similarly, a broader experimental study might shed light on the role of gender and age, as our subject pool was not sufficiently diverse in this regard.
Finally, all visual hand renderings received low and high rank rates from different participants, suggesting that users should be able to choose and personalize some aspects of the visual hand rendering according to their preferences or needs, and this should be also be evaluated.
@@ -139,6 +140,10 @@ It remains to be explored how to support rendering for different and larger area
% systematic exploration of the parameter space of the haptic rendering to determine the most important parameters their influence on the perception
% measure the difference in sensitivity to the haptic feedback and how much it affects the perception of the object properties
%- Visio en réalité mixte : ar avec avatars distants, vr pour se retrouver dans l'espace de l'autre ou un espace distant, et besoin de se faire toucher des objets à distance
%- Ou bien en cours, voir l'échantillon à toucher dans lenv de travail ou en contexte en passant en VR
%- Ex : médecin palpation, design d'un objet, rénovation d'un logement (AR en contexte courant, VR pour voir et toucher une fois terminé)
\subsection*{Responsive Visuo-Haptic Augmented Reality}
%Given these three points, and the diversity of haptic actuators and renderings, one might be able to interact with the \VOs with any haptic device, worn anywhere on the body and providing personalized feedback on any other part of the hand, and the visuo-haptic system should be able to support such a adapted usage.s