From 26fd971474be308f89a9f8d5fa2692b6032c11d8 Mon Sep 17 00:00:00 2001 From: Erwan Normand Date: Tue, 15 Oct 2024 10:24:25 +0200 Subject: [PATCH] Complete perspectives --- 4-conclusion/conclusion.tex | 29 +++++++++++++++++++++-------- references.bib | 21 +++++++++++++++++++++ 2 files changed, 42 insertions(+), 8 deletions(-) diff --git a/4-conclusion/conclusion.tex b/4-conclusion/conclusion.tex index 101c952..08b2189 100644 --- a/4-conclusion/conclusion.tex +++ b/4-conclusion/conclusion.tex @@ -146,8 +146,8 @@ They also allow us to outline out longer-term research perspectives. We have reviewed how complex the sense of touch is (\secref[related_work]{haptic_hand}). Multiple sensory receptors all over the skin allow us to perceive different properties of objects, such as their texture, temperature, weight or shape. Particularly concentrated in the hands, their sensory feedback is crucial, along with the muscles, for grasping and manipulating objects. -In this manuscript, we have shown how wearable haptic devices can provide virtual tactile sensations to support direct hand interaction in immersive \AR. -In particular, we have investigated the visuo-haptic perception of texture augmenting real surfaces (\partref{perception}) as well as the manipulation of \VOs with visuo-haptic feedback of hand contacts with \VOs (\partref{manipulation}). +In this manuscript, we have shown how wearable haptic devices can provide virtual tactile sensations to support direct hand interaction in immersive \AR: +We have investigated the visuo-haptic perception of texture augmenting real surfaces (\partref{perception}) as well as the manipulation of \VOs with visuo-haptic feedback of hand contacts with \VOs (\partref{manipulation}). However, unlike for the visual sense, which can be completely immersed in the virtual using an \AR/\VR headset, there is no universal wearable haptic device that can reproduce all the haptic properties perceived by the hand (\secref[related_work]{wearable_haptics}). Thus, the haptic renderings and augmentations we studied were limited to specific properties of roughness (\chapref{vhar_system}) and contact (\chapref{visuo_haptic_hand}) using vibrotactile feedback. @@ -166,13 +166,26 @@ These results would make it possible to design more universal wearable haptic de \subsection*{Responsive Visuo-Haptic Augmented Reality} -Nous avons vu la diversité des affichages de réalité augmentée et virtuelle ainsi que les implications sur le rendu des objets virtuel, sa perception (\secref[related_work]{ar_displays}) et sa manipulation avec la main (\chapref{visual_hand}). -La diversité des dispositifs haptiques portables et des rendus tactiles qu'ils peuvent fournir est plus importante encore et un sujet de recherche actif (\secref[related_work]{wearable_haptics}). -Le couplage de l'haptique portable avec la RA immersive demande aussi de placer l'haptique ailleurs sur le corps qu'aux points de contact de la main avec les objets virtuels (\secref[related_work]{vhar_haptics}). -En particulier, nous avons étudié la perception d'augmentations haptiques de textures avec un dispositif vibrotactile sur la phalange médiane (\secref{vhar_system}) et comparés plusieurs positionnements de l'haptique sur la main pour la manipulation d'objets virtuels (\secref{visuo_haptic_hand}). +We have reviewed the diversity of \AR and \VR reality displays and their respective characteristics in rendering, their perception (\secref[related_work]{ar_displays}) and their manipulation with the hand (\chapref{visual_hand}). +The diversity of wearable haptic devices and the different sensations they can provide is even more important (\secref[related_work]{wearable_haptics}) and an active research topic \cite{pacchierotti2017wearable}. +Coupling wearable haptics with immersive \AR also requires the haptic actuator to be placed on the body other than at the hand contact points (\secref[related_work]{vhar_haptics}). +In particular, in this thesis, we have investigated the perception of haptic texture augmentation using a vibrotactile device on the median phalanx (\chapref{vhar_system}) and also compared different positions of the haptics on the hand for manipulating \VOs (\chapref{visuo_haptic_hand}). + +Haptic feedback should be provided close to the point of contact of the hand with the virtual, for a better realism of texture augmentation (\chapref{vhar_textures}) and for rendering contact with virtual objects (\chapref{visuo_haptic_hand}), \eg rendering fingertip contact with a haptic ring worn on the middle or proximal phalanx. +However, the task at hand, the user's sensitivity and preferences, the constraints of the tracking system, or the ergonomics of the haptic device may require the use of other form factors and positions, such as the wrist or the arm. +Similarly, collaborative and transitional experiences between \AR and \VR are becoming more common, and would involve different setups and modalities \cite{roo2017onea}. +Novel \AR/\VR displays are already capable of transitioning from augmented to virtual environments \cite{feld2024simple}, and the haptic feedback should also adapt to these transitions as well (\chapref{xr_perception}). + +Therefore, a visuo-haptic augmented reality system should be able to adapt to any \AR/\VR display, any wearable haptic device worn anywhere on the body, and support personalization of the haptic feedback. +In other words, the visuo-haptic rendering system should be designed to be responsive to the context of use. +This would require the development and validation of methods to automatically calibrate the haptic feedback to the user's perception in accordance of what it was been designed to represent. +Methods should also be developed to allow the user to easily adjust the haptic feedback to their preferences at runtime, but without exposing all the design parameters \cite{kim2020defining}. + +Finally, different use cases and applications of visuo-haptic \AR should be explored and evaluated. +For example, capturing and sharing visuo-haptic perceptions of objects in a \AR/\VR visioconference, or projecting and touching a sample in a real wall for interior design then switching to \VR to see and touch the complete final result. +A museum object could be manipulated by a real proxy object in visuo-haptic \AR, or even as it was in the past in its original state in \VR. +Another example could be a medical teleconsultation, where the doctor could palpate a distant patient with haptic augmentation, but in \VR. -Un rendu haptique fourni proche du point de contact de la main avec le virtuel semble donc préférable pour augmenter la perception d'objets réels (\secref{vhar_textures}) et améliorer la manipulation d'objets virtuels (\secref{visuo_haptic_hand}). -%Given these three points, and the diversity of haptic actuators and renderings, one might be able to interact with the \VOs with any haptic device, worn anywhere on the body and providing personalized feedback on any other part of the hand, and the visuo-haptic system should be able to support such a adapted usage. % design, implement and validate procedures to automatically calibrate the haptic feedback to the user's perception in accordance to what it has been designed to represent % + let user free to easily adjust (eg can't let adjust whole spectrum of vibrotactile, reduce to two or three dimensions with sliders using MDS) diff --git a/references.bib b/references.bib index 66d8a50..8717f1f 100644 --- a/references.bib +++ b/references.bib @@ -1111,6 +1111,16 @@ doi = {10/f3pj8m} } +@article{feld2024simple, + title = {Simple and {{Efficient}}? {{Evaluation}} of {{Transitions}} for {{Task-Driven Cross-Reality Experiences}}}, + shorttitle = {Simple and {{Efficient}}?}, + author = {Feld, Nico and Bimberg, Pauline and Weyers, Benjamin and Zielasko, Daniel}, + date = {2024}, + journaltitle = {IEEE Trans. Vis. Comput. Graph.}, + pages = {1--18}, + doi = {10/g8fg3f} +} + @article{fernandes2012tactual, title = {Tactual Perception: A Review of Experimental Variables and Procedures}, shorttitle = {Tactual Perception}, @@ -1945,6 +1955,17 @@ doi = {10/gfm7wv} } +@inproceedings{kim2020defining, + title = {Defining {{Haptic Experience}}: {{Foundations}} for {{Understanding}}, {{Communicating}}, and {{Evaluating HX}}}, + shorttitle = {Defining {{Haptic Experience}}}, + booktitle = {{{CHI Conf}}. {{Hum}}. {{Factors Comput}}. {{Syst}}.}, + author = {Kim, Erin and Schneider, Oliver}, + date = {2020-04-21}, + pages = {1--13}, + doi = {10/gg2tfb}, + isbn = {978-1-4503-6708-0} +} + @article{kim2021synthesizing, title = {Synthesizing the {{Roughness}} of {{Textured Surfaces}} for an {{Encountered-Type Haptic Display Using Spatiotemporal Encoding}}}, author = {Kim, Yaesol and Kim, Siyeon and Oh, Uran and Kim, Young J.},