erwan/phd-thesis
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Style

Browse Source
This commit is contained in:
Erwan Normand
2024-09-29 14:09:31 +02:00
parent f2d82affe7
commit d5956ad176
14 changed files with 25 additions and 25 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
1-background/introduction/introduction.tex
View File

@@ -144,7 +144,7 @@ It is possible instead to place the haptic actuator close to the point of contac
Therefore, when touching a virtual or augmented object, the real and virtual visual sensations are seen as co-localized, but the virtual haptic feedback is not.
It remains to be investigated how such potential discrepancies affect the overall perception to design visuo-haptic renderings adapted to \AR.
So far, \AR can only add visual and haptic sensations to the user's overall perception of the environment, but conversely it is very difficult to remove sensations.
So far, \AR can only add visual and haptic sensations to the user's overall perception of the environment, but conversely it is more difficult to remove sensations.
These added virtual sensations can therefore be perceived as out of sync or even inconsistent with the sensations of the \RE, for example with a lower rendering quality, a temporal latency, a spatial shift, or a combination of these.
It is therefore unclear to what extent the real and virtual visuo-haptic sensations will be perceived as realistic or plausible, and to what extent they will conflict or complement each other in the perception of the \AE.

2
1-background/related-work/1-haptic-hand.tex
View File

@@ -226,7 +226,7 @@ For patterned textures, as illustrated in \figref{delhaye2012textureinduced}, th
\lambda \sim \frac{v}{f_p}
\end{equation}
The vibrations generated by exploring everyday textures are also very specific to each texture and similar between individuals, making them identifiable by vibration alone \cite{manfredi2014natural,greenspon2020effect}.
The vibrations generated by exploring everyday textures are also specific to each texture and similar between individuals, making them identifiable by vibration alone \cite{manfredi2014natural,greenspon2020effect}.
This shows the importance of vibration cues even for macro textures and the possibility of generating virtual texture sensations with vibrotactile rendering.
\fig[0.55]{delhaye2012textureinduced}{Speed of finger exploration (horizontal axis) on grating textures with different periods $\lambda$ of spacing (in color) and frequency of the vibration intensity peak $f_p$ propagated in the wrist (vertical axis) \cite{delhaye2012textureinduced}.}

8
1-background/related-work/2-wearable-haptics.tex
View File

@@ -38,7 +38,7 @@ However, it cannot constrain the movements of the wrist and the reaction force i
Such \emph{body-grounded} devices are often heavy and bulky and cannot be considered wearable.
\textcite{pacchierotti2017wearable} defined that : \enquote{A wearable haptic interface should also be small, easy to carry, comfortable, and it should not impair the motion of the wearer}.
An approach is then to move the grounding point very close to the end-effector (\figref{pacchierotti2017wearable_3}): the interface is limited to cutaneous haptic feedback, but its design is more compact, lightweight, comfortable and portable, \eg in \figref{grounded_to_wearable}.
An approach is then to move the grounding point close to the end-effector (\figref{pacchierotti2017wearable_3}): the interface is limited to cutaneous haptic feedback, but its design is more compact, lightweight, comfortable and portable, \eg in \figref{grounded_to_wearable}.
Moreover, as detailed in \secref{object_properties}, cutaneous sensations are necessary and often sufficient for the perception of the haptic properties of an object explored with the hand, as also argued by \textcite{pacchierotti2017wearable}.
\begin{subfigs}{grounded_to_wearable}{Haptic devices for the hand with different wearability levels. }[][
@@ -73,7 +73,7 @@ However, these platforms are specifically designed to provide haptic feedback to
\subsubsection{Pin and Pneumatic Arrays}
\label{array_actuators}
A pin-array is a surface made up of small, rigid pins arranged very close together in a grid and that can be moved individually.
A pin-array is a surface made up of small, rigid pins arranged close together in a grid and that can be moved individually.
When placed in contact with the fingertip, it can create sensations of edge, pressure and texture.
The \figref{sarakoglou2012high} shows an example of a pin-array consisting of \numproduct{4 x 4} pins of \qty{1.5}{\mm} diameter and \qty{2}{\mm} height, spaced at \qty{2}{\mm} \cite{sarakoglou2012high}.
Pneumatic systems use a fluid such as air or water to inflate membranes under the skin, creating sensations of contact and pressure \cite{raza2024pneumatically}.
@@ -153,7 +153,7 @@ A voice-coil actuator is a \LRA but capable of generating vibration at two \DoF,
They are larger in size than \ERMs and \LRAs, but can generate more complex renderings.
Piezoelectric actuators deform a solid material when a voltage is applied.
They are very small and thin and provide two \DoFs of amplitude and frequency control.
They are small and thin and provide two \DoFs of amplitude and frequency control.
However, they require high voltages to operate, limiting their use in wearable devices.
\begin{subfigs}{lra}{Diagram and performance of \LRAs. }[][
@@ -219,7 +219,7 @@ More complex models have also been developed to be physically accurate and repro
\paragraph{Data-driven Models}
Because simulations of realistic virtual textures can be very complex to design and to render in real-time, direct capture and models of real textures have been developed \cite{culbertson2018haptics}.
Because simulations of realistic virtual textures can be complex to design and to render in real-time, direct capture and models of real textures have been developed \cite{culbertson2018haptics}.
\textcite{okamura1998vibration} were the first to measure the vibrations produced by the interaction of a stylus dragged over sandpaper and patterned surfaces.
They found that the contact vibrations with patterns can be modeled as exponentially decaying sinusoids (\eqref{contact_transient}) that depend on the normal force and the scanning velocity of the stylus on the surface.

2
1-background/related-work/3-augmented-reality.tex
View File

@@ -188,7 +188,7 @@ The \emph{system control tasks} are changes to the system state through commands
In \AR and \VR, the state of the system is displayed to the user as a \ThreeD spatial \VE.
In an immersive and portable \AR system, this \VE is experienced at a 1:1 scale and as an integral part of the \RE.
The rendering gap between the real and virtual elements, as described on the interaction loop in \figref[introduction]{interaction-loop}, is thus experienced as very narrow or even not consciously perceived by the user.
The rendering gap between the real and virtual elements, as described on the interaction loop in \figref[introduction]{interaction-loop}, is thus experienced as narrow or even not consciously perceived by the user.
This manifests as a sense of presence of the virtual, as described in \secref{ar_presence}.
As the gap between real and virtual rendering is reduced, one could expects a similar and seamless interaction with the \VE as with a \RE, which \textcite{jacob2008realitybased} called \emph{reality based interactions}.

4
1-background/related-work/4-visuo-haptic-ar.tex
View File

@@ -179,14 +179,14 @@ They can also simulate a texture sensation by rapidly rotating in and out.
In a user study not in \AR, but directly touching images on a tablet, Fingeret was found to be more realistic (4/7) than a \LRA at \qty{100}{\Hz} on the nail (3/7) for rendering buttons and a patterned texture (\secref{texture_rendering}), but not different from vibrations for rendering high-frequency textures (3.5/7 for both).
However, as with \textcite{teng2021touch}, finger speed was not taken into account when rendering vibrations, which may have been detrimental to texture perception, as described in \secref{texture_rendering}.
Finally, \textcite{preechayasomboon2021haplets} (\figref{preechayasomboon2021haplets}) and \textcite{sabnis2023haptic} designed Haplets and Haptic Servo, respectively: These are very compact and lightweight vibrotactile \LRA devices designed to provide both integrated finger motion sensing and very low latency haptic feedback (\qty{<5}{ms}).
Finally, \textcite{preechayasomboon2021haplets} (\figref{preechayasomboon2021haplets}) and \textcite{sabnis2023haptic} designed Haplets and Haptic Servo, respectively: These are compact and lightweight vibrotactile \LRA devices designed to provide both integrated finger motion sensing and low latency haptic feedback (\qty{<5}{ms}).
However, no proper user study has been conducted to evaluate these devices in \AR.
\begin{subfigs}{ar_wearable}{Nail-mounted wearable haptic devices designed for \AR. }[][
%\item A voice-coil rendering a virtual haptic texture on a real sheet of paper \cite{ando2007fingernailmounted}.
\item Touch\&Fold provide contact pressure and vibrations on demand to the fingertip \cite{teng2021touch}.
\item Fingeret is a finger-side wearable haptic device that pulls and pushs the fingertip skin \cite{maeda2022fingeret}.
\item Haplets is a very compact nail device with integrated sensing and vibrotactile feedback \cite{preechayasomboon2021haplets}.
\item Haplets is a compact nail device with integrated sensing and vibrotactile feedback \cite{preechayasomboon2021haplets}.
]
\subfigsheight{33mm}
%\subfig{ando2007fingernailmounted}

4
2-perception/vhar-textures/3-results.tex
View File

@@ -63,7 +63,7 @@ No statistical significant effect of \textit{Visual Texture} was found (\anova{8
%
Almost all the texture pairs in the \textit{Haptic Textures Ranking} results were statistically significantly different (\chisqr{8}{20}{146}, \pinf{0.001}; \pinf{0.05} for each comparison), except between (Metal Mesh, Sandpaper~100), (Cork, Brick~2), (Cork, Sandpaper~320) (Plastic Mesh~1, Velcro Hooks), and (Plastic Mesh~1, Terra Cotta).
%
Average Kendall's Tau correlations between the participants indicated a very high consensus (\kendall{0.82}, \ci{0.81}{0.84}) showing that participants perceived similarly the roughness of the haptic textures.
Average Kendall's Tau correlations between the participants indicated a high consensus (\kendall{0.82}, \ci{0.81}{0.84}) showing that participants perceived similarly the roughness of the haptic textures.
%
Most of the texture pairs in the \textit{Visual Textures Ranking} results were also statistically significantly different (\chisqr{8}{20}{119}, \pinf{0.001}; \pinf{0.05} for each comparison), except for the following groups: \{Metal Mesh, Cork, Plastic Mesh~1\}; \{Sandpaper~100, Brick~2, Plastic Mesh~1, Velcro Hooks\}; \{Cork, Velcro Hooks\}; \{Sandpaper~320, Terra Cotta\}; and \{Sandpaper~320, Coffee Filter\}.
%
@@ -71,7 +71,7 @@ Even though the consensus was high (\kendall{0.61}, \ci{0.58}{0.64}), the roughn
%
Also, almost all the texture pairs in the \textit{Visuo-Haptic Textures Ranking} results were statistically significantly different (\chisqr{8}{20}{140}, \pinf{0.001}; \pinf{0.05} for each comparison), except for the following groups: \{Sandpaper~100, Cork\}; \{Cork, Brick~2\}; and \{Plastic Mesh~1, Velcro Hooks, Sandpaper~320\}.
%
The consensus between the participants was also very high \kendall{0.77}, \ci{0.74}{0.79}.
The consensus between the participants was also high \kendall{0.77}, \ci{0.74}{0.79}.
%
Finally, calculating the similarity of the three rankings of each participant, the \textit{Visuo-Haptic Textures Ranking} was on average highly similar to the \textit{Haptic Textures Ranking} (\kendall{0.79}, \ci{0.72}{0.86}) and moderately to the \textit{Visual Textures Ranking} (\kendall{0.48}, \ci{0.39}{0.56}).
%

2
2-perception/vhar-textures/4-discussion.tex
View File

@@ -31,7 +31,7 @@ The rankings (\figref{results_matching_ranking}, right) confirmed that the parti
%
These results made it possible to identify and name groups of textures in the form of clusters, and to construct confusion matrices between these clusters and between visual texture ranks with haptic clusters, showing that participants consistently identified and matched haptic and visual textures (\figref{results_clusters}).
%
Interestingly, 30\% of the matching variance was captured with a second dimension, opposing the roughest textures (Metal Mesh, Sandpaper~100), and to a lesser extent the smoothest (Coffee Filter, Sandpaper~320), with all other textures.
30\% of the matching variance was also captured with a second dimension, opposing the roughest textures (Metal Mesh, Sandpaper~100), and to a lesser extent the smoothest (Coffee Filter, Sandpaper~320), with all other textures.
%
One hypothesis is that this dimension could be the perceived stiffness of the textures, with Metal Mesh and smooth textures appearing stiffer than the other textures, whose granularity could have been perceived as bumps on the surface that could deform under finger pressure.
%

2
2-perception/xr-perception/5-discussion.tex
View File

@@ -17,7 +17,7 @@ On average, participants responded faster (\percent{-16}), explored textures at
The \level{Mixed} rendering was always in between, with no significant difference from the other two.
This suggests that touching a virtual vibrotactile texture on a tangible surface with a virtual hand in \VR is different from touching it with one's own hand: users were more cautious or less confident in their exploration in \VR.
This does not seem to be due to the realism of the virtual hand or the environment, nor to the control of the virtual hand, all of which were rated high to very high by the participants (\secref{results_questions}) in both the \level{Mixed} and \level{Virtual} renderings.
Very interestingly, the evaluation of the vibrotactile device and the textures was also the same between the visual rendering, with a very high sense of control, a good realism and a very low perceived latency of the textures (\secref{results_questions}).
The evaluation of the vibrotactile device and the textures was also the same between the visual rendering, with a high sense of control, a good realism and a low perceived latency of the textures (\secref{results_questions}).
Conversely, the perceived latency of the virtual hand (\response{Hand Latency} question) seemed to be related to the perceived roughness of the textures (with the \PSEs).
The \level{Mixed} rendering had the lowest \PSE and highest perceived latency, the \level{Virtual} rendering had a higher \PSE and lower perceived latency, and the \level{Real} rendering had the highest \PSE and no virtual hand latency (as it was not displayed).

2
3-manipulation/visual-hand/2-method.tex
View File

@@ -146,7 +146,7 @@ Participants signed an informed consent, including the declaration of having no
Inspired by \textcite{laviolajr20173d}, we collected the following metrics during the experiment:
\begin{itemize}
\item \response{Completion Time}, defined as the time elapsed between the very first contact with the virtual cube and its correct placement inside the target volume; as subjects were asked to complete the tasks as fast as possible, lower completion times mean better performance.
\item \response{Completion Time}, defined as the time elapsed between the first contact with the virtual cube and its correct placement inside the target volume; as subjects were asked to complete the tasks as fast as possible, lower completion times mean better performance.
\item \response{Contacts}, defined as the number of separate times the user's hand makes contact with the virtual cube; in both tasks, a lower number of contacts means a smoother continuous interaction with the object.
\item \response{Time per Contact}, defined as the total time any part of the user's hand contacted the cube divided by the number of contacts; higher values mean that the user interacted with the object for longer non-interrupted periods of time.
\item \response{Grip Aperture} (solely for the grasp-and-place task), defined as the average distance between the thumb's fingertip and the other fingertips during the grasping of the cube; lower values indicate a greater finger interpenetration with the cube, resulting in a greater discrepancy between the real hand and the visual hand rendering constrained to the cube surfaces and showing how confident users are in their grasp \cite{prachyabrued2014visual, al-kalbani2016analysis, blaga2017usability, chessa2019grasping}.

2
3-manipulation/visual-hand/4-discussion.tex
View File

@@ -10,7 +10,7 @@ Indeed, participants found the \level{None} and \level{Occlusion} renderings les
To understand whether the participants' previous experience might have played a role, we also carried out an additional statistical analysis considering \VR experience as an additional between-subjects factor, \ie \VR novices vs. \VR experts (\enquote{I use it every week}, see \secref{participants}).
We found no statistically significant differences when comparing the considered metrics between \VR novices and experts.
Interestingly, all visual hand renderings showed \response{Grip Apertures} very close to the size of the virtual cube, except for the \level{None} rendering (\figref{results/Grasp-GripAperture-Hand-Overall-Means}), with which participants applied stronger grasps, \ie less distance between the fingertips.
All visual hand renderings showed \response{Grip Apertures} close to the size of the virtual cube, except for the \level{None} rendering (\figref{results/Grasp-GripAperture-Hand-Overall-Means}), with which participants applied stronger grasps, \ie less distance between the fingertips.
Having no visual hand rendering, but only the reaction of the cube to the interaction as feedback, made participants less confident in their grip.
This result contrasts with the wrongly estimated grip apertures observed by \textcite{al-kalbani2016analysis} in an exocentric VST-AR setup.
Also, while some participants found the absence of visual hand rendering more natural, many of them commented on the importance of having feedback on the tracking of their hands, as observed by \textcite{xiao2018mrtouch} in a similar immersive OST-AR setup.

6
3-manipulation/visuo-haptic-hand/2-method.tex
View File

@@ -83,14 +83,14 @@ This design led to a total of 5 vibrotactile positionings \x 2 vibration contact
\subsection{Apparatus and Procedure}
\label{apparatus}
Apparatus and experimental procedure were very similar to the \chapref{visual_hand}, as described in \secref[visual_hand]{apparatus} and \secref[visual_hand]{protocol}, respectively.
Apparatus and experimental procedure were similar to the \chapref{visual_hand}, as described in \secref[visual_hand]{apparatus} and \secref[visual_hand]{protocol}, respectively.
We report here only the differences.
We employed the same vibrotactile device used by \cite{devigne2020power}.
It is composed of two encapsulated \ERM (\secref[related_work]{vibrotactile_actuators}) vibration motors (Pico-Vibe 304-116, Precision Microdrive, UK).
They are small and very light (\qty{5}{\mm} \x \qty{20}{\mm}, \qty{1.2}{\g}) actuators capable of vibration frequencies from \qtyrange{120}{285}{\Hz} and
They are small and light (\qty{5}{\mm} \x \qty{20}{\mm}, \qty{1.2}{\g}) actuators capable of vibration frequencies from \qtyrange{120}{285}{\Hz} and
amplitudes from \qtyrange{0.2}{1.15}{\g}.
They have a latency of \qty{20}{\ms} that we partially compensated for at the software level with slightly larger colliders to trigger the vibrations very close the moment the finger touched the cube.
They have a latency of \qty{20}{\ms} that we partially compensated for at the software level with slightly larger colliders to trigger the vibrations close the moment the finger touched the cube.
These two outputs vary linearly together, based on the tension applied.
They were controlled by an Arduino Pro Mini (\qty{3.3}{\V}) and a custom board that delivered the tension independently to each motor.
A small \qty{400}{mAh} Li-ion battery allowed for 4 hours of constant vibration at maximum intensity.

4
3-manipulation/visuo-haptic-hand/5-conclusion.tex
View File

@@ -9,8 +9,8 @@ In a user study, we compared sixteen visuo-haptic renderings of the hand as the
Results showed that delocalized vibrotactile haptic hand rendering improved the perceived effectiveness, realism, and usefulness when it is provided close to the contact point.
%However, the farthest positioning on the contralateral hand gave the best performance even though it was disliked: the unfamiliarity of the positioning probably caused the participants to take more effort to consider the haptic stimuli and to focus more on the task.
The visual hand rendering was perceived less necessary than the vibrotactile haptic hand rendering, but still provided a useful feedback on the hand tracking.
This study provide evidence that moving away the feedback from the inside of the hand is a simple but very promising approach for wearable haptics in \AR.
This study provide evidence that moving away the feedback from the inside of the hand is a simple but promising approach for wearable haptics in \AR.
If integration with the hand tracking system allows it, and if the task requires it, a haptic ring worn on the middle or proximal phalanx seems preferable.
However, a wrist-mounted haptic device will be able to provide richer feedback by embedding more diverse haptic actuators with larger bandwidths and maximum amplitudes, while being less obtrusive than a ring.
Finally, we think that the visual hand rendering complements the haptic hand rendering very well by providing continuous feedback on the hand tracking, and that it can be disabled during the grasping phase to avoid redundancy with the haptic feedback of the contact with the \VO.
Finally, we think that the visual hand rendering complements the haptic hand rendering well by providing continuous feedback on the hand tracking, and that it can be disabled during the grasping phase to avoid redundancy with the haptic feedback of the contact with the \VO.

2
4-conclusion/conclusion.tex
View File

@@ -82,7 +82,7 @@ More generally, many other haptic feedbacks could be investigated in \AR \vs \VR
The visual hand renderings we evaluated were displayed on the Microsoft HoloLens~2, which is a common \OST-\AR headset \cite{hertel2021taxonomy}.
We purposely chose this type of display as it is with \OST-\AR that the lack of mutual occlusion between the hand and the \VO is the most challenging to solve \cite{macedo2023occlusion}.
We thus hypothesized that a visual hand rendering would be more beneficial to users with this type of display.
However, the user's visual perception and experience is very different with other types of displays, such as \VST-\AR, where the \RE view is seen through a screen (\secref[related_work]{ar_displays}).
However, the user's visual perception and experience is different with other types of displays, such as \VST-\AR, where the \RE view is seen through a screen (\secref[related_work]{ar_displays}).
While the mutual occlusion problem and the hand tracking latency can be overcome with \VST-\AR, the visual hand rendering could still be beneficial to users as it provides depth cues and feedback on the hand tracking, and should be evaluated as such.
\paragraph{More Ecological Conditions}

8
config/template.tex
View File

@@ -21,7 +21,7 @@
\usepackage[english]{babel} % Typographical rules
\usepackage{microtype} % Micro-typography improvements (slightly more compact, better to read)
% Use sans-serif font for sections
% Fonts: Use sans-serif for sections
\usepackage{titlesec}
\titleformat{\part}[display]{\normalfont\Huge\sffamily\bfseries\centering}{\partname\ \thepart}{20pt}{}
\titleformat{\chapter}[display]{\normalfont\Huge\sffamily\bfseries}{\chaptertitlename\ \thechapter}{10pt}{}
@@ -30,9 +30,7 @@
\titleformat{\subsubsection}{\normalfont\normalsize\sffamily\bfseries}{\thesubsubsection}{1em}{}
\titleformat{\paragraph}[runin]{\normalfont\normalsize\sffamily\bfseries}{\theparagraph}{1em}{}
\setcounter{secnumdepth}{3} % Number subsubsections
% Parts and chapters
% Parts, chapters and sections
\NewCommandCopy{\oldchapter}{\chapter}
\RenewDocumentCommand{\chapter}{s o m}{% #1 = star (no number), #2 = short title, #3 = title
\IfBooleanTF{#1}{%
@@ -57,6 +55,8 @@
\includefrom{#1}{#2}% Include with relative paths \input in the chapter
}
\setcounter{secnumdepth}{3} % Number subsubsections
% Headers
\usepackage{emptypage} % Remove headers and footers on empty pages
\usepackage{etoolbox} % Patching commands