From 202455e9e846e8abe686041b9f6a202be8c11378 Mon Sep 17 00:00:00 2001 From: Erwan Normand Date: Thu, 26 Dec 2024 19:27:47 +0100 Subject: [PATCH] Improve results report in vhar-textures --- 3-perception/vhar-textures/2-experiment.tex | 2 +- 3-perception/vhar-textures/3-results.tex | 20 +++++++++++--------- 2 files changed, 12 insertions(+), 10 deletions(-) diff --git a/3-perception/vhar-textures/2-experiment.tex b/3-perception/vhar-textures/2-experiment.tex index 1d62f8e..ef098e5 100644 --- a/3-perception/vhar-textures/2-experiment.tex +++ b/3-perception/vhar-textures/2-experiment.tex @@ -95,7 +95,7 @@ Each modality level was ranked once per participant following the fixed order li \subsection{Participants} \label{participants} -Twenty participants took part in the user study (12 males, 7 females, 1 preferred not to say), aged between 20 and 60 years (\mean{29.1}, \sd{9.4}). +Twenty participants took part in the user study (12 males, 7 females, 1 preferred not to say), aged between 20 and 60 years (\median{26.5}, \iqr{9}{}). One participant was left-handed, all others were right-handed; they all performed the user study with their dominant hand. All participants had normal or corrected-to-normal vision and none of them had a known hand or finger impairment. They rated their experience with haptics, \AR, and \VR (\enquote{I use it every month or more}); 10 were experienced with haptics, 2 with \AR, and 10 with \VR. diff --git a/3-perception/vhar-textures/3-results.tex b/3-perception/vhar-textures/3-results.tex index fa4282b..09b22d9 100644 --- a/3-perception/vhar-textures/3-results.tex +++ b/3-perception/vhar-textures/3-results.tex @@ -38,32 +38,33 @@ To verify that the difficulty with all the visual textures was the same on the \ As the \response{Completion Time} results were Gamma distributed, they were transformed with a log to approximate a normal distribution. A \LMM on the log \response{Completion Time} with the \factor{Visual Texture} as fixed effect and the participant as random intercept was performed. Normality was verified with a QQ-plot of the model residuals. -No statistical significant effect of \factor{Visual Texture} was found (\anova{8}{512}{1.9}, \p{0.06}) on \response{Completion Time} (\geomean{44}{\s}, \ci{42}{46}), indicating an equal difficulty and participant behaviour for all the visual textures. +No statistical significant effect of \factor{Visual Texture} was found (\anova{8}{512}{1.9}, \p{0.06}) on \response{Completion Time} (\geomean{44}{\s} \ci{42}{46}), indicating an equal difficulty and participant behaviour for all the visual textures. \subsection{Textures Ranking} \label{results_ranking} \figref{results/ranking_mean_ci} presents the results of the three rankings of the haptic textures alone, the visual textures alone, and the visuo-haptic texture pairs. +For each ranking, a Friedman test was performed with post-hoc Wilcoxon signed-rank tests and Holm-Bonferroni adjustment. \paragraph{Haptic Textures Ranking} Almost all the texture pairs in the haptic textures ranking results were statistically significantly different (\chisqr{8}{20}{146}, \pinf{0.001}; \pinf{0.05} for each comparison), except between (\level{Metal Mesh}, \level{Sandpaper~100}), (\level{Cork}, \level{Brick~2}), (\level{Cork}, \level{Sandpaper~320}) (\level{Plastic Mesh~1}, \level{Velcro Hooks}), and (\level{Plastic Mesh~1}, \level{Terra Cotta}). -Average Kendall's Tau correlations between the participants indicated a high consensus (\kendall{0.82}, \ci{0.81}{0.84}) showing that participants perceived similarly the roughness of the haptic textures. +Average Kendall's Tau correlations between the participants indicated a high consensus (\kendall{0.82} \ci{0.81}{0.84}) showing that participants perceived similarly the roughness of the haptic textures. \paragraph{Visual Textures Ranking} Most of the texture pairs in the visual textures ranking results were also statistically significantly different (\chisqr{8}{20}{119}, \pinf{0.001}; \pinf{0.05} for each comparison), except for the following groups: \{\level{Metal Mesh}, \level{Cork}, \level{Plastic Mesh~1}\}; \{\level{Sandpaper~100}, \level{Brick~2}, \level{Plastic Mesh~1}, \level{Velcro Hooks}\}; \{\level{Cork}, \level{Velcro Hooks}\}; \{\level{Sandpaper~320}, \level{Terra Cotta}\}; and \{\level{Sandpaper~320}, \level{Coffee Filter}\}. -Even though the consensus was high (\kendall{0.61}, \ci{0.58}{0.64}), the roughness of the visual textures were more difficult to estimate, in particular for \level{Plastic Mesh~1} and \level{Velcro Hooks}. +Even though the consensus was high (\kendall{0.61} \ci{0.58}{0.64}), the roughness of the visual textures were more difficult to estimate, in particular for \level{Plastic Mesh~1} and \level{Velcro Hooks}. \paragraph{Visuo-Haptic Textures Ranking} Also, almost all the texture pairs in the visuo-haptic textures ranking results were statistically significantly different (\chisqr{8}{20}{140}, \pinf{0.001}; \pinf{0.05} for each comparison), except for the following groups: \{\level{Sandpaper~100}, \level{Cork}\}; \{\level{Cork}, \level{Brick~2}\}; and \{\level{Plastic Mesh~1}, \level{Velcro Hooks}, \level{Sandpaper~320}\}. -The consensus between the participants was also high \kendall{0.77}, \ci{0.74}{0.79}. -Finally, calculating the similarity of the three rankings of each participant, the \textit{Visuo-Haptic Textures Ranking} was on average highly similar to the \textit{Haptic Textures Ranking} (\kendall{0.79}, \ci{0.72}{0.86}) and moderately to the \textit{Visual Textures Ranking} (\kendall{0.48}, \ci{0.39}{0.56}). +The consensus between the participants was also high \kendall{0.77} \ci{0.74}{0.79}. +Finally, calculating the similarity of the three rankings of each participant, the \textit{Visuo-Haptic Textures Ranking} was on average highly similar to the \textit{Haptic Textures Ranking} (\kendall{0.79} \ci{0.72}{0.86}) and moderately to the \textit{Visual Textures Ranking} (\kendall{0.48} \ci{0.39}{0.56}). A Wilcoxon signed-rank test indicated that this difference was statistically significant (\wilcoxon{190}, \p{0.002}). These results indicate that the two haptic and visual modalities were integrated together, the resulting roughness ranking being between the two rankings of the modalities alone, but with haptics predominating. -\fig[0.6]{results/ranking_mean_ci}{Means with bootstrap \percent{95} \CI of the three rankings of the haptic textures alone, the visual textures alone, and the visuo-haptic texture pairs. }[ +\fig[0.7]{results/ranking_mean_ci}{Means with bootstrap \percent{95} \CI of the three rankings of the haptic textures alone, the visual textures alone, and the visuo-haptic texture pairs. }[ A lower rank means that the texture was considered rougher, a higher rank means smoother. ] @@ -137,13 +138,14 @@ This shows that the participants consistently identified the roughness of each v \figref{results_questions} presents the questionnaire results of the \level{Matching} and \level{Ranking} tasks. A non-parametric \ANOVA on \ART models were used for the \response{Difficulty} and \response{Realism} question results. The other question results were analyzed using Wilcoxon signed-rank tests, with Holm-Bonferroni adjustment. +The results are shown as mean $\pm$ standard deviation. On \response{Difficulty}, there were statistically significant effects of \factor{Task} (\anova{1}{57}{13}, \pinf{0.001}) and of \factor{Modality} (\anova{1}{57}{8}, \p{0.007}), but no interaction effect. % \factor{Task} \x \factor{Modality} (\anova{1}{57}{2}, \ns). -The \level{Ranking} task was found easier (\mean{2.9}, \sd{1.2}) than the \level{Matching} task (\mean{3.9}, \sd{1.5}), and the Haptic textures were found easier to discriminate (\mean{3.0}, \sd{1.3}) than the Visual ones (\mean{3.8}, \sd{1.5}). +The \level{Ranking} task was found easier (\num{2.9 \pm 1.2}) than the \level{Matching} task (\num{3.9 \pm 1.5}), and the Haptic textures were found easier to discriminate (\num{3.0 \pm 1.3}) than the Visual ones (\num{3.8 \pm 1.5}). -Both haptic and visual textures were judged moderately realistic for both tasks (\mean{4.2}, \sd{1.3}), with no statistically significant effect of \factor{Task}, \factor{Modality} or their interaction on \response{Realism}. +Both haptic and visual textures were judged moderately realistic for both tasks (\num{4.2 \pm 1.3}), with no statistically significant effect of \factor{Task}, \factor{Modality} or their interaction on \response{Realism}. No statistically significant effects of \factor{Task} on \response{Textures Match} and \response{Uncomfort} were found either. -The coherence of the texture pairs was considered moderate (\mean{4.6}, \sd{1.2}) and the haptic device was not felt uncomfortable (\mean{2.4}, \sd{1.4}). +The coherence of the texture pairs was considered moderate (\num{4.6 \pm 1.2}) and the haptic device was not felt uncomfortable (\num{2.4 \pm 1.4}). \begin{subfigs}{results_questions}{Boxplots of the questionnaire results for each visual hand rendering.}[ Pairwise Wilcoxon signed-rank tests with Holm-Bonferroni adjustment: * is \pinf{0.05}, ** is \pinf{0.01} and *** is \pinf{0.001}.