erwan/phd-thesis
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Improve results report in vhar-textures

Browse Source
This commit is contained in:
Erwan Normand
2024-12-26 19:27:47 +01:00
parent f75de4439e
commit 202455e9e8
2 changed files with 12 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
3-perception/vhar-textures/2-experiment.tex
View File

@@ -95,7 +95,7 @@ Each modality level was ranked once per participant following the fixed order li
\subsection{Participants} \subsection{Participants}
\label{participants} \label{participants}
Twenty participants took part in the user study (12 males, 7 females, 1 preferred not to say), aged between 20 and 60 years (\mean{29.1}, \sd{9.4}). Twenty participants took part in the user study (12 males, 7 females, 1 preferred not to say), aged between 20 and 60 years (\median{26.5}, \iqr{9}{}).
One participant was left-handed, all others were right-handed; they all performed the user study with their dominant hand. One participant was left-handed, all others were right-handed; they all performed the user study with their dominant hand.
All participants had normal or corrected-to-normal vision and none of them had a known hand or finger impairment. All participants had normal or corrected-to-normal vision and none of them had a known hand or finger impairment.
They rated their experience with haptics, \AR, and \VR (\enquote{I use it every month or more}); 10 were experienced with haptics, 2 with \AR, and 10 with \VR. They rated their experience with haptics, \AR, and \VR (\enquote{I use it every month or more}); 10 were experienced with haptics, 2 with \AR, and 10 with \VR.

20
3-perception/vhar-textures/3-results.tex
View File

@@ -38,32 +38,33 @@ To verify that the difficulty with all the visual textures was the same on the \
As the \response{Completion Time} results were Gamma distributed, they were transformed with a log to approximate a normal distribution. As the \response{Completion Time} results were Gamma distributed, they were transformed with a log to approximate a normal distribution.
A \LMM on the log \response{Completion Time} with the \factor{Visual Texture} as fixed effect and the participant as random intercept was performed. A \LMM on the log \response{Completion Time} with the \factor{Visual Texture} as fixed effect and the participant as random intercept was performed.
Normality was verified with a QQ-plot of the model residuals. Normality was verified with a QQ-plot of the model residuals.
No statistical significant effect of \factor{Visual Texture} was found (\anova{8}{512}{1.9}, \p{0.06}) on \response{Completion Time} (\geomean{44}{\s}, \ci{42}{46}), indicating an equal difficulty and participant behaviour for all the visual textures. No statistical significant effect of \factor{Visual Texture} was found (\anova{8}{512}{1.9}, \p{0.06}) on \response{Completion Time} (\geomean{44}{\s} \ci{42}{46}), indicating an equal difficulty and participant behaviour for all the visual textures.
\subsection{Textures Ranking} \subsection{Textures Ranking}
\label{results_ranking} \label{results_ranking}
\figref{results/ranking_mean_ci} presents the results of the three rankings of the haptic textures alone, the visual textures alone, and the visuo-haptic texture pairs. \figref{results/ranking_mean_ci} presents the results of the three rankings of the haptic textures alone, the visual textures alone, and the visuo-haptic texture pairs.
For each ranking, a Friedman test was performed with post-hoc Wilcoxon signed-rank tests and Holm-Bonferroni adjustment.
\paragraph{Haptic Textures Ranking} \paragraph{Haptic Textures Ranking}
Almost all the texture pairs in the haptic textures ranking results were statistically significantly different (\chisqr{8}{20}{146}, \pinf{0.001}; \pinf{0.05} for each comparison), except between (\level{Metal Mesh}, \level{Sandpaper~100}), (\level{Cork}, \level{Brick~2}), (\level{Cork}, \level{Sandpaper~320}) (\level{Plastic Mesh~1}, \level{Velcro Hooks}), and (\level{Plastic Mesh~1}, \level{Terra Cotta}). Almost all the texture pairs in the haptic textures ranking results were statistically significantly different (\chisqr{8}{20}{146}, \pinf{0.001}; \pinf{0.05} for each comparison), except between (\level{Metal Mesh}, \level{Sandpaper~100}), (\level{Cork}, \level{Brick~2}), (\level{Cork}, \level{Sandpaper~320}) (\level{Plastic Mesh~1}, \level{Velcro Hooks}), and (\level{Plastic Mesh~1}, \level{Terra Cotta}).
Average Kendall's Tau correlations between the participants indicated a high consensus (\kendall{0.82}, \ci{0.81}{0.84}) showing that participants perceived similarly the roughness of the haptic textures. Average Kendall's Tau correlations between the participants indicated a high consensus (\kendall{0.82} \ci{0.81}{0.84}) showing that participants perceived similarly the roughness of the haptic textures.
\paragraph{Visual Textures Ranking} \paragraph{Visual Textures Ranking}
Most of the texture pairs in the visual textures ranking results were also statistically significantly different (\chisqr{8}{20}{119}, \pinf{0.001}; \pinf{0.05} for each comparison), except for the following groups: \{\level{Metal Mesh}, \level{Cork}, \level{Plastic Mesh~1}\}; \{\level{Sandpaper~100}, \level{Brick~2}, \level{Plastic Mesh~1}, \level{Velcro Hooks}\}; \{\level{Cork}, \level{Velcro Hooks}\}; \{\level{Sandpaper~320}, \level{Terra Cotta}\}; and \{\level{Sandpaper~320}, \level{Coffee Filter}\}. Most of the texture pairs in the visual textures ranking results were also statistically significantly different (\chisqr{8}{20}{119}, \pinf{0.001}; \pinf{0.05} for each comparison), except for the following groups: \{\level{Metal Mesh}, \level{Cork}, \level{Plastic Mesh~1}\}; \{\level{Sandpaper~100}, \level{Brick~2}, \level{Plastic Mesh~1}, \level{Velcro Hooks}\}; \{\level{Cork}, \level{Velcro Hooks}\}; \{\level{Sandpaper~320}, \level{Terra Cotta}\}; and \{\level{Sandpaper~320}, \level{Coffee Filter}\}.
Even though the consensus was high (\kendall{0.61}, \ci{0.58}{0.64}), the roughness of the visual textures were more difficult to estimate, in particular for \level{Plastic Mesh~1} and \level{Velcro Hooks}. Even though the consensus was high (\kendall{0.61} \ci{0.58}{0.64}), the roughness of the visual textures were more difficult to estimate, in particular for \level{Plastic Mesh~1} and \level{Velcro Hooks}.
\paragraph{Visuo-Haptic Textures Ranking} \paragraph{Visuo-Haptic Textures Ranking}
Also, almost all the texture pairs in the visuo-haptic textures ranking results were statistically significantly different (\chisqr{8}{20}{140}, \pinf{0.001}; \pinf{0.05} for each comparison), except for the following groups: \{\level{Sandpaper~100}, \level{Cork}\}; \{\level{Cork}, \level{Brick~2}\}; and \{\level{Plastic Mesh~1}, \level{Velcro Hooks}, \level{Sandpaper~320}\}. Also, almost all the texture pairs in the visuo-haptic textures ranking results were statistically significantly different (\chisqr{8}{20}{140}, \pinf{0.001}; \pinf{0.05} for each comparison), except for the following groups: \{\level{Sandpaper~100}, \level{Cork}\}; \{\level{Cork}, \level{Brick~2}\}; and \{\level{Plastic Mesh~1}, \level{Velcro Hooks}, \level{Sandpaper~320}\}.
The consensus between the participants was also high \kendall{0.77}, \ci{0.74}{0.79}. The consensus between the participants was also high \kendall{0.77} \ci{0.74}{0.79}.
Finally, calculating the similarity of the three rankings of each participant, the \textit{Visuo-Haptic Textures Ranking} was on average highly similar to the \textit{Haptic Textures Ranking} (\kendall{0.79}, \ci{0.72}{0.86}) and moderately to the \textit{Visual Textures Ranking} (\kendall{0.48}, \ci{0.39}{0.56}). Finally, calculating the similarity of the three rankings of each participant, the \textit{Visuo-Haptic Textures Ranking} was on average highly similar to the \textit{Haptic Textures Ranking} (\kendall{0.79} \ci{0.72}{0.86}) and moderately to the \textit{Visual Textures Ranking} (\kendall{0.48} \ci{0.39}{0.56}).
A Wilcoxon signed-rank test indicated that this difference was statistically significant (\wilcoxon{190}, \p{0.002}). A Wilcoxon signed-rank test indicated that this difference was statistically significant (\wilcoxon{190}, \p{0.002}).
These results indicate that the two haptic and visual modalities were integrated together, the resulting roughness ranking being between the two rankings of the modalities alone, but with haptics predominating. These results indicate that the two haptic and visual modalities were integrated together, the resulting roughness ranking being between the two rankings of the modalities alone, but with haptics predominating.
\fig[0.6]{results/ranking_mean_ci}{Means with bootstrap \percent{95} \CI of the three rankings of the haptic textures alone, the visual textures alone, and the visuo-haptic texture pairs. }[ \fig[0.7]{results/ranking_mean_ci}{Means with bootstrap \percent{95} \CI of the three rankings of the haptic textures alone, the visual textures alone, and the visuo-haptic texture pairs. }[
A lower rank means that the texture was considered rougher, a higher rank means smoother. A lower rank means that the texture was considered rougher, a higher rank means smoother.
] ]
@@ -137,13 +138,14 @@ This shows that the participants consistently identified the roughness of each v
\figref{results_questions} presents the questionnaire results of the \level{Matching} and \level{Ranking} tasks. \figref{results_questions} presents the questionnaire results of the \level{Matching} and \level{Ranking} tasks.
A non-parametric \ANOVA on \ART models were used for the \response{Difficulty} and \response{Realism} question results. A non-parametric \ANOVA on \ART models were used for the \response{Difficulty} and \response{Realism} question results.
The other question results were analyzed using Wilcoxon signed-rank tests, with Holm-Bonferroni adjustment. The other question results were analyzed using Wilcoxon signed-rank tests, with Holm-Bonferroni adjustment.
The results are shown as mean $\pm$ standard deviation.
On \response{Difficulty}, there were statistically significant effects of \factor{Task} (\anova{1}{57}{13}, \pinf{0.001}) and of \factor{Modality} (\anova{1}{57}{8}, \p{0.007}), but no interaction effect. % \factor{Task} \x \factor{Modality} (\anova{1}{57}{2}, \ns). On \response{Difficulty}, there were statistically significant effects of \factor{Task} (\anova{1}{57}{13}, \pinf{0.001}) and of \factor{Modality} (\anova{1}{57}{8}, \p{0.007}), but no interaction effect. % \factor{Task} \x \factor{Modality} (\anova{1}{57}{2}, \ns).
The \level{Ranking} task was found easier (\mean{2.9}, \sd{1.2}) than the \level{Matching} task (\mean{3.9}, \sd{1.5}), and the Haptic textures were found easier to discriminate (\mean{3.0}, \sd{1.3}) than the Visual ones (\mean{3.8}, \sd{1.5}). The \level{Ranking} task was found easier (\num{2.9 \pm 1.2}) than the \level{Matching} task (\num{3.9 \pm 1.5}), and the Haptic textures were found easier to discriminate (\num{3.0 \pm 1.3}) than the Visual ones (\num{3.8 \pm 1.5}).
Both haptic and visual textures were judged moderately realistic for both tasks (\mean{4.2}, \sd{1.3}), with no statistically significant effect of \factor{Task}, \factor{Modality} or their interaction on \response{Realism}. Both haptic and visual textures were judged moderately realistic for both tasks (\num{4.2 \pm 1.3}), with no statistically significant effect of \factor{Task}, \factor{Modality} or their interaction on \response{Realism}.
No statistically significant effects of \factor{Task} on \response{Textures Match} and \response{Uncomfort} were found either. No statistically significant effects of \factor{Task} on \response{Textures Match} and \response{Uncomfort} were found either.
The coherence of the texture pairs was considered moderate (\mean{4.6}, \sd{1.2}) and the haptic device was not felt uncomfortable (\mean{2.4}, \sd{1.4}). The coherence of the texture pairs was considered moderate (\num{4.6 \pm 1.2}) and the haptic device was not felt uncomfortable (\num{2.4 \pm 1.4}).
\begin{subfigs}{results_questions}{Boxplots of the questionnaire results for each visual hand rendering.}[ \begin{subfigs}{results_questions}{Boxplots of the questionnaire results for each visual hand rendering.}[
Pairwise Wilcoxon signed-rank tests with Holm-Bonferroni adjustment: * is \pinf{0.05}, ** is \pinf{0.01} and *** is \pinf{0.001}. Pairwise Wilcoxon signed-rank tests with Holm-Bonferroni adjustment: * is \pinf{0.05}, ** is \pinf{0.01} and *** is \pinf{0.001}.