erwan/phd-thesis
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Complete comments

Browse Source
This commit is contained in:
Erwan Normand
2025-04-18 11:21:19 +02:00
parent 3de6ad37df
commit 0a0e1ff4b5
11 changed files with 35 additions and 38 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
3-perception/vhar-system/1-introduction.tex
View File

@@ -2,7 +2,7 @@
\label{intro}
One approach to render virtual haptic textures consists in simulating the roughness of a periodic grating surface as a vibrotactile sinusoidal (\secref[related_work]{texture_rendering}).
The vibrations are rendered to a voice-coil actuator embedded in a hand-held tool or worn on the finger.
The vibrations are rendered to a voice-coil actuator embedded in a hand-held tool or worn on the finger (\secref[related_work]{vhar_haptics}).
To create the illusion of touching a pattern with a fixed spatial period, the frequency of signal must be modulated according to the finger movement.
Previous work either used mechanical system to track the movement at high frequency \cite{strohmeier2017generating,friesen2024perceived}, or required the user to move at a constant speed to keep the signal frequency constant \cite{asano2015vibrotactile,ujitoko2019modulating}.
However, this method has not yet been integrated in an \AR headset context, where the user should be able to freely touch and explore the visuo-haptic texture augmentations.
@@ -10,10 +10,12 @@ However, this method has not yet been integrated in an \AR headset context, wher
%which either constrained hand to a constant speed to keep the signal frequency constant \cite{asano2015vibrotactile,friesen2024perceived}, or used mechanical sensors attached to the hand \cite{friesen2024perceived,strohmeier2017generating}
In this chapter, we propose a \textbf{system for rendering visual and haptic virtual textures that augment real surfaces}.
It is implemented with the \OST-\AR headset Microsoft HoloLens~2 and a wearable vibrotactile (voice-coil) device worn on the outside of finger (not covering the fingertip, \secref[related_work]{vhar_haptics}).
The visuo-haptic augmentations can be \textbf{viewed from any angle} and \textbf{explored freely with the bare finger}, as if they were real textures.
It is implemented with the \OST-\AR headset Microsoft HoloLens~2 and a wearable vibrotactile (voice-coil) device worn on the outside of finger (not covering the fingertip).
The visuo-haptic augmentations rendered with this design allow a user to \textbf{see the textures from any angle} and \textbf{explore them freely with the bare finger}, as if they were real textures.
To ensure both real-time and reliable renderings, the hand and the real surfaces are tracked using a webcam and marker-based pose estimation.
The haptic textures are rendered as a vibrotactile signal representing a patterned grating texture that is synchronized with the finger movement on the augmented surface.
The goal of this design is to enable new \AR applications capable of augmenting real objects with virtual visuo-haptic textures in a portable, on-demand manner, and without impairing with the interaction of the user with the \RE.
\comans{SJ}{The rationale behind the proposed design is not provided. Since there are multiple ways to implement mechanically transparent haptic devices, the thesis should at least clarify why this design is considered optimal for a specific purpose at this stage.}{This has been better explained in the introduction.}
\noindentskip The contributions of this chapter are:
\begin{itemize}

6
3-perception/vhar-system/2-method.tex
View File

@@ -1,8 +1,6 @@
\section{Concept}
\label{principle}
\comans{SJ}{The rationale behind the proposed design is not provided. Since there are multiple ways to implement mechanically transparent haptic devices, the thesis should at least clarify why this design is considered optimal for a specific purpose at this stage.}{TODO}
The visuo-haptic texture rendering system is based on:
\begin{enumerate}[label=(\arabic*)]
\item a real-time interaction loop between the finger movements and a coherent visuo-haptic feedback simulating the sensation of a touched texture,
@@ -49,7 +47,6 @@ Finally, this filtered finger velocity is transformed into the augmented surface
\subsection{Virtual Environment Registration}
\label{virtual_real_registration}
\comans{JG}{The registration process between the external camera, the finger, surface and HoloLens could have been described in more detail. Specifically, it could have been described clearer how the HoloLens coordinate system was aligned (e.g., by also tracking the fiducials on the surface and or finger).}{This has been better described.}
Before a user interacts with the system, it is necessary to design a \VE that will be registered with the \RE during the experiment.
Each real element tracked by a marker is modelled virtually, \eg the hand and the augmented surface (\figref{device}).
In addition, the pose and size of the virtual textures were defined on the virtual replicas.
@@ -58,12 +55,13 @@ First, the coordinate system of the headset is manually aligned with that of the
This resulted in a \qty{\pm .5}{\cm} spatial alignment error between the \RE and the \VE.
While this was sufficient for our use cases, other methods can achieve better accuracy if needed \cite{grubert2018survey}.
The registration of the coordinate systems of the camera and the headset thus allows the use of the marker estimation poses performed with the camera to display in the headset the virtual models aligned with their real-world counterparts.
\comans{JG}{The registration process between the external camera, the finger, surface and HoloLens could have been described in more detail. Specifically, it could have been described clearer how the HoloLens coordinate system was aligned (e.g., by also tracking the fiducials on the surface and or finger).}{This has been better described.}
\comans{JG}{A description if and how the offset between the lower side of the fingertip touching the surface and the fiducial mounted on the top of the finger was calibrated / compensated is missing}{This has been better described.}
An additional calibration is performed to compensate for the offset between the finger contact point and the estimated marker pose \cite{son2022effect}.
The current user then places the index finger on the origin point, whose respective poses are known from the attached fiducial markers.
The transformation between the marker pose of the finger and the finger contact point can be estimated and compensated with an inverse transformation.
This allows to detect if the calibrated real finger touches a virtual texture using a collision detection algorithm (Nvidia PhysX).
\comans{JG}{A description if and how the offset between the lower side of the fingertip touching the surface and the fiducial mounted on the top of the finger was calibrated / compensated is missing}{This has been better described.}
In our implementation, the \VE is designed with Unity (v2021.1) and the Mixed Reality Toolkit (v2.7)\footnoteurl{https://learn.microsoft.com/windows/mixed-reality/mrtk-unity}.
The visual rendering is achieved using the Microsoft HoloLens~2, an \OST-\AR headset with a \qtyproduct{43 x 29}{\degree} \FoV, a \qty{60}{\Hz} refresh rate, and self-localisation capabilities.