Learning to Adapt: Test-Time Personalized Gaze Estimation with Weak and Self-Supervised Learning for In-Vehicle scenarios｜Vikrant Nagpure｜Research Insights

著　者: Vikrant Nagpure, Tanisha Jain, Rongali Sai Bhargav, Ashwin K Krishna，Kenji Okuma

要　旨: Abstract—Personalized gaze estimation in in-vehicle settings is crucial for driver monitoring but remains challenging due to individual anatomical differences and environmental variability.Existing approaches often rely on fully supervised fine-tuning,which requires labeled calibration data and limits practical deployment. To overcome this limitation, we propose a Test-Time Personalized Gaze Estimation framework that enables in-vehicle gaze personalization with minimal supervision. Our method builds upon GazeDPTR V2, which extends gaze estimation to gaze zone classification by leveraging positional features from point projection and visual attributes from images. We introduce a two-stage adaptation strategy: (1) Self-Supervised Test-Time Adaptation, where augmentation-based self-supervision is applied to adapt the model to freely available unlabeled test-time data, facilitated by Model-Agnostic Meta-Learning (MAML) for effective unsupervised adaptation; (2)
Weakly-Supervised Personalization, where a small number of gaze zone samples collected from the driver during an initial setup phase are used for fine-tuning. We evaluate our approach on the IVGaze dataset, which provides both gaze direction vectors and gaze zone labels. Experimental results show that our method significantly outperforms baseline models, demonstrating the effectiveness of self-supervised meta-learning and weak supervision for real-world in-vehicle gaze estimation.

Vikrant Nagpure

収　録: IEEE/CVF Winter Conference on Applications of Computer Vision (WACV 2023)

要　旨: For aiming at a more accurate appearance-based gaze estimation, a series of recent works propose to use transformers or high-resolution networks in several ways which achieve state-of-the-art, but such works lack efficiency for real-time applications on edge computing devices. In this paper, we propose a compact model to precisely and efficiently solve gaze estimation. The proposed model includes 1) a Neural Architecture Search(NAS)-based multiresolution feature extractor for extracting feature maps with global and local information which are essential for this task and 2) a novel multi-resolution fusion transformer as the gaze estimation head for efficiently estimating gaze values by fusing the extracted feature maps. We search our proposed model, called GazeNAS-ETH, on the ETH-Xgaze dataset. We confirmed through experiments that GazeNASETH achieved state-of-the-art on Gaze360, MPIIFaceGaze,RTGENE, and EYEDIAP datasets, while having only about 1M parameters and using only 0.28 GFLOPs, which is significantly less compared to previous state-of-the-art models,making it easier to deploy for real-time applications.

Learning to Adapt: Test-Time Personalized Gaze Estimation with Weak and Self-Supervised Learning for In-Vehicle scenarios