Open Access
Gaze estimation based on machine learning
(2024) Larumbe Bergera, Andoni; Villanueva Larre, Arantxa; Ingeniería Eléctrica, Electrónica y de Comunicación; Ingeniaritza Elektrikoa, Elektronikoaren eta Telekomunikazio Ingeniaritzaren; Universidad Pública de Navarra / Nafarroako Unibertsitate Publikoa
Esta tesis, desarrollada en el marco del grupo GI4E, se centra en el desarrollo de un algoritmo de estimación de la mirada para sistemas de videooculografía (Video-oculography, VOG) que hacen uso de componentes genéricos. En primer lugar, se resaltan las limitaciones de la tecnología de seguimiento ocular existente, así como los métodos comúnmente utilizados para la estimación de la mirada en sistemas que emplean este tipo de componentes. La contribución central de la tesis es el desarrollo de un algoritmo de estimación de la mirada dividido en dos grandes bloques: un primer bloque para la detección de puntos de referencia faciales y un segundo bloque que, a partir de un vector de características generado usando esos puntos de referencia, estima el punto de la mirada (Point of Gaze, PoG, en inglés). Debido a los grandes avances en el campo del aprendizaje automático (machine learning), se ha decidido emplear este tipo de técnicas para ambos bloques. Se realiza una revisión de los métodos del estado del arte que hacen uso de técnicas de aprendizaje automático y deep learning para la detección de puntos de referencia faciales. También se exploran y resumen los algoritmos del estado del arte aplicados a la estimación de la mirada. Para el primer bloque de detección de puntos de referencia faciales, se implementan dos modelos, uno basado en métodos de regresión en cascada y un otro basado en redes neuronales. Ambos modelos son comparados sobre diversas bases de datos, analizando las virtudes y defectos de cada uno de ellos. Además, se realiza una comparación del método propuesto con el estado del arte en la que se demuestra la superioridad de nuestro método. En cuanto al segundo bloque, en primer lugar se presenta un método para la generación de un vector de características que incluya informaci ón relevante para poder realizar la estimación de la mirada. Además, se proponen varios modelos basados en redes neuronales y se investiga el uso de datos sintéticos para su entrenamiento. Por último, se plantea un método para adaptar y calibrar los modelos entrenados con usuarios sintéticos, a datos de sujetos reales. La tesis concluye con un resumen de sus contribuciones y principales hallazgos. La integración de técnicas de aprendizaje automático, algoritmos avanzados y datos sintéticos presenta perspectivas prometedoras para futuras investigaciones en este campo.
Open Access
Low cost gaze estimation: knowledge-based solutions
(IEEE, 2020) Martinikorena Aranburu, Ion; Larumbe Bergera, Andoni; Ariz Galilea, Mikel; Porta Cuéllar, Sonia; Cabeza Laguna, Rafael; Villanueva Larre, Arantxa; Ingeniería Eléctrica, Electrónica y de Comunicación; Ingeniaritza Elektrikoa, Elektronikoaren eta Telekomunikazio Ingeniaritzaren
Eye tracking technology in low resolution scenarios is not a completely solved issue to date. The possibility of using eye tracking in a mobile gadget is a challenging objective that would permit to spread this technology to non-explored fields. In this paper, a knowledge based approach is presented to solve gaze estimation in low resolution settings. The understanding of the high resolution paradigm permits to propose alternative models to solve gaze estimation. In this manner, three models are presented: a geometrical model, an interpolation model and a compound model, as solutions for gaze estimation for remote low resolution systems. Since this work considers head position essential to improve gaze accuracy, a method for head pose estimation is also proposed. The methods are validated in an optimal framework, I2Head database, which combines head and gaze data. The experimental validation of the models demonstrates their sensitivity to image processing inaccuracies, critical in the case of the geometrical model. Static and extreme movement scenarios are analyzed showing the higher robustness of compound and geometrical models in the presence of user’s displacement. Accuracy values of about 3◦ have been obtained, increasing to values close to 5◦ in extreme displacement settings, results fully comparable with the state-of-the-art.
Open Access
Synthetic gaze data augmentation for improved user calibration
(Springer, 2021) Garde Lecumberri, Gonzalo; Larumbe Bergera, Andoni; Porta Cuéllar, Sonia; Cabeza Laguna, Rafael; Villanueva Larre, Arantxa; Ingeniaritza Elektrikoa, Elektronikoaren eta Telekomunikazio Ingeniaritzaren; Institute of Smart Cities - ISC; Ingeniería Eléctrica, Electrónica y de Comunicación
In this paper, we focus on the calibration possibilitiesó of a deep learning based gaze estimation process applying transfer learning, comparing its performance when using a general dataset versus when using a gaze specific dataset in the pretrained model. Subject calibration has demonstrated to improve gaze accuracy in high performance eye trackers. Hence, we wonder about the potential of a deep learning gaze estimation model for subject calibration employing fine-tuning procedures. A pretrained Resnet-18 network, which has great performance in many computer vision tasks, is fine-tuned using user’s specific data in a few shot adaptive gaze estimation approach. We study the impact of pretraining a model with a synthetic dataset, U2Eyes, before addressing the gaze estimation calibration in a real dataset, I2Head. The results of the work show that the success of the individual calibration largely depends on the balance between fine-tuning and the standard supervised learning procedures and that using a gaze specific dataset to pretrain the model improves the accuracy when few images are available for calibration. This paper shows that calibration is feasible in low resolution scenarios providing outstanding accuracies below 1.5 ∘ ∘ of error.
Open Access
Accurate pupil center detection in off-the-shelf eye tracking systems using convolutional neural networks
(MDPI, 2021) Larumbe Bergera, Andoni; Garde Lecumberri, Gonzalo; Porta Cuéllar, Sonia; Cabeza Laguna, Rafael; Villanueva Larre, Arantxa; Ingeniería Eléctrica, Electrónica y de Comunicación; Ingeniaritza Elektrikoa, Elektronikoaren eta Telekomunikazio Ingeniaritzaren; Universidad Pública de Navarra / Nafarroako Unibertsitate Publikoa
Remote eye tracking technology has suffered an increasing growth in recent years due to its applicability in many research areas. In this paper, a video-oculography method based on convolutional neural networks (CNNs) for pupil center detection over webcam images is proposed. As the first contribution of this work and in order to train the model, a pupil center manual labeling procedure of a facial landmark dataset has been performed. The model has been tested over both real and synthetic databases and outperforms state-of-the-art methods, achieving pupil center estimation errors below the size of a constricted pupil in more than 95% of the images, while reducing computing time by a 8 factor. Results show the importance of use high quality training data and well-known architectures to achieve an outstanding performance.
Open Access
Low-cost eye tracking calibration: a knowledge-based study
(MDPI, 2021) Garde Lecumberri, Gonzalo; Larumbe Bergera, Andoni; Bossavit, Benoît; Porta Cuéllar, Sonia; Cabeza Laguna, Rafael; Villanueva Larre, Arantxa; Ingeniería Eléctrica, Electrónica y de Comunicación; Ingeniaritza Elektrikoa, Elektronikoaren eta Telekomunikazio Ingeniaritzaren
Subject calibration has been demonstrated to improve the accuracy in high-performance eye trackers. However, the true weight of calibration in off-the-shelf eye tracking solutions is still not addressed. In this work, a theoretical framework to measure the effects of calibration in deep learning-based gaze estimation is proposed for low-resolution systems. To this end, features extracted from the synthetic U2Eyes dataset are used in a fully connected network in order to isolate the effect of specific user’s features, such as kappa angles. Then, the impact of system calibration in a real setup employing I2Head dataset images is studied. The obtained results show accuracy improvements over 50%, probing that calibration is a key process also in low-resolution gaze estimation scenarios. Furthermore, we show that after calibration accuracy values close to those obtained by high-resolution systems, in the range of 0.7°, could be theoretically obtained if a careful selection of image features was performed, demonstrating significant room for improvement for off-the-shelf eye tracking systems
Open Access
Gaze estimation problem tackled through synthetic images
(Association for Computing Machinery (ACM), 2020) Garde Lecumberri, Gonzalo; Larumbe Bergera, Andoni; Bossavit, Benoît; Cabeza Laguna, Rafael; Porta Cuéllar, Sonia; Villanueva Larre, Arantxa; Ingeniería Eléctrica, Electrónica y de Comunicación; Ingeniaritza Elektrikoa, Elektronikoaren eta Telekomunikazio Ingeniaritzaren
In this paper, we evaluate a synthetic framework to be used in the field of gaze estimation employing deep learning techniques. The lack of sufficient annotated data could be overcome by the utilization of a synthetic evaluation framework as far as it resembles the behavior of a real scenario. In this work, we use U2Eyes synthetic environment employing I2Head datataset as real benchmark for comparison based on alternative training and testing strategies. The results obtained show comparable average behavior between both frameworks although significantly more robust and stable performance is retrieved by the synthetic images. Additionally, the potential of synthetically pretrained models in order to be applied in user's specific calibration strategies is shown with outstanding performances.