Mental fatigue is a complex state that results from prolonged cognitive activity. Symptoms of mental fatigue can include change in mood, motivation, and temporary deterioration of various cognitive functions involved in goal-directed behavior. Extensive research has been done to develop methods for recognizing physiological and psychophysiological signs of mental fatigue. This has allowed the development of many AI-based models to classify different levels of fatigue, using data extracted from eye-tracking device, EEG, or ECG. In this paper, we present an experimental protocol which aims to both generate/measure mental fatigue and provide effective strategies for recuperation via VR sessions paired with EEG and eye tracking devices. This paper first provides a comprehensive state-of-the-art of mental fatigue predictive factors, measurement methods, and recuperation strategies. Then the paper presents an experimental protocol resulting from the state-of-the-art to 1) generate and measure mental fatigue and 2) evaluate the effectiveness of virtual therapy for fatigue recuperation, using a virtual reality (VR) simulated environment. In our work, we successfully generated mental fatigue through completion of cognitive tasks in a virtual simulated environment. Participants showed significant decline in pupil diameter and theta/alpha score during the various cognitive tasks. We trained an RBF SVM classifier from Electroencephalogram (EEG) data classifying mental fatigue with 95% accuracy on the test set. Finally, our results show that the time allocated for virtual therapy did not improve pupil diameter in post-relaxation period. Further research on the impact of relaxation therapy on relaxation therapy should allocate time closer to the standard recovery time of 60 min.
Ishii, A., Tanaka, M. and Watanabe, Y. (2014) Neural Mechanisms of Mental Fatigue. Reviews in the Neuroscience, 25, 469-479.
Hopstaken, J.F., van der Linden, D., Bakker, A.B. and Kompier, M.A.J. (2015) The Window of My Eyes: Task Disengagement and Mental Fatigue Covary with Pupil Dynamics. Biological Psychology, 110, 100-106. https://doi.org/10.1016/j.biopsycho.2015.06.013
Hopstaken, J.F., van der Linden, D., Bakker, A.B. and Kompier, M.A.J. (2015) A Multifaceted Investigation of the Link between Mental Fatigue and Task Disengagement. Psychophysiology, 52, 305-315. https://doi.org/10.1111/psyp.12339
Parekh, V., Shah, D. and Shah, M. (2020) Fatigue Detection Using Artificial Intelligence Framework. Augmented Human Research, 5, Article No. 5. https://doi.org/10.1007/s41133-019-0023-4
Åkerstedt, T., Nilsson, P.M., Kecklund, G., Sonnentag, S., Perrewé, P.L. and Ganster, D.C. (2009) Sleep and Recovery. In: Sonnentag, S., Perrewé, P.L. and Ganster, D.C., Eds., Current Perspectives on Job-Stress Recovery (Research in Occupational Stress and Well Being, Vol. 7), Emerald Group Publishing Limited, Bingley, 205-247. https://doi.org/10.1108/S1479-3555(2009)0000007009
Horvat, M., Dobrinic, M., Novosel, M. and Jercic, P. (2018) Assessing Emotional Responses Induced in Virtual Reality Using a Consumer EEG Headset: A Preliminary Report. Proceedings of the 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), Opatija, 21-25 May 2018, 1006-1010. https://doi.org/10.23919/MIPRO.2018.8400184
Tarrant, J., Viczko, J. and Cope, H. (2018) Virtual Reality for Anxiety Reduction demonstrated by Quantitative EEG: A Pilot Study. Frontiers in Psychology, 9, 1280. https://doi.org/10.3389/fpsyg.2018.01280
Amores, J., Richer, R., Zhao, N., Maes, P. and Eskofier, B. (2018) Promoting Relaxation Using Virtual Reality, Olfactory Interfaces and Wearable EEG. Proceedings of the 2018 IEEE 15th International Conference on Wearable and Implantable Body Sensor Networks (BSN), Las Vegas, 4-7 March 2018, 98-101. https://doi.org/10.1109/BSN.2018.8329668
Kamińska, D., Smólka, K. and Zwoliński, G. (2021) Detection of Mental Stress through EEG Signal in Virtual Reality Environment. Electronics, 10, 2840. https://doi.org/10.3390/electronics10222840
Ren, Z.W., Li, R.H., Chen, B., et al. (2021) EEG-Based Driving Fatigue Detection Using a Two-Level Learning Hierarchy Radial Basis Function. Frontiers in Neurorobotics, 15, Article ID: 618408. https://doi.org/10.3389/fnbot.2021.618408
Chirico, A., Cipresso, P., Yaden, D.B., Biassoni, F., Riva, G. and Gaggioli, A. (2017) Effectiveness of Immersive Videos in Inducing Awe: An Experimental Study. Scientific Reports, 7, Article No. 1218. https://doi.org/10.1038/s41598-017-01242-0
Weiß, S., Klassen, N. and Heuten, W. (2021) Effects of Image Realism on the Stress Response in Virtual Reality. Proceedings of the 27th ACM Symposium on Virtual Reality Software and Technology, Osaka, 8-10 December 2021, 1-10. https://doi.org/10.1145/3489849.3489885
Marín-Morales, J., Higuera-Trujillo, J., Greco, A., Guixeres, J., Llinares, C., Scilingo, E. and Valenza, G. (2018) Affective Computing in Virtual Reality: Emotion Recognition from Brain and Heartbeat Dynamics Using Wearable Sensors. Scientific Reports, 8, Article No. 13657. https://doi.org/10.1038/s41598-018-32063-4
Xu, T., Yin, R., Shu, L. and Xu, X. (2019) Emotion Recognition Using Frontal EEG in VR Affective Scenes. Proceedings of the 2019 IEEE MTT-S International Microwave Biomedical Conference (IMBioC), Nanjing, 6-8 May 2019, 1-4. https://doi.org/10.1109/IMBIOC.2019.8777843
Csathó, á., Linden, D.V., Hernádi, I., Buzás, P. and Kalmár, G. (2012) Effects of Mental Fatigue on the Capacity Limits of Visual Attention. Journal of Cognitive Psychology, 24, 511-524. https://doi.org/10.1080/20445911.2012.658039
Chaouachi, M., Jraidi, I. and Frasson, C. (2011) Modeling Mental Workload Using EEG Features for Intelligent Systems. 19th International Conference, UMAP 2011, Girona, 11-15 July 2011, 50-61. https://doi.org/10.1007/978-3-642-22362-4_5
Fan, J.L. and Smith, A.P. (2017) The Impact of Workload and Fatigue on Performance. 1st International Symposium, H-WORKLOAD 2017, Dublin, 28-30 June 2017, 90-105. https://doi.org/10.1007/978-3-319-61061-0_6
Desmond, P.A. and Hancock, P.A. (2001) Active and Passive Fatigue States. In: Hancock, P.A. and Desmond, P.A., Eds., Stress, Workload, and Fatigue, CRC Press, Boca Raton, 455-465. https://doi.org/10.1201/b12791-3.1
Saxby, D.J., Matthews, G., Hitchcock, E.M., Warm, J.S., Funke, G.J. and Gantzer, T. (2008) Effect of Active and Passive Fatigue on Performance Using a Driving Simulator. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 52, 1751-1755. https://doi.org/10.1177/154193120805202113
Bafna, T., Bækgaard, P. and Hansen, J.P. (2021) Mental Fatigue Prediction during Eye-Typing. PLOS ONE, 16, e0246739.
Dalveren, G.G.M., Cagiltay, N.E., Ozcelik, E. and Maras, H. (2018) Insights from Pupil Size to Mental Workload of Surgical Residents: Feasibility of an Educational Computer-Based Surgical Simulation Environment (ECE) Considering the Hand Condition. Surgical Innovation, 25, 616-624. https://doi.org/10.1177/1553350618800078
Pfleging, B., Fekety, D.K., Schmidt, A. and Kun, A.L.. (2016) A Model Relating Pupil Diameter to Mental Workload and Lighting Conditions. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI’16), Association for Computing Machinery, New York, 5776-5788. https://doi.org/10.1145/2858036.2858117
Borghini, G., Astolfi, L., Vecchiato, G., Mattia, D. and Babiloni, F. (2014) Measuring Neurophysiological Signals in Aircraft Pilots and Car Drivers for the Assessment of Mental Workload, Fatigue and Drowsiness. Neuroscience and Biobehavioral Review, 44, 58-75. https://doi.org/10.1016/j.neubiorev.2012.10.003
Zhao, C., Zhao, M., Liu, J.P. and Zheng, C.X. (2012) Electroencephalogram and Electrocardiograph Assessment of Mental Fatigue in a Driving Simulator. Accident Analysis and Prevention, 45, 83-90. https://doi.org/10.1016/j.aap.2011.11.019
Pope, A., Bogart, E.H. and Bartolome, D.S. (1995) Biocybernetic System Evaluates Indices of Operator Engagement in Automated Task. Biological Psychology, 40, 187-195. https://doi.org/10.1016/0301-0511(95)05116-3
Laurent, F., Valderrama, M., Besserve, M., Guillard, M., Lachaux, J.-P., Martinerie, J. and Florence, G. (2012) Multimodal Information Improves the Rapid Detection of Mental Fatigue. Biomedical Signal Processing and Control, 8, 400-408. https://doi.org/10.1016/j.bspc.2013.01.007
Rojas, R.F., Valderrama, M., Besserve, M., Guillard, M., Lachaux, J.-P., Martinerie, J. and Florence, G. (2020) Electroencephalographic Workload Indicators during Teleoperation of an Unmanned Aerial Vehicle Shepherding a Swarm of Unmanned Ground Vehicles in Contested Environments. Frontiers in Neuroscience, 14, 40. https://doi.org/10.3389/fnins.2020.00040
Cao, T., Wan, F., Wong, C.M., da Cruz, J.N. and Hu, Y. (2014) Objective Evaluation of Fatigue by EEG Spectral Analysis in Steady-State Visual Evoked Potential-Based Brain-Computer Interfaces. BioMedical Engineering OnLine, 13, Article No. 28. https://doi.org/10.1186/1475-925X-13-28
Magnuson, J.R., Doesburg, S.M. and McNeil, C.J. (2021) Development and Recovery Time of Mental Fatigue and Its Impact on Motor Function. Biological Psychology, 161, Article ID: 108076. https://doi.org/10.1016/j.biopsycho.2021.108076
Ben Abdessalem, H., Byrns, A., Cuesta, M., Manera, V., Robert, P., Bruneau, M.A., Belleville, S. and Frasson, C. (2020) Application of Virtual Travel for Alzheimer’s Disease. SENSORNET 2020, the 9th International Conference on Sensor Networks, Valletta, 28-29 February 2020, 52-60. https://doi.org/10.5220/0008976700520060
Hopstaken, J.F., van der Linden, D., Bakker, A.B., et al. (2016) Shifts in Attention during Mental Fatigue: Evidence from Subjective, Behavioral, Physiological, and Eye-Tracking Data. Journal of Experimental Psychology: Human Perception and Performance, 42, 878-889. https://doi.org/10.1037/xhp0000189