nav emailalert searchbtn searchbox tablepage yinyongbenwen piczone journalimg journalInfo searchdiv qikanlogo popupnotification paper paperNew
2021, 04, v.34 339-348
基于脑电信号的认知负荷评估综述
基金项目(Foundation): 国家自然科学基金(61876082,61861130366); 国家重点研发计划(2018YFC2001600,2018YFC2001602)
邮箱(Email): wuxia@bnu.edu.cn;dqzhang@nuaa.edu.cn;
DOI: 10.16289/j.cnki.1002-0837.2021.04.008
摘要:

过高的认知负荷是导致人因事故的重要原因之一,认知负荷的准确评估十分重要。脑电信号具有时间分辨率高、便携性好等优点,已成为评估认知负荷的重要技术手段。随着机器学习的发展,越来越多的研究采用机器学习方法实现基于脑电的认知负荷评估,以获得更稳定和准确的结果。首先介绍常见的认知负荷评估实验范式及脑电预处理流程;其次,详细综述基于脑电认知负荷评估的常用方法,包括脑电特征提取、脑电特征选择、认知负荷分类预测;随后,对现阶段基于脑电的认知负荷评估应用领域进行总结;最后指出当前研究中仍然存在跨个体、跨时间、跨任务以及脑电数据来源等问题,并对发展趋势进行展望。

Abstract:

Excessive cognitive workload is considered as important cause of human accidents.Therefore,it is necessary to assess cognitive workload accurately.Due to the high temporal resolution and portability of electroencephalogram(EEG) signal,the EEG-based cognitive workload assessment has become a hot topic.On the other hand,with the development of machine learning,researchers have increasingly devoted to using machine learning methods for cognitive workload assessment,to acquire robust and accurate results.In this paper,we first introduce the commonly used experimental paradigms of cognitive workload assessment and the procedure of EEG preprocessing.According to the orders of EEG feature extraction,feature selection,cognitive workload classification and prediction,we then review common methods for EEG based cognitive assessment in details,respectively.Then,we summarize the current applications of cognitive workload assessment.Finally,we point out the issues in the current research,such as cross-subject,cross-session,cross-task,and EEG data sources.And we discuss the future trends of this field.

参考文献

[1]Babiloni F.Mental workload monitoring:new perspectives from neuroscience[C]//International Symposium on Human Mental Workload:Models and Applications.Rome,Italy:Springer,Cham,2019:3-19.

[2]Puma S,Matton N,Paubel PV,et al.Using theta and alpha band power to assess cognitive workload in multitasking environments[J].International Journal of Psychophysiology,2018,123:111-120.

[3]Chin ZY,Zhang X,Wang C,et al.EEG-based discrimination of different cognitive workload levels from mental arithmetic[C]//2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society(EMBC).Hawaii,USA:IEEE,2018:1984-1987.

[4]Baig MZ,Kavakli M.Analyzing novice and expert user’s cognitive load in using multi-modal interface system[C]//2018 26th International Conference on Systems Engineering(ICSEng).Sydney,Australia:IEEE,2018:1-7.

[5]Kramer AF,Sirevaag EJ,Braune R.psychophysiological assessment of operator workload during simulated flight missions[J].Human factors,1987,29(2):145-160.

[6]明东,柯余峰,何峰,等.基于生理信号的脑力负荷检测及自适应自动化系统研究:40年回顾与最新进展[J].电子测量与仪器学报,2015,29(1):1-13.Ming D,Ke YF,He F,et al.Psychophysiological measures based studies on mental workload assessment and adaptive automation:Review of the last 40 years and the latest developments[J].Journal of Electronic Measurement and Instrument,2015,29(1):1-13

[7]Heard J,Harriott CE,Adams JA.survey of workload assessment algorithms[J].IEEE Transactions on HumanMachine Systems,2018,48(5):434-451.

[8]Charles RL,Nixon J.Measuring mental workload using physiological measures:systematic review[J].Applied Ergonomics,2019,74:221-232.

[9]Aghajani H,Garbey M,Omurtag A.Measuring mental workload with EEG+f NIRS[J].Frontiers in Human Neuroscience,2017,11:359.

[10]Pergher V,Wittevrongel B,Tournoy J,et al.Mental workload of young and older adults gauged with ERPs and spectral power during N-Back task performance[J].Biological Psychology,2019,146(9):107726.

[11]Zhang Y,Shen Y.Parallel mechanism of spectral featureenhanced maps in EEG-based cognitive workload classification[J].Sensors,2019,19(4):808.

[12]Zhang D,Zhao H,Bai W,et al.Functional connectivity among multi-channel EEGs when working memory load reaches the capacity[J].Brain Research,2016,1631:101-112.

[13]So WKY,Wong SWH,Mak JN,et al.An evaluation of mental workload with frontal EEG[J].Plo SOne,2017,12(4):e0174949.

[14]Radüntz T.new method for the objective registration of mental workload[C]//Advances in Neuroergonomics and Cognitive Engineering.Florida,USA:Springer,Cham,2017:279-290.

[15]Blanco JA,Johnson MK,Jaquess KJ,et al.Quantifying cognitive workload in simulated flight using passive,dry EEG measurements[J].IEEE Transactions on Cognitive and Developmental Systems,2016,10(2):373-383.

[16]Kim NY,House R,Yun MH,et al.Neural correlates of workload transition in Multitasking:An ACT-R model of hysteresis effect[J].Frontiers in Human Neuroscience,2019,12:535.

[17]Zhang J,Cui X,Li J,et al.Imbalanced classification of mental workload using cost-sensitive majority weighted minority oversampling strategy[J].Cognition,Technology Work,2017,19(4):633-653.

[18]Almogbel MA,Dang AH,Kameyama W.Cognitive workload detection from raw EEG-signals of vehicle driver using deep learning[C]//2019 21st International Conference on Advanced Communication Technology(ICACT).PyeongChang,Korea:IEEE,2019:1-6.

[19]Delorme A,Makeig S.EEGLAB:An open source toolbox for analysis of single-trial EEG dynamics including independent component analysis[J].Journal of Neuroscience Methods,2004,134(1):9-21.

[20]Causse M,Fabre E,Giraudet L,et al.EEG/ERP as measure of mental workload in simple piloting task[J].Procedia Manufacturing,2015,3:5230-5236.

[21]Dehais F,Duprès A,Blum S,et al.Monitoring pilot’s mental workload using ERPs and spectral power with six-dry-electrode EEG system in real flight conditions[J].Sensors,2019,19(6):1324.

[22]Wang S,Gwizdka J,Chaovalitwongse WA.Using wireless EEG signals to assess memory workload in the n-back task[J].IEEE Transactions on Human-Machine Systems,2015,46(3):424-435.

[23]Wang F,Wu S,Zhang W,et al.Multiple nonlinear features fusion based driving fatigue detection[J].Biomedical Signal Processing and Control,2020,62:102075.

[24]Chaudhuri A,Routray A.Driver fatigue detection through chaotic entropy analysis of cortical sources obtained from scalp EEG signals[J].IEEE Transactions on Intelligent Transportation Systems,2019,21(1):185-198.

[25]Yang S,Yin Z,Wang Y,et al.Assessing cognitive mental workload via EEG signals and an ensemble deep learning classifier based on denoising autoencoders[J].Computers in Biology and Medicine,2019,109:159-170.

[26]Cheema BS,Samima S,Sarma M,et al.Mental workload estimation from EEG signals using machine learning algorithms[C]//International Conference on Engineering Psychology and Cognitive Ergonomics.Las Vegas,USA:Springer,Cham,2018:265-284.

[27]Bashivan P,Yeasin M,Bidelman GM.Single trial prediction of normal and excessive cognitive load through EEGfeature fusion[C]//2015 IEEE Signal Processing in Medicine and Biology Symposium(SPMB).Philadelphia,USA:IEEE,2015:1-5.

[28]Calcagno A,Coelli S,Couceiro R,et al.EEG monitoring during software development[C]//2020 IEEE 20th Mediterranean Electrotechnical Conference(MELECON).Palermo,Italy:IEEE,2020:325-329.

[29]Gudmundsson S,Runarsson TP,Sigurdsson S,et al.Reliability of quantitative EEG features[J].Clinical Neurophysiology,2007,118(10):2162-2171.

[30]Rojas RF,Debie E,Fidock J,et al.Electroencephalographic workload indicators during teleoperation of an unmanned aerial vehicle shepherding swarm of unmanned ground vehicles in contested environments[J].Frontiers in Neuroscience,2020,1:1-10.

[31]Christensen JC,Estepp JR,Wilson GF,et al.The effects of day-to-day variability of physiological data on operator functional state classification[J].Neuro Image,2012,59(1):57-63.

[32]Plechawska-Wójcik M,Tokovarov M,Kaczorowska M,et al.three-class classification of cognitive workload based on EEG spectral data[J].Applied Sciences,2019,9(24):5340.

[33]Gao Z,Gong Z,Cai Q,et al.Complex Network Analysis of Experimental EEG Signals for Decoding Brain Cognitive State[J].IEEE Transactions on Circuits and Systems II:Express Briefs,2020,68(1):531-535.

[34]Dimitrakopoulos GN,Kakkos I,Dai Z,et al.Task-independent mental workload classification based upon common multiband EEG cortical connectivity[J].IEEETransactions on Neural Systems and Rehabilitation Engineering,2017,25(11):1940-1949.

[35]Kakkos I,Dimitrakopoulos GN,Gao L,et al.Mental workload drives different reorganizations of functional cortical connectivity between 2D and 3D simulated flight experiments[J].IEEE Transactions on Neural Systems and Rehabilitation Engineering,2019,27(9):1704-1713.

[36]He B,Astolfi L,Valdés-Sosa PA,et al.Electrophysiological brain connectivity:theory and implementation[J].IEEE Transactions on Biomedical Engineering,2019,66(7):2115-2137.

[37]Arvaneh M,Umilta A,Robertson IH.Filter bank common spatial patterns in mental workload estimation[C]//2015 37th Annual International Conference of the IEEEEngineering in Medicine and Biology Society(EMBC).Milan,Italy:IEEE,2015:4749-4752.

[38]Fu R,Han M,Tian Y,et al.Improvement motor imagery EEG classification based on sparse common spatial pattern and regularized discriminant analysis[J].Journal of Neuroscience Methods,2020,343:108833.

[39]Zhang P,Wang X,Chen J,et al.Spectral and temporal feature learning with two-stream neural networks for mental workload assessment[J].IEEE Transactions on Neural Systems and Rehabilitation Engineering,2019,27(6):1149-1159.

[40]Cai J,Luo J,Wang S,et al.Feature selection in machine learning:new perspective[J].Neurocomputing,2018,300:70-79.

[41]Vergara JR,Estévez PA.review of feature selection methods based on mutual information[J].Neural Computing and Applications,2014,24(1):175-186.

[42]Kosti MV,Georgiadis K,Adamos DA,et al.Towards an affordable brain computer interface for the assessment of programmers’mental workload[J].International Journal of Human-Computer Studies,2018,115:52-66.

[43]Arico P,Borghini G,Di Flumeri G,et al.Reliability over time of EEG-based mental workload evaluation during Air Traffic Management(ATM) tasks[C]//2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society(EMBC).Milan,Italy:IEEE,2015:7242-7245.

[44]AricòP,Borghini G,Di Flumeri G,et al.Adaptive automation triggered by EEG-based mental workload index:passive brain-computer interface application in realistic air traffic control environment[J].Frontiers in Human Neuroscience,2016,10:539.

[45]Sciaraffa N,AricòP,Borghini G,et al.On the use of machine learning for EEG-based workload assessment:algorithms comparison in realistic task[C]//International Symposium on Human Mental Workload:Models and Applications.Rome,Italy:Springer,Cham,2019:170-185.

[46]Hefron RG,Borghetti BJ,Christensen JC,et al.Deep long short-term memory structures model temporal dependencies improving cognitive workload estimation[J].Pattern Recognition Letters,2017,94:96-104.

[47]Zhang L,Wade J,Bian D,et al.Cognitive load measurement in virtual reality-based driving system for autism intervention[J].IEEE Transactions on Affective Computing,2017,8(2):176-189.

[48]Yin Z,Zhang J.Cross-session classification of mental workload levels using EEG and an adaptive deep learning model[J].Biomedical Signal Processing and Control,2017,33:30-47.

[49]Di Flumeri G,Borghini G,AricòP,et al.EEG-based mental workload neurometric to evaluate the impact of different traffic and road conditions in real driving settings[J].Frontiers in Human Neuroscience,2018,12:509

[50]Kim HS,Yoon DS,Lee SJ,et al.study on the cognitive workload characteristics according to the driving behavior in the urban road[C]//2018 International Conference on Electronics,Information,and Communication(ICEIC).Honolulu,USA:IEEE,2018:1-4.

[51]Dimitrakopoulos GN,Kakkos I,Dai Z,et al.Functional connectivityanalysisofmentalfatiguerevealsdifferent network topological alterations between driving and vigilance tasks[J].IEEE Transactions on Neural Systems and Rehabilitation Engineering,2018,26(4):740-749.

[52]Plazak J,Di Govannii DA,Collins DL,et al.Cognitive load associations when utilizing auditory display within image-guided neurosurgery[J].International Journal of Computer Assisted Radiology and Surgery,2019,14(8):1431-1438.

[53]Morales JM,Ruiz-Rabelo JF,Diaz-Piedra C,et al.Detecting mental workload in surgical teams using wearable single-channel electroencephalographic device[J].Journal of Surgical Education,2019,76(4):1107-1115.

[54]Richer N,Downey RJ,Hairston WD,et al.Motion and muscle artifact removal validation using an electrical head phantom,robotic motion platform,and dual layer mobile EEG[J].IEEE Transactions on Neural Systems and Rehabilitation Engineering,2020,28(8):1825-1835.

[55]Zammouri A,Moussa AA,Mebrouk Y.Brain-computer interface for workload estimation:Assessment of mental efforts in learning processes[J].Expert Systems with Applications,2018,112:138-147.

[56]Baldwin CL,Penaranda BN.Adaptive training using an artificial neural network and EEG metrics for within-and cross-task workload classification[J].Neuro Image,2012,59(1):48-56.

[57]Zhang P,Wang X,Zhang W,et al.Learning spatialspectral-temporal EEG features with recurrent 3D convolutional neural networks for cross-task mental workload assessment[J].IEEE Transactions on Neural Systems and Rehabilitation Engineering,2018,27(1):31-42.

[58]Debie E,Rojas RF,Fidock J,et al.Multimodal fusion for objective assessment of cognitive workload:review[J].IEEE Transactions on Cybernetics,2019:51(3):1542-1555.

[59]Craik A,He Y,Contreras-Vidal JL.Deep learning for electroencephalogram(EEG) classification tasks:review[J].Journal of Neural Engineering,2019,16(3):031001.

基本信息:

DOI:10.16289/j.cnki.1002-0837.2021.04.008

中图分类号:R318;TN911.7

引用信息:

[1]许子明,牛一帆,温旭云等.基于脑电信号的认知负荷评估综述[J].航天医学与医学工程,2021,34(04):339-348.DOI:10.16289/j.cnki.1002-0837.2021.04.008.

基金信息:

国家自然科学基金(61876082,61861130366); 国家重点研发计划(2018YFC2001600,2018YFC2001602)

检 索 高级检索

引用

GB/T 7714-2015 格式引文
MLA格式引文
APA格式引文