Brain Training with Cognitive Simulations

Today we will con­tin­ue our review of the ben­e­fits of brain train­ing for spe­cif­ic occu­pa­tions: in this case, pilots and bas­ket­ball play­ers. The lessons can be rel­e­vant not only for cor­po­rate train­ing but also for edu­ca­tion and brain health & wellness.

To do so, we will select quotes from our inter­view last year with one of the major sci­en­tists in the field of cog­ni­tive sim­u­la­tions, Pro­fes­sor Daniel Gopher. You can read the full inter­view here.

Prof. Gopher pub­lished an award-win­ning arti­cle in 1994, Gopher, D., Weil, M. and Baraket, T. (1994), Trans­fer of skill from a com­put­er game train­er to flight, Human Fac­tors 36, 1–19., that con­sti­tutes a key mile­stone in the cog­ni­tive engi­neer­ing field.

On Cog­ni­tive Train­ing and Cog­ni­tive Simulations

AF: Tell us a bit about your over­all research interests

DG: My main inter­est has been how to expand the lim­its of human atten­tion, infor­ma­tion pro­cess­ing and response capa­bil­i­ties which are crit­i­cal in com­plex, real-time deci­sion-mak­ing, high-demand tasks such as fly­ing a mil­i­tary jet or play­ing pro­fes­sion­al bas­ket­ball. Using a ten­nis anal­o­gy, my goal has been, and is, how to help devel­op many “Wimbledon”-like cham­pi­ons. Each with their own styles, but per­form­ing to their max­i­mum capac­i­ty to suc­ceed in their environments.

What research over the last 15–20 years has shown is that cog­ni­tion, or what we call think­ing and per­for­mance, is real­ly a set of skills that we can train sys­tem­at­i­cal­ly. And that com­put­er-based cog­ni­tive train­ers or “cog­ni­tive sim­u­la­tions” are the most effec­tive and effi­cient way to do so.

This is an impor­tant point, so let me empha­size it. What we have dis­cov­ered is that a key fac­tor for an effec­tive trans­fer from train­ing envi­ron­ment to real­i­ty is that the train­ing pro­gram ensures “Cog­ni­tive Fideli­ty”, this is, it should faith­ful­ly rep­re­sent the men­tal demands that hap­pen in the real world. Tra­di­tion­al approach­es focus instead on phys­i­cal fideli­ty, which may seem more intu­itive, but less effec­tive and hard­er to achieve. They are also less effi­cient, giv­en costs involved in cre­at­ing expen­sive phys­i­cal sim­u­la­tors that faith­ful­ly repli­cate, let’s say, a whole mil­i­tary heli­copter or just a sig­nif­i­cant part of it.

AF: Very inter­est­ing. In the Seri­ous Games Sum­mit this week we are see­ing a num­ber of sim­u­la­tions for mil­i­tary train­ing that try to be as real­is­tic as pos­si­ble. Are you say­ing that they may not be the best approach for training?

DG: The need for phys­i­cal fideli­ty is not based on research, at least for the type of high-per­for­mance train­ing we are talk­ing about. In fact, a sim­ple envi­ron­ment may be bet­ter in that it does not cre­ate the illu­sion of real­i­ty. Sim­u­la­tions can be very expen­sive and com­plex, some­times even cost­ing as much as the real thing, which lim­its the access to train­ing. Not only that, but the whole effort may be futile, giv­en that some impor­tant fea­tures can not be repli­cat­ed (such as grav­i­ta­tion free tilt­ed or invert­ed flight), and even result in neg­a­tive trans­fer, because learn­ers pick up on spe­cif­ic train­ing fea­tures or sen­sa­tions that do not exist in the real situation.

Main stud­ies and applications

AF: What are the main stud­ies have you conducted?

DG: in this field of work, I would men­tion two. In one, which con­sti­tut­ed the basis for the 1994 paper, we showed that 10 hours of train­ing for flight cadets, in an atten­tion train­er instan­ti­at­ed as a com­put­er game-Space Fortress- result­ed in 30% improve­ment in their flight per­for­mance. The results led the train­er to be inte­grat­ed into the reg­u­lar train­ing pro­gram of the flight school. It was used in the train­ing of hun­dreds of flight cadets for sev­er­al years. In the oth­er one, spon­sored by NASA, we com­pared the results of the cog­ni­tive train­er vs. a sophis­ti­cat­ed, pic­to­r­i­al and high-lev­el-graph­ic and phys­i­cal-fideli­ty-based com­put­er sim­u­la­tion of a Black­hawk heli­copter. The result: the Space Fortress cog­ni­tive train­er was very suc­cess­ful in improv­ing per­for­mance, while the alter­na­tive was not. The study was pub­lished in the pro­ceed­ings of the Human Fac­tors and Ergonom­ic Soci­ety: Hart S. G and Bat­tiste V. (1992), Flight test of a video game train­er. Pro­ceed­ings of the Human Fac­tors Soci­ety 26th Meet­ing (pp. 1291–1295).

AF: What have been to date the main appli­ca­tions of your com­put­er-based cog­ni­tive simulations?

DG: in sum­ma­ry, I’d say

- Fly­ing high-per­for­mance air­planes: in 10 hours, we showed an increase in 30% flight performance

- Fly­ing with HMD (hel­met mount­ed displays)

- Touch-typ­ing skills

- Teach­ing old adults to cope with high work­load atten­tion demands.

- Devel­op­ing Bas­ket­ball “game-intel­li­gence” for pro­fes­sion­al play­ers, to improve the per­for­mance of indi­vid­u­als and teams

Train­er for bas­ket­ball “game-intel­li­gence”

AF: talk to us about the bas­ket­ball exam­ple. I am sure many read­ers will find that fascinating.

DG: I served as a sci­en­tif­ic advi­sor to ACE, who devel­oped the pro­gram called Intel­li­Gym. Although the con­text is dif­fer­ent, the approach and basic prin­ci­ples are the same of those of devel­op­ing a train­er for the task of fly­ing a high per­for­mance jet air­plane. First, one needs to ana­lyze what cog­ni­tive skills are involved in play­ing at top lev­el, and then devel­op a com­put­er-based cog­ni­tive sim­u­la­tion that trains those skills. What most peo­ple don’t real­ize is that top play­ers are not born top play­ers. We are not just talk­ing about instincts. We are talk­ing about skills that can be trained.

AF: what are the results of the pro­gram so far?

DG: Well, first let me say that the com­pa­ny has had to over­come huge cul­tur­al bar­ri­ers to get adop­tion by a good num­ber of uni­ver­si­ty teams and some NBA play­ers. Coach­es see the val­ue of this tool very quick­ly, but admin­is­tra­tors are hard­er to con­vince in the begin­ning. We have seen that the teams and indi­vid­u­als using Intel­li­gym have improved their per­for­mance sig­nif­i­cant­ly. From the cog­ni­tive train­ing, or skill devel­op­ment point of view, we have seen that play­ers improve their posi­tion­al aware­ness-of them­selves, their mates and oppo­nents, and abil­i­ty to pre­dict what is going on in the game and to make fast and good deci­sions. Play­ers quick­ly devel­op atten­tion allo­ca­tion strate­gies that enable them bet­ter par­tic­i­pate in the game, and also improve their spa­tial orientation.

Sum­ma­ry of key findings

AF: Fas­ci­nat­ing real-world expe­ri­ence. Can you sum­ma­rize your research find­ings across all these exam­ples and fields, and how you see the field evolving?

DG: In short, I’d sum­ma­rize by say­ing that

- Cog­ni­tive per­for­mance can be sub­stan­tial­ly improved with prop­er training.

- It is not rigid­ly con­strained by innate, fixed abilities.

- Cog­ni­tive task analy­sis enables us to extract major cog­ni­tive skills involved in any task.

- Atten­tion con­trol and atten­tion allo­ca­tion strate­gies are a crit­i­cal deter­mi­nants in per­form­ing at top lev­el in com­plex, real-time deci­sion-mak­ing environments

- Those skills, and oth­er asso­ci­at­ed, can be improved through training

- Research shows that stand-alone, inex­pen­sive, PC-based train­ing is effec­tive to trans­fer and gen­er­al­ize per­for­mance.

- The key for suc­cess is to ensure Cog­ni­tive fideli­ty, this is, that the cog­ni­tive demands in train­ing resem­ble those of the real life task.

I can think of many oth­er appli­ca­tions. Prob­a­bly cur­ren­cy and options traders would ben­e­fit from a sys­tem like this. Now, we will need to increase aware­ness, and will need to find cham­pi­ons will­ing to take risks. The cog­ni­tive sim­u­la­tion approach is less intu­itive that tra­di­tion­al ones.

Pro­fes­sor Wayne She­bilske, at Wright State Uni­ver­si­ty Psy­chol­o­gy depart­ment, is con­duct­ing addi­tion­al research on appli­ca­tions, such as out­lined on the paper She­bilske, Wayne L., et al, “Revised Space Fortress: A Val­i­da­tion Study” (accept­ed for Behav­ior Research Meth­ods, Instru­ments and computers).

(Pro­fes­sor She­bilske was kind enough to write a great com­ment below, giv­ing us 2 detailed references:

She­bilske, W. L., Volz, R. A., Gildea, K. M., Work­man, J. W., Nan­janath, M., Cao, S., & Whet­zel, J. (2005). Revised Space Fortress: A val­i­da­tion study. Behav­ior Research Meth­ods, 37, 591–601.

Volz, R.A., John­son, J.C., Cao, S., Nan­janath, M., Whet­zel, J., Ioerg­er, T.R., Raman, B., She­bilske, W.L., and Xu, Dianx­i­ang (2005). Fine-Grained data acqui­si­tion and agent ori­ent­ed tools for dis­trib­uted train­ing pro­to­col research: Revised Space Fortress. Down Load Tech­ni­cal Sup­ple­ment, Psy­cho­nom­ic Soci­ety Web-based Archive (see 37,591–601).

AF: are you doing some­thing to spread the word?

DG: apart from con­fer­ences and jour­nals, I have writ­ten the chap­ter Empha­sis change as a train­ing pro­to­col for high demands tasks, in the book Applied Atten­tion: From The­o­ry to Prac­tice, A. Kramer, D. Wieg­man, A. Kir­lik (Eds): Oxford Psy­chol­o­gy Press, about to be released.

A more in-depth view of his cog­ni­tive sim­u­la­tion approach

AF: Great. For read­ers who may be inter­est­ed in more spe­cif­ic details about your spe­cif­ic approach to cog­ni­tive train­ing, could you give us some lessons learned?

DG: Good ques­tion. There are dif­fer­ent types of cog­ni­tive train­ing. The one we have spe­cial­ized in focus­es on the devel­op­ment of atten­tion-con­trol, atten­tion-allo­ca­tion strate­gies, which are bot­tle­neck in some high-per­form­ing, high-men­tal-work­load- envi­ron­ments. Our approach is called Empha­sis Change Pro­to­col, and is based on the intro­duc­tion of sys­tem­at­ic vari­abil­i­ty in train­ing, while main­tain­ing the over­all task intact. We just change the empha­sis on sub-com­po­nents of a com­plex task dur­ing per­for­mance. In our research, this has proven to be the most effec­tive way to train atten­tion man­age­ment skills, task switch­ing and con­trol process­es, such as the abil­i­ty to ini­ti­ate, coor­di­nate, syn­chro­nize and reg­u­late goal-direct­ed behavior.

This “whole task” approach increas­es trans­fer and adap­ta­tion capa­bil­i­ties, vs. tra­di­tion­al part task train­ing, which decom­pos­es the com­plex task and trains ele­ments in iso­la­tion. How­ev­er, whole task train­ing is hard­er at the begin­ning-there is slow­er progress at ear­ly stages of training.

Oth­er prin­ci­ples we use, based on our and oth­ers lit­er­a­ture, is the need for inter­mit­tent sched­ules of feed­back (vs. full one), to help reten­tion and trans­fer (at the cost of mak­ing learn­ing slow­er), and the encour­age­ment to explore alter­na­tives to reach a gen­er­al opti­mum. This explo­ration is impor­tant: we want to help the user find a flex­i­ble, and per­son­al best, match between his abil­i­ties and task demands, out of local­ized peaks. Com­ing back to the ten­nis exam­ple, we know that McEn­roe and Boris Beck­er have dif­fer­ent styles, but both are Wim­ble­don win­ners. We want to make sure the user increas­es sen­si­tiv­i­ty to real-time changes in the envi­ron­ment and expands his or her abil­i­ty to cope with them.

AF: Pro­fes­sor Gopher, it has been a plea­sure to talk to you. Thank you for your time.

————————————–

After we pub­lished this inter­view, Pro­fes­sor She­bilske wrote the great fol­low­ing comment:

“Your excel­lent inter­view with Dr. Gopher remind­ed me why so many of us have fol­lowed his lead in train­ing com­plex skills. I hope that your inter­view inspires oth­ers. They will find that he is gen­er­ous with his ideas, time, ener­gy, and infec­tious pos­i­tive spir­it. Work­ing with him to repli­cate exper­i­ments and extend ideas is both pro­duc­tive and enjoyable.
Your inter­view includes a ref­er­ence to an arti­cle by my col­leagues and me. I want to update the ref­er­ence and pro­vide a relat­ed ref­er­ence to a Web-based Archive.

She­bilske, W. L., Volz, R. A., Gildea, K. M., Work­man, J. W., Nan­janath, M., Cao, S., & Whet­zel, J. (2005). Revised Space Fortress: A val­i­da­tion study. Behav­ior Research Meth­ods, 37, 591–601.

Volz, R.A., John­son, J.C., Cao, S., Nan­janath, M., Whet­zel, J., Ioerg­er, T.R., Raman, B., She­bilske, W.L., and Xu, Dianx­i­ang (2005). Fine-Grained data acqui­si­tion and agent ori­ent­ed tools for dis­trib­uted train­ing pro­to­col research: Revised Space Fortress. Down Load Tech­ni­cal Sup­ple­ment, Psy­cho­nom­ic Soci­ety Web-based Archive (see 37,591–601). .

The jour­nal arti­cles’ abstract describes both:

Abstract
We describe briefly the rede­vel­op­ment of Space Fortress (SF), a research tool wide­ly used to study train­ing of com­plex tasks involv­ing both cog­ni­tive and motor skills, to exe­cute on cur­rent gen­er­a­tion sys­tems with sig­nif­i­cant­ly extend­ed capa­bil­i­ties, and then com­pare the per­for­mance of human par­tic­i­pants on an orig­i­nal PC ver­sion of SF with the Revised Space Fortress (RSF). Par­tic­i­pants trained on SF or RSF for 10 sets of 8 3‑min prac­tice tri­als and 2 3‑min test tri­als. They then took tests on reten­tion, resis­tance to sec­ondary task inter­fer­ence, and trans­fer to a dif­fer­ent con­trol sys­tem. They then switched from SF to RSF or from RSF to SF for two sets of final tests and com­plet­ed rat­ing scales com­par­ing RSF and SF. Slight dif­fer­ences were pre­dict­ed based on a scor­ing error in the orig­i­nal ver­sion of SF used and on slight­ly more pre­cise joy­stick con­trol in RSF. The pre­dic­tions were sup­port­ed. The SF group start­ed bet­ter, but did worse when they trans­ferred to RSF. Despite the dis­ad­van­tage of hav­ing to be cau­tious in gen­er­al­iz­ing from RSF to SF, RSF has many advan­tages, which include accom­mo­dat­ing new PC hard­ware and new train­ing tech­niques. A mono­graph that presents the method­ol­o­gy used in cre­at­ing RSF, details on its per­for­mance and val­i­da­tion, and direc­tions on how to down­load free copies of the sys­tem may be down­loaded from www.psychonomic.org/archive/.

The extend­ed capa­bil­i­ties for RSF include a) being exe­cutable on cur­rent gen­er­a­tion plat­forms, b) being writ­ten in a most­ly plat­form inde­pen­dent man­ner, c) being exe­cutable in a dis­trib­uted envi­ron­ment, d) hav­ing hooks built in for the incor­po­ra­tion of intel­li­gent agents to play var­i­ous roles, such as part­ners and coach­es e) pro­vid­ing a gen­er­al exper­i­ment def­i­n­i­tion mech­a­nism, f) sup­port­ing team­work through being able to flex­i­bly assign dif­fer­ent input con­trols to dif­fer­ent mem­bers of a team, g) main­tain­ing all data in a cen­tral data­base rather than hav­ing to man­u­al­ly merge data sets after the fact, and h) hav­ing play­back capa­bil­i­ty, which enables researchers to review all actions that occurred dur­ing an exper­i­ment and to take new mea­sures. Exper­i­menters can design mea­sures before an exper­i­ment to test spe­cif­ic hypothe­ses with a rig­or­ous lab­o­ra­to­ry task. They can also use play­back to dis­cov­er and explore unan­tic­i­pat­ed events. Although sim­pler and more com­plex syn­thet­ic task envi­ron­ments can be advan­ta­geous for some goals, Dan­ny Gopher, our col­leagues, and I believe that Space Fortress remains an impor­tant tool for sci­en­tists and train­ers. Please feel free to con­tact me (wayne.shebilske@wright.edu) for addi­tion­al help down­load­ing and using RSF.”