owen

Another victim of the BBC/Nature “brain training” experiment

by

Have you read the cov­er sto­ry of the New Sci­en­tist this week: Men­tal mus­cle: six ways to boost your brain?

The arti­cle, which includes good infor­ma­tion on brain food, the val­ue of med­i­ta­tion, etc., starts by say­ing that: “Brain train­ing does­n’t work, but there are lots of oth­er ways to give your grey mat­ter a quick boost.” Fur­ther in the arti­cle you can read “… brain train­ing soft­ware has now been con­signed to the shelf of tech­nolo­gies that failed to live up to expectations.”

Such claims are based on the one study wide­ly pub­li­cized ear­li­er this year: the BBC “brain train­ing” exper­i­ment, pub­lished by Owen et al. (2010) in Nature.

What hap­pened to the sci­en­tif­ic rig­or asso­ci­at­ed with the New Scientist?

As expressed in one of our pre­vi­ous posts: “Once more, claims seem to go beyond the sci­ence back­ing them up … except that in this case it is the researchers, not the devel­op­ers, who are respon­si­ble.” (See BBC “Brain Train­ing” Exper­i­ment: the Good, the Bad, the Ugly).

Read our two pre­vi­ous posts to get to the heart of the BBC study and what it real­ly means. As Alvaro Fer­nan­dez and Dr. Zelin­s­ki explore the poten­tial sci­en­tif­ic flaws of the study, they both point out that there are very promis­ing pub­lished exam­ples of brain train­ing method­olo­gies that seem to work.

BBC “Brain Train­ing” Exper­i­ment: the Good, the Bad, the Ugly

Sci­en­tif­ic cri­tique of BBC/ Nature Brain Train­ing Experiment

Scientific critique of BBC/ Nature Brain Training Experiment

by

logo-bbcThere has been quite a bit of com­ment about the Owen et al study in Nature avail­able online on April 20, 2010. A quick syn­op­sis of the study is that the BBC show Bang Goes the The­o­ry worked with the study authors to pro­vide a test of the hypoth­e­sis that com­mer­cial­ly avail­able brain train­ing pro­grams trans­fer to gen­er­al cog­ni­tive abil­i­ties. The con­clu­sion was that, despite improve­ments on the trained tasks, “no evi­dence was found for trans­fer effects to untrained tasks, even when those tasks were cog­ni­tive­ly close­ly related.”

The exper­i­ment

The study was con­duct­ed through the show’s web site. Of 52,617 par­tic­i­pants who reg­is­tered, approx­i­mate­ly 20% (11,430) com­plet­ed full par­tic­i­pa­tion in the study, which con­sist­ed of two bench­mark­ing assess­ments 6 weeks apart with vari­ants of neu­ropsy­cho­log­i­cal tests and at least two train­ing ses­sions. Peo­ple were ran­dom­ly assigned to one of three groups that were asked to train for about 10 min a day three times a week for the 6‑week peri­od, though they could train either more or less fre­quent­ly. One of the two exper­i­men­tal groups was a “brain train­ing” group that com­plet­ed tasks includ­ing sim­ple arith­metic, find­ing miss­ing pieces, match­ing sym­bols to a tar­get, order­ing rotat­ing num­bers by numer­i­cal val­ue, updat­ing, and mem­o­ry for items. Most of the train­ing ses­sions were 90 sec each; the rotat­ing num­bers tasks was 3 min. These activ­i­ties are sim­i­lar to those used in “edu­tain­ment” pro­grams that can be played online or with a hand­held device. The oth­er exper­i­men­tal group was trained on rea­son­ing tasks that involved iden­ti­fy­ing rel­a­tive weights of objects based on a visu­al “see­saw”, select­ing the “odd” item in a con­cept for­ma­tion type task, a task involv­ing think­ing through the effects of one action on cur­rent and future states, and three plan­ning tasks includ­ing draw­ing a con­tin­u­ous line around a grid while ascer­tain­ing that the line will not hin­der lat­er moves, a ver­sion of the Tow­er of Hanoi task, and a tile slid­ing game. The con­trol group spent time answer­ing ques­tions about obscure facts and orga­niz­ing them chrono­log­i­cal­ly based on any avail­able online resource. Results indi­cat­ed that the two exper­i­men­tal groups per­formed bet­ter than the con­trol group on only one out­come test of gram­mat­i­cal rea­son­ing; there were no dif­fer­ences between either exper­i­men­tal group and the con­trols on the remain­ing test. The exper­i­men­tal groups had improved on the trained tasks but not on the trans­fer tasks.

Sci­en­tif­ic concerns

Although some news reports sug­gest that these find­ings are defin­i­tive, there are a num­ber of con­cerns, many of which have to do with whether the find­ings have been over­gen­er­al­ized to all forms of brain train­ing because only a few tests were used. Sec­ond, there have been ques­tions raised about the amount of time allo­cat­ed to train­ing and the issue of test­ing in the home envi­ron­ment. The study report­ed [Read more…] about Sci­en­tif­ic cri­tique of BBC/ Nature Brain Train­ing Experiment