I was really inter­ested in the recent cri­tique of the BBC brain train­ing exper­i­ment by Dr. Eliz­a­beth Zelin­ski. I think Owens et al (2010) was a crit­i­cal piece of research which was not con­ducted in the right way and was focus­ing on the wrong sam­ple pop­u­la­tion.  I totally agree with the com­ments by Dr. Zelin­ski regard­ing the poten­tial for sam­ple bias and the use of some ques­tion­able cog­ni­tive mea­sures. How­ever, I would like to take this cri­tique fur­ther and ques­tion whether the study was value for money when there are other stud­ies which can­not achieve fund­ing but would, in my opin­ion, show the criticism/scepticism of the use-it-or-lose-it theory.

I think there is not enough crit­i­cism about the age of the sam­ple pop­u­la­tion used in Owens et al. (2010). We have con­clu­sive cog­ni­tive and neu­ro­log­i­cal evi­dence that cognitive/neurological plas­tic­ity exists in young adults. There is also ade­quate evi­dence that neu­ro­plas­tic­ity is evi­dent in older adults. The crit­i­cal point which I want to make about the sam­ple pop­u­la­tion in Owens et al. study is that it did not tar­get the cor­rect sam­ple pop­u­la­tion, that is, older adults who are at risk of cognitive/neuronal atro­phy. It does not mat­ter if younger adults improve on brain train­ing tasks, or if skills picked up by younger adults from brain train­ing are not trans­ferred to other cog­ni­tive domains, sim­ply because younger adults are good at these skills/cognitive func­tions. There­fore there is a pos­si­bil­ity that ceil­ing or scal­ing effects mask the true find­ings in Owens et al. (2010), as indi­cated by Zelinski.

The recruit­ment of the sam­ple pop­u­la­tion is also very con­cern­ing and I do not feel that their con­trol group was appro­pri­ate. Read the rest of this entry »

You may already have read the hun­dreds of media arti­cles today titled “brain train­ing doesn’t work” and sim­i­lar, based on the BBC “Brain Test Britain” experiment.

Once more, claims seem to go beyond the sci­ence back­ing them up … except that in this case it is the researchers, not the devel­op­ers, who are responsible.

Let’s recap what we learned today.

The Good Sci­ence

The study showed that putting together a vari­ety of brain games in one web­site and ask­ing peo­ple who hap­pen to show up to play around for a grand total of 3–4 hours over 6 weeks (10 min­utes 3 times a week for 6 weeks) didn’t result in mean­ing­ful improve­ments in cog­ni­tive func­tion­ing. This is use­ful infor­ma­tion for con­sumers to know, because in fact there are web­sites and com­pa­nies mak­ing claims based on sim­i­lar approaches with­out sup­port­ing evi­dence. And this is pre­cisely the rea­son Sharp­Brains exists, to help both con­sumers (through our book) and orga­ni­za­tions (through our report) to make informed deci­sions. The paper only included peo­ple under 60, which is sur­pris­ing, but, still, this is use­ful infor­ma­tion to know.

A TIME arti­cle sum­ma­rizes the lack of trans­fer well:

“But the improve­ment had noth­ing to do with the interim brain-training, says study co-author Jes­sica Grahn of the Cog­ni­tion and Brain Sci­ences Unit in Cam­bridge. Grahn says the results con­firm what she and other neu­ro­sci­en­tists have long sus­pected: peo­ple who prac­tice a cer­tain men­tal task — for instance, remem­ber­ing a series of num­bers in sequence, a pop­u­lar brain-teaser used by many video games — improve dra­mat­i­cally on that task, but the improve­ment does not carry over to cog­ni­tive func­tion in general.”

The Bad Sci­ence

The study, which was not a gold stan­dard clin­i­cal trial, con­tained obvi­ous flaws both in method­ol­ogy and in inter­pre­ta­tion, as some neu­ro­sci­en­tists have started to point out. Back to the TIME article:

“Kling­berg (note: Torkel Kling­berg is a cog­ni­tive neu­ro­sci­en­tist who has pub­lished mul­ti­ple sci­en­tific stud­ies on the ben­e­fits of brain train­ing, and founded a com­pany on the basis of that pub­lished work)…criticizes the design of the study and points to two fac­tors that may have skewed the results.

On aver­age the study vol­un­teers com­pleted 24 train­ing ses­sions, each about 10 min­utes long — for a total of three hours spent on dif­fer­ent tasks over six weeks. “The amount of train­ing was low,” says Kling­berg. “Ours and oth­ers’ research sug­gests that 8 to 12 hours of train­ing on one spe­cific test is needed to get a [gen­eral improve­ment in cognition].”

Sec­ond, Read the rest of this entry »