After the mirror neurons saga, the neuromania/neurophobia affair and the dyslexia debate, another topic is worthy of attention and will fascinate the readers: bilingualism and its supposed cognitive benefits!
The beginning was a meta-analysis of the published studies from conference abstracts, showing a publication bias towards a bilingual advantage compared to a disadvantage or no effect of bilingualism.
Then, a couple of weeks ago Psychological Science published two Commentaries: a reply by Bialystok et al. and a counter-response by de Bruin et al.
I've selected some passages from these two commentaries, putting them together as in a direct dispute.
I. The Beginning
Cognitive Advantage in Bilingualism: An Example of Publication Bias?
de Bruin A, Treccani B, Della Sala S
Psychological Science OnlineFirst, published on December 4, 2014 as doi:10.1177/0956797614557866
We analyzed conference abstracts presented between 1999 and 2012 on the topic of bilingualism and executive control. Conference abstracts were classified on the basis of their outcome. We observed an effect of result type on publication: Studies were published relatively often (68%) if the data demonstrated a bilingual advantage. In contrast, only 29% of the studies that showed no effect of bilingualism or even a bilingual disadvantage were published.
This difference in publication percentage based on the outcomes of the study could be the result of a bias during several steps of the publication process: Authors, reviewers, and editors can decide to submit or accept only studies that showed positive results.
While we agree that bilingualism should be conceived, a priori, as a positive and desirable achievement, we are also convinced that educational and political debates addressing the relevance of bilingualism should not be promoted by ignoring null or negative results. Instead of selecting exclusively those tasks and results that support current theories, investigators should attempt to include all conducted tasks and reported findings. On the other hand, reviewers and editors should be more open to studies that challenge the existing theories, especially when these are not yet fully established.
II. The Reply
Publication Bias and the Validity of Evidence: What’s the Connection?
Bialystok E , Kroll JF, Green DW, MacWhinney B, Craik FIM
Psychological Science OnlineFirst, published on May 5, 2015 as doi:10.1177/0956797615573759
III. The Counter-response
The Connection Is in the Data: We Should Consider Them All
de Bruin A, Treccani B, Della Sala S
Psychological Science OnlineFirst, published on May 5, 2015 as doi:10.1177/0956797615583443
Bialystok et al. The implication of their argument is that this publication bias invalidates the credibility of the positive published evidence. These are serious charges and need to be carefully scrutinized.
De Bruin et al. We are not part of a camp that is for or against the bilingual-advantage hypothesis and have in fact published a study supporting this hypothesis ourselves (Treccani, Argyri, Sorace, & Della Sala, 2009).
Bialystok et al. Their conclusion, however, is undermined by three errors in reasoning that concern (a) the relation between conference abstracts and published articles.,
[The authors] used conference abstracts as a proxy, but there is no way of knowing how many abstracts that reported positive results and how many that reported negative results were submitted for publication.
De Bruin et al. […] in our article, we sent out a short ad hoc questionnaire to all first authors of unpublished abstracts. Unfortunately, 33 of the 52 authors contacted did not reply or refused to fill in the questionnaire... [...] This suggests that more than half of the null or negative findings had not been submitted.
Bialystok et al. (b) the difference between null and negative findings.
If the main point of the article is to demonstrate that journals prefer to publish articles that show significant effects over those that show no effects, then their point is correct, and no further analyses are needed.
[…] It seems, therefore, that the real purpose of the de Bruin et al. article is to use publication bias as a means of discrediting evidence for bilingual effects on cognition.
De Bruin et al. In our article, we clearly acknowledged that an equal number of disadvantages and advantages should be found if there is no advantage at all.
[…] in the absence of a theoretical motivation for monolingual executive-control advantages, a bilingual disadvantage is likely to be interpreted as the result of a Type I error.
Bialystok et al. (c) the differential effects of bilingualism on verbal and nonverbal task performance.
The finding of fewer cases of published negative results is not bias but rather a reflection of the use of verbal tasks.
De Bruin et al. [...] we did not include conference abstracts that discuss lexical tasks without an executive-control component. We did include executive-control tasks with verbal materials, in which, according to the studies mentioned by Bialystok et al. themselves (e.g., Bialystok, 2009), bilinguals should outperform monolinguals.
However, when we do exclude all studies using verbal materials from our analysis [...] The outcome is not affected by the nature of the materials.
Bialystok et al. [...] a large and growing body of literature not only indicates that these changes are real but also increasingly points to the brain mechanisms by which these changes occur (Green & Abutalebi, 2013).
De Bruin et al. We agree that there are many interesting and compelling data supporting an advantage (e.g., Pliatsikas, Moschopoulou, & Saddy, 2015). Yet evidence for a bilingual advantage is far from accepted wisdom. Indeed, the number of studies challenging such an advantage has recently been increasing (e.g., Lawton, Gasquoine, & Weimer, 2014; Paap, Johnson, & Sawi, 2014).
And to conclude:
Bialystok et al. Not every study of climate change that samples weather conditions from a specific time and place finds that temperatures are rising, but researchers (mostly) accept that climate change is occurring because of global warming.
De Bruin et al. Publication biases may be worsened by researchers’ own prejudices or agenda. Bialystok and her colleagues have doubted the value of null results, asserting that “the considerable literature that reports group differences between monolingual and bilingual participants is greatly more informative than the attempted replications that fail to find significance” (Kroll & Bialystok, 2013, p. 502). [...] We disagree with the logic underlying this supposition.
I highly recommend to read these articles.
On this topic:
- Is bilingualism really an advantage by Maria Konnikova
- My conversation with Barbara Treccani (in Italian)