How to Become a Critical Reader of Academic Research
While social science has provided enormous insight to help managers be more effective, not all research is good research. Here’s how to be a smart consumer of new ideas.
Carolyn Geason-Beissel/MIT SMR | Getty Images
The gravitas of published academic research can lead organizational leaders to trust social science researchers’ findings and adjust their own practices accordingly. But scandals within the academic community have diminished some of the credibility of management research. Cautious leaders can adopt three practices to vet advice: Read the research critically, find out whether the results have been replicated, and test ideas before implementing them broadly.
On July 15, 2021, I received an email from three coauthors of a data-sleuthing website called Data Colada. They informed me that they would soon be publishing compelling evidence that a field experiment in my coauthored 2012 research paper “Signing at the Beginning Makes Ethics Salient and Decreases Dishonest Self-Reports in Comparison to Signing at the End” was based on fabricated data.1
What had popularly and prominently become known as the “signing first” paper claimed to show that when people sign a statement promising to tell the truth before they fill out a form, such as a tax or insurance form, they provide more honest information than when asked to sign such a statement after providing the requested information. The signing-first paper was widely seen as an effective means of “nudging” more ethical behavior. The paper was widely disseminated in the business world, and many organizations (including Slice Insurance, a consulting client of mine) had taken our advice and moved the signature lines in their forms to try to improve people’s honesty.
Data Colada’s allegations of fraud shocked me. Though the possibility of fraud had never occurred to me, I had tried and failed in the years before I heard from Data Colada to replicate the signing-first effect with colleagues not connected to the original paper. In 2020, after inviting the original authors to join us, we published a paper about this failure to replicate the results, and I tried to convince my coauthors on the original paper that we should retract the 2012 paper.2 The majority were against retraction.
My 2012 coauthor Dan Ariely of Duke University had claimed to have gathered the data for the field experiment from Hartford Insurance, but this now seemed in doubt. The Data Colada authors confided that they also had suspicions about one of the paper’s other experiments, whose data had been provided by my Harvard Business School colleague Francesca Gino. Gino had been my advisee and had become my coauthor, friend, and peer. I was shaken to hear that Data Colada suspected she had committed fraud on this and at least three other papers.
References
1. L.L. Shu, N. Mazar, F. Gino, et al., “Signing at the Beginning Makes Ethics Salient and Decreases Dishonest Self-Reports in Comparison to Signing at the End,” Proceedings of the National Academy of Sciences 109, no. 38 (Sept. 18, 2012): 15197-15200.
2. A.S. Kristal, A.V. Whillans, M.H. Bazerman, et al., “Signing at the Beginning Versus at the End Does Not Decrease Dishonesty,” Proceedings of the National Academy of Sciences 117, no. 13 (March 16, 2020): 7103-7107.
3. R. MacCoun and S. Perlmutter, “Blind Analysis: Hide Results to Seek the Truth,” Nature 526, no. 7572 (Oct. 8, 2015): 187-189.
4. These journals were Psychological Science, the Journal of Personality and Social Psychology, and the Journal of Experimental Psychology: Learning, Memory, and Cognition.
5. A.A. Aarts, J.E. Anderson, C.J. Anderson, et al., “Estimating the Reproducibility of Psychological Science,” Science 349, no. 6251 (Aug. 28, 2015).
6. D.T. Gilbert, G. King, S. Pettigrew, et al., “Comment on ‘Estimating the Reproducibility of Psychological Science,’” Science 351, no. 6277 (March 4, 2016): 1037.
7. M.H. Bazerman, “Inside an Academic Scandal: A Story of Fraud and Betrayal” (Cambridge, Massachusetts: MIT Press, 2025).
8.Ibid.