UW Researchers Find Writing Errors Can Lower Economists’ Perceptions of Research Quality

How does writing style influence economists’ perceptions of research quality?

University of Wyoming researchers discovered that even the slightest of spelling or punctuation errors in a manuscript can create a negative viewpoint from U.S. academic economists.

What’s more, Alexander James says many economists, at least in the international sample group, believe papers prepared using LaTeX are viewed more favorably rather than those prepared with Microsoft Word.

“The paper finds that academic economists view research papers with a spelling and/or grammar error less favorably than those without errors. In fact, economists even view the underlying research question being asked less interesting and important if the research includes an error,” says James, an associate professor in the UW Department of Economics. “More interesting, and surprising, is that the choice of document editor appears to matter -- among an international sample of academic economists, at least.”

James was lead author of a paper titled “Does LaTeX enhance perceptions of research quality in economics?” published in the October issue of the Journal of Economic Behavior and Organization. The journal is devoted to theoretical and empirical research concerning economic decision-making, organization and behavior, and to economic change in all its aspects.

 

Tyler Kjorstad, a UW assistant lecturer of economics and undergraduate director in the College of Business, was a co-author of the paper.

 

During the study, James and Kjorstad conducted two online survey experiments examining how writing style influences economists’ perceptions of research quality. According to the paper, participants were randomly assigned to evaluate manuscript cover pages that varied by document editor -- Microsoft Word versus LaTeX -- and by the presence of a spelling error, grammar error or no error.

 

In the original experiment, a sample of 441 U.S. academic economists -- out of 4,076 originally sent an email to participate in the survey -- were exposed to one of six cover pages crafted from an actual, accepted economics manuscript that had not yet been published. Spelling and grammar errors significantly reduced perceived research quality, whereas the choice of document editor had little effect, according to the paper.

 

In a follow-up experiment, conducted among an international sample of 311 economists who fully responded to the survey, an expanded design incorporating a second manuscript abstract was used. Cover pages varied according to whether they were prepared in Microsoft Word or LaTeX, and whether they included a spelling error, grammar error or no error, the study says.

 

In the follow-up experiment, LaTeX use was substantially more common, and manuscripts prepared in LaTeX were generally evaluated more favorably than those prepared in Microsoft Word, according to the study.

 

Subjects in both experiments evaluated only a manuscript’s cover page rather than the full paper.

 

“To the extent that treatment effects are attenuated when readers have access to the complete manuscript, our estimates should be interpreted as upper bounds on the effects of document editor and writing quality,” James wrote in the paper. “Accordingly, our experimental design is best understood as identifying how these features shape readers’ initial impressions.”

 

The study was designed to inform how subtle cues in writing quality and document preparation influence economists’ perceptions of research quality.

 

“Taken together, our findings suggest that careful proofreading and preparing manuscripts in LaTeX are prudent best practices,” James says. “Depending on the reader, both may improve perceptions of research quality while carrying little or no downside risk.”

James cautions, however, that the study notes it is unclear whether these findings extend to settings in which economists evaluate complete manuscripts rather than just abstracts. This remains an open question and represents a promising avenue for future research.