Long-time readers of this blog will know that plagiarism is a topic that comes up with some regularity, sometimes fueled by “kids today!” stories from the mainstream media, and sometimes due to actual research on plagiarism in different educational and professional spheres.

Today, let’s have a look of a report of one such investigation, “Plagiarism in Residency Application Essays,” published July 20, 2010 in Annals of Internal Medicine. The investigators looked at the personal statements applicants wrote (or, in some cases, “wrote”) as part of their application to residency programs at Brigham and Women’s hospital. As they describe their study:

The primary goals of this investigation were to estimate the prevalence of plagiarism in applicants’ personal statements at our institution and to determine the association of plagiarism with demographic, educational, and experience related characteristics of the applicants. (112)

The people applying to residency programs have already successfully completed medical school. The residency is an additional part of their training to help them prepare to practice a particular medical specialty. And, the personal statement is a standard part of what’s involved in applying for a residency:

All applicants to U.S. residency programs must complete an original essay known as the “personal statement.” The format is free-form, the content is not specified, and expectations may vary by specialty. Common themes include the motivation for seeking training in a chosen specialty, the factors that affect suitability for a field or program, a critical incident that affected the applicant’s career choice, and circumstances that distinguish the applicant from others. (112)

There are some fairly commonsense reasons to expect that these personal statements ought to be original work, written by the applicant rather than copied from some other source. After all, the personal essay represents the applicant to the residency program, not as a transcript or a set of test scores but as a person. The essay gives insight into why the applicant is interested in a particular medical specialty, what training experiences and life experiences might bear on his or her motivation or likelihood of success, what kind of personal qualities he or she will bring to the table.

Also, since plagiarism is explicitly forbidden, these essays may give insight into the applicant’s personal and academic integrity, or at least into his or her grasp of rudimentary rules of scholarship:

The ERAS [Electronic Residency Application Service] also warns applicants that “any substantiated findings of plagiarism may result in reporting of such findings to the programs to which [they] apply now and in the future”. Applicants must certify that work is accurate and original before an ERAS application is complete. (112)

In the study, the investigators performed an analysis of the personal statements in residency program applications to Brigham and Women’s Hospital over an interval of about 18 months. They analyzed 4975 essays using software that compared them with a database that included previously submitted essays, published works, and Internet pages.

For the purposes of the study, the researchers defined evidence of plagiarism as a match of more than 10% of an essay to an existing work. Since the software was flagging matching strings of words between the essays and the sources in the database, this methodology may well have missed instances of plagiarism where the plagiarist changed a word here or there.

It’s also worth noting that the authors point, in the Discussion section of the paper, to the following definition of plagiarism:

Plagiarism may be defined as “the action or practice of taking someone else’s work, idea, etc., and passing it off as one’s own; literary theft”. (114)

This definition seems (at least to my eye) to make intent an element of the crime. As we’ve discussed before, this requirement is by no means a standard part of the definition of plagiarism.

What did this research find? In the 4975 essays analyzed, they detected evidence of plagiarism (i.e., a match of more than 10%) in 5.2% of the essays, for an incidence of a little more than one plagiarized paper in 20. Rather than relying solely on the software analysis, the researchers examined the essays the software flagged for plagiarism to rule out false positives. (They found none.)

I’m not sure whether this frequency of plagiarism is unusually high (or unusually low). However, for a personal statement, I reckon this is higher than it should be. Again, what better source could there be for your personal statement than yourself? Still, we might want some data on the frequency of plagiarism in personal statements for other sorts of things to get a better sense of whether the results of this study indicate a special problem with people applying for medical residencies, or whether they reflect a basic human frailty of which people applying for medical residencies also partake.

The authors also report demographic trends that emerged in their results. They found a higher incidence of plagiarism among the applicants who were:

• international (which included non-U.S. citizens and those who had attended medical school outside the U.S.)
• older
• fluent in languages other than English
• applying for a residency with previous residency training under their belts

They found a lower incidence of plagiarism among the applicants who:

• were members of Alpha Omega Alpha (a medical honor society)
• had research experience
• had volunteer experience
• had higher scores on the U.S. Medical Licensing Exam Step 1

The authors offer no hypotheses about causal mechanisms that might account for these correlations, and it seems likely that more research is required to tease out the factors that might contribute to these demographic differences, not to mention strategies that might address them. (I’m guessing that the applicants with research experience and/or volunteer experience had an easier time finding stuff to write about in their personal essays.)

One might reasonably ask whether plagiarism in these personal essays is a problem that ought to worry those training the next generation of physicians. The authors of this study argue that it is. They write:

First, residency selection committees would probably find misrepresentation on the application to be a strong negative indicator of future performance as a resident. The Accreditation Council for Graduate Medical Education has deemed professionalism 1 of the 6 core competencies to be taught and assessed in undergraduate and graduate medical education. We believe that program directors would find a breach of professionalism in an application to be an unacceptable baseline from which to begin residency. Second, lapses in professionalism in medical school and residency training can be predictive of future disciplinary action by state medical boards. Third, increasing public scrutiny of physicians’ ethical behavior is likely to put pressure on training programs to enforce strict rules of conduct, beginning with the application process. (114-115)

The presumption is that honesty is a quality that physicians (and those training to be physicians) ought to display — that there is something wrong with lying not only to the patients you are treating but also to other members of your professional community. Indeed, the “professionalism” to which the authors refer is important in large part because it allows member of the larger public to recognize the professional community of physicians as possessing the necessary skills, judgment, and trustworthiness. Without this recognition, why should your average patient trust an M.D. any more than a snake-oil salesman?

In this study, as in all studies with human subjects, the researchers were required to look out for the interests of their human subjects — here, the applicants to the residency programs who wrote the personal essays that were analyzed. Protecting their interests included maintaining the anonymity of the authors of the essays in the context of the study. This, in turn, means that it’s possible that the plagiarism identified in the study may not have been identified by the residency selection committees who were also reading these essays.

Finally, near the end of the paper, the authors offer recommendations for how to address the general problem of plagiarism in applications for residency programs:

Ideally, the submission of applicant essays for comparison in a centralized database would occur at the level of ERAS, which would make this process unavoidable for applicants.This method also would eliminate the difficulties inherent in having multiple institutions using plagiarism detection software programs simultaneously, because submitted essays become part of the database for future submissions. Furthermore, manual inspection of the similarity report itself rather than simply reporting the score would allow individual program directors to make independent judgments about the seriousness of any putative offense. Finally, the mere knowledge that essays are being screened by plagiarism-detection software may substantially deter would-be plagiarizers. (119)

These recommendations are clearly leaning toward detecting plagiarism that has been committed, rather than being weighted towards prevention efforts. As they note, and as other researchers have found, an expectation that there will be a plagiarism screening may discourage applicants from committing plagiarism, but it’s possible that prevention efforts that depend on fear of detection may just end up separating the risk averse applicants from the gamblers.

Segal S, Gelfand BJ, Hurwitz S, Berkowitz L, Ashley SW, Nadel ES, & Katz JT (2010). Plagiarism in residency application essays. Annals of internal medicine, 153 (2), 112-20 PMID: 20643991