2024 Nepal USMLE Cheating Scandal Explained

An unprecedented medical school exam scandal left both US and international students wondering about the future of their medical education. Here is everything you need to know about the Nepal USMLE score scandal.

The United States Medical Licensing Examination, USMLE for short, has incredibly strict practices that ensure every medical student taking their board exams is on an equal playing field.

They go much further than checking if photo ID matches test registration and proctors to monitor unusual behavior. For USMLE, they go as far as to measure how long it takes to answer each question, in addition to comparing your correct and incorrect answer choices with other test takers in your vicinity.

They are extreme with their anti-cheating tactics because the USMLE is designed to determine who is qualified to be a physician and treat patients in the United States. Not having strict anti-cheating measures in place could result in patients being harmed if unqualified students become practicing doctors.

 

Nepal Scandal Background: Getting Everyone Up to Speed

At the beginning of 2024, the USMLE announced that a large number of test takers from Nepal had unusually high Step exam scores, and many had selected the exact same answer choices for the vast majority of questions, regardless of whether it was the correct answer or not. The odds of this occurring without cheating are near-impossible.

To catch everyone up to speed, we first need to look into how the Nepali test takers were cheating to get a sense of the scale of their operation. The evidence the USMLE released to the public suggests the examinees were most likely using “recalls”.

This is when examinees write down the questions they saw during their exam to later sell to others. This has been occurring since the dawn of the USMLE and will likely always continue. However, these recalls are usually fairly low quality, as they’re based on the test taker’s memory, and memories are fallible.

The other important detail to note is that the USMLE likely has fewer test questions than we think they do. Writing and validating questions that tens of thousands of people will interpret in nearly the same way, regardless of their individual biases, is not a simple task. It takes years of data and hours per question, which makes creating test questions a labor intensive and expensive process.

There is a great chance questions will be recycled from one exam to another, which makes recalls an unethical but lucrative business.

Some defending the Nepali test takers have argued these recalls are no different from resources like First Aid and UWorld; however, those resources are used to teach students medical concepts, whereas recalls are live USMLE test questions that are sold to classmates and future test takers. Since recalls are exact exam questions, they are often memorized rather than used as a tool to deeply understand the correct answer.

Naturally, creating, selling, and buying these recall questions is strictly against the rules of taking board exams, as is even speaking about questions you saw with friends. It’s part of the terms every test taker agrees to before they step foot inside the testing center and again right before they take the test.

In USMLE’s bulletin of information, they clearly state: “If you reproduce and/or distribute any examination materials, by any means, including by memorizing and reconstructing them, you are violating the legal rights of the USMLE program. The USMLE program will use every legal means available to protect the copyrighted materials and secure redress against those who violate copyright law.”

If you’re caught, you run the risk of losing your medical license and being barred from practicing medicine ever again.

Nepali test takers joined WhatsApp groups designed to help others earn the highest possible test score. The scale of this cheating operation is unprecedented, and at the release of this video, 832 test takers have had at least one of their board exams invalidated. The document, titled “Savior,” was over a thousand pages long and included over 90% of the live USMLE test questions.

The people who bought these perfect recalls, which were unauthorized materials, were able to memorize the questions and the correct answers before they ever entered the testing center. Looking at this graph, you can see how Nepal’s average score skyrocketed to well above any other country as a result of “Savior.”

Fortunately, the NBME was tipped off in January 2023. Due to the high number of suspicious scores from Nepali students, NBME focused its initial investigation on Nepal before launching a full investigation. They were able to join these WhatsApp groups to discover first-hand how accurate and how widespread the cheating was. In the year since those initial tip offs, the NBME notified individuals whose exams were invalidated.

And this is where things get interesting. One test taker chose to sue the NBME for discrimination after being identified as a potential cheater, claiming many Nepali students were being wrongfully targeted and that it violated their civil rights. This case went to court, and the judge ruled in favor of the NBME, saying the choice to look at Nepali students was based on data, not discrimination.

USMLE’s calculations found there was less than 1 in a hundred million chance that these test takers from the same region would have answers that matched so similarly. This agreement analysis proved that there was some sort of coordinated effort by cheating test takers, as while they disproportionately got the same answers right, they also disproportionately got the same answers wrong.

 

The Trial and Verdict

So, how did they get caught, and why was the motion to restore scores denied? The simple answer is math.

The emergency motion to restore score validity was denied by a Washington, D.C., district court judge due to the clear evidence of cheating.

Each USMLE question is designed to take 90 seconds, as that’s generally the amount of time it takes to fully comprehend the question and answer. The data shows this particular test taker got every single question right that they spent fewer than 20 seconds on while performing much worse on questions they took longer on.

Additionally, the test taker claimed that the reason it only took them 20 seconds was because they were quickly guessing. If this were true, they naturally would not have guessed correctly 100% of the time when choosing between 5 different answer choices.

Due to the strict anti-cheating measures USMLE exams have in place, the performance of test takers on stolen questions shared in WhatsApp groups can be compared to their performance on the non-compromised test questions.

If they perform approximately the same across all questions, it suggests the test taker was likely not cheating and raises the probability of a false positive in the USMLE’s system. However, if the test taker performs much worse on the non-compromised test questions, it’s a huge red flag that indicates access to past test questions.

With all of their data, it’s extremely easy for the USMLE to find what they called “a pattern of anomalous test performance,” suggesting “prior unauthorized access to secure exam content.”

For the suing test taker, USMLE found there was less than a 1 in a hundred million chance that they took the exam without having access to prior test questions. In real life, this is as close to an absolute guarantee they cheated as we can ever get.

 

What Happened to Suspected Cheaters?

The students who were suspected of having unauthorized access to past test questions have had their exams invalidated. Having their scores invalidated doesn’t mean they cheated; instead, they now need to prove they didn’t cheat.

Those who received a message about an invalidated score were given three options to choose from.

1. They could request that NBME reconsider its decision. This is a process that could take up to 10 weeks.
2. They could choose to retake the test(s), or
3. They could choose to do nothing.

For both options 1 and 2, the right to sue NBME would be waived.

There are many different outcomes, depending on where someone is in their medical education process.

If the identified cheater was aiming to match this year, they now will no longer be eligible for this cycle as they no longer have a validated test score. If they retake their exams, and if they legitimately pass, they will be allowed to apply to the Match again next year.

Of course, when they apply, program directors will be able to look at their entire USMLE transcript and see the highly suspect, invalidated score, indicating they likely cheated on their prior exam attempt.

However, the USMLE was able to go several years back to identify current residents and even attendings from Nepal who they think had access to secure exam content before they took their board exams. Because their ECFMG medical license, which is the licensing foreign medical graduates need to be able to practice in the US, depends on having valid, passing scores on USMLE Step 1, Step 2, and Step 3, these individuals will immediately lose their ability to practice in the US.

Without their licenses, the resident will be cut from the residency program, lose their work visa, and be required to leave the country, which could very likely end their medical career in the United States.

 

What This Means for Current and Future Students

So, what does this all mean for future med students?

If you’ve already taken your Step exams, you don’t need to worry that you might have received a worse score due to the abnormally high number of cheaters in the past few years. The curve the USMLE uses is only based on MD students in the US and Canada who are first time test takers. They set the curve for everyone else.

DOs, IMGs, FMGs, and even repeat test takers have no influence on the curve. If you already took the test honestly, you can be rest assured that the score you received is actually yours, and will not be invalidated.

Due to USMLE’s extremely high threshold and degree of certainty for catching cheating students, there will likely be some false negatives, which means some cheaters will slip through the cracks. However, this degree of certainly all but ensures no false positives will occur—meaning no one will be falsely accused of cheating.

Lastly, the unfortunate victims of this scandal are the patients who have been treated by unqualified residents and physicians, as well as the IMGs applying to match this year who were not involved in the cheating scandal.

Although there was no discrimination found in choosing to look at the Nepali students first, there could be unfortunate biases against other Nepali students who did not cheat.

Program directors may be more hesitant to accept residents from a certain region of the world, as they might fear their resident could be kicked out of their program years down the road. It’s not a risk many medical programs will want to take.

The USMLE will continue to heavily scrutinize test takers to ensure there is no variation in score reports. They have already collected large amounts of suspicious data in countries like Pakistan, India, and Jordan.

The organization will be aggressive in ensuring cheating is impossible on board exams, as this is one of the best ways to make sure the physicians practicing in the US are qualified to take care of patients here. In the wake of this scandal, it’s expected that they will create a great deal of new test questions, along with removing the compromised ones from circulation.

At Med School Insiders, we always aim to bring you the unbiased facts, but if you want my unfiltered take on this cheating scandal, check out my reaction video over on the Kevin Jubbal M.D. channel.