Should USMLE Step 1 Be Pass/Fail?

Per an official statement, the USMLE Step 1 may in the future be moving to a pass-fail scoring system. Would USMLE Step 1 becoming pass-fail be a good or bad thing? What are the pros and cons of each side of the argument?

What is USMLE Step 1?

The United States Medical Licensing Examination, or USMLE, is a 3-part test designed to assess one’s medical knowledge and application of said knowledge to patient care.

Step 1, as it currently stands, is arguably the most important test a future physician will ever take – even more important than the MCAT. It is typically taken at the end of the second year in medical school, and is followed by Step 2 in the third or fourth year. If you’d like to know more about the USMLE Step exams and their timelines, check out my 4 years of medical school explained post.

The USMLE was officially designed to assist state authorities in granting medical licenses, but it has grown in significance far beyond the intended pass-fail indication of competence. USMLE scores are now heavily weighted during residency applications to assess a candidate’s strength, and low scores often prohibit students from pursuing highly competitive specialties, such as dermatology or plastic surgery.

Before the $h!% Storm Starts

Before we dive in, I hope that this video serves as a lesson in how to approach controversial topics. When people asked me about where I stand on the issue, I said I don’t have a strong opinion yet. I need to first do my research, think through the subject, and follow up with questions I have. Too many people first choose a side based on limited information and then seek out data to bolster their argument – that’s the confirmation bias. Learn to think for yourself and not just echo what people around you are saying.

Also, a disclaimer. I am the founder of Med School Insiders, which offers pre-med and medical student admissions consulting and tutoring services. We do offer tutoring for Step 1, amongst other pre-med and medical student exams. That being said, it won’t influence my argument, as you’ll see. One of the biggest reasons I left plastic surgery was to have a positive impact on the medical education process, as I described in my video explaining my decision to leave the hyper competitive and highly lucrative field. My priority is helping medical trainees. I am a firm believer in objective and rational discourse not clouded by biases. Seek facts first, then develop an argument, not the other way around.

How Did We Get Here?

The bubbling conversation boiled over and blew up after the CEOs of the NBME and FSMB released this:

If students reduce time and effort devoted to preparing for Step 1, they may indeed devote attention to other activities that will prepare them to be good physicians… However, if students were to devote more time to activities that make them less prepared to provide quality care, such as binge-watching the most recent Netflix series or compulsively updating their Instagram account, this could negatively impact residency performance and ultimately patient safety.

How Dr.’s Katsufrakis and Chaudhry could release such a tactless statement as CEO’s to alienate their audience is beyond me. The ensuing conversation blew up in their face, with large criticism over the rapidly increasing annual compensation of the NBME executives. In short, given the current importance of USMLE Step 1, medical students and medical schools have been strongly incentivized to purchase more and more prep materials, such as practice tests, from the NBME, drastically increasing the organization’s revenue year after year.

As it stands, there are now two camps: those in favor of transitioning the USMLE to pass-fail, and those wishing to maintain the status quo with continuous scoring.

Arguments For Pass-Fail Grading

We’ve already touched on the outrageous profiteering of the not-for-profit NBME. But beyond that, there are several other reasons:

1 | Rift in Medical Education

First, the current Step 1 climate is highly stressful and creates a rift in medical education. A recent article in Academic Medicine states “many students opt to disengage from institutional curricula in favor of intensive exam preparation … and are rewarded with a high Step 1 score.”

2 | Impact on Disparities

Second, the importance of Step 1 scores is argued to reduce diversity for a few reasons. It contributes to a destructive culture of hierarchy among specialties, where you’re perceived as scoring higher on Step and therefore being “smarter” if you chose a surgical sub-specialty rather than primary care. Additionally, some claim that to perform well requires spending $1,000 on test prep materials that not all students are able to afford. Lastly, underrepresented minorities and those coming from low income families on average score lower on Step 1.

3 | Impact on Well-Being

Third, and in my opinion most importantly, Step 1 may be contributing to a harmful learning environment. Some say the Step 1 climate contributes to the mental health crisis affecting the medical community. As you all probably know, I’m incredibly passionate about addressing the epidemic of burnout, depression, and suicide amongst medical trainees and is why I started the #SaveOurDoctors movement.

4 | Not an Effective Predictor of Physician Ability

Despite Step 1 being a heavily weighted factor in residency admissions, it’s not a reliable indicator of one’s abilities as a physician. Rather than a USMLE Step 1 score, other important qualities, such as clinical reasoning, professionalism, and the ability to work in a team should be prioritized.

Arguments for Continuous Scoring (Status Quo)

Those in favor of maintaining the status quo with a continuous scoring system raise several counter-points:

1 | Step 1 Isn’t Perfect, but This Makes it Worse

Status quo proponents admit that USMLE Step 1 is far from perfect, but moving to a pass-fail system doesn’t address the underlying issue, and could actually make the residency selection process worse. To be an effective physician, we would need to reliably measure communication skills, empathy, professionalism, and other factors, but we currently do not have any test to adequately assess these. Pass-fail proponents point to emphasizing heavier weighting on clinical grades for evaluation, which brings us to the second point.

2 | Lack of Other Objective Measurements

In residency admissions, your Step 1 score is heavily weighted, likely weighted more than it should be. However, status quo proponents argue that Step 1 is the only objective measurement we have. With pass-fail during pre-clinical courses and highly subjective clinical evaluations that may be heavily influenced by chance, there is no other reliable objective measurement to replace it. There is also a complete lack of standardization among clinical clerkships, with some schools practicing hard quotas, but others handing out honors and high pass to almost every student, making the clerkship grade close to meaningless and a terribly inaccurate indicator of true performance.

3 | Levels the Playing Field

Those in favor of continuous grading argue that moving to a pass-fail grading system would lead to nepotism, making residency selection more about who you know or what school you came from. With continuous grading, students from less prestigious schools are able to get noticed should they excel on the exam. Under a pass-fail system, school prestige, your personal connections, and other far less equitable factors would be prioritized.

4 | Decreased Medical Knowledge

Some argue that with a pass-fail system, students would lose the motivation to study hard for Step and ultimately lack medical knowledge. While I understand this line of reasoning, it’s purely speculation and I’d argue we don’t actually know how medical knowledge would be influenced one way or the other.

Dr. Jubbal’s Take on Step 1

After considering both sides of the argument and upon closer examination, it’s apparent that the truth is somewhere in the middle, and I’m glad we’re having this conversation. It points to a larger issue. Medicine and medical education is broken, and we need to do something about it. Proponents of moving Step 1 to a pass-fail system are fed up, and we all should be, but that frustration is misguided. Moving Step 1 to a pass-fail system would only make the issues within medical education worse, not better.

If you’re confused, allow me to explain.

First, my most important consideration is the wellbeing of medical students and residents. Some would argue that patient care and safety is the first priority, but I say if you don’t have healthy doctors, you don’t get healthy patients. You must first take care of yourself before you’re able to take care of others. The fact that suicide rates are the highest amongst physicians is appalling. Again, I’d point you again to the #SaveOurDoctors movement. But while Step 1 is highly stressful, it’s not the cause of the burnout, depression, and suicide epidemic. It’s one element of a larger cultural issue, but pointing to a single standardized exam as the root cause is not only a naïve and misguided interpretation of the facts, but could ultimately prove harmful.

There are countless stories of students being devastated by a suboptimal performance on Step 1. Rather than blaming the test, take responsibility for your life. Figure out what went wrong, and act on it. Sometimes it’s factors outside of your control and life’s not fair. I get it. You can still take responsibility. Learn how to learn more effectively, learn how to be a better test taker, and then go crush your shelf exams, honor your clerkships, and ace Step 2CK. The 100% free advice on this blog has helped thousands of students radically improve their performance. Why wouldn’t it help you? The victim mentality is becoming more and more prevalent. Trust me, making Step 1 pass-fail won’t solve your issues. Residency will still be competitive under a different set of metrics, and you’ll blame those new metrics for your problems as well. I recommend you start with my Stoicism for Students post, which many of you have commented on and told me was my best ever post. Check out Jocko Willink’s Extreme Ownership while you’re at it too.

Several other arguments from the pass-fail proponents don’t hold any water. They state Step 1 implements a destructive hierarchy of specialties and a negative stigma. News flash, self-victimizing Victor, competitive things are competitive because of supply and demand. More competitive specialties will always be more competitive, regardless of Step 1. Making Step 1 pass-fail doesn’t change competitiveness. It just makes it so that other metrics replace Step 1. Even more importantly, why does it matter that someone assumes your Step score is lower because you went into primary care? Your self worth shouldn’t be tied to your Step score, let alone what someone assumes your Step score to be. Don’t blame medicine, that’s on you.

Pass-fail proponents quote that to perform well on Step 1, students must each spend $1,000, and this is a significant barrier to those from less financially privileged backgrounds. This doesn’t make sense for two reasons. First, the average medical student graduates with $200,000 in debt. There are many students who graduate without debt, further skewing the average, meaning the average debt for students that need to take out loans is actually much higher than $200,000. At that point, $1,000 is just a drop in the bucket. Plus, these costs are taken into account in your medical school budget and student loan offerings. Second, who says you need to spend $1,000? Between 6 months of UWorld, Pathoma, First Aid, and Sketchy, I spent less than $500. You could spend even less if you’re frugal and purchase second hand. I don’t come from a financially privileged background – I had to pay for my own college and medical school, and I had to take out loans. I was financially strapped yet still performed quite well on my USMLE.

Possibly the most frustrating aspect of this entire debate is how short-sighted it is. The problem isn’t Step 1, but rather the medical education and training system. Why do residencies place such emphasis and screen with Step 1? They have so many applicants to sift through and need an objective measurement to cut the pile down. Even with pass-fail grading, program directors face the same issue and will find other metrics to pare down the list. You don’t solve the problem with pass-fail, you simply shift it to something else. Pass-fail would place more weight on clerkships, which are much more subjective and biased. Which preceptor did you get? Were they the easy or hard one? Were you the type of personality that usually meshes well with them or did you get unlucky? With pass-fail, you place more importance on the number of publications, who you know, and what school you come from. Brown nosing would be even worse than it already is. Would you rather have meritocracy or nepotism? I’ve even heard some pass-fail proponents say we should place higher emphasis on Step 2CK or the shelf exams. The exact same problems would simply transfer over – it doesn’t take a rocket surgeon to know that.

Proponents also point to the fact many medical schools have transitioned the pre-clinical classes from graded to pass-fail, without any negative effect on academic achievement. I shouldn’t have to explain why that’s comparing apples to oranges. And lastly, while Step 1 used to be a pass-fail exam, that doesn’t mean it should be again. The residency selection process is an entirely different beast now, and no Step 1 didn’t cause those changes.

My Suggestions

I agree the system isn’t perfect and we need change. Here are my suggestions:

1 | Keep Step 1 as it Currently Stands

To change USMLE Step 1 to pass-fail would be short-sighted and harmful to future medical trainees.

2 | Put the NBME in Their Place

It’s appalling that the NBME, a not-for-profit organization, is making Scrooge McDuck look charitable. The fact the CEO tripled his salary in 15 years and is making over 1.2 million dollars is absurd. NBME resources, such as practice tests, should be offered for free or highly subsidized, as should the cost of exams. Their monopoly on medical education is being blatantly abused.

3 | Investigate Other Objective Assessments

Step 1 is far from perfect. While making it pass-fail is obviously not the answer, we should seek to find other objective metrics to more accurately assess a medical student’s aptitude. Only then can we consider making Step 1 pass-fail.

What are your thoughts? Are you for or against Step 1 being pass-fail? Let’s have a mature and rational discourse in the comments.