Physician Ranking Websites – The Bad, The Worse and the Ugly

On the internet, there is a website that ranks just about anything you would to be ranked, reviewed, prioritized, commented on, etc. Turns out that Physicians are no different, but as is often the case in healthcare, these websites are just getting started. I’m not sure all of the reasons why doctors weren’t being ranked and reviewed on websites before, but it definitely came later than many other industries. One reason this might be the case is that it’s a bad idea to try and create a website for physician reviews.

I realize this will be a position that many will disagree with (particularly those websites who review physicians). In fact, Jennifer Dennard wrote on the sister site to EMR and HIPAA the following comment:

Healthcare will become more affordable once consumers start making an effort to patronize providers that have a reputation for high patient satisfaction and quality scores. Get engaged via websites like Healthgrades.com to start sifting through local MDs’ scores and reviews.

While I agree with Jennifer that consumer involvement in their healthcare could eventually make healthcare more affordable, I don’t see websites like Healthgrades or similar sites as achieving that goal. The problem is that these physician ranking websites don’t have the right data to be able to rate physician’s effectively.

The first challenge is that most people don’t use these websites. The closest physician review website that comes close to having a critical mass of reviews is probably Yelp in San Francisco. That might be an interesting case study if we want to evaluate the value of user reviews of doctors. Although, I know people who have been “gaming” that system for a long time. The gaming elements aside for now, is it fair to grade a physician based on a small handful of patient reviews when the doctor is seeing 10-15 patients a day? The answer is no.

Let’s hypothetically say that we overcome the issues of not enough patient reviews. Unfortunately, the reviews still have a huge number of problems associated with them. Let’s take a look at some of the larger ones.

Patients Rate Customer Service, Not Quality of Care
One of the biggest problems with review sites is that most of the ratings and reviews reflect the customer service that a patient was offered. The ratings and reviews almost never reflect the quality of the care that the patient received. The customer service that a patient received does matter and should influence which doctor you see. A physician’s bedside manner should be an important part of the decision of which doctor you see. The problem is that most physician review sites give the impression to users that the ranking is more than just customer service. In fact, I expect many who read those sites equate a high ranking with the quality of care a physician provides. This is just not the case.

Can a Patient Rate a Physician on Quality of Care?
I can’t answer this for all patients, but I know the answer for me is no. I even work in this industry and if I’m being honest I really have very little idea of whether a doctor is providing better care than I would have gotten somewhere else. Other than really egregious stuff, how would I know? The whole reason I go to the doctor is because they know something that I don’t know. Sure, you can find a lot of info online and have some really great, informed discussions with doctors, but I expect most patients don’t know the quality of care they’re getting from a doctor.

Even many doctors and nurses don’t know how good their colleagues are at what they do. They certainly know better than patients, but unless they’ve worked with them on a regular basis across multiple patients how would they really know either? Plus, it’s amazing how little things can bias someone to how good or bad someone is at what they do. This is particularly true when you’re analysis of someone is based on only a few data points.

Positive and Negative Review Bias
One challenge with review sites in general is what I call positive and negative review bias. Take a look at any review site and you’ll see what I mean. The only people who take the time to leave a review are those who are extremely displeased with something (negative review) or those who had an extremely positive experience (positive review). That’s such a small percentage of the people who are seen that we have to be careful to interpret the skewed data.

The reality is that the majority of patients fall in the middle. They didn’t have an amazingly good or bad experience. They just got what they needed and moved on with their life. None of these people are motivated to review a doctor.

Gaming the Review Sites
Another enormous challenge to review sites is from people trying to game the reviews. I once saw the founder of Travel Advisor talk about this challenge on their site (they reviewed hotels) and the enormous amount of resources they put to combat it. They never got this perfect, but they also had enough scale that they could largely overcome this problem. Think about how hard it is for a website to know how authentic a physician review is. How do they know a positive review is not the doctor or practice manager using a fake email account? On the other side, a negative review might be a competitor using a fake email. Plus, there are a lot of other tricks to “game” physician ranking websites that are impossible for a website to detect.

Patient Perspective
Many healthcare visits are complex situations where the doctor has to inform you of something terrible. If your doctor had just told you about your STD diagnoses or just put you through a painful (albeit necessary) procedure, are you likely to go and give that doctor a high rating? In some cases you might, but most people are very emotional people and it’s hard for them to separate the situation from the person. Sometimes the treatments doctors provide might hurt in the short term in order to achieve some long term benefit. A patient suffering in the short term might not include the long term planning that a quality doctor is providing them when reviewing a doctor.

Phyisican’s Dislike Review Sites
Search Health IT recently wrote about an ACPE study where doctors almost universally thought doctor review sites were not useful. Here’s an excerpt from the article:

Only 12% of physicians believe websites where patients can review doctors are useful and should be made more available to patients, according to a study by the American College of Physician Executives (ACPE). A majority of respondents offered critical views of such review sites, with 29% saying they weren’t used enough by patients to be relevant, and 26% called them a “nuisance that provides no benefit.”

One physician explained he doesn’t check his online ratings because: “They don’t accurately reflect the competency of a physician.”

I don’t know anyone who likes to be reviewed, so the results shouldn’t be a surprise. Hopefully this post will help to add some depth to why many doctors don’t like physician review sites as opposed to the natural knee jerk reaction of not wanting to be reviewed.

The interesting challenge is that patients are going to continuing to look for ways to differentiate one doctor from another. It’s an incredible challenge. I love what Fred Trotter is doing with docGraph. It’s not a perfect model either, but at least he’s using data to try and differentiate doctors. I’d love to hear other ideas people have on how a patient can be sure they’re getting the doc that provides the highest quality care. I’ve wondered if use of technology and/or EMR choice/use could be an possible differentiator.