Rankings and report cards

This column was originally published in AMA eVoice on July 26, 2007. Dr. Davis is president of the American Medical Association.

Readers love rankings and report cards. Every week seems to bring us another list of the best or worst of whatever.

Michael Moore's new movie Sicko highlighted the World Health Organization's ranking of the best health care systems in the world. In my review of the movie in an earlier column, I cited a few compelling critiques of the WHO's ranking scheme.

On June 13, the Commonwealth Fund issued a ranking of health system performance in the 50 states, based on 32 indicators related to access, quality, avoidable hospital use and costs, equity, and healthy lives. Hawaii and Iowa ranked No. 1 and 2, respectively, whereas Mississippi and Oklahoma tied for last.

The United Health Foundation published a 2006 ranking of the health status of the 50 states, from the healthiest to the least healthy. It showed Minnesota and Vermont as the two healthiest states and Mississippi and Louisiana as the two unhealthiest.

States get grades for specific areas within health care, health policy, and public health. The National Alliance on Mental Illness has issued an analysis of states' mental health care systems; no states received an "A," five earned a "B," and eight got an "F." In January, the American Lung Association released its fifth annual report card on states' tobacco control policies; 15 states received an "A" but 23 got an "F." The AMA's medical liability crisis map shows 17 states in crisis, 25 states shaded yellow for caution, and eight states with a stable liability climate.

For the past nine years, Men's Fitness magazine has ranked the top 25 "fittest and fattest cities in America," based on factors such as the availability of public recreational facilities, how much exercise people get, and people's diet and TV–viewing habits. In 2004, Detroit (where I work) was ranked as the "fattest" city—a label I used with local colleagues to obtain grant funding for childhood obesity prevention programs.

U.S. News and World Report ranks the best colleges, graduate schools, health plans, and hospitals. In its July 23–30 issue, the magazine published its 18th annual "exclusive rankings" of America's best hospitals and medical centers.

Some health care accreditation organizations issue report cards. The Joint Commission gives four different grades to hospitals and other health care institutions for specific services and programs: a star ("best possible results"); or a plus, a checkmark, or a minus (better than, similar to, or worse than the performance of most accredited organizations, respectively). The National Committee for Quality Assurance issues health plan report cards, comparing a health plan's performance to that of other plans in the region or nation, and giving it ratings (from one to four stars) across several domains, including access and service, qualified providers, chronic disease management, and prevention.

Of course, there are many rankings of physicians. Books and Web sites provide listings of "America's top doctors," and many local magazines—for example, New York Magazine, Chicago Magazine, and Hour Detroit—rank the "top doctors" in the region within various specialties. Modern Physician and Modern Healthcare have ranked the "50 most powerful physician executives" and the "100 most powerful people in healthcare," based on online voting by the magazines' readers.

Athenahealth ranks health insurer performance based on several metrics including speed of payment, denial rate, and deviation from national coding standards. A magazine article about the study noted, "They've been ranking physicians for years now. Isn't it time someone ranked them?" The three leading national payers, according to the 2006 ranking, were Cigna, Aetna, and Medicare Part B.

On July 16, USA Today presented a list of "the top 25 medical developments" since 1982. In January, the BMJ conducted an online poll to determine "the most important medical advance since 1840" (sanitation was the winner). As the 20th century came to a close, the Centers for Disease Control and Prevention's Morbidity and Mortality Weekly Report profiled "10 great public health achievements" in the United States from 1900 to 1999.

Some of these ratings are intended to increase accountability in health care and public health, and to stimulate performance and policy improvement. Other scores are designed to sell publications, titillate readers, boost advertising revenue, or garner publicity for the sponsoring organizations. In most cases, the impact of these rating systems has not been evaluated. Do they stimulate positive change? Do they have unintended negative consequences? Are their criteria valid and their grades accurate?

What do you think about this medley of measurement? Which ranking system is the best and which is the worst? What needs to be ranked that isn't already? Perhaps pay-for-performance programs should be graded based on the AMA's detailed principles and guidelines.

Please send comments, questions, and replies to amaprez@ama-assn.org.