U.S. News & World Report has established itself as a well known arbiter of measuring how colleges and universities stack up against each other. And now it’s in the process of developing a method for evaluating law schools based on their “scholarly impact” — how often faculty members publish scholarly articles and how many citations their work generates over a number of years.
Although U.S. News is jumping into this particular research eld now, comparing law schools by how often their faculty publish and get cited isn’t new. Brian Leiter of the University of Chicago is known for launching this type of study of law professors. Gregory Sisk of the University of St. Thomas has carried on the study with a group of co-authors, and he estimated he’s been working on it for about 10 years.
Sisk and his colleagues use Westlaw to compile their research, which they use to update their rankings every three years.
“It’s appropriate to evaluate whether those law professors who are claim- ing to be doing scholarship and trying to make a difference in that regard are actually accomplishing something,” he said. “Is somebody really reading their work, or are they just writing articles that end up on a shelf somewhere that no one ever takes a look at?”
Sisk said the scholarly citations research isn’t trying to hold itself out as the single best way to measure the quality of law schools, or as the only
way to measure scholarly impact. “The idea was not to come up with a ranking that would be everything to everybody, which U.S. News tried to do, but rather speak to one aspect of legal education,” he said. “In contrast with U.S. News, which suggests their ranking is universal and pulls in everything that’s of value to evaluating a law school. … Our view is this is one part of the cathedral that should betaken a look at.”
According to initial reports, U.S. News will keep its scholarly impact ranking separate for its existing “Best Law Schools” ranking — at least at first. U.S. News announced it will partner with William S. Hein & Co. Inc to produce the research. It’s still unclear whether U.S. News will eventually integrate data on scholarly impact into the factors for its existing ranking. Bruce Smith, dean of the University of Denver Strum College of Law, spoke about the challenges of deciding who in a law school’s faculty should be included in the data collection on scholarly impact. Framing it by tenured and tenure-track faculty, he said as an ex- ample, risks under-reporting the impact of faculty members who publish often but aren’t tenured. It also can over-represent the impact on a law school’s overall score of tenured faculty, whose responsibilities are mostly teaching.
“Teaching faculty … really shouldn’t be measured along the lines of their scholarly productivity,” Smith said.