A Machine Learning Model Finds Supreme Court Justices Conceding to Rivals They Publicly Attack

Somewhere in the 34,112 passages a pair of political scientists pulled out of six decades of Supreme Court opinions is a sentence in which Justice John Paul Stevens, midway through tearing apart a mandatory sentencing ruling, stops to say the majority happens to be right about something. Not as a rhetorical feint that anyone flagged at the time. Just a concession, dropped into a dissent, and then the argument moves on. Multiply that moment by tens of thousands and you get a pattern nobody had counted.

Donald Snyder and Caitlyn Pierce, political scientists at the University of Massachusetts Amherst, have now counted it. Their measure, published in American Politics Research, puts a number on something court-watchers have always sensed but could never quite pin down.

They call it appeasement, which is a slightly loaded word for what it actually describes: the ratio of moments when a dissenting or concurring Supreme Court justice grants the majority a point to the moments when they refuse to grant anything at all. The pair assembled 9,049 non-majority opinions written between 1946 and 2011, then wrote software to yank out 50-word windows around every mention of “the Court” or “the majority.” That produced 60,637 candidate passages. Human coders worked through 2,500 of them by hand, sorting concession from non-concession, and a second coder checking a subset agreed 97 per cent of the time.

Then the hand-coded set went into LEGAL-BERT, a language model pre-trained on legal text and, one assumes, better at parsing judicial syntax than a general-purpose model would be. It reached 98.6% accuracy on held-out cases.

What emerged is a portrait of the court that looks nothing like the one on cable news. Justices who savage each other in public do, quite routinely, hand each other things.

“Our measure is unique in that we’re not just looking at how harsh justices’ language is, but how they change their language over time,” Pierce says.

The Bridges You Might Need Again

The distinction matters more than it might sound. An earlier and widely used measure, developed by Michael Zilis and Justin Wedeking, tallies harsh and unpleasant words, and it works by dictionary lookup: count the nasty terms, get a score. Appeasement and that harshness measure turn out to be almost entirely unrelated, correlating at r = -0.037, which is to say not at all. They’re picking up different things. Snyder and Pierce show why with a case that ought to trouble anyone relying on word counting alone: Justice Lewis Powell‘s separate opinion in Coker v. Georgia, the 1977 death penalty case, scores extremely high on harsh rhetoric largely because the case concerns rape and capital punishment and those words live in the negative dictionary, while Powell’s actual argument is patient, careful and full of concessions to his colleagues. The dictionary was reading the subject matter, not the man.

Once you have the measure, the patterns start doing work. Justice William O. Douglas, blunt to the point of legend and threatened with impeachment more than any of his colleagues, sits near the bottom of the whole dataset. No surprise there, which is sort-of the point; a measure that failed to notice that Douglas was Douglas wouldn’t be worth much.

William Rehnquist is the more interesting one. From 1972 he behaves like an ordinary associate justice. Then in 1986 he becomes Chief, and his appeasement figures start climbing and swinging about, reaching highs he never touched before. The institutional burden of the chief’s chair appears to be visible in his prose.

What the Numbers Can and Can’t Carry

Two other relationships hold up statistically. Appeasement rises as the majority coalition gets bigger (t = 10.07), and rises again when that coalition spans a wider ideological range. Justices who are themselves further from the ideological center concede less, a result significant at p = 0.003. Which fits a fairly intuitive story about persuasion: the more colleagues you might yet win over, and the closer some of them sit to you, the more you have to lose by scorching the earth.

The caveat worth stating is that these are real but modest effects. The regression models explain very little of the overall variance, with R-squared values between 0.002 and 0.034. Appeasement is one thread among many in a justice’s rhetorical choices, not the loom.

There is also the awkward matter of the calendar. The dataset stops in 2011, well before the current court, before Dobbs, before the fights over presidential immunity and nationwide injunctions that have justices accusing one another of rewriting statutes outright. Pierce is open that she cannot speak to the present bench from this data. “If I had to make an educated guess, I would assume that it has changed, and we’ve probably seen appeasement drop,” she says, though she adds that some justices still seem to be using conciliatory language to keep working relationships intact.

Snyder sees a forward use for it. “One benefit of a measure like appeasement is that higher levels could serve as a potential signal that a justice may be more sensitive to the opinions of their peers and perhaps even open to ideological change,” he says. If that holds, appeasement scores become a kind of early-warning system for the swing votes that decide everything, which is precisely what everyone spends their time guessing about anyway. Pierce thinks the method travels further still, to congressional floor debate and executive branch communications, anywhere persuasion runs on the balance between yielding a point and holding one.

“You don’t want to burn every bridge that you cross, because eventually you might need to go back,” Pierce says.

Frequently Asked Questions