Supreme Court Justice Anthony Kennedy is retiring. Donald Trump, aided by a pliant and hypocritical Senate Republican caucus, will fill his seat, likely with someone even more conservative than Kennedy. But it is worth asking: Will we miss Justice Kennedy?

The answer is: well, yes and no.

At least within some segments of the queer community, Kennedy walks with a halo. He wrote the big gay rights decisions at the Supreme Court. Romer v. Evans held that a state cannot pass a law discriminating against gay people simply because the people in that state hate gays. Lawrence v. Texas declared laws that criminalized sodomy unconstitutional. United States v. Windsor struck down the Defense of Marriage Act, which denied federal recognition to otherwise lawfully married gay couples. And Obergefell v. Hodges finally put an end to state bans on same-sex marriage.

In the course of these decisions, Kennedy created an equality- or due process-based (scholars disagree) jurisprudence on the “dignity” of persons. In Kennedy’s queer rights cases, the dignity due each person qua person placed three significant limitations on the power of government: (1) the state cannot act out of mere distaste, hatred, or “animus” toward gays and other minorities, (2) the state cannot outlaw forms of intimacy in the queer community, and (3) the state cannot treat same-sex unions differently from opposite-sex ones.

But Kennedy’s decision to retire now, when he knows his replacement will look a lot less like him and a lot more like the late-Antonin Scalia and the spry Neil Gorsuch, imperils this legacy. After all, not even a month into his new term on the Court, Gorsuch wrote a dissent that argued that even though a state has to allow gays to marry under Obergefell, that state should not have to provide any of the benefits that normally go along with marriage. Kennedy knows this, obviously. So I can only conclude that he’s either woefully naive (doubtful) or he cares a lot more about his conservative legacy than his progressive one.

Kennedy is, after all, a conservative. He always has been.

He brought unlimited shady money into the political sphere with Citizens United. He sided with the conservatives in Bush v. Gore, which essentially installed George W. Bush as president. He thought ensuring health care for all was unconstitutional in NFIB v. Sebelius. He concluded that racism is over when he voted to gut the Voting Rights Act in Shelby County v. Holder. He believed that the Second Amendment, which has a clear reference to a “well organized militia,” guarantees a right to every individual to own a gun when he voted to overturn Washington, D.C.’s handgun ban in District of Columbia v. Heller. And in the last two weeks alone, Kennedy sided with the conservatives in Hawaii v. Trump (Muslim ban upheld), Masterpiece Cakeshop (a Christian baker was unfairly treated during a hearing about his refusal to serve a gay couple), and Janus v. AFCSME (upending 40 years of precedent to destroy public sector unions).

These decisions will survive him because his replacement will cement the five-to-four majority that decided them. Every single progressive decision to which he attached his name is at risk: abortion, queer rights, defendants’ rights, and powers of the police are poised to be overturned. That Kennedy would retire knowing this prospect makes him a partisan like the rest of them.

And Kennedy plays politics well. We saw it recently. Compare his decision in Masterpiece Cakeshop to his vote with the conservatives in Hawaii v. Trump. In Masterpiece, Kennedy wrote a narrow decision that vindicated a Christian baker who refused service to a gay couple who wanted a cake for their wedding. Although the Court neither adopted the baker’s argument that anti-discrimination laws are per se unconstitutional because they violate the Free Exercise Clause of the First Amendment nor concluded that baking a cake is an exercise of free speech, it did reject the Colorado Civil Rights Commission’s decision against the baker because Commissioners made comments that suggested hostility to religion. Those comments, Kennedy implied, revealed their true prejudices and the real meaning behind the Commission’s decision, which was supposed to be impartial.

Kennedy hypocritically abandoned that logic in Hawaii v. Trump. That case was a challenge to Trump’s “Muslim ban.” If that ban was discrimination on the basis of religion, it would violate the Establishment Clause, which prohibits the government from favoring one religion over another (Islam). Also, if the ban was instituted out of mere animus toward a marginalized group (Muslims), it would violate the principle of Romer. Petitioners brought in a ton of evidence, including Donald Trump’s own statements, to prove the real purpose and meaning behind the Muslim ban was to indeed ban Muslims, and that inclusion of Venezuela and North Korea on the list was really just window dressing. That kind of evidence, which Kennedy found relevant in Masterpiece, was irrelevant in Hawaii. There is no intellectually honest way to explain that hypocrisy. Kennedy’s empathy simply doesn’t extend to Muslims.

Indeed, Justice Kennedy’s vision of compassion is decidedly myopic. He seems to have cared about the LGBTQ community; Romer, Lawrence, Windsor, and Obergefell suggest as much. But he can’t care too much or he wouldn’t resign when his replacement is likely to imperil every one of those decisions. Kennedy’s empathy also never extended to religious minorities, only to Christians. He ignored real evidence of racism and black voter suppression, so his empathy didn’t extend to persons of color, either.

So, again I ask, will we miss Justice Kennedy? The only reason not to say no is that with Kennedy, at least the marginalized, the poor, the weak, and the vulnerable had an ear that was listening. As I have shown, the downtrodden only very occasionally won with Kennedy. But they will almost never win when his likely replacement takes the bench. We will miss Justice Kennedy only because what will happen to the Court after he leaves, and not for any real, progressive legacy he leaves behind.