DENVER (AP) — Attorneys for a Colorado baker who refused to make a wedding cake for a gay couple on religious grounds say the state is punishing him again for not baking a cake celebrating a gender transition.

Lawyers for Jack Phillips argued Tuesday in U.S. court in Denver to try to stop the state from taking action against him over the new discrimination allegation.

The U.S. Supreme Court issued a limited ruling for the baker in the gay couple’s 2012 case.

Phillips is suing to overturn the Colorado Civil Rights Commission’s June finding that he discriminated against Denver attorney Autumn Scardina because she’s transgender.

The state has asked the lawsuit to be dismissed, but Senior Judge Wiley Y. Daniel said he was inclined to let it go forward. He said he’d issue a ruling later.