โš–๏ธ Supreme Court Oral Argument Arena

Compare justice responses from top AI models and vote for your preference.

You will begin by selecting an annotator and a specific context ID. For each context, you will be presented with:

  • The case facts, legal question and the conversation in the oral argument so far
  • A given Supreme Court justice that speaks next in the conversation
  • 36 different pairs of potential responses the supreme court justice could have said. (There's actually only 9 distinct responses, but you are giving preference ratings for each possible pair)

For each pair, your task is to choose the response that is more realistic for the given Supreme Court justice to have said in response to the conversation so far.

Choosing the context ID: You should have been assigned a specific set of context ids to annotate (check with Nimra/Kylie if not sure).

Begin by selecting your initials as the annotator and a specific context ID.

๐Ÿ‘ค Select Annotator
๐Ÿ“‹ Select Context