Objective

You will be given a set of tables covering a specific topic (e.g., Education or Banking) and templates of an ambiguous question and its interpretations.Your task is to format and paraphrase the question and its interpretations to appear natural and grammatical.
  • You need to understand the information presented in the tables to ensure the template and your question have the same meaning, i.e., they must ask about the same thing.
  • You also need to preserve ambiguity when writing the question.

❗️Please carefully read more detailed instructions below. You can also download the instructions in PDF format here.

Context

You will be given a database (a set of tables) in some domain (e.g., Education, Banking) and their content.

Example: the following tables are representative of the Fitness domain:  

Fitness_Studio

id name address
1FitLife123 Main St
2Wellness Zone456 Market St
3Active Life678 Pine St

Exercise_Program

id program_name description
1YogaA mind and body practice.
2ZumbaDance for fitness.
3Crossfit Intense strength training.

Loyalty_Program

id member_id
(→ Member.id)
points
11100
22NULL
33200
44150

Fitness_Studio_Exercise_Program

fitness_studio_id
(→ Fitness_Studio.id)
exercise_program_id
(→ Exercise_Program.id)
11
12
21
31
32
33

  • Usually, there is at least one table connecting two tables in the database. This table and two connected tables are central to the template and its corresponding question.
    For example, each fitness studio is connected to one or more fitness classes through Fitness_Studio_Exercise_Program.

Your task is to write a question (based on the provided template) which is essentially a request to extract an answer from the database. When writing the question, you must ensure that it has the same intent (asks about the same columns) as the template.

    Example: the three templates below reflect different intents:
  • What program_name of Exercise_Program do we have for every Fitness_Studio?
  • What description of Exercise_Program do we have for every Fitness_Studio?
  • What program_name and description of Exercise_Program do we have for every Fitness_Studio?
    Please note that you will only see only one of the possible intents each time.

Templates

You will be given a template of an ambiguous question (due to words “every” or “each”).

    Example:

Ambiguous question template:
What
program_name of Exercise_Program do we have for every Fitness_Studio?

This means that the same question has two different interpretations:

  1. The request to show something common for all items.
    Example:
        Template of interpretation 1:
        Give me
    program_name of Exercise_Program that is common for all Fitness_Studio.
        i.e., we want to retrieve Yoga as all studios in the database offer yoga classes (see table Fitness_Studio_Exercise_Program).
  1. The request to show items and a list of corresponding properties.
    Example:
        Template of interpretation 2:
        For each Fitness_Studio
    , what is the corresponding program_name of Exercise_Program?
        i.e., we want to retrieve that fitness studio FitLife offers Yoga, Zumba and CrossFit classes, fitness studio Wellness Zone offers only Yoga, fitness studio Active LifeYoga and Zumba.

Interface

Interface

Guidelines for annotations

Your task is to write an ambiguous question and its interpretations based on the given templates that will sound natural and grammatical.

  1. All unformatted table and column names must be replaced with plain words, preferably synonyms.
    Example:
    Let's consider writing ambiguous question based on the ambiguous question template above:
        ✅ What exercise programs does each fitness studio offer?
        ❌ What Exercise_Programs does each Fitness_Studio offer?
  1. You’re welcome to simplify the structure (e.g. remove unnecessary words or paraphrase) but you must check the relevant tables to ensure that the question and its interpretations express the same request as the templates and would yield the same answer.
    Example: You can use “training program” and omit the unnecessary word “fitness”:
        ✅ What training programs are available at each studio?
    but you can’t further simplify “training programs” to “programs” because then the question can be interpreted as a request to show loyalty programs instead of exercise programs (the database has a table Loyalty_Program):
        ❌ What programs are available at each studio?
    Note that if there is no Loyalty_Program table in the database (or similar), this question would be allowed.
    You can also write a question in the form of an instruction or command (as you would ask an assistant):
        ✅ Give me the exercise programs that every fitness studio has.
  1. You must maintain ambiguity when writing the question and a reading of each interpretation.
    Example: The following paraphrase of the question template is not ambiguous as it has only one reading (”show smth common“):
        ❌ What training programs are offered at all fitness studios?

The same rules apply to writing interpretations.

Example of a completed annotation:

Interface

Process:

  1. Read the template question. 
  2. Replace table and column names with their common-language equivalents. Check formatting.
  3. Rephrase to a more natural question.
  4. Ensure the question reflects the same request as the original template: consider the context of the tables.
  5. Ensure the question is ambiguous.
  6. Repeat steps 1-4 to write the two interpretations the question has.