Open Response Assessment (ORA) Basic Walkthrough

The Open Response Assessment (ORA) allows learners to answer open ended questions in an essay form or upload files as a part of their response. Responses are then graded by either the learners themselves, their peers, or course instructors (or a combination of the three). 

There are many different ways to configure an ORA. This article is meant to provide a quick overview of the problem type so you can get up and running.

Creating an Open Response Assessment

From Open edX Studio, navigate to the course unit where you wish to add the ORA. Then, go to the Add New Component section at the bottom of the unit, click Problem, then select Open Response Assessment.

NOTE: You will see both the terms Peer Assessment and Open Response Assessment - Know that these terms are mean the same thing and are used by edX interchangeably. At Appsembler, we will always use Open Response Assessment to avoid confusion.

You will now see a demo version of the ORA. Click the Edit button in order to begin making your changes.

The first page you will be looking at is the PROMPT panel. Here you can create the prompt for you ORA. The prompt is the question your learners will be trying to answer. You can have either 1 or multiple prompts. 

If you're done writing your prompt, you can either click Save, or click on RUBRIC to configure how the learners' responses will be graded.

Note: There is an + Add Prompt button at the bottom of the prompt screen that can be used to add multiple prompts to an ORA. 

Configure the Rubric

The rubric of an ORA defines how learners' responses are graded. A rubric is made up of Criterion, which are characteristics that a learner's response should include. Each criterion contains a set of Options, which describe how well the response meets the criterion. These usually include things like Poor, Fair, & Good which each then have a point value assigned. Each Criterion requires a name and then a prompt.

Under each criterion are the options for that criterion. Here you include a name for the option, an explanation for it, and how many points are awarded for that option.

At the bottom of each criterion, you can add more options & choose whether written feedback for that criterion is optional or required

Whenever you have all your criteria and options in place, click Settings to finish setting up your ORA.  

Configure the settings of an ORA

From the settings panel, there are multiple options including...Choose the name and availability dates of response submissions.

Choose whether files should be allowed to be uploaded or LaTeX written as a part of the response. Click here to see more about allowing file uploads as a part of learner responses.

Choose the steps for learners to take

In the next part of the Settings, there are multiple optional steps that a learner can take. They are as follows:

  • Learner Training
    • An option for learners to see what a sample response looks like and how it should be graded
  • Peer Assessment
    • Have learners use the rubric to assess one another's responses
  • Self Assessment
    • Have learners use the rubric to assess their own responses
  • Staff Assessment
    • Have a course staff member use the rubric to assess learner responses

Each step can be turned on or off by checking the checkbox next to the step.

For this example, the only step turned on is Staff Assessment.

Responding to an ORA

Once an ORA is live, this is how it will look to a learner: 

After a learner has written their response, they can click Submit to move on to the next step. 

After the response has been submitted, and the learner is waiting for feedback it looks like this:

Grading an ORA

After a learner has submitted a response to an Open Response Assessment, a Staff member can navigate to the ORA and begin grading responses. 

Here, a Staff member can begin reading learner responses. 

Staff can add Feedback, then choose to either submit only the one assessment or submit it and continue grading other responses.