Skip to main content

Grading Guide

This page will teach you how to use AI grading on HiTA. The information here is applicable to both internal and external assignments.

High Level Overview​

There are three key pieces to AI grading on HiTA. At minimum you will need:

  1. Assignment definition or instructions - this is what you will have created in the assignment editor for internal assignments, or in the assignment details step for external assignments.
  2. A submission - either submitted directly by students on HiTA for internal assignments or uploaded by you in external assignments.
  3. Grading criteria - These are the criteria the AI will use to judge the submission. We will discuss these in much greater detail later on this page.

With these three pieces in place, the AI grader will examine each submission, determine which criteria apply to it and calculate a score based on that. When you open the grading view, you will see what it decided for each submission. One at a time, you can review the AI grade for each submission and make any manual changes you would like. You can then submit the grade when you are satisfied. This will save the grade in the HiTA gradebook and make it visible to students when you decide to release all the grades. If Canvas integration is enabled and you have linked this HiTA assignment to a Canvas assignment, this is when the scores will be pushed to the Canvas gradebook.

Optional supplemental resources​

In addition to the three necessary pieces above, grading configuration allows you to upload some additional files if you have them. These are not strictly necessary, but are encouraged especially for larger and more complex assignments.

Rubric file​

The first optional supplemental material you can upload is a rubric file. This must be a PDF file. The rubric file can be any kind of grading instructions; it does not strictly need to be a rubric. The AI will consider this file when generating the grading criteria. Having a rubric file will help align the criteria it generates to meet your grading goals.

Solution file​

This optional supplement must also be a PDF file. The AI will regard the solution file as an ideal student response. It can be an answer key for assignments with clearly defined right and wrong answers, or just a typical example of a superb submission. The AI will consider this both in criteria generation, as well as during grading. Student submissions will be compared against it if present. It is recommended to include a solution file for technically advanced assignments where the AI may not always get the right answer on its own. This helps to prevent mistakes in grading where the AI grader determines a correct response is wrong or vice-versa.

Grading Criteria​

Grading criteria in HiTA are the basic atomic units of evaluation the AI grader uses. You can think of this like a checklist where a perfect scoring submission would check all the items. Each criterion contains a description and a score. The description describes some aspect that a perfect submission would exhibit. The AI grader will look at the description and the submission and determine if the submission meets the description. If the criterion description is met, then the score for the submission is incremented by the associated criterion score. The total points possible for an assignment is the sum of all criteria scores.

Example: grading criteria for a math worksheet​

Maria is teaching a lower-level math course. She has assigned her students a simple five-question worksheet about using the quadratic formula to solve quadratic equations. Each question is worth 3 points: 1 point for attempting to answer at all, 1 point for correctly applying the quadratic formula, and 1 point for the correct answer. The grading criteria she creates has 15 criteria:

  • (+1) Attempted problem 1
  • (+1) Correctly applied quadratic formula in problem 1
  • (+1) Got the correct answer for problem 1
  • (+1) Attempted problem 2
  • (+1) Correctly applied quadratic formula in problem 2
  • etc...

With these criteria created, she can run the AI grading and get the results that align with her grading objectives.

Automatically Generating Grading Criteria​

Empty grading criteria builder with AI generation button.

Click the "Generate with AI" button to start building your grading criteria

Creating criteria manually, as shown in the example above, can be quite tedious. Fortunately, we can leverage AI to speed up this process. In the grading configuration pages for assignments, you will see a "Generate with AI" button. Clicking this brings up a modal where you can specify the sum total of points you want the criteria to add up to and any additional instructions specific to the criteria generation.

When generating the criteria, the AI will consider the assignment instructions along with any rubric and solutions files you pass it. The AI will do its best to transform whatever style of rubric is passed in to the grading criteria paradigm. In the case of Maria and the math worksheet above, she doesn't have a rubric file, but she has a clear idea of what the grading criteria should be. She can therefore skip the rubric file upload and just put her instructions in the grading criteria generation custom instructions.

Grading criteria generation modal with 15 max points and custom instructions of 'For each question create 3 criteria: 1 for attempting the problem, 1 for correctly applying the quadratic formula and 1 for getting the right answer.'

Maria fills out the criteria generation modal so the AI can generate the criteria she wants.

After generating some grading criteria, take some time to review what the AI has generated and decide if it meets your needs. You may make manual changes if only small tweaks are required, or you can try again with some different instructions or rubric files if deeper changes are required. Once you're happy, you can save the criteria and start grading!

Grading criteria for the math worksheet worth 15 points that align with Maria's goals.

Maria decides she is satisfied with these grading criteria and clicks the "Save Criteria" button.

Grading​

Grading view with AI grading in progress banner

If you see this banner, wait a minute or two for the AI to finish grading this submission.

Once you have configured grading, you can click the "Begin Grading" button on your assignment's configuration page. This will trigger the AI to get to work grading submissions. Each submission is evaluated independently. This may take a little time. If you visit the grading page of a particular submission before the AI is done grading that submission, you will see a banner letting you know to wait. Once the AI is done, this banner will go away and you will see the grading view filled out with what the AI grader has decided for this submission.

Grading view for math worksheet with AI selections

After the AI grader is done, you will see the score it suggests and its decisions on the criteria.

Once the AI grader has finished with each submission, you must review its decisions and decide whether to submit it or make whatever manual changes you would like first. The criteria for the assignment are laid out like a checklist. If a criterion is checked, that means the AI has decided that it applies to this submission. If it is unchecked, the AI grader decided that the submission failed to meet this criterion. You can manually check or uncheck the criteria, and the score will change by the value of that criterion.

Along with each criterion, there is a box for feedback comments. The AI will put its reasoning here for why the submission did or did not meet the criterion. This feedback will be visible to students once grades are released.

Underneath all the criteria, the AI grader will leave high level global feedback for the submitter. This will also be visible to students and are part of the grade export .csv you can download after you're done grading.

You can edit any of these feedback fields if needed. Students will only see the feedback fields as they are when you submit the grade for this submission, so take this opportunity to correct any mistakes the AI grader may make and to give personalized human feedback to each student.

Editing grading criteria during grading

If during grading, you decide you want to tweak the point values of criteria, or to create and/or delete some criteria, you may. HiTA will keep consistency for the submissions you've already graded by updating their scores appropriately.

Say, for example, you have already submitted a grade for John Doe's submission and are grading Mary Whatsit's submission. John's submission met criteria A, but several others didn't. While grading Mary Whatsit's submission, you decide criterion A should actually be worth 2 more points. You edit the criterion's value, incrementing its score by 2 points. John Doe's score will then also go up by 2 points to reflect the change in the criterion value. The other submissions which failed this criterion only have the max possible score updated. Since criterion A was not applied to those submissions, their score does not change otherwise.

Likewise, deleting a criterion will decrease the max possible score and all submissions it was applied to will have their score decremented by the criterion's value.

Once you are happy with the criteria selection and feedback for each submission, click the "Submit and Next" button. This will post the submission's grade to the HiTA gradebook. If Canvas integration is enabled and the assignment is deeplinked, this is when the score will be posted to the Canvas gradebook.

Once you are on the last submission, the "Submit" button will change to "Submit and Finish." Clicking this will take you to the grade summary page where you can review the results and download the grades.