Scoring criteria

 

When using Scoring mode you will need to create scoring criteria. These are the criteria which your reviewers will use to assess entries and enter a score. You can have as many or as few as you want and criteria can be specific to categories and fields. They can also have different weightings. For instance, one criteria might be worth twice as much as another.

To create scoring criteria go to Reviewing > Settings > Scoring criteria in the Manage workspace and click the New criterion button. There are several configuration options including the type of control you'd like your reviewers to use, the weightings for each criterion, and even the number of decimal places you'd like to have for each score. We'll go through some of these options below.

Controls

Controls are how your reviewers will score. There are three options to choose from - Select, Slider, and Keyboard input.

Select score

The select score control uses a drop-down menu from which reviewers can select their score. We recommend this control as most reviewers find it the easiest to use.

Select score example

Slider

The slider is a sliding scale and reviewers will need to move the circle into the correct position.

Slider example

Keyboard input

Keyboard input example

The keyboard input control lets judges enter their score using a keyboard. This is great for when you require a score with decimals.

Categories and fields

Scores can be assigned to specific categories and fields.

Categories and fields

Important: if you align a scoring criteria with a field which doesn't apply to all categories, it will not be visible to the reviewers (since it does not exist in the application). This means the application will not be considered 'Complete' even though the reviewers have scored all visible criteria.
Important: if you align a scoring criteria with a conditional field, it may or may not appear to your judges and you'll see the message outlined below. Should this be intentional, we recommend utilising the "Mean" calculation within your score set configuration. For more details, see: What is the difference between 'Sum' and 'Mean' result calculation?. If you do not wish for reviewers to see scoring criteria for fields that may not be applicable, select the Don't score when unanswered checkbox.

If you choose to align the score with a field it will appear to the right of the field:

Criteria aligned with a field example

If the scoring criteria are not aligned to any fields they'll appear at the bottom of the application:

Criteria not aligned with a field example

Weightings

You can give scoring criteria different weightings. For example, let's say you want to create three scoring criteria - Creativity, Function, and Usability - but you'd like to give Usability twice as much weight as the other two criteria. In this example the breakdown of weightings is as follows:

Creativity 25%
Function 25%
Usability 50%

To create this example you'd give Creativity and Function a weight of 0.25 and Usability a weight of .50.

Weight example

Note: you can apply a weight of 0 only if the score set calculation is set to 'Mean'.

Minimum and maximum scores

Each criteria requires a minimum and maximum score. If the maximum score is ten then reviewers will give a score out of ten. If the maximum is five then reviewers will give a score out of five.

Important: deleting a scoring criteria and recalculating the leaderboard will permanently delete scoring data collected from your reviewers. 

Commenting

Depending on your needs, you can opt to allow reviewers to optionally comment on this scoring criteria or make that comment compulsory. 

Commenting on individual scores.png

Video overview

Was this article helpful?
0 out of 0 found this helpful

Articles in this section