Configure Scoring mode

Numerically reviewing applications using is referred to as the Scoring in Good Grants. Our Scoring reviewing method features configurable criteria to set your rubric. Each stage of Scoring (you can have multiple if necessary; see: Moving applications between reviewing stages and selecting winners) requires four things to be configured:

• A reviewing round to control the access period
• A score set to collate scores
• Individual scoring criteria
• One or more reviewing panels to assign which reviewers score which applications

To configure a Scoring reviewing stage, you can follow the steps below or simply utilise the reviewing fast start. 

Set up a reviewing round

1. In the Manage workspace, go to Reviewing > Settings > Rounds
2. Create a new round, setting the type to Reviewing
3. Set 'Starts' and 'Ends' dates for reviewing period
4. Click Save when finished

For more detailed instructions, see: Round configuration.

Create a Scoring score set

All reviewing stages require a score set. Score sets allow you to have multiple discrete stages of reviewing, even running at the same time. For example, if you have two rounds of reviewing, the first round might be a process of scoring to produce a short list for each category, the second round might a fresh round of reviewing with another score set and new scoring criteria designed to produce a winner for each category.

1. From the Manage workspace, navigate to Reviewing > Settings > Score sets
2. Create a new score set, setting the reviewing mode to Scoring
3. The Score set name is displayed to reviewers and applicants in some contexts, so name the score set clearly, e.g. "First stage reviewing"
4. Under the 'Display' tab you have other optional configurable settings, such as:
   • Category quick filter: to enable reviewers to filter by a category quickly
   • Order applications: choose the order in which you want to display applications for reviewers
   • List length default: choose the number of applications your reviewers see in their reviewing list view. This setting makes it easier for reviewers who have a lot of applications to view everything at once. Options include 10, 25, 50, and 100 applications per page.
   • Abstention and Conflict of interest: allow/disallow reviewers to abstain from reviewing an application or declaring a conflict of interest. In the latter, the grant manager can choose to include their scores in the final result
   • Commenting options: choose if reviewers can leave a comment on the overall application or individual scores and if the comments are mandatory. You can also select if reviewers should be able to see each other's comments
   • Attachment options: choose if attachments should be displayed and/or available for download. This setting refers to files uploaded via the Attachments tab.
   • Images: this setting refers only to images uploaded via File upload (single) fields. If Display gallery of images from file fields' is checked, images will be displayed as thumbnails at the top of each application and the images included in the application body.
   • Share scores: control whether you display scores from another score set to reviewers on this score set. See more about displaying score matrixes to reviewers.
   • Slideshow and Display: choose if you want to create a slideshow for all media included in the application and choose to show/hide application and applicant name. You can also choose to include scoring boxes on the reviewer PDF.
5. Click Save when finished

Configure scoring criteria

Scoring criteria are required for the scoring mode only. A scoring criterion is like a question for the reviewer, to which they can apply a score. Each scoring criterion created is directly linked to a score set. All scoring criteria associated with a score set will together result in a total score.

For example, criterion A is worth 10 points and criterion B is worth 30. Both criteria are associated with a score set called "Round 1 Reviewing", therefore the maximum possible score for Round 1 Reviewing is 40.

1. In the Manage workspace, go to Reviewing > Settings > Scoring criteria
2. Create a New criterion
3. Provide a Score title (maximum 32 characters), this is required
4. Provide a Shortcode, this is used in summary tables and exports where space is limited— the full score title may be too long and take up too much space
5. The Label, Help text, and Hint text are all optional
6. If you have multiple Score sets, select the one just created
7. Give the scoring criteria a Maximum score, the default 10 is a good option
8. The Weight value adjusts the contribution of this scoring criterion to the total score. The default weight of 1 makes no adjustment to the score. Weighted scores are calculated as score multiplied by weight.
   
   Note: you can apply a weight of 0 only if the score set calculation is set to 'Mean'.
9. If the scoring criteria you're creating should be displayed alongside a specific application field, select the application field from the Align this score with a field, otherwise it is simply positioned at the end of the application
   
   Important: if you align a scoring criteria with a conditional field, it may or may not appear to your reviewers. Should this be intentional, we recommend utilising the "Mean" calculation within your score set configuration. For more details, see: What is the difference between 'Sum' and 'Mean' result calculation?. 
10. Provide an Order for the scoring criteria, this simply controls the placement of the scoring criteria on screen for the reviewer, in score matrix and in exports. It is optional but advisable to use so you have control of its sequence.
11. Select a Control type: Select score, the default is a dropdown for choosing from the range of integer scores; or Slider is a horizontal drag and drop control that is a more visual method of scoring and is in increments of one decimal place.
12. Finally, chose which Categories (all, or some) this scoring criteria should be used for
13. Save when finished

Create reviewing panel(s)

Panels help you easily manage which application go to which reviewers. You can configure as many panels as necessary for a given score set— you might typically have one panel per category. It's highly recommended that you add yourself to any panel you create so that you can check your configuration progress. To learn about adding reviewers to your program, see our dedicated guide: Adding reviewers.

1. In the Manage workspace, go to Reviewing > Settings > Panels
2. Create a New panel
3. Choose Scoring as the reviewing mode, and select the score set created earlier
4. In the Applications section, check the categories / chapters / tags for applications that are to be reviewing. There are two additional settings in this section: 'Only include moderation approved applications' and 'Include archived applications'. If moderation is in use, checking the first box will ensure only approved applications go through to reviewing. Checking the second box will include archived applications.
5. In the Who section select who will be allowed to review
   If you want to allow anybody with a specific role to review, select Any users with a role and then check the role
6. In the When section, check the round configured earlier, for when reviewing will be permitted
7. Click Save when you've completed setup

Wrapping up

Once all the above is done, open the Review workspace and assuming a) the reviewing round is currently open; and b) you're in at least one panel with applications; you will see a list of applications assigned to you for reviewing.

If you're setting up reviewing before there are applications in the system, you can create and submit a couple of 'test' applications for each category so that you can apply scores to test applications. You can delete your test applications (or moderation 'reject' them) to remove them from any real scoring results.