Rating in OpenWater

Overview

Rating is completed by member company representatives using two separate processes. The first process allows the many internal stakeholders in a company to participate in the decision on selections in a secure, collaborative environment. This is handled by providing ratings and comments in a company-specific clone of the solicitation. It will be labeled with your company name. The second step is for the designated rater to submit the company's final selection decisions to the official SRC call.

Rater Team Manager

Each company is assigned a single rater per solicitation. This person is called the Rater Team Manager. The rater team manager is responsible for

• Communicating with internal stakeholders within their company
• Providing links to internal stakeholders to use the OpenWater collaborative rating platform
• Providing a single rating for their company to the SRC official solicitation

Rater team managers will have access to both the SRC official solicitation and a company-specific clone of the solicitation that can be used for internal collaboration. Rater team managers are solely in charge of providing access to the company-specific clone to their colleagues. This process happens outside of SRC.

How does a Rating Team Manager add a rater to their team?

Find the invitation link by accessing the company-specific solicitation. The link will be at the top of the list of submissions. Send this link to any collaborators. When they follow the link, they will request to be added as a collaborative rater to your company-specific solicitation.

Raters

Internal stakeholders at a company are welcome to participate as a rater using the OpenWater collaborative rating platform. An OpenWater account will be required for this. To be added as a collaborator on the company-specific solicitation, ask your voting Advisory Board member to send you the invitation link.

Step 1. Collaborative rating using the company-specific clone solicitations

Log into the OpenWater platform at https://src.secure-platform.com/a/organizations/main/home

Choose 'My Rating Assignments' from the menu on the left. Find the solicitation you are looking for, and choose 'Open' under the Action header. Select a category or all categories. Click 'View Application' to select a specific application.

The submission will display a thumbnail of the document. If the thumbnail image does not work for you, there is a Download to PDF link on the page.

Shared Rater Comments

Shared rater comments are where internal team members can discuss a specific submission. These comments will not be shared with SRC. They are private to your company.

NOTE: You cannot enter Shared Rater Comments and a score at the same time. These actions should be performed in serial.

Step 1.5 Compiling your company-specific ratings

All raters are able to take advantage of a feature that allows you to see how others have rated a particular submission by clicking on See Other Rater Scores. This feature is only available in the company-specific clone solicitation.

The Rater Team Manager is able to use the Rater Team Manager Panel in OpenWater. Find this in the local navigation and select the appropriate round.

Manage Raters

Here you can track the progress your raters have made on their assigned ratings. You do not actively manage raters using this panel; instead it's about what your team is doing.

Manage Raters: Export Scores

There are three different variations of this report.

• Show score summaries only
  Spreadsheet of all raters with submissions that they have rated and the summary score.
• Show all rating questions, scores and comments
  Same as above but with more detail about individual answers to each question and any comments.
• Export rater conversations
  Instead of displaying scores, this is going to display the Shared Rater Comments for each submission. This helps you understand what people are saying about any particular submission.

Scores and Results: Judging Results

This is focused on the submissions rather than the people; it gives you a corporate view of the ratings. This includes an online grid which allows you to see scores by submission. This can be exported to allow you to manipulate the data.

Step 2. Official SRC Solicitation

Only the rater team manager will have access to the SRC Official call. This is where the final votes will be submitted.

Log into the OpenWater platform at https://src.secure-platform.com/a/organizations/main/home

Choose 'My Rating Assignments' from the menu on the left. Find the solicitation you are looking for, and choose 'Open' under the Action header. Select a category or all categories. Click 'View Application' to select a specific application.

The submission will display a thumbnail of the document. If the thumbnail image does not work for you, there is a Download to PDF link on the page.

Comments for SRC Program Managers

Comments left in this field will be shared with SRC.

Step 1 Alternative: Offline Evaluation

If you prefer to perform the evaluation offline, you can. The rater team manager can download the Master Report and distribute this to internal stakeholders. This spreadsheet will contain the PI submission information as well as a link to the submissions. This file can be distributed to the internal stakeholders and will allow them to access the submissions.

Frequently Asked Questions

Why are you using OpenWater?

We switched to OpenWater because we needed to modernize our tech stack. At the same time, we wanted to support an online rating process for our member community. We also wanted to offer better tools for collaborating than we had offered in the past.

Do I use my SRC website account?

No. In order to submit a paper or review a solicitation, you do not need an SRC website account. However, if you are selected as a contract performer you will need to create an SRC website account.

Do I need to create an account at OpenWater?

Yes. In order to submit a paper or review a solicitation, you will need to sign up for an OpenWater account. Raters and collaborators will be invited to sign up. Researchers can sign up at their convenience.

Am I a Rater or Rater Team Manager?

The voting Advisory Board member is assigned as a rater team manager. Colleagues and subject matter experts who are invited to collaborate on the rating process are raters.

Where do I go to rate?

The SRC OpenWater portal can be found here: https://src.secure-platform.com/

What can I download, and where?

You can download each submission individually, a monolithic PDF of all the submissions, or a Master Report spreadsheet including links to all the submissions. These downloads are available from within the list of submissions, or within the submission itself.

Can I track a proposal back to the white paper?

Yes. The application code for the submission will persist from the white paper stage to the proosal stage.

I don't see any solicitations when I'm logged in

This may be because you have not been assigned as a rater to the solicitation. Please contact the proposal administrator or email proposal.admin@src.org for help.

Share