There are few aspects of speaker management that can be more painful than the process of conducting a peer review from an open Call for Papers. If the review is conducted in the classical way, the planners will find themselves buried in a bewildering collection of PDFs, Word docs and spreadsheets. Hours upon hours can be spent communicating with the review team, collating information and somehow trying to make sense of it all. In between all of this manual work is the need to give proper consideration to all the submissions and being fair in the final judgments of which presentation is accepted, which is declined, and which can be modified for other uses such as poster sessions.
Fortunately, there are better ways to conduct your review beyond mass document exchange. High quality Call for Papers and Speaker Management software should include a Peer Review module that can save you a ton of time, simplify communications, and guarantee that all submissions are treated fairly. Here are some of the key features of good peer review management software.
- Each reviewer should have his or her own “portal page” for performing all the tasks that need to be accomplished.
- On the portal page the reviewer should be able to mark areas of expertise as well as update contact information.
- The software should be able to automatically assign reviewers to submissions based on these areas of expertise and any other rules. Those rules can include such things as maximum number of submissions per reviewer, or other priorities.
- The system should have a robust administrative back-end where you can modify the assignments as needed.
- When the actual reviewing begins, the reviewers should be able to just see those parts of the submission that you wish to display. In particular, blind reviews should be supported.
- The software should support reviews based on the kind of scoring scale you pick – such as 0-10, 1-3 or whatever. The reviewer should be able to see the average score he or she gives per submission.
- The review questionnaire should support any other kind of questions you may need to provide including open ended-comments, a choice-list of categories the review may cover, and so forth.
- If you have a peer review committee, chairs should be able to access a portal page where they can manage the reviewers including seeing what is left to be completed and how papers were scored.
- On the administrative back-end you should be able to run real-time reports that provide critical information such as the progress of the scoring effort (such as which reviewers have not completed the questionnaires), aggregate scores and custom reports.
- You should be able to Email blast the reviewers throughout the process, including who has not completed the assignments.
- The administrators should be able to close off the reviewing portal when it is time to start evaluating the results.
- One or more higher-level leaders or committee chairs should be able to see the results of their teams of reviewers. The results should be sortable and filtered by key fields such as track.
- The back-end should make it straightforward for you to mark which presentations have been accepted or declined or even undecided.
- At the end of the review process, the system should make it easy to notify the prospective presenters of the acceptance status, with the appropriate Email for each.
- If desired, the presenters should be able to see key areas in which their submission was scored, all the while maintaining the anonymity of the reviewers.
Good peer review management software can free planners to perform the many other tasks needed for a successful conference, all the while making life simpler for the reviewers, presenters, and committee chairs.