Center for Scientific Review: Peer Review at NIH Fungai Chanetsa, PhD, MPH Scientific Review Officer Population Sciences and Epidemiology IRG Division of AIDS and Population Sciences
Center for Scientific Review Focal Point for Initial Review at NIH Serves as central receipt point for grant applications submitted to NIH and some other DHHS agencies Assigns applications to CSR Integrated Review Groups/Study Sections or Institute Scientific Review Groups Assigns applications to NIH Institute(s) as potential funding component(s) Conducts initial scientific merit review of most research applications submitted to the NIH
CSR organization Office of the Director (OD) Dr. Richard Nakamura, Director Division of Basic & Integrative Biological Sciences (DBIB) Division of Neuroscience, Development & Aging (DNDA) Division of Physiological & Pathological Sciences (DPPS) Division of AIDS, Behavior & Population Sciences (DABP) Dr. Noni Brynes, Director Dr. René Etcheberrigaray, Director Dr. Sy Garte, Director Dr. Karyl Swartz, Director Division of Management Services (DMS) Executive Officer Division of Receipt & Referral (DRR) Dr. Cathleen Cooper, Director 2
The Scientific Review Process Dual Review System for Grant Applications First Level of Review CSR or Institute Review by Scientific Review Group For scientific merit Recommend level of support/duration Do not make funding decisions Second Level of Review NIH Institute/Center Council Reviews the review Recommends funding to institute/center Evaluates priorities/relevance Advises on policy
Separation of Funding and Review Program Staff Identify and promote research priorities Recommend projects for funding (based on score, budget, priorities) Manage portfolio of projects Work with applicants up to review and after review Review Staff Manage study section meetings to evaluate scientific and technical merit Provide a fair, thorough and competent review for each application Work with applicants before review
Applications Are Assigned to: Integrated Review Groups based on Specific review guidelines for each Integrated Review Group (IRG) Institutes or Centers based on Overall mission and guidelines of the Institute or Center Specific programmatic mandates and interests of the Institute or Center
Assignment to CSR Study Sections Within an IRG, applications are assigned to: Standing Study Sections When subject matter of application matches the referral guidelines for the study section or Special Emphasis Panels (SEPs) When the subject matter does not fit into any study section When assignment of an application to the most appropriate study section creates a conflict of interest When certain types of grants are sought (e.g., fellowships, SBIRs, AREAS)
Your Scientific Review Officer Takes Charge Your SRO is a doctoral-level scientist with expertise relevant to your field who manages the overall peer review of your application.
Your SRO Assigns at Least Three Reviewers to Your Application
What Your SRO Looks for When Recruiting Reviewers Demonstrated scientific expertise/research support Doctoral degree or equivalent Mature judgment Work effectively in a group context Breadth of perspective Impartiality Diversity Geographic distribution
The Study Section Meeting Your SRO Convenes the Study Section Meeting
Study Section Chair Partners with their Scientific Review Officer to conduct the meeting Guides and summarizes study section discussion Ensures all study section member opinions are given careful consideration Manages scientific discussions at the meeting, e.g., timeliness and thoroughness
Your Application Could Be Reviewed Electronically Electronic reviews are used to facilitate reviewer participation Electronic Review Platforms Telephone Assisted Meetings Internet Assisted Meetings Video Assisted Meetings
Discussions Focus on the Best Applications Reviewers typically discuss the top half of the applications The panel will discuss any application a reviewer wants to discuss
At the Meeting: Application Discussion Any member in conflict with an application leaves the room Reviewer 1 introduces the application and presents critique Reviewers 2 and 3 highlight new issues and areas that significantly impact scores All eligible members are invited to join the discussion and then vote on the final overall impact score
Career Stage Consideration New Investigator or Early Stage Investigator Applications R01 grant applications: Your status is formally considered and NIH is committed to funding a significant number of these applications. Other grant applications: Your career stage is factored into the Investigator critique. NIH must have correct info on your career stage
Main Review Criteria Overall Impact Assessment of the likelihood for the project to exert a sustained, powerful influence on the research field(s) involved Core Review Criteria
Core Review Criteria Significance
Core Review Criteria Significance Investigator(s)
Core Review Criteria Significance Investigator(s) Innovation
Core Review Criteria Significance Investigator(s) Innovation Approach
Core Review Criteria Significance Investigator(s) Innovation Approach Environment
Additional Criteria Contribute to Overall Impact Scores Protections for human subjects Inclusions of women, minorities and children Appropriate use of vertebrate animals Management of biohazards
Other Considerations that Do Not Affect Overall Impact Scores Resource Sharing Plans: Data Model Organisms Genome Wide Association Studies Foreign Organizations Select Agents Budget
CRITERION SCORES OF RESEARCH APPLICATIONS Impact Score Descriptor 1 Exceptional High Impact 2 Outstanding 3 Excellent 4 Very Good Moderate Impact 5 Good 6 Satisfactory 7 Fair Low Impact 8 Marginal 9 Poor
Overall Impact: The likelihood that a project will have a sustained and powerful influence on science. Overall Impact High Medium Low Score 1 2 3 4 5 6 7 8 9 Evaluating Overall Impact: Consider the 5 criteria: significance, investigator, innovation, approach, environment (weighted based on reviewer s judgment) and other score influences, e.g. human subjects, animal welfare, inclusion plans and biohazards e.g. Applications are addressing a problem of high importance/interest in the field. May have some or no weaknesses.. e.g. Applications may be addressing a problem of high importance in the field, but weaknesses in the criteria bring down the overall impact to medium. e.g. Applications may be addressing a problem of moderate importance in the field, with some or no weaknesses e.g. Applications may be addressing a problem of moderate/high importance in the field, but weaknesses in the criteria bring down the overall impact to low. e.g. Applications may be addressing a problem of low or no importance in the field, with some or no weaknesses. 5 is a good medium-impact application, and the entire scale (1-9) should always be considered.
Scoring Each panel member provides an overall impact score. Range of Scores After discussion, assigned reviewers state final Overall Impact Scores, defining the score range. Panel members may vote outside this range although any intent to do so must be declared.
After Your Review Your SRO Prepares summary statements Provides information to NIH Institutes and Centers
Your Summary Statement Scores for each review criterion Critiques from assigned reviewers Administrative notes if any If your application is discussed, you also will receive: An overall impact/priority score and percentile ranking A summary of review discussion Budget recommendations
Check the Status of Your Application in NIH Commons
When Preparing an Application Read instructions Never assume that reviewers will know what you mean Refer to pertinent literature State rationale of proposed investigation Include well-designed tables and figures Present an organized, lucid write-up Obtain pre-review from faculty at your institution NIH Grant Writing Tips http://grants.nih.gov/grants/grant_tips.htm
What Reviewers Look for in Applications Significance and impact Exciting ideas Clarity Ideas they can understand -- Don t assume too much Realistic aims and timelines -- Don t be overly ambitious Brevity with things that everybody knows Noted limitations of the study A clean, well-written application
Common Problems in Applications Lack of new or original ideas Absence of an acceptable scientific rationale Lack of experience in the essential methodology Questionable reasoning in experimental approach Uncritical approach Diffuse, superficial, or unfocused research plan Lack of sufficient experimental detail Lack of knowledge of published relevant work Unrealistically large or overly ambitious amount of work Unrealistic recruitment goals Uncertainty concerning future directions
Who Can Answer Your Questions? Before You Submit Your Application A Program Officer at an NIH Institute or Center Scientific Review Officer After You Submit Your Scientific Review Officer After Your Review Your Assigned Program Officer
Key NIH Review and Grants Web Sites NIH Center for Scientific Review http://www.csr.nih.gov NIH Office of Extramural Research http://grants.nih.gov/
If you have questions Pre-Submission o A Program Officer at an NIH Institute or Center o Scientific Review Officer Post submission o Your Scientific Review Officer Post review o Your Assigned Program Officer GrantsInfo: GrantsInfo@nih.gov 301 435-0714
Got a Question Today? Send It to askexperts@csr.nih.gov
Early Career Reviewer (ECR) Program Train and educate qualified scientists without significant prior review experience to become effective reviewers Help emerging researchers advance their careers through exposure to the peer review process Enrich the existing pool of NIH reviewers by including scientists from less research-intensive institutions as well as those from traditionally research-intensive institutions.
Eligibility Requirements Has not reviewed for NIH beyond one mail review and has not been to a face-to-face meeting Demonstrates training, experience or qualifications in the discipline and fields of the scientific areas under review as evidenced by: A faculty appointment or equivalent An active independent research program and recent publications in good research journals Other relevant credentials or experience Does not necessarily have NIH or equivalent funding
Responsibilities of an ECR Attend study section meeting Assigned 2-4 applications as 3 rd reviewer Write a full critique of each application Participate in no more than one study section per year and no more than twice total
How to Apply to the ECR Program Send a current CV and a list of terms that describe your scientific expertise to CSREarlyCareerReviewer@mail.nih.gov Nominations of qualified researchers Self-nominations are welcomed CSR maintains an database of eligible ECR Invitation extended to serve when specific expertise is needed for particular applications or study section http://wwhttp://public.csr.nih.gov/reviewerresources/becomearevi ewer/pages/overview-of-ecr-program.aspx