Center for Scientific Review: Peer Review at NIH

Size: px

Start display at page:

Download "Center for Scientific Review: Peer Review at NIH"

Teresa Taylor
6 years ago
Views:

1 Center for Scientific Review: Peer Review at NIH Fungai Chanetsa, PhD, MPH Scientific Review Officer Population Sciences and Epidemiology IRG Division of AIDS and Population Sciences

2 Center for Scientific Review Focal Point for Initial Review at NIH Serves as central receipt point for grant applications submitted to NIH and some other DHHS agencies Assigns applications to CSR Integrated Review Groups/Study Sections or Institute Scientific Review Groups Assigns applications to NIH Institute(s) as potential funding component(s) Conducts initial scientific merit review of most research applications submitted to the NIH

3 CSR organization Office of the Director (OD) Dr. Richard Nakamura, Director Division of Basic & Integrative Biological Sciences (DBIB) Division of Neuroscience, Development & Aging (DNDA) Division of Physiological & Pathological Sciences (DPPS) Division of AIDS, Behavior & Population Sciences (DABP) Dr. Noni Brynes, Director Dr. René Etcheberrigaray, Director Dr. Sy Garte, Director Dr. Karyl Swartz, Director Division of Management Services (DMS) Executive Officer Division of Receipt & Referral (DRR) Dr. Cathleen Cooper, Director 2

support/duration Do not make funding decisions Second Level of Review NIH Institute/Center

4 The Scientific Review Process Dual Review System for Grant Applications First Level of Review CSR or Institute Review by Scientific Review Group For scientific merit Recommend level of support/duration Do not make funding decisions Second Level of Review NIH Institute/Center Council Reviews the review Recommends funding to institute/center Evaluates priorities/relevance Advises on policy

5 Separation of Funding and Review Program Staff Identify and promote research priorities Recommend projects for funding (based on score, budget, priorities) Manage portfolio of projects Work with applicants up to review and after review Review Staff Manage study section meetings to evaluate scientific and technical merit Provide a fair, thorough and competent review for each application Work with applicants before review

6 Applications Are Assigned to: Integrated Review Groups based on Specific review guidelines for each Integrated Review Group (IRG) Institutes or Centers based on Overall mission and guidelines of the Institute or Center Specific programmatic mandates and interests of the Institute or Center

7 Assignment to CSR Study Sections Within an IRG, applications are assigned to: Standing Study Sections When subject matter of application matches the referral guidelines for the study section or Special Emphasis Panels (SEPs) When the subject matter does not fit into any study section When assignment of an application to the most appropriate study section creates a conflict of interest When certain types of grants are sought (e.g., fellowships, SBIRs, AREAS)

8 Your Scientific Review Officer Takes Charge Your SRO is a doctoral-level scientist with expertise relevant to your field who manages the overall peer review of your application.

9 Your SRO Assigns at Least Three Reviewers to Your Application

10 What Your SRO Looks for When Recruiting Reviewers Demonstrated scientific expertise/research support Doctoral degree or equivalent Mature judgment Work effectively in a group context Breadth of perspective Impartiality Diversity Geographic distribution

11 The Study Section Meeting Your SRO Convenes the Study Section Meeting

12 Study Section Chair Partners with their Scientific Review Officer to conduct the meeting Guides and summarizes study section discussion Ensures all study section member opinions are given careful consideration Manages scientific discussions at the meeting, e.g., timeliness and thoroughness

13 Your Application Could Be Reviewed Electronically Electronic reviews are used to facilitate reviewer participation Electronic Review Platforms Telephone Assisted Meetings Internet Assisted Meetings Video Assisted Meetings

14 Discussions Focus on the Best Applications Reviewers typically discuss the top half of the applications The panel will discuss any application a reviewer wants to discuss

15 At the Meeting: Application Discussion Any member in conflict with an application leaves the room Reviewer 1 introduces the application and presents critique Reviewers 2 and 3 highlight new issues and areas that significantly impact scores All eligible members are invited to join the discussion and then vote on the final overall impact score

Career Stage Consideration New Investigator or Early Stage

status is formally considered and NIH is committed to

Other grant applications: Your career stage is factored

16 Career Stage Consideration New Investigator or Early Stage Investigator Applications R01 grant applications: Your status is formally considered and NIH is committed to funding a significant number of these applications. Other grant applications: Your career stage is factored into the Investigator critique. NIH must have correct info on your career stage

17 Main Review Criteria Overall Impact Assessment of the likelihood for the project to exert a sustained, powerful influence on the research field(s) involved Core Review Criteria

18 Core Review Criteria Significance

19 Core Review Criteria Significance Investigator(s)

20 Core Review Criteria Significance Investigator(s) Innovation

21 Core Review Criteria Significance Investigator(s) Innovation Approach

22 Core Review Criteria Significance Investigator(s) Innovation Approach Environment

23 Additional Criteria Contribute to Overall Impact Scores Protections for human subjects Inclusions of women, minorities and children Appropriate use of vertebrate animals Management of biohazards

24 Other Considerations that Do Not Affect Overall Impact Scores Resource Sharing Plans: Data Model Organisms Genome Wide Association Studies Foreign Organizations Select Agents Budget

25 CRITERION SCORES OF RESEARCH APPLICATIONS Impact Score Descriptor 1 Exceptional High Impact 2 Outstanding 3 Excellent 4 Very Good Moderate Impact 5 Good 6 Satisfactory 7 Fair Low Impact 8 Marginal 9 Poor

26 Overall Impact: The likelihood that a project will have a sustained and powerful influence on science. Overall Impact High Medium Low Score Evaluating Overall Impact: Consider the 5 criteria: significance, investigator, innovation, approach, environment (weighted based on reviewer s judgment) and other score influences, e.g. human subjects, animal welfare, inclusion plans and biohazards e.g. Applications are addressing a problem of high importance/interest in the field. May have some or no weaknesses.. e.g. Applications may be addressing a problem of high importance in the field, but weaknesses in the criteria bring down the overall impact to medium. e.g. Applications may be addressing a problem of moderate importance in the field, with some or no weaknesses e.g. Applications may be addressing a problem of moderate/high importance in the field, but weaknesses in the criteria bring down the overall impact to low. e.g. Applications may be addressing a problem of low or no importance in the field, with some or no weaknesses. 5 is a good medium-impact application, and the entire scale (1-9) should always be considered.

27 Scoring Each panel member provides an overall impact score. Range of Scores After discussion, assigned reviewers state final Overall Impact Scores, defining the score range. Panel members may vote outside this range although any intent to do so must be declared.

28 After Your Review Your SRO Prepares summary statements Provides information to NIH Institutes and Centers

Your Summary Statement Scores for each review criterion Critiques from assigned reviewers Administrative notes if any If your application is

29 Your Summary Statement Scores for each review criterion Critiques from assigned reviewers Administrative notes if any If your application is discussed, you also will receive: An overall impact/priority score and percentile ranking A summary of review discussion Budget recommendations

30 Check the Status of Your Application in NIH Commons

31 When Preparing an Application Read instructions Never assume that reviewers will know what you mean Refer to pertinent literature State rationale of proposed investigation Include well-designed tables and figures Present an organized, lucid write-up Obtain pre-review from faculty at your institution NIH Grant Writing Tips

32 What Reviewers Look for in Applications Significance and impact Exciting ideas Clarity Ideas they can understand -- Don t assume too much Realistic aims and timelines -- Don t be overly ambitious Brevity with things that everybody knows Noted limitations of the study A clean, well-written application

33 Common Problems in Applications Lack of new or original ideas Absence of an acceptable scientific rationale Lack of experience in the essential methodology Questionable reasoning in experimental approach Uncritical approach Diffuse, superficial, or unfocused research plan Lack of sufficient experimental detail Lack of knowledge of published relevant work Unrealistically large or overly ambitious amount of work Unrealistic recruitment goals Uncertainty concerning future directions

34 Who Can Answer Your Questions? Before You Submit Your Application A Program Officer at an NIH Institute or Center Scientific Review Officer After You Submit Your Scientific Review Officer After Your Review Your Assigned Program Officer

35 Key NIH Review and Grants Web Sites NIH Center for Scientific Review NIH Office of Extramural Research

36 If you have questions Pre-Submission o A Program Officer at an NIH Institute or Center o Scientific Review Officer Post submission o Your Scientific Review Officer Post review o Your Assigned Program Officer GrantsInfo: GrantsInfo@nih.gov

37 Got a Question Today? Send It to askexperts@csr.nih.gov

38 Early Career Reviewer (ECR) Program Train and educate qualified scientists without significant prior review experience to become effective reviewers Help emerging researchers advance their careers through exposure to the peer review process Enrich the existing pool of NIH reviewers by including scientists from less research-intensive institutions as well as those from traditionally research-intensive institutions.

39 Eligibility Requirements Has not reviewed for NIH beyond one mail review and has not been to a face-to-face meeting Demonstrates training, experience or qualifications in the discipline and fields of the scientific areas under review as evidenced by: A faculty appointment or equivalent An active independent research program and recent publications in good research journals Other relevant credentials or experience Does not necessarily have NIH or equivalent funding

40 Responsibilities of an ECR Attend study section meeting Assigned 2-4 applications as 3 rd reviewer Write a full critique of each application Participate in no more than one study section per year and no more than twice total

41 How to Apply to the ECR Program Send a current CV and a list of terms that describe your scientific expertise to CSREarlyCareerReviewer@mail.nih.gov Nominations of qualified researchers Self-nominations are welcomed CSR maintains an database of eligible ECR Invitation extended to serve when specific expertise is needed for particular applications or study section ewer/pages/overview-of-ecr-program.aspx

Review of Small Business Applications at the National Institutes of Health

Review of Small Business Applications at the National Institutes of Health Gagan Pandya, Ph.D. Scientific Review Officer, Infectious Diseases Microbiology, IRG Center for Scientific Review, NIH November