RHICTS Junior Investigator Program 1/16/08 Tips from the Trenches John Lochman jlochman@ua.edu
Initial Review (Peer Review) THAT S IT? THAT S PEER REVIEW?
Dual Review System for Grant Applications First Level of Review Scientific Review Group (SRG) Provides Initial Scientific Merit Review of Grant Applications Rates Applications and Makes Recommendations for Appropriate Level of Support and Duration of Award Second Level of Review Council Assesses Quality of SRG Review of Grant Applications Makes Recommendation to Institute Staff on Funding Evaluates Program Priorities and Relevance Advises on Policy
Types of Scientific Review Groups Where are Applications Reviewed? GROUPS CSR IRGs Study Sections Special Emphasis Panels INSTITUTES Scientific Review Groups Contract Review Committees APPLICATIONS REVIEWED Research Projects Academic Research Enhancement Awards Postdoctoral Fellowships Small Business Innovation Research Shared Instrumentation Program Projects Centers Institutional Training Grants Conference Grants Career Awards Small Grants RFAs Contracts
CSR Study Sections Each CSR standing study section has 20-30 members who are primarily from academia Members selected because they work effectively in groups, are impartial, have a broad perspective, but CSR standing study sections convene face-toface meetings Each study section is managed by a Scientific Review Administrator
Who/What is a Scientific Review Administrator? Scientific Review Administrator is a scientist and administrator manages the review of grants, contracts, cooperative agreements (provides orientation; explains NIH review policy) appoints members to initial review groups/study sections/special emphasis panels (permanent, ad hoc) responds to questions about review at Advisory Councils and Board meetings prepares summary statements reflecting IRG recommendations
Grant Review Issues: NIH Grant Review Committees CSR committees meet 3 times per year, reviewing 60-100 grants over a two-day period; [Before submitting a grant, check the committee membership for the likely committee to review your grant, and use that information as you prepare your grant, for references, conceptual framework, etc] Each committee reviews grants on certain topic areas, for a variety of institutes For example, PDRP (Psychosocial Development, Risk and Prevention formerly RPHB-1) reviews grants for NIMH, NIDA, NIAAA, NICHD & NIA
Initial Review (Peer Review) Initial Review (Peer Review) SRA selects reviewers Who are the Reviewers? They all have day jobs Active researchers Review applications in spare time Will review many applications; careful application preparation is valued
Grant Review Issues: Priority Scores for Applications Applications are divided by reviewers into scored (1.0-???) and unscored ; 50% are unscored (in the???-5.0 range) 1.0-1.5: virtually flawless 1.5-2.0: significant, few weaknesses, easy fix 2.0-2.5: very promising, need to consider critiques seriously 2.5-3.0: very good, some notable weaknesses that need to be addressed 3.0-5.0: serious weaknesses in conceptual framework, methods and/or design
Grant Review Issues: Priority Scores Priority scores are converted into percentiles, using a modestly arcane system, based on this review cycle plus the 2 prior review cycles Can be a problem when a committee recalibrates (due to a process similar to grade inflation) Percent funded varies depending in budget However, different institutes have different cutpoints for funding, using these percentiles
Grant Review Issues: The Review of a Particular Grant Each application has 2-3 reviewers who prepare written critiques When the grant is reviewed at the meeting, first the reviewers give the temperature of the grant (e.g. 1.3, 1.8, 2.0) Primary reviewer presents the grant to the committee, then the other 2 reviewers add, followed by committee discussion Primary reviewers revisit scores (e.g 1.8-2.1) If a committee member votes outside of the range, they must inform the committee
Grant Review Issues: Review Criteria Significance: Does the study address an important problem? If the aims of the application are achieved, how will scientific knowledge be advanced? Approach: Are the conceptual framework, design, methods, and analyses adequately developed, wellintegrated and appropriate to the project s aims? Innovation: Does the project employ novel aims, concepts, approaches and methods? Investigator: Is the investigator appropriately trained? Environment: Does the scientific environment contribute to the probability of success?
Grant Writing Tips (The Scientist,10/02; etc) Follow the guidelines, etc (font size, # of pages, misspellings, incomprehensible text) Make contact with the grant sponsor (success rate is lower when skip this) Focus on RFAs subscribe to services like ScienceWise, or AbelesR@OD.NIH.GOV re Behavioral and Social Sciences Research Guide to NIH Grants Try, try again Take the advice that comes from review s critiques; in revisions, carefully and clearly respond to reviewer s issues; successful grant writing is a matter of perseverance and a thick skin
Grant Writing Tips (continued): Develop a proposal that FITS Fills an important gap in knowledge Is Interesting to you, your field, and the funding agency; researchers need to tailor their ideas to the mission of the granting agency Tests a hypothesis; descriptive fishing-expedition proposals are not viewed as highly as experimental, hypothesis-driven ones Has a Short-term attainable goal, but also meshes with the granting agency s long-term goals; evidence of feasibility is important; don t overpromise (e.g. promise to cure cancer in 3 years); avoid The Christmas Tree Light Effect, where if one experiment in a series fails, the rest of the project fails
Tips from the trenches Make it readable (breaks between paragraphs, use clear transitions and headings and subheadings, number and bold the headings and subheads Summarize at end of sections: what are the important gaps this application will address, what is the significance, what is the innovation BE SHAMELESS HERE Aims and testable hypotheses: these aims/hypotheses should go from B&S to measures to data analyses
Tips from the trenches (cont) Conceptual framework must be present and specific to this application Define key constructs with brief lit. review and make sure they are assessed specifically by the measures used Make sure that terms like mediators and moderators are used correctly If moderators are proposed, don t just provide background on the main effects of the moderator variables on the outcomes
Tips from the trenches (cont) Make sure details are in agreement throughout (e.g.sample size, names of conditions..) Use multi-source, multi-method measures when possible Special attention to providing details in the Data Analytic section (examples, clear power estimates) Importance of pilot data and preliminary studies, when appropriate Importance of the investigative TEAM
Sources of Information on the Grants Process Video of a mock study section meeting Inside the Grants Process http://www.csr.nih.gov/video/video.asp
Where to Find Information CSR website (http://www.csr.nih.gov) Forms (PHS 398, PHS 416) SRG Structures and Rosters Video of study section in action (staged) Institute websites (http://www.nida.nih.gov) Program Announcements Institute Divisions and Offices SRG Structures and Rosters
NIH Websites HTTP://WWW.NIDA.NIH.GOV/FUNDING/FAQS.HTML HTTP://GRANTS.NIH.GOV/GRANTS/INTRO2OER.HTM HTTP://GRANTS.NIH.GOV/GRANTS/GRANT_TIPS.HTM HTTP://CMS.CSR.NIH.GOV/PEERREVIEWMEETINGS/CSRIR GDESCRIPTION/ HTTP://CMS.CSR.NIH.GOV/RESOURCESFORAPPLICANTS/ POLICYPROCEDUREREVIEW+GUIDELINES/ HTTP://GRANTS.NIH.GOV/GRANTS/PEER/PEER.HTM HTTP://GRANTS.NIH.GOV/GRANTS/OER.HTM HTTP://CRISP.CIT.NIH.GOV/