Research Data and Reporting Ben Jacobs, Manager, Research Data and Reporting Research Services Office (RSO)
What is ERA? Excellence in Research for Australia (ERA) is an Australian government framework that evaluates the quality of the research undertaken in Australian universities against national and international benchmarks (an example of which are RCIs).
What is ERA? Personally I think it is an Exercise in Reviewing Acronyms
Research Data & Reporting Team Service charter: Provide services and support processes that involve the collection and reporting of research data
External data reporting Service Charter: Coordinating regular statutory reporting processes such as submissions for the Excellence in Research for Australia (ERA) program and the Higher Education Research Data Collection (HERDC) Other research data reporting processes include: NSRC (KCI) Survey ABS Survey of R&D Expenditure
HERDC Research Income ACGR (Category 1) OPS (Category 2) IOR (Category 3) CRCs (Category 4). (Category 3 includes Australian Contracts, philanthropic and foundation research which is not eligible to be reported in Category 1, International (NIH etc.) and International HDR or RHD fees. The NIH grant submission system is called era not to be confused with our ERA.) I could list a whole list of schemes in each category, but that would just be OTT (See SO s presentation on RP at Lunch for more information on funding opportunities and to possibly learn more acronyms).
HERDC Publications A1, B1, C1, E1 (A1 = Articles? B1 = Books? C1 = Chapters / Conference Papers?) WRONG! A1 = Books B1 = Chapters C1 = Articles E1 = Conference Papers
The Changing Environment Previously HEPs were eligible for RBG schemes such as RIBG, IGS, and HDR students were funded by the RTS, APAs and IPRS then came the JRE and SRE which included a TC component and an EI pool. Change was inevitable. Clearly there were just too many acronyms to remember!
The Changing Environment
The Changing Environment National Innovation and Science Agenda ( NISA ) Emphasis on business, industry, community organisation engagement, innovation and commercialisation Changes to block funding: New Research Support Block Grant drivers: 50% Cat 1 (ACGR), 50% Cat 2-4 (Cat 4 = CRCs) New Research Training Block Grant drivers: 50% completions, 25% Cat 1, 25% Cat 2-4
Research Engagement and Impact Assessment A range of new initiatives to support research, incentivise innovation and entrepreneurship, reward risk taking, and promote science, maths and computing (STEM) in schools Experiment Bravely Funding to conduct a national assessment measuring impact and engagement in university research ARC and Department of Education to develop methodology in 2016, including extensive sector consultation Pilot assessment in 2017 Full assessment in 2018 as a companion to ERA
Internal data capture and reporting Service charter: Support Flinders researchers, administrative staff and managers by providing advice, reports and analysis on research activity to inform strategic decision making processes.
Council level KPIs Research income: currently approximately $50 million pa, will rise to approximately $125 million pa by 2025 Research Quality: Improve the quality of publications (as measured by publications in the top 10% journals) by 50% by 2021
Council level KPIs
ERA recognises Quality (but critical mass / quantity is still important) Apportioned publications (adjusted for multiple Flinders authors) 0 2,000 4,000 6,000 8,000 10,000 Contributions to ERA Rounds - Flinders ERA2010 ERA2012 ERA2015
Council level KPIs ( SJR ) proxy for publication quality
Internal data capture and reporting Publications quantity and quality proxies / metrics IPP SNIP CPP SJR JCR, contains IFs
Systems
Systems Projects, Contracts and Publication details are recorded in the Research Management Information System (RMIS) [not to be confused with RMS or RGMS] Publications are currently entered into Research Master (RM) not to be confused with the OTHER RM (Records Manager) However, RSO will enter publications from CVs (in the first instance)
Types of research data captured Publications Projects Projects in the RMIS contain a listing of relevant personnel and these listings are used to allocate HERDC reported income equally between Flinders CIs and PIs, but not AIs, except when the PIs are really AIs, in which case only the CIs are attributed income.. (as clear as mud?) Contracts But not RHD / HDR students
RePortal
Other contributions We work with many other areas of the university to report and publicise research activity: Staff business pages (publications) Data for Academic Promotions eresearch @ Flinders ERSA, RDA, RDSI, etc. etc. etc. ORCID Public profiles ROPE (Research Opportunity & Performance Evidence) contains pubs and projects
Other contributions Service Charter: Maintain the Research section of the University website.
Other contributions Service charter: Publish the RSO s Inspiring Research newsletter, including interviewing of subjects for articles IR (Inspiring Research) not to be confused with IRs (Institutional Repositories). We do push some data through to the University s repository [FAC] for processes such as ERA.
Finally (last acronym I promise!) Q&A
Web Links http://research.flinders.edu.au (RePortal) http://www.orcid.org http://www.arc.gov.au/excellence-research-australia http://www.arc.gov.au/nisa https://www.scopus.com/ https://www.journalmetrics.com/ http://www.scimagojr.com/ https://incites.thomsonreuters.com/ https://jcr.incites.thomsonreuters.com/
Research in different ERAs Past, Present and Future Ben Jacobs, Manager, Research Data and Reporting Research Services Office (RSO)
What is ERA? Excellence in Research for Australia (ERA) is an Australian government framework that evaluates the quality of the research undertaken in Australian universities against national and international benchmarks (an example of which are RCIs).
What is ERA? The reporting process is administered by the Australian Research Council ( ARC ) and the outcomes / ratings are determined and moderated by committees of distinguished researchers from Australia and overseas ( RECs ). The unit of evaluation (UoE) in ERA is broadly defined as the Fields of Research (FoRs) within an institution based on the Australia and New Zealand Standard Classification (ANZSRC) which was developed by the Australian Bureau of Statistics (ABS).
What is ERA? Indicators used in ERA include a range of metrics such as citation profiles ( CA disciplines) which are common to disciplines in the natural sciences, and peer review of a sample of research outputs ( PR disciplines) which is more broadly common in the humanities (HCA) and social sciences (together = HASS). Peer review disciplines allow a larger range of research outputs to be submitted including Non-Traditional Research Outputs (NTROs) ERA also includes esteem measures such as membership of AIATSIS or fellows of ACOLA partners such as ATSE
What is ERA? Some of these metrics such as income and publications are/were also reported in the Higher Education Research Data Collection ( HERDC ); FTE were reported in the HESDC and patents were reported in the National Survey of Research Commercialisation ( NSRC ) ERA is a comprehensive collection. The data submitted by universities covers all eligible researchers and their research outputs. Every discipline meeting the volume threshold is assessed. And a few more just for good measure UoE, UoA, RQF, RAE, REF, PBRF, THE, SEO, TOA
Fields of Research (FoR)
The ERA Rating Scale
High performing disciplines at Flinders (Rated 5 )
ERA Indicators - Citation Analysis 25 Benchmark Citation Numbers by Age of Output Average Number of Citations 20 15 10 5 0 2 Years 3 Years 4 Years 5 Years 6 Years 7 Years Age of Output ERA2010 Benchmark ERA2012 Benchmark ERA2015 Benchmark ERA2015 Flinders (3)
ERA Indicators - ERA Peer Review Rationale and Process The quality of a research output or of the work of a group of scholars is most appropriately determined by a community of peers Relative quality Importance Impact Institutions nominate representative sample 30% Rating = REC Committee s evaluation informed by reviewer assessments + indicator information
SCImago Journal Rank (SJR)
InCites Journal Citation Reports (JCR) - IFs