Updates from the Review Committee for Internal Medicine

Transcription

1 Updates from the Review Committee for Internal Medicine APDEM June 22, 2014 Jerry Vasilias, PhD, Executive Director, RC IM

2 What s new NAS is here Brief summary of NAS Highlights from last RC meeting Workflow Citations and non citations Highlights Different types of visits One step back

3 Why NAS? To achieve promise of outcomes based accreditation Annual review of programs to identify problem programs to help them improve Reduce the burden of accreditation Some key elements of NAS: Most data in NAS already in place Annual ADS data entry replaces PIFs Self studies every 10 years Site visited only when issues arise Internal Reviews no longer required Programs in good standing can innovate with detail PRs Significant % of common and specialty PRs are detail Citations, Areas for Improvement 2013 Accreditation Council for Graduate Medical Education Information Current as of December 2, 2013 The Data Elements of NAS The following are the primary annual data elements: 1) Program Attrition 2) Program Changes 3) Scholarly Activity 4) Board Pass Rate * 5) Clinical Experience Data 6) Resident Survey 7) Faculty Survey 8) Milestones Understand that this is a work-in-progress

4 Clinical Experience Data The following are the primary annual data elements: 1) Program Attrition 2) Program Changes 3) Scholarly Activity 4) Board Pass Rate 5) Clinical Experience Data 6) Resident Survey 7) Faculty Survey 8) Milestones Annual Data Element #5: Clinical Experience Data Proxy for case/procedure logs Broad + Brief, included in fellow/survey Assesses fellows perceptions of clinical preparedness experience w variety of clinical problems/stages of disease experience w patients of both genders & ages continuity experience will competently perform medical/ diagnostic procedures of subspecialist will provide patient care that is compassionate, appropriate and effective for the treatment of health problems and promotion of health Implemented in this year s administration of fellow survey Sample survey available on website

5 Where did most of the NAS data elements come from? In 2009, data modeling project began to identify factors that predicted high and low program performance Model was replicated, results were reproducible Selection of Elements needed to be Obtainable Meaningful Correlates w/ prior decisions Passed statistical muster Used in combination Understand that this is a work in progress New data elements likely in future Role of Review Committees in NAS Use data and judgment to: concentrate efforts on problem programs determine whether accreditation standards are violated and provide useful feedback for programmatic improvement determine whether these violations (citations) rise to a level requiring alteration in accreditation status motivate programs to rapidly improve, rather than playing the accelerating accreditation action game over time, understand and refine the nuances of the process

6 Two steps forward. NAS Year 1: Ground Rules Basic operational principle of NAS: RCs will take an accreditation action on every program annually. All programs will receive notice regarding accreditation status btw January and July. At January and May 2014 meetings, RC reviewed NAS data submitted in AY ADS annual update information submitted in fall of 2012 Faculty and Resident survey data from spring of 2013 Responses to previous citations, major changes in program,?s related to evaluation, duty hours and clinical learning environment were current

7 NAS Year 1: Ground Rules All programs on warning or probation seen by reviewers All programs identified by NAS data as troubled underwent review by RC staff and then members What data elements were triggered? Not all data elements have same importance/weight Are programs still getting used to data elements (e.g., scholarly activity table)? Are there patterns/trends in data?

8 What did we expect? 84% of core internal medicine residency programs had a review cycle between 3 5 years * * ACGME Data Resource Book , based on 378 core programs. Book available on NAS Conceptual Model Expected Outcomes STANDARDS Initial Accreditation Accreditation with Warning New Programs, Accredited Programs with Major Concerns Continued Accreditation Accredited Programs without Major Concerns New Programs Probationary Continued Accreditation with Accreditation Commendation 2-4% 15% 75% Structure Resources Core Process Detailed Process Outcomes Structure Resources Core Process Detailed Process Outcomes Structure Resources Core Process Detailed Process Outcomes Structure Core Process Resources Detailed Process Outcomes Withhold Accreditation Withdrawal of Accreditation 6-8%

9 NAS Conceptual Model Expected Outcomes New (Initial) 2-4% Warning/ Probation 15% Good Standing 75% Withheld/ Withdrawn 6-8% NAS Year 1: Expected vs Actual Outcomes Warning/Probation 5% New Programs (Initial) 3% Withheld: One Program * Site visit scheduled 3% * NAS Projections 75% 15% 6-8% 2-4% Continued Accreditation (Good Standing) 89% 390 Core Internal Medicine Residency Programs

10 What happened in year ONE of NAS What will happen in year TWO of NAS

11 RC Work in 2014 All accredited programs will be reviewed annually. All programs will receive notice of annual review before end of AY. Accreditation Status Options Continued Accreditation Accredited Program Continued Accreditation (CA) CA w/warning Other (e.g. egregious) Site Visit Probationary Accreditation* Withdrawal of Accreditation** CA w/warning CA * Probation cannot exceed 2 years ** Does not require Probation first

12 Citations in NAS Citations are not new Identify areas of non compliance Linked to specific requirements Responses required in ADS Citations are given and removed by RC (not by staff) Citations received in NAS (after July 1, 2013): will require an RC member to review annually. Citations received in OAS (given prior to July 1, 2013): will go away after two cycles of continued accreditation in NAS with no new citations. Areas for Improvement (AFI) New in NAS General concerns May be given/removed by staff (RC rules) or by RC Might not be specifically linked to a requirement Do not require written response in ADS Will be monitored locally Will be tracked by RC Will likely be used more frequently than citations in hopes that AFI s trigger appropriate local response

13 Highlights Subs dependent on cores Self study = RC will see entire department Probation for core = probation for subs Subs annual review data affect core s review No review cycles anymore; because Annual Accreditation. Except for newly accredited programs they get 2 year cycle + a site visit Who will receive a LON? Core always receives LON Subs always copied/listed on core s LON Subs sometimes receive individual LONs if no new citations/afi, sub will not receive separate LON No longer propose adverse actions Core and Subs have same status options Probation now an option for subs Site Visits in NAS Full site visits Focused site visits for an issue Self study visits every ten years

14 NAS Site Visits: Full vs Focused Full Application for a new core program At the end of the initial accreditation period RRC identifies broad issues/concerns Serious conditions or situations identified by the RRC Notification given ~ 60 days Focused Potential problems identified during annual review To diagnose reason for deterioration in performance To evaluate complaint Minimal notification given (~ 1 month) Both Minimal document preparation expected Team of site visitors NAS Site Visits: Self Study Visit Not fully developed Will review core + subs together Scheduled every ten years Conducted by a team of visitors Minimal document preparation Interview residents, faculty, leadership Begin in summer of 2015 for IM Examine annual program evaluations Response to citations Faculty development Focus: Continuous improvement in program Learn future goals of program Will verify compliance with core requirements 09SelfStudy.pdf 2013 Accreditation Council for Graduate Medical Education (ACGME)

15 Questions? Jerry Vasilias (312) Karen Lambert (312) Billy Hart (312) Jessalynn Watanabe (312) Thank you. You can t teach an old dogma new tricks. Dorothy Parker