Remedial education Assessor feedback

Transcription

1 Remedial education Assessor feedback IPAC Conference 2013 Dr Steven Lillis Medical Adviser, MCNZ Nikita Takai Professional Standards Coordinator, MCNZ

2 Background Results of remediation

3 New Zealand 13,500 active medical practitioners All are covered by the same legislation (Health Practitioners Competency Assurance Act 2003) All required to have Annual Practicing Certificate issues by the responsible authority (The Medical Council of New Zealand)

4 The Act (HPCAA 2003) Requires the authority to review the competence of a health practitioner when concerns have been raised about competence Requires the authority to decide if the practitioner meets or does not meet the required standard on the basis of the review.

5 The Act (HPCAA 2003) Permits the authority (MCNZ) to order Competence program Recertification program

6 Competence/recertification program Pass any examination or assessment or both Complete a period of practical training Undertake a course of instruction Undertake a period of supervised practice Allow another health practitioner to examine clinical notes.

7 Who gets notified? H.D.C 47% Other 24% MCNZ Public 13% Health practitioner 16%

8 Process... Notification (200) Complaints triage Considered by Council (70) Assessment ordered (40)

9 Structuring the assessment Council orders assessment with areas of concern Converted into Terms of Reference that guide the assessment.

10 Performance Assessment Team 3 people, 2 peers and a lay person over 1 2 days. At the place of work of the doctor

11 Performance Assessment Team Always includes: Interview Notes review Observation of consultations Case based oral Closing interview

12 Performance Assessment Team Always includes: Interview Notes review Observation of consultations Case based oral Closing interview

13 Performance Assessment Team Depending on type of concerns, may include: Observation of procedures Prescribing addictive medications review Professional interactions assessment Complementary and Alternative Medicine

14 Outcome RECOMMENDATIONS: Dr XXX needs remediation in communication skills and in understanding how to record accurate clinical notes.

15 Failed assessments...the results of 24 consecutive cases of failed assessments.

16 Distribution Category 1 (no concerns) Category 2 (concerns in some areas of practice) 14 Category 3 (widespread concerns) - 10

17 Age

18 Age Variable This study % National data % Age (p < ) <

19 Gender

20 Gender Variable This study % National data % Gender (p = ) Male Female 21 40

21 Scope of practice Variable This study % National data % General practice Psychiatry 8 5 General Surgery 8 2 Internal medicine 4 7 Radiology 4 3 Not vocationally registered 50 32

22 Nature of concerns (top 6) Nature % of doctors Clinical records 65 Communication 52 Organisational 48 Clinical knowledge 43 Clinical reasoning 43 Prescribing 39

23 Progress 24 failed assessment 19 entered remediation 5 did not engage in remediation

24 12 months Education program Detailed remediation plan Monthly education supervisor meetings and reports A range of possible tasks: Audit Communication skills course Joining peer groups Notes reviews

25 Education supervisors Experienced clinicians Same or very similar scope Experience in delivering education Attend training with MCNZ Supported by case managers and advisers Remedial education is fundamentally different

26 Progress 24 failed assessment 5 did not engage in remediation 19 entered remediation 13 had satisfactory monthly reports 6 had formal 2 nd assessment

27 Same format as 1 st 2 nd Assessment One member of previous team included

28 Progress 24 failed assessment 5 did not engage in remediation 19 entered remediation 13 had satisfactory monthly reports 6 had formal 2 nd assessment 5 were found not to be competent

29 Doctor 1st assessment 2nd Assessment A Category 2 Category 3 B Category 2 Category 1 C Category 3 Category 2 D Category 3 Category 2 E Category 3 Category 3 F Category 3 Category 3

30 Doctor 1st assessment 2nd Assessment A Category 2 Category 3 B Category 2 Category 1 C Category 3 Category 2 D Category 3 Category 2 E Category 3 Category 3 F Category 3 Category 3

31 Doctor 1st assessment 2nd Assessment A Category 2 Category 3 B Category 2 Category 1 C Category 3 Category 2 D Category 3 Category 2 E Category 3 Category 3 F Category 3 Category 3

32 Doctor 1st assessment 2nd Assessment A Category 2 Category 3 B Category 2 Category 1 C Category 3 Category 2 D Category 3 Category 2 E Category 3 Category 3 F Category 3 Category 3

33 Conclusion Assessment and remediation dependent on legislation Workplace based assessment is feasible and can be robust Multiple modes of assessment in workplace based assessment Remediation requires good structure and process

34 Assessors Dr Steven Lillis Medical Adviser, MCNZ Nikita Takai Professional Standards Coordinator, MCNZ

35 Why? Assessors are contractors to MCNZ Users of methods of assessment Many are College assessors Lay assessors have unique perspective Independent and informed opinion on assessment methods

36 Process of the assessment Variable Appropriate Reasonable Thorough Unobtrusive Decision Likert scale 5 point scale used

37 Process Feedback survey requested of all 37% response rate

38

39 Assessment tools Tools General assessment Retrospective notes review Case based oral Prescribing Professional interactions Communication skills Peer ratings Observation of consults Practice systems Variable Appropriate Instructions Apply Reliability

40 Assessment tools Tools General assessment Observation of consults Retrospective notes review Case based oral Prescribing Professional interactions Communication skills Peer ratings Practice systems Variable Appropriate Instructions Apply Reliability

41 General assessment Opening interview Observation of consultations Retrospective notes review Case based oral Peer ratings Closing interview.

42

43 General assessment

44 Record review 20 sets of clinical notes (letters to GPs and to other referrers for hospital doctors) Randomly chosen Last 3 months

45 Record review

46 Lasts 1 2 hours Case based oral Based on any clinical material uncovered in the retrospective notes review or observation of consultations Uses principles of Script concordance testing

47 Script concordance testing You saw a 48 year old male with 3 days of cough, temperature of 37.5, heart rate of 92, respiratory rate of 16 and no respiratory findings. If the temperature had been 38.5 and the respiratory rate 24, how would this have changed your management?

48 Case based oral

49 Prescribing Analysis of prescribing data Modified case based oral Role plays The newly diagnosed hypertensive man aged 45. Etc

50 Prescribing

51 Communication skills Explorative semi-structured interview Covers areas of provision of information to patients patient consent power sharing patient rights privacy issues cultural sensitivity interpersonal boundaries.

52 Easy to game Communication skills Useful for International Medical Graduates (IMG)

53 Communication skills

54 Communication skills Not commonly used IMG with English as second language

55 Peer ratings

56 Peer ratings Removed due to concerns from assessors Difficult to administer Replaced by standard Multi-source feedback

57 Observation of consultations

58 Observation of consultations Standard Mini-CEX form Applicable to ward round, out-patient clinic, General practitioner type consult

59 Practice systems The objective of this assessment tool is to assess the medical practitioner s systems essential to patient safety with regard to: Premises Equipment and its safe use and maintenance Access and availability Record systems.

60 Practice systems - example Criteria for medical assessors met part met The practice has and uses correctly either a steam steriliser or autoclave for sterilisation of reusable instruments and materials (or makes appropriate alternative arrangements) Sterile or surgically clean instruments are stored in a manner that ensures maintenance of their sterility Lighting is adequate for the procedures performed not met n/a

61 Practice systems

62

63

64

65 Conclusion Limited by respondent numbers Confidence in process Confidence in the methods Believe the outcome is both fair and helpful