Pediatric Primary Care Nurse Practitioner Role Delineation Study Summary Report

Table of Contents ACKNOWLEDGEMENTS... 3 BACKGROUND... 4 ROLE DELINEATION STUDY OVERVIEW... 4 UPDATED TEST CONTENT OUTLINE... 4 ROLE OF THE CONTENT EXPERT PANEL... 4 SURVEY METHODOLOGY... 5 SURVEY CHRONOLOGY... 5 SAMPLE SELECTION... 5 SURVEY DEVELOPMENT AND MEASURES... 6 DATA COLLECTION... 7 DATA ANALYSIS... 8 SURVEY RESULTS... 10 DEMOGRAPHIC INFORMATION... 10 PRACTICE DESCRIPTIONS... 11 APPENDICES APPENDIX A: WORK ACTIVITIES STATEMENTS... 14 APPENDIX B: DEMOGRAPHIC DATA SUMMARY... 17 APPENDIX C: WORK ACTIVITIES DESCRIPTIVE STATISTICS... 31 APPENDIX D: WORK ACTIVITIES MEAN OVERALL CRITICALITY RANK ORDER... 35 Summary Report 2

Acknowledgements The American Nurses Credentialing Center (ANCC) wishes to thank the content experts who served on the 2014 Pediatric Primary Care Nurse Practitioner Role Delineation Study Content Expert Panel (CEP) for sustaining this effort and producing a role delineation study of such high caliber. Without their numerous hours of input and feedback, the study would not be possible. We also acknowledge the ANCC staff members, who contributed to the preparation and implementation of the Role Delineation Study and the writing of this report: Measurement Services Department Leaders Tim Sares, PhD, Assistant Director Jaehoon Seol, PhD, Assistant Director Lee Skinner, MA, Director Measurement Services Staff Contributors Cheray Jones, Data Specialist Teresa Myers, Administrative Assistant Denise Mahone, Senior Administrative Assistant Gossie Nworu, MS, Data Specialist Chie Ohba, PhD, Survey & Data Analyst Meeting Facilitators & Presenters Daria Ellis, PhD, Research Scientist Nichole Thomas, PhD, Research Scientist Report Author Nichole Thomas, PhD, Research Scientist 3

Background The American Nurses Credentialing Center (ANCC), which was incorporated in 1991 as a subsidiary of the American Nurses Association (ANA), is the largest nursing credentialing organization in the United States of America. ANCC s vision is to drive nursing excellence, quality care, and improved outcomes. Currently, ANCC offers 25 examinations at various levels including diploma and associate degree, baccalaureate, and advanced practice for nurse practitioners, clinical nurse specialists, and other disciplines. More than 22,000 candidates took an ANCC certification examination or submitted a portfolio for ANCC certification in 2014. In addition to certification, ANCC provides services such as the Magnet and Pathways to Excellence recognition programs for hospitals and other facilities that demonstrate excellence in nursing services, accreditation of continuing education programs, education and consultation services, and outreach to nursing organizations around the globe. Role Delineation Study Overview Role delineation or job analysis studies are typically carried out at the national level with the goal of describing current practice expectations, performance requirements, and environments. ANCC s current goal is to conduct a study of each advanced practice specialty approximately every four years, in order to capture changes in work activities and the knowledge and skill areas required to perform those activities. The findings of a role delineation study are used to update the content of a certification examination. Updated Test Content Outline The results of this role delineation study were used in developing and updating the test content outline for the Pediatric Primary Care Nurse Practitioner Examination. Examination forms produced based on the Pediatric Primary Care Nurse Practitioner Test Content Outline developed through this study are scheduled to go into effect on February 9, 2016. Role of the Content Expert Panel Throughout the study, ANCC invited professionals in practice and educators who teach courses relevant to pediatric primary care nurse practitioners to serve on content expert panel (CEP) for this study. The internal qualifications and assembly criteria for the Pediatric Primary Care Nurse Practitioner CEP were met. All of the content experts serving on the CEP were certified by ANCC as Pediatric Primary Care Nurse Practitioners and were invited to serve on the panel based upon their expertise in the specialty. The CEP developed the work activities and demographic items for the survey, linked knowledge and skill areas to the work activities list, and finalized the test content outline. 4

Survey Methodology The 2014 Pediatric Primary Care Nurse Practitioner Role Delineation Study involved two sets of activities that ran concurrently: a national web-based survey and a linking activity. The purpose of the development and administration of the national survey was to collect information on the work activities pediatric primary care nurse practitioners actually perform in practice. The role delineation study panel met for three days from February 19-21, 2014, to draft a pilot version of the survey and construct the initial map of knowledge, skills, and abilities (KSAs) relevant to the work activities included in the survey. A linking activity was also conducted to identify the major KSA domains and subdomains required to successfully perform the work activities listed in the survey. The results of both the survey and linking activity were used as foundational documents for updating the test content outline and item distribution. Survey Chronology The survey development, administration, and review spanned 10 months. February June 2014 The role delineation study panel along with ANCC staff drafted the survey. The survey was pilot tested and revised. September - October 2014 The CEP completed an activity to link all of the work activity survey items to the domains and subdomains in the KSA map. The final survey was administered on the web. November December 2014 The survey activity results were analyzed, and activity weights were determined. The panel met to review the survey results and activity weights; and to discuss revisions to the test content outline and item distribution for the certification examination. Sample Selection In April 2014, there were a total of 962 actively certified ANCC pediatric primary care nurse practitioners with mailing addresses in the United States. One hundred of these nurse practitioners were selected via a stratified random sample, based on region, to participate in the pilot survey and all other ANCC-certified pediatric primary care nurse practitioners were invited to participate in the national survey. Table 1 presents the ANCC-certified pediatric primary care nurse practitioners selected to participate in the national survey from each region. 5

Table 1. Number of ANCC-certified Pediatric Primary Care Nurse Practitioners Selected per Geographic Region Geographic Region Number of Selected ANCC Certified Percent of Total Population Midwest IA, NE, KS, OH, MO, MN, SD, ND, MI, IL, IN, WI 163 16.94 Northeast NY, CT, MA, NJ, ME, PA, NH, VT, RI 335 34.82 Other AE, AP, APO 1 0.00 South TN, MS, TX, FL, LA, AL, GA, AR, OK, VA, MD, SC, DC, NC, WV, DE, KY 342 35.55 West WA, AZ, CA, OR, CO, AK, ID, NM, UT, HI, NV, WY, MT 121 12.58 Total 962 100.00 Survey Development and Measures From February 19-21, 2014 the CEP met in Silver Spring, Md., to draft the national Pediatric Primary Care Nurse Practitioner Role Delineation Study Survey for the 2014 Role Delineation Study. The CEP reviewed and discussed the National Association of Pediatric Nurse Practitioners, Society of Pediatric Nurses, and American Nurses Association s Pediatric Nursing: Scope and Standards of Practice (2008), as well as the work activities that were used in ANCC's 2011 Role Delineation Survey of Pediatric Primary Care Nurse Practitioners. Across multiple sessions in the three-day meeting, the CEP discussed and deliberated on additions, deletions, and revisions that should be made to update the 2014 work activity list to reflect current pediatric primary care nurse practices. As a result of this meeting, the panel reached consensus on a list of 83 work activities to be used in the 2014 survey. These work activities were divided into four domains: Assessment, Diagnosis, Clinical Management, and Role. The complete text of the work activities list is presented in Appendix A. The workgroup also identified and finalized a set of 18 demographic questions, which are presented in Appendix B. During this meeting, the panel also reviewed and approved three scales that survey respondents would use to rate the work activities listed in the survey: Frequency (the frequency with which a work activity is performed), Performance Expectation (how soon on the job the performance of an activity is expected), and Consequence (the consequence of performing an activity incorrectly). The three questions associated with each work activity, and the instructions for answering them, are presented in Table 2. Table 2. Survey Questions for Rating Work Activity Statements Please respond to each activity with three separate responses, one response in each category. When considering a response for one category, do not consider the other categories. For example: when considering the consequences of incorrect performance of an activity, do not worry about how frequently the activity is performed or whether it is expected to be performed. The possibility exists that an activity has severe consequences, even if it is rarely performed. Performance Expectation: When is the nurse newly certified as a pediatric primary care nurse practitioner first expected to perform this activity? -- Within the first 6 months of certification. -- After the first 6 months of certification. -- Never expected to perform this activity. Frequency: Considering your setting, how often does the nurse newly certified in psychiatric-mental health nursing perform this activity? -- Not performed (The nurse newly certified in psychiatric-mental health nursing never performs this activity) -- Seldom (The nurse newly certified as a pediatric primary care nurse practitioner performs this activity a few times a year) -- Monthly (The nurse newly certified as a pediatric primary care nurse practitioner performs this activity approximately once a month) -- Weekly (The nurse newly certified as a pediatric primary care nurse practitioner performs this activity approximately once or twice a week) -- Daily (The nurse newly certified as a pediatric primary care nurse practitioner performs this activity approximately every day or multiple times a day) 6

Consequences: Incorrect performance of this activity could cause the patient: -- Little or no physical or psychological harm. -- Moderate physical or psychological harm. -- Severe physical or psychological harm. The study design included combining each respondent s responses to each of the three rating scales in a hierarchical manner into one overall ranking of criticality. To select a procedure for combining the three scales, the importance of each scale to the performance of the work activity was considered. The performance expectation scale was determined to be more critical than the other two scales for representing entry-level practice, and the performance expectation scale was specifically designed to distinguish entry-level skills from advanced skills. The consequence scale was regarded as less critical than the performance expectation scale, but more critical than the frequency scale. Therefore, the scales were combined so that a particular value on the performance expectation scale would outweigh or outrank all values on the consequence and frequency scales. This hierarchical scheme emphasized the work activities that are required of newly certified specialists and have the greatest impact on public health or safety. Thus, this hierarchical scheme was used as the organizing mechanism for combining responses from the three survey scales into an overall measure of criticality. Data Collection Pilot Testing. Using the same procedures intended for administering the final survey, the survey was piloted in May and June of 2014. One-hundred ANCC-certified Pediatric Primary Care Nurse Practitioners were selected using a stratified random sample from the entire population of board certified nurse practitioners in the specialty area. Fourteen of the Pediatric Primary Care Nurse Practitioners invited to take the pilot survey completed the survey. The results of the pilot test indicated that the work activities were appropriate and reflective of the job of the Pediatric Primary Care Nurse Practitioners. However, a post-pilot survey conference call was conducted to present the results of the pilot survey and provide CEP members with an opportunity to make any needed revisions to the survey. National Survey. In September and October of 2014, the 962 ANCC-certified Pediatric Primary Care Nurse Practitioners invited to take the national web-based survey were sent at least two of three notifications via the United States Postal Service: an alert letter, and two follow-up reminders. The alert letter explained the purpose and importance of the study, the eligibility criteria of the study, and stated how to access the survey via the internet. The letter indicated that the participant s responses would be kept confidential. The letter also noted that respondents completing the survey would receive a five hour reduction of their continuing education requirement for ANCC recertification in the specialty area. The first follow-up reminder letter was sent about two weeks after the alert letter to all respondents. The reminder included language thanking individuals who had already submitted their completed survey, as well as language encouraging individuals to complete the survey, if they had not already done so. The final follow-up reminder letter was sent out only to individuals who had not yet responded to the survey 7

and was sent out approximately two weeks prior to the end of the survey administration period. Data Analysis The three rating scales were combined into a single measure of overall criticality using a hierarchical method. As agreed by the CEP, the three rating scales were combined into a single measure in such a manner that a particular value on the performance expectation scale would outweigh or outrank all values on the consequence and frequency scales, and that a particular value on the consequence scale would outweigh or outrank all values on the frequency scale. Table 3 displays how the values of the overall criticality rating were constructed according to all the possible survey response patterns that might be given to rate an individual work activity by its frequency, performance expectation, and consequence. For example, if a respondent indicated that a particular work activity was expected to be performed within the first six months of certification as a pediatric primary care nurse practitioner, could cause severe negative consequences if it was performed incorrectly, and is performed occasionally, the overall criticality rating for that response pattern would be 39. A score between 37.00 and 41.00 indicated that a work activity is generally expected to be performed within the first six months of certification as a pediatric primary care nurse practitioner and could cause severe negative consequences if incorrectly performed. A score between 32.00 and 36.99 suggests that a work activity is generally expected to be performed within the first six months of certification as a pediatric primary care nurse practitioner and could cause moderate negative consequences if incorrectly performed. Work activities with scores of 32 or higher on the overall criticality variable may be considered as highly critical. When a work activity was rated as never expected on the performance expectation scale, it received an overall criticality score of 1 as the bottom row in Table 3 indicates. 8

Table 3. Construction of the Overall Criticality Variable Survey Response Options Overall Performance Expectations Consequences Frequency Criticality Ranking Within the first 6 months of certification within the specialty Always 41 After the first 6 months of certification within the specialty After the first 6 months of certification within the specialty (Continued) Severe negative consequences Moderate negative consequences Mild negative consequences No negative consequences Severe negative consequences Moderate negative consequences Mild negative consequences No negative consequences Frequently 40 Occasionally 39 Seldom 38 Never 37 Always 36 Frequently 35 Occasionally 34 Seldom 33 Never 32 Always 31 Frequently 30 Occasionally 29 Seldom 28 Never 27 Always 26 Frequently 25 Occasionally 24 Seldom 23 Never 22 Always 21 Frequently 20 Occasionally 19 Seldom 18 Never 17 Always 16 Frequently 15 Occasionally 14 Seldom 13 Never 12 Always 11 Frequently 10 Occasionally 9 Seldom 8 Never 7 Always 6 Frequently 5 Occasionally 4 Seldom 3 Never 2 Never expected to perform this activity within the specialty 1 9

Survey Results The total sample of the national survey included 962 ANCC certified pediatric primary care nurse practitioners. A total of 244 surveys were returned, with 187 surveys completed yielding an overall response rate of 25.36% percent and a total usable response rate of 19.44%. Table 4 includes the percent of surveys per population returned in each geographic region compared to the number of ANCC-certified pediatric primary care nurse practitioners selected within the region. Table 4. Number of Surveys Returned per Geographic Region Geographic Region Midwest IA, NE, KS, OH, MO, MN, SD, ND, MI, IL, IN, WI Number Selected (Percent of total pop.) Number Return (percent of total pop.) 163 (16.94) 29 (15.51) Northeast NY, CT, MA, NJ, ME, PA, NH, VT, RI 335 (34.82) 69 (36.90) Other AE, AP, APO 1 (0.10) 0 (0.00) South TN, MS, TX, FL, LA, AL, GA, AR, OK, VA, MD, SC, DC, NC, WV, DE, KY 342 (35.55) 60 (32.09) West WA, AZ, CA, OR, CO, AK, ID, NM, UT, HI, NV, WY, MT 121 (12.58) 29 (15.51) Total 962 (100.00) 187 (100.00) Demographic Information Appendix B includes the survey responses to the 18 demographic questions, which included inquiry about the respondents background and practice settings. Demographic Background Results from the demographic portion of the survey indicated that approximately 98% of the survey respondents were female, and 2% were male. Most respondents, 35%, were in the 55 to 64 age range; the 45 to 54 age range, in which 31% of respondents fell; or in the 35 to 44 age range, in which 17% of the respondents fell. In addition, 11% of respondents were in the 25 to 34 age range, and 7% of respondents were 65 or older. Eighty-seven percent of respondents identified as White, 5% of respondents identified as Hispanic or Latino, 4% of respondents identified as Black or African-American, 1% of respondents identified as American Indian or Alaska Native, and 1% of respondents identified as Native Hawaiian or Other Pacific Islander. Less than 1% of respondents identified as Middle Eastern or other. Thirteen percent of respondents had earned a professional doctorate, with 6% having earned a Doctorate in Nursing Practice, 4% having earned a Doctorate in Nursing Research, and 3% having earned a doctorate in a field other than nursing. Eighty-one percent of respondents held a Master s in Nursing, and 6% of respondents held a Master s degree in a field other than nursing. In addition, 9% of respondents had earned other educational credentials. The average number of years of experience the respondents had as a registered nurse and nurse practitioner were 23 and 18 years respectively. 10

Practice Settings Thirty-seven percent of respondents indicated that the setting of their primary clinical practice is in a private practice, 27% indicated that an outpatient hospital is the setting of their primary clinical practice, and a community or public health agency was the primary clinical practice setting of 14% of respondents. In addition, 6% of respondents indicated the setting of their primary clinical practice is a school or college health, 4% of respondents indicated that the setting of their primary clinical practice is managed care, 3% of respondents indicated that an independent nurse practitioner practice is the setting of their primary clinical practice, and a school or college of nursing was selected as the setting of primary clinical practice for 3% of respondents. Two percent of respondents indicated that a medical school is the setting of their primary clinical practice, and 8% of respondents selected other as the setting of their primary clinical practice. Forty-three percent of respondents indicated that their primary clinical practice location is in a city. A metropolitan area was the primary clinical practice location of 20% of respondents; a greater metropolitan area was the primary clinical practice location of 17% or respondents; a town was the primary clinical practice location of 17% of respondents; and a rural area was the primary clinical practice location of 4% of respondents. Sixty-one percent of respondents did not have hospital admitting privileges, and 39% of respondents did have hospital admitting privileges. Ninety-seven percent of respondents had privileges to prescribe medications in their primary clinical practices, and 3% of respondents did not have privileges to prescribe medication. Seventy-eight percent of respondents indicated that they spend zero percent of their time during an average week in research, and 49% percent of respondents indicated that they spend zero percent of their time engaged in management, supervision and administration, during an average week. Forty-six percent of respondents indicated that they spend 1% to 19% of their time during an average week engaged in teaching, and 43% of respondents indicated that they spend 20% to 39% of their time during an average week engaged in documentation. Forty percent of respondents indicated that they spend 60% to 70% of their time during an average week in direct care, and 40% of respondents indicated that they spend 1% to 19% of their time during an average week in consultation. Thirty percent of respondents indicated that they admit patients to the hospital; twenty-four percent of respondents indicated that they manage patients during hospitalization; and seventeen percent of respondents indicated that they discharge patients from the hospital. Eighty-two percent of respondents were required to have a physician collaborator/supervisor, and seventeen percent of respondents were not required to have a physician collaborator. Practice Descriptions Descriptive statistics (total count, means, and standard deviations) for the three ratings of all 83 work activities, performance expectation, consequence, and frequency, as well as and mean overall criticality are listed in Appendix C. The scales were highly reliable. Cronbach s coefficient alpha estimates for the performance expectation, consequence, and frequency scales when applied to all data were 0.9414, 0.9890, and 11

0.9667, respectively. Cronbach's coefficient alpha, a measure of internal stability, ranges in value between zero and one. In Appendix D, the overall criticality statistics are presented in rank order of criticality. As indicated in Table 5, 83 work activity statements were rated by the 187 respondents as highly critical (with a mean overall criticality rank of 32 or above). Eighty-three percent of these work activities fell into one of the following three domains: Clinical Management (40.00%); Assessment (23.33%); and Role (20.00%). Table 5. Number of Work Activities by Mean Overall Criticality Range for Pediatric Primary Care Nurse Practitioner Mean Overall Criticality Score Between 37.0 and 41 Between 32.0 and 36.9 Between 27.0 and 31.9 Between 22.0 and 26.9 Between 17.0 and 21.9 Between 12.0 and 16.9 Between 7.0 and 11.9 6.9 and under Total number above 32.0 Number of Work Activities 2 28 30 9 9 5 0 0 30 Tables 6 and 7 include the 20 highest-ranked and the 20 lowest-ranked work activities by mean overall criticality, respectively. The highest ranked task (#40 Reports suspected abuse, exploitation and/or neglect) received a criticality rank of 37.83. This work activity highlighted in Table 6 is listed under the domain Clinical Management. The lowest ranked work activity (#81 Conducts formal or informal peer reviews) received a criticality ranking of 13.09 and is highlighted in Table 7. This work activity is found within the domain Role. 12

Table 6. Top 20 Work Activities Ranked by Mean Overall Criticality Overall Criticality Standard Work Activity Number and Name Mean Deviation 40. Reports suspected abuse, exploitation and/or neglect 37.83 5.55 76. Maintains confidentiality and privacy according to regulatory standards (e.g., electronic health records, ***(**HIPAA) 37.79 5.78 37. Prescribes medications 36.89 6.20 20. Differentiates between normal and abnormal physiologic changes 36.37 6.35 19. Performs a focused physical exam 36.35 5.44 67. Refers patient with conditions beyond scope of practice 35.86 7.66 18. Performs a comprehensive physical exam 35.61 5.91 2. Obtains a focused health history 35.43 4.95 51. Evaluates effectiveness and safety of pharmacologic regimen 35.35 7.29 73. Documents patient related activities (e.g. referrals, telephone triage) 35.24 6.61 27. Formulates diagnoses 34.75 6.73 1. Obtains a comprehensive health history 34.73 5.34 43. Manages episodic disease 34.51 6.37 39. Manages health maintenance and health promotion interventions (e.g. nutrition, immunizations) 34.45 6.38 42. Plans follow-up care 34.07 5.62 16. Assesses patient for pain (e.g. acute, chronic) 34.07 6.11 74. Documents in accordance with regulatory process and payor source 33.96 8.25 38. Prescribes non-pharmacologic interventions 33.74 6.25 25. Develops differential diagnoses 33.71 7.17 75. Bills for services according to level of care 33.37 8.36 Table 7. Bottom 20 Work Activities Ranked by Mean Overall Criticality Overall Criticality Standard Work Activity Number and Name Mean Deviation 71. Promotes the role of the nurse practitioner 25.11 11.83 82. Engages in self-evaluations 24.90 11.69 36. Implements plan of care that incorporates cultural, spiritual, psychosocial considerations 24.80 12.03 41. Facilitates transitions across health care settings 24.11 11.67 13. Performs a community assessment (e.g. safety, transportation resources, nutrition resources, availability of 24.06 10.99 ***(**social services) 68. Understands the impact of policy on practice 23.83 12.64 8. Performs a spiritual assessment 21.44 12.54 66. Engages in academic and professional activities 20.57 11.80 63. Serves as an interprofessional resource for patient care 20.55 11.58 65. Participates in quality improvement 20.39 11.92 62. Advocates for improved access, quality, and cost-effective health care 20.07 12.18 77. Evaluates the impact of health care delivery on patients, providers, and stakeholders 19.28 12.85 17. Assesses for advance care planning/advanced directives (e.g. DNR, living will ) 18.79 14.26 45. Participates in palliative care (e.g. chronic pain management, symptom management) 18.42 12.50 10. Performs a genetic assessment 17.64 12.40 33. Facilitates the development of advance care planning/advance directives 16.96 12.44 78. Recognizes the impact of globalization on healthcare policy development (e.g. access to health care, disease 16.63 11.50 ***(**transmission, international adoption) 35. Develops population-focused plan of care based on epidemiologic data 15.89 12.34 46. Provides end of life care 13.14 11.97 81. Conducts formal or informal peer reviews 13.09 9.96 13

Appendix A Work Activities Statements 14

2014 Pediatric Primary Care Nurse Practitioner Exam National Survey Tasks/Activities List Role Delineation Study I. Assessment 1. Obtains a comprehensive health history 2. Obtains a focused health history 3. Performs appropriate screenings (e.g., developmental, hearing, vision, oral health, genetics) 4. Performs a psychosocial evaluation 5. Performs a lifestyle assessment (e.g., sleep, exercise, sexual behaviors, drug and alcohol use, safety, environmental) 6. Performs a nutritional assessment 7. Performs a cultural assessment 8. Performs a spiritual assessment 9. Performs an individual risk assessment (e.g., cardiac, cancer, diabetes) 10. Performs a genetic assessment 11. Performs a mental health assessment 12. Performs a family/caregiver assessment (e.g. family dynamics, communication patterns, support systems, caregiver burden) 13. Performs a community assessment (e.g. safety, transportation resources, nutrition resources, availability of social services) 14. Performs a functional assessment (e.g. mobility, cognition) 15. Assesses patient s capacity for decision-making 16. Assesses patient for pain (e.g. acute, chronic) 17. Assesses for advance care planning/advanced directives (e.g. DNR, living will ) 18. Performs a comprehensive physical exam 19. Performs a focused physical exam II. Diagnosis 20. Differentiates between normal and abnormal physiologic changes 21. Orders diagnostic tests 22. Performs diagnostic tests 23. Interprets results from diagnostic tests 24. Synthesizes data to inform clinical reasoning 25. Develops differential diagnoses 26. Prioritizes differential diagnoses 27. Formulates diagnoses 28. Prioritizes diagnoses 29. Develops a comprehensive problem list III. Clinical Management 30. Develops a patient-centered plan of care 31. Evaluates patient, family and/or caregiver s knowledge of plan of care 32. Educates patient, family and/or caregiver regarding plan of care include testing, diagnosis, treatments and/or implications. 33. Facilitates the development of advance care planning/advance directives 34. Prioritizes plan of care considering safety, risk reduction, and comorbidities 35. Develops population-focused plan of care based on epidemiologic data 36. Implements plan of care that incorporates cultural, spiritual, psychosocial considerations 37. Prescribes medications 38. Prescribes non-pharmacologic interventions 15

IV. Role 39. Manages health maintenance and health promotion interventions (e.g. nutrition, immunizations) 40. Reports suspected abuse, exploitation and/or neglect 41. Facilitates transitions across health care settings 42. Plans follow-up care 43. Manages episodic disease 44. Manages chronic disease 45. Participates in palliative care (e.g. chronic pain management, symptom management) 46. Provides end of life care 47. Performs common primary care procedures 48. Coaches patient, family, and/or caregiver regarding lifestyle and behavioral changes 49. Counsels on family planning, sexuality, and/or reproductive health 50. Evaluates patient responses to interventions 51. Evaluates effectiveness and safety of pharmacologic regimen 52. Evaluates effectiveness and safety of nonpharmacologic interventions 53. Evaluates adherence to treatment plan 54. Evaluates the impact of diagnosis and treatment on patient, family, and caregiver 55. Evaluates treatment and educational outcomes related to pain 56. Evaluates treatment and educational outcomes related to nutrition 57. Evaluates plan of care considering safety, risk reduction and comorbidities 58. Modifies plan of care to meet the needs of patient, families, and/or caregivers 59. Maintains a comprehensive problem list 60. Participates in interprofessional collaboration 61. Advocates for individual patient needs 62. Advocates for improved access, quality, and cost-effective health care 63. Serves as an interprofessional resource for patient care 64. Provides a climate of patient- and family-centered care 65. Participates in quality improvement 66. Engages in academic and professional activities 67. Refers patient with conditions beyond scope of practice 68. Understands the impact of policy on practice 69. Identifies ethical dilemmas and seeks resources for resolution 70. Identifies legal dilemmas and seeks resources for resolution 71. Promotes the role of the nurse practitioner 72. Integrates theory, current evidence, professional standards, and clinical guidelines 73. Documents patient related activities (e.g. referrals, telephone triage) 74. Documents in accordance with regulatory process and payor source 75. Bills for services according to level of care 76. Maintains confidentiality and privacy according to regulatory standards (e.g., electronic health records, HIPAA) 77. Evaluates the impact of health care delivery on patients, providers, and stakeholders 78. Recognizes the impact of globalization on healthcare policy development (e.g. access to health care, disease transmission, international adoption) 79. Prepares for emergency and/or disaster situations 80. Integrates informatics and/or health care technology into practice 81. Conducts formal or informal peer reviews 82. Engages in self-evaluations 83. Engages in interprofessional collaboration 16

Appendix B Demographic Data Summary 17

2014 Pediatric Primary Care Nurse Practitioner Exam National Survey Demographic Data Role Delineation Study Q1. Where is your primary clinical practice location? Recruitment Respondents Count Percent Count Percent Northeast 335 34.82% 69 36.90% South 342 35.55% 60 32.09% Midwest 163 16.94% 29 15.51% West 121 12.58% 29 15.51% Other 1 0.10% 0 0.00% Total 962 100.00% 187 100.00% Q2a. Which nurse practitioner certification best describes your current practice? (Choose one) Count Percent Family Nurse Practitioner 10 5.35% Pediatric Primary Care Nurse Practitioner 177 94.65% Total 187 100.00% Q2b. By which organization are you certified as a nurse practitioner in the above area by? (Choose all that apply) Other Responses (3 responses) Count Percent Count ANCC 186 99.47% NCC 3 1.60% IBCLC 1 PNCB 12 6.42% NAPNAP 1 AANP 1 0.53% palliative 1 Other 3 1.60% *N = 187 was used to compute the percentage 18

Q3a. Do you hold other nurse practitioner certification(s)? Count Percent Yes 31 16.58% No 156 83.42% Total 187 100.00% Q3b. List any other subspecialty certifications (e.g., Diabetic Certification, Lactation Consultant, Mental Health) Certification Title Certification Organization Count ACLS 1 Adolescent Medicine Children's hospital Boston 1 adult nurse practitioner ANCC 1 Adult Nurse Practitioner ANCC (dual certifications in Peds and Adults, one year apart) 1 Adult Primary Nurse Practitioner ANCC 1 AE-C Association of Asthma Educators 1 Asthma Educator NACEB 1 BC-ADM Diabetes Educators 1 BCLS Instructor American Heart Association 1 CDE NCBCDE 1 CDE AADE 1 Certified Diabetic Educator American Association of Diabetic Educators 1 Certified Pediatric Nurse (CPN) PCNB 1 Chemotherapy provider APHON 1 CORLN Society of Head & Neck nurses 1 CPHON APHON 1 CPHON ONS 1 CPN PNP 1 CPNP-AC PNCB 1 CPON Oncology Nurse Credentialing Center 1 CPON APHON 1 CPR INSTRUCTOR AMERICAN HEALTH INSTITUE 1 DNP Northeastern University 1 Education Masters in Ed. Special Education and Rehab. Counseling 1 Emergency Nurse Emergency Nurses Association 1 Family ANCC 1 (continued) 19

Certification title Certification organization Count Family American academy of nurse practitioners 1 Family Nurse Practitioner ANCC 3 Family Practice ANCC 2 Family Primary Care NP ANCC 1 FNP AANP 1 FNP ANCC 4 HIV/AIDS Counselor MDCH 1 Holistic ANCC 1 IBCLC ILCA 1 IBCLC 1 Lactation Consultant International Board of Lactation Consultant Examiners 1 lactation consultant IBCLCE 1 Lactation Consultant IBCLC 1 Lactation Consultant IBCLE 1 Mental Health First AID USA National Council for Behavioral Health 1 NNP NCC 1 PA NCCPA 1 Pain Management ANCC 1 palliative care NBCHPN/HPNA 1 PALS 1 Pediatric Nurse ANCC 1 PMHS PCNB 1 PNP 1 PNP ANCC 2 Professor/adjunct faculty no certification requirements. Based on experience 1 Public Health MPH/ tropical and infection diseases 1 sane iafn 1 School ANCC 1 School Health NP ANCC 1 SOHN Society of Otorhinolaryngology Head- Neck Nurses 1 travel medicine International Society of Travel Medicine 1 Women s Health Care Post masters course 1 Women's Health Care NCC 1 20

Q3c. List any other certifications that you hold other than nurse practitioner Certification title Certification organization Count Academy of Certified Social Workers National Assoc. of Social Workers 1 Acute Care PNP PNCB 1 Advanced Cardiac Life Support AHA 1 ALNC RN Market 1 BLS AHA 1 BLS Provider American Heart Association 1 CEN 1 Child passenger safety Technician Safe Kids Worldwide 1 clinical nurse specialist grandfather in 1 CNS BRN of CA 1 CPN PNCB 1 CPR American Heart Association 1 DEA # DEA 1 DNP Northeastern University 1 environmental toxicology southwest naturopathic college 1 FAMILY NP ANCC 1 Health Care Provider American Heart Association 1 herbal medicine private certifying body 1 Lactation Consultant IBBLC 1 Lamaze 1 licensed massage therapist national certification 1 Maternal Exercise Physiology Temple University 1 Narcan administration New York State Health Department 1 NP Furnishing # BRN of CA 1 Nurse Executive ANCC 1 PALS American Heart Association 3 PCEN 1 Pediatric nurse ANCC 2 perinatal Bereavement Counselor RTS Bereavement Services 1 PHN BRN of CA 1 pnp pncb 1 Public Health Nurse State of California 1 Red Cross Instructor American Red Cross 1 RN-BC general nursing practice 1 TNCC ENA 1 21

Q4. What is your gender? Q5. What is your age? (Choose one) Count Percent Count Percent Female 184 98.40% 25-34 years old 20 10.70% Male 3 1.60% 35-44 years old 32 17.11% 45-54 years old 57 30.48% Total 187 100.00% 55-64 years old 65 34.76% 65 and older 13 6.95% Total 187 100.00% Q6. What is your racial/ethnic background? (Choose all that apply) Count Percent White 163 87.17% Black or African American 7 3.74% American Indian and Alaska Native 2 1.07% Asian 6 3.21% Native Hawaiian and other Pacific Islander 2 1.07% Hispanic or Latino 10 5.35% Middle Eastern 1 0.53% Other 1 0.53% *N = 187 was used to compute the percentage Other Responses (1 response) Count South East Asian 1 22

Q7 How many years of experience do you have as: a registered nurse a nurse practitioner Count Percent Count Percent Less than 1 3 1.60% 1 0.53% 1 to 9 38 20.32% 37 19.79% 10 to 19 34 18.18% 70 37.43% 20 to 29 39 20.86% 50 26.74% 30 to 39 50 26.74% 26 13.90% 40 to 49 21 11.23% 2 1.07% Over 50 2 1.07% 1 0.53% Total 187 100.00% 187 100.00% Mean 23.35 18.44 Q8. Indicate the highest educational level you have completed (Choose all that apply) Count Percent Masters in Nursing 152 81.28% Masters in field other than Nursing 10 5.35% Doctorate in Nursing Research (e.g., Ph.D., DNS, DSN) 7 3.74% Doctorate in Nursing Practice (DNP) 12 6.42% Doctorate in field other than Nursing 5 2.67% Other 17 9.09% *N = 187 was used to compute the percentage Other Responses (17 responses) Count ADN, certificate PNP 1 Bachelor of science and certificate in advance practice 1 Bachelor's In Psychology 1 BSN/Certificate 1 certification program 1 (continued) Count 23

Doctor Educational Leadership 1 Family and Community 1 MPH, M. Ed 1 MPH, Masters Public Policy, Masters of Science 1 PhD behavioral pediatrics and community child health 1 PNP certificate program 1 Post masters 2 Post masters certificate in Health Professional Education 1 Post Masters Certificate in Nursing Education 1 Post Masters Nurse Practitioner Certificate 1 R.N.,. C.R.N.P.thru a certification program 1 Q9. Which one of the following best characterizes your primary clinical practice location? (Choose one) Count Percent Rural (population less than 2,500) 7 3.74% Town (population 2,500-49,999) 32 17.11% City (population 50,000-249,999) 80 42.78% Metropolitan (population 250,000-999,999) 37 19.79% Greater Metropolitan (population greater than 999,999) 31 16.58% Total 187 100.00% Q10. Which of the following describes your primary clinical practice setting? (Choose all that apply) Count Percent Community/Public Health (city/county/state/federal agency) 26 13.90% Home Health Care 0 0.00% Hospice Facility 0 0.00% Hospital, Inpatient 30 16.04% Hospital, Outpatient 50 26.74% Managed Care (HMO, Blue Cross/Blue Shield) 8 4.28% Medical School 3 1.60% Nursing Home/Long-term Care 0 0.00% Independent Nurse Practitioner Practice 6 3.21% (continued) Count Percent 24

Occupational Health 0 0.00% Private Practice 70 37.43% Retail Based Clinic 1 0.53% School or College Health 12 6.42% School/College of Nursing 5 2.67% VA/Armed Forces 0 0.00% Other 14 7.49% *N = 187 was used to compute the percentage Other Responses (14 responses) and output 1 community based clinic 1 Emergency room 1 federally qualified health care center 1 Federally qualified healthcare facility 1 FQHC 2 Free Pediatric Clinic for uninsured and those having difficulty accessing care 1 Hospital and Clinic practice combined 1 I do discharge babies from inpatient on the weekends 1 military 1 Physician owned group practice 1 Primary Care Clinic that is family medicine residency site & University affiliated 1 Private Non-profit 1 Count 25

Q11. Estimate the percentage of time (during an average week) that you provide direct patient care in each of the age groups listed below. (Note that you should only enter whole numbers without the percent sign (e.g., 32; 15) and that your total percentage should equal 100.) Infant Child Adolescent (birth to 1 year) (2 to 12 years) (13 to 17 years) Count Percent Count Percent Count Percent 0% 24 12.83% 8 4.28% 8 4.28% 1% to 19% 40 21.39% 13 6.95% 29 15.51% 20% to 39% 84 44.92% 76 40.64% 117 62.57% 40% to 59% 29 15.51% 68 36.36% 24 12.83% 60% to 79% 5 2.67% 15 8.02% 4 2.14% 80% to 100% 5 2.67% 7 3.74% 5 2.67% Total 187 100.00% 187 100.00% 187 100.00% Mean 25.52% 37.23% 27.95% Adult Aging Adult (18 to 65 years) (over 65 years) Count Percent Count Percent 0% 90 48.13% 173 92.51% 1% to 19% 71 37.97% 9 4.81% 20% to 39% 14 7.49% 4 2.14% 40% to 59% 5 2.67% 1 0.53% 60% to 79% 3 1.60% 0 0.00% 80% to 100% 4 2.14% 0 0.00% Total 187 100.00% 187 100.00% Mean 8.22% 1.07% 26

Q12. Estimate the percentage of time (during an average week) that you provide patients with: (Note that you should only enter whole numbers without the percent sign (e.g., 32; 15) and that your total percentage should equal 100.) Health maintenance Maternity care Acute illness care Chronic illness care Count Percent Count Percent Count Percent Count Percent 0% 10 5.35% 175 93.58% 16 8.56% 31 16.58% 1% to 19% 33 17.65% 10 5.35% 28 14.97% 78 41.71% 20% to 39% 57 30.48% 2 1.07% 58 31.02% 46 24.60% 40% to 59% 60 32.09% 0 0.00% 64 34.22% 17 9.09% 60% to 79% 19 10.16% 0 0.00% 13 6.95% 10 5.35% 80% to 100% 8 4.28% 0 0.00% 8 4.28% 5 2.67% Total 187 100.00% 187 100.00% 187 100.00% 187 100.00% Mean 35.34% 0.47% 33.53% 19.40% End-of-life care Mental health care Other Count Percent Count Percent Count Percent 0% 161 86.10% 71 37.97% 164 87.70% 1% to 19% 25 13.37% 92 49.20% 11 5.88% 20% to 39% 1 0.53% 21 11.23% 6 3.21% 40% to 59% 0 0.00% 1 0.53% 2 1.07% 60% to 79% 0 0.00% 2 1.07% 1 0.53% 80% to 100% 0 0.00% 0 0.00% 3 1.60% Total 187 100.00% 187 100.00% 187 100.00% Mean 0.78% 7.10% 3.38% 27

Q13. Estimate the percentage of time (during an average week) that you provide patients with: (Note that you should only enter whole numbers without the percent sign (e.g., 32; 15) and that your total percentage should equal 100.) Direct patient care Management, supervision, and administration Teaching Research Count Percent Count Percent Count Percent Count Percent 0% 1 0.53% 92 49.20% 77 41.18% 146 78.07% 1% to 19% 6 3.21% 80 42.78% 85 45.45% 31 16.58% 20% to 39% 13 6.95% 11 5.88% 17 9.09% 6 3.21% 40% to 59% 45 24.06% 2 1.07% 4 2.14% 3 1.60% 60% to 79% 74 39.57% 1 0.53% 4 2.14% 0 0.00% 80% to 100% 48 25.67% 1 0.53% 0 0.00% 1 0.53% Total 187 100.00% 187 100.00% 187 100.00% 187 100.00% Mean 63.08% 5.59% 7.66% 2.81% Consultation Documentation Other Count Percent Count Percent Count Percent 0% 108 57.75% 21 11.23% 176 94.12% 1% to 19% 74 39.57% 75 40.11% 9 4.81% 20% to 39% 5 2.67% 79 42.25% 1 0.53% 40% to 59% 0 0.00% 12 6.42% 0 0.00% 60% to 79% 0 0.00% 0 0.00% 0 0.00% 80% to 100% 0 0.00% 0 0.00% 1 0.53% Total 187 100.00% 187 100.00% 187 100.00% Mean 3.06% 16.76% 1.03% 28

Q14. Do you have privileges to prescribe medications in your primary clinical practice setting? (Choose one) Count Percent Yes 182 97.33% No 5 2.67% Total 187 100.00% Q15. Do you have hospital privileges in your primary clinical practice setting? (Choose one) Count Percent Yes 73 39.04% No 114 60.96% Total 187 100.00% Q16. On average, how many hours per week do you work as a Family Nurse Practitioner nurse? Count Percent 0 hour 0 0.00% 1-9 hours 9 4.81% 10-19 hours 17 9.09% 20-29 hours 32 17.11% 30-39 hours 61 32.62% 40-49 hours 52 27.81% 50-59 hours 9 4.81% 60-69 hours 1 0.53% 70-79 hours 2 1.07% 80-89 hours 0 0.00% 90-99 hours 1 0.53% Over 100 hours 3 1.60% Total 187 100.00% Mean 32.94 29

17. Do you: (Choose all that apply) Count Percent Admit patients to the hospital 57 30.48% Manage patients during hospitalization 44 23.53% Discharge patients from the hospital 31 16.58% *N = 187 was used to compute the percentage Q18. Are you required to have a physician collaborator/supervisor? (Choose one) Count Percent Yes 153 82.26% No 33 17.74% Total 186 100.00% Not Answered 1 30

Appendix C Work Activities Descriptive Statistics 31

Performance Expectation Frequency Consequences Overall Rank Activity/Task Statement N Mean Std Dev N Mean Std Dev N Mean Std Dev N Mean Std Dev 1. Obtains a comprehensive health history 187 1.98 0.15 187 3.55 0.54 187 1.92 0.80 187 34.73 5.34 2. Obtains a focused health history 187 1.98 0.13 187 3.73 0.46 187 2.01 0.83 187 35.43 4.95 3. Performs appropriate screenings (e.g., developmental, hearing, vision, oral *****health, genetics) 187 1.93 0.26 187 3.37 0.62 187 1.79 0.74 187 32.94 6.32 4. Performs a psychosocial evaluation 187 1.83 0.41 187 3.20 0.72 187 1.59 0.73 187 29.93 8.87 5. Performs a lifestyle assessment (e.g., sleep, exercise, sexual behaviors, drug *****and alcohol use, safety, environmental) 187 1.95 0.23 187 3.24 0.67 187 1.63 0.77 187 32.33 6.19 6. Performs a nutritional assessment 187 1.95 0.23 187 3.24 0.69 187 1.49 0.71 187 31.63 6.23 7. Performs a cultural assessment 187 1.82 0.44 187 2.74 0.98 187 1.16 0.70 187 27.26 9.42 8. Performs a spiritual assessment 187 1.54 0.71 187 2.04 1.17 187 0.89 0.73 187 21.44 12.54 9. Performs an individual risk assessment (e.g., cardiac, cancer, diabetes) 187 1.84 0.37 187 3.02 0.82 187 1.60 0.75 187 29.83 9.10 10. Performs a genetic assessment 187 1.26 0.65 187 1.84 1.09 187 1.32 0.83 187 17.64 12.40 11. Performs a mental health assessment 187 1.74 0.50 187 2.85 0.90 187 1.89 0.80 187 29.56 10.97 12. Performs a family/caregiver assessment (e.g. family dynamics, communication * ******* patterns, support systems, caregiver burden) 187 1.74 0.48 187 3.00 0.87 187 1.50 0.76 187 27.72 10.61 13. Performs a community assessment (e.g. safety, transportation resources, **** *******nutrition resources, availability of social services) 187 1.64 0.53 187 2.53 1.01 187 1.25 0.74 187 24.06 10.99 14. Performs a functional assessment (e.g. mobility, cognition) 187 1.82 0.40 187 3.06 0.97 187 1.59 0.77 187 29.50 9.71 15. Assesses patient's capacity for decision-making 187 1.77 0.48 187 2.95 1.14 187 1.76 0.90 187 29.65 10.63 16. Assesses patient for pain (e.g. acute, chronic) 187 1.96 0.19 187 3.43 0.76 187 1.88 0.82 187 34.07 6.11 17. Assesses for advance care planning/advanced directives (e.g. DNR, living will) 187 1.24 0.80 187 1.51 1.27 187 1.31 1.11 187 18.79 14.26 18. Performs a comprehensive physical exam 187 1.95 0.21 187 3.56 0.54 187 2.20 0.80 187 35.61 5.91 19. Performs a focused physical exam 187 1.97 0.18 187 3.74 0.48 187 2.25 0.76 187 36.35 5.44 20. Differentiates between normal and abnormal physiologic changes 187 1.94 0.24 187 3.79 0.42 187 2.35 0.78 187 36.37 6.35 21. Orders diagnostic tests 187 1.90 0.30 187 3.20 0.64 187 2.01 0.75 187 33.30 7.57 22. Performs diagnostic tests 187 1.63 0.58 187 2.52 1.09 187 1.75 0.92 187 26.91 12.22 23. Interprets results from diagnostic tests 187 1.78 0.41 187 3.28 0.65 187 2.23 0.81 187 32.05 10.13 24. Synthesizes data to inform clinical reasoning 187 1.78 0.42 187 3.54 0.60 187 2.09 0.81 187 31.48 10.21 25. Develops differential diagnoses 187 1.90 0.30 187 3.66 0.53 187 1.99 0.74 187 33.71 7.17 26. Prioritizes differential diagnoses 187 1.80 0.40 187 3.60 0.56 187 1.96 0.76 187 31.43 9.29 27. Formulates diagnoses 187 1.93 0.26 187 3.66 0.50 187 2.12 0.80 187 34.75 6.73 28. Prioritizes diagnoses 187 1.82 0.38 187 3.58 0.55 187 2.01 0.78 187 32.07 8.92 29. Develops a comprehensive problem list 187 1.72 0.45 187 3.39 0.68 187 1.70 0.78 187 28.22 10.50 30. Develops a patient-centered plan of care 187 1.88 0.32 187 3.55 0.66 187 1.67 0.78 187 31.54 8.69 31. Evaluates patient, family and/or caregiver's knowledge of plan of care 187 1.88 0.33 187 3.55 0.69 187 1.79 0.82 187 32.05 8.71 32. Educates patient, family and/or caregiver regarding plan of care include testing, *****diagnosis, treatments and/or implications. 187 1.90 0.30 187 3.65 0.61 187 1.87 0.82 187 33.08 7.74 32

Performance Expectation Frequency Consequences Overall Rank Activity/Task Statement N Mean Std Dev N Mean Std Dev N Mean Std Dev N Mean Std Dev 33. Facilitates the development of advance care planning/advance directives 187 1.17 0.72 187 1.71 1.27 187 1.33 0.96 187 16.96 12.44 34. Prioritizes plan of care considering safety, risk reduction, and comorbidities 187 1.72 0.47 187 3.11 0.85 187 1.67 0.82 187 28.10 10.78 35. Develops population-focused plan of care based on epidemiologic data 187 1.18 0.65 187 2.04 1.23 187 1.16 0.80 187 15.89 12.34 36. Implements plan of care that incorporates cultural, spiritual, psychosocial ******considerations 187 1.63 0.54 187 2.86 1.00 187 1.39 0.77 187 24.80 12.03 37. Prescribes medications 187 1.94 0.24 187 3.47 0.55 187 2.52 0.70 187 36.89 6.20 38. Prescribes non-pharmacologic interventions 187 1.96 0.19 187 3.40 0.61 187 1.82 0.85 187 33.74 6.25 39. Manages health maintenance and health promotion interventions (e.g. nutrition, ******immunizations) 187 1.95 0.24 187 3.63 0.57 187 1.94 0.82 187 34.45 6.38 40. Reports suspected abuse, exploitation and/or neglect 187 1.96 0.19 187 3.18 1.11 187 2.68 0.68 187 37.83 5.55 41. Facilitates transitions across health care settings 187 1.58 0.54 187 2.59 1.02 187 1.49 0.79 187 24.11 11.67 42. Plans follow-up care 187 1.96 0.19 187 3.55 0.58 187 1.86 0.78 187 34.07 5.62 43. Manages episodic disease 187 1.95 0.23 187 3.49 0.55 187 2.02 0.78 187 34.51 6.37 44. Manages chronic disease 187 1.65 0.48 187 3.12 0.76 187 2.05 0.72 187 28.43 10.77 45. Participates in palliative care (e.g. chronic pain management, symptom ******management) 187 1.22 0.66 187 1.78 1.19 187 1.62 0.94 187 18.42 12.50 46. Provides end of life care 187 0.89 0.72 187 1.13 1.16 187 1.44 1.16 187 13.14 11.97 47. Performs common primary care procedures 187 1.78 0.44 187 3.05 0.83 187 1.84 0.77 187 30.01 9.78 48. Coaches patient, family, and/or caregiver regarding lifestyle and behavioral ******changes 187 1.88 0.32 187 3.23 0.69 187 1.48 0.72 187 30.28 7.82 49. Counsels on family planning, sexuality, and/or reproductive health 187 1.80 0.43 187 2.73 0.88 187 1.66 0.80 187 29.27 9.54 50. Evaluates patient responses to interventions 187 1.93 0.28 187 3.46 0.69 187 1.80 0.90 187 33.18 7.43 51. Evaluates effectiveness and safety of pharmacologic regimen 187 1.93 0.26 187 3.59 0.55 187 2.25 0.79 187 35.35 7.29 52. Evaluates effectiveness and safety of nonpharmacologic interventions 187 1.93 0.26 187 3.43 0.67 187 1.77 0.79 187 32.89 7.18 53. Evaluates adherence to treatment plan 187 1.92 0.27 187 3.49 0.59 187 1.83 0.78 187 33.03 7.10 54. Evaluates the impact of diagnosis and treatment on patient, family, and caregiver 187 1.82 0.39 187 3.30 0.74 187 1.61 0.79 187 29.74 9.07 55. Evaluates treatment and educational outcomes related to pain 187 1.80 0.44 187 2.88 0.96 187 1.70 0.79 187 29.71 9.86 56. Evaluates treatment and educational outcomes related to nutrition 187 1.85 0.36 187 3.10 0.68 187 1.57 0.75 187 29.97 8.65 57. Evaluates plan of care considering safety, risk reduction and comorbidities 187 1.79 0.41 187 3.10 0.82 187 1.68 0.81 187 29.33 10.01 58. Modifies plan of care to meet the needs of patient, families, and/or caregivers 187 1.79 0.42 187 3.22 0.79 187 1.63 0.77 187 29.20 9.96 59. Maintains a comprehensive problem list 187 1.75 0.43 187 3.38 0.79 187 1.57 0.78 187 28.19 9.98 60. Participates in interprofessional collaboration 187 1.77 0.43 187 3.10 0.74 187 1.48 0.84 187 27.98 10.73 61. Advocates for individual patient needs 187 1.87 0.34 187 3.42 0.75 187 1.71 0.86 187 31.38 9.19 62. Advocates for improved access, quality, and cost-effective health care 187 1.41 0.53 187 2.74 1.01 187 1.36 0.85 187 20.07 12.18 63. Serves as an interprofessional resource for patient care 187 1.48 0.50 187 2.82 0.87 187 1.24 0.80 187 20.55 11.58 64. Provides a climate of patient- and family-centered care 187 1.89 0.32 187 3.67 0.58 187 1.52 0.84 187 31.02 8.42 65. Participates in quality improvement 187 1.45 0.51 187 2.83 0.89 187 1.29 0.81 187 20.39 11.92 33