A National Role Delineation Study of the Pediatric Emergency Nurse Executive Summary Conducted for the Board of Certification for Emergency Nursing Prepared by Lawrence J. Fabrey, PhD, Sr. Vice President, Psychometrics and Christopher S. Bialko, MA, Associate Psychometrician, Psychometrics December 2017 Copyright 2017 by the Board of Certification for Emergency Nursing (BCEN ). PROPRIETARY. All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopy and recording, or any information and retrieval system, without permission in writing from BCEN.
Introduction The purpose of this study was to identify the responsibilities of pediatric emergency nurses as a first step in the development of a job-related certification examination. The BCEN requested the services of PSI to design and conduct a study that would provide the support necessary to develop specifications upon which a content valid certification examination could be built. The title of this study includes the use of the term role delineation study. However, other equally appropriate terms could be used to describe this study, for example, job analysis or practice analysis. A role delineation study suggests breadth of focus; however, the term role delineation has sometimes been used to describe a strictly judgmental process that does not make use of the data collection methodology used in the present study. While these three terms could be considered synonymous, there may be some subtle differences. Job analysis is the traditional technical term that is consistent with traditional standards of practice used to describe validation procedures for certification examinations. Practice analysis is a contemporary term that provides an appropriate description of the present study, as practice analysis suggests that the focus of a study is broader than a single job. Again, role delineation is the term primarily used in this report, although the terms should be considered to be interchangeable in relation to this study. The BCEN appointed a Role Delineation Study AC to conduct the activities necessary to identify responsibilities of pediatric emergency nurses and develop Examination Specifications. The AC was reflective of the pediatric emergency nurse profession in all relevant respects, for example: geographic, gender, and education. All AC members had demonstrated expertise in their field. The composition of the AC is shown in Table 1. PSI is grateful to these committee members for their guidance and expertise, as well as the time committed to this project. Without the AC's effort and expertise, this project would not have been accomplished. In addition, special mention should be made of the valuable contributions of BCEN staff. Table 1. Advisory Committee Members Name Renee Carroll Madeline Gehrig Chenoa Hanson Dede Koenekamp Cassandra Rennick Dana Shoemaker Rebecca Steinmann Brian Wharton State WA SC LA MN PA OH IL DE In the next section of this report, the methodology of the study is discussed. In particular, the design of the survey instrument is described, including the method of defining topics, tasks, rating scales, and demographic questions. Also discussed in the methodology section is the sampling plan and the distribution of a web-based survey. The results section of this report discusses the respondents and their demographics, the adequacy of the instrument, and a summary of the responses. The final section of this report discusses the development of the Examination Specifications based on these data. ~ 1 ~
Methodology The AC considered various resource materials that could be useful in gaining an understanding of the responsibilities of pediatric emergency nurses, including job descriptions, journal articles, performance evaluation forms, and textbooks. Materials assembled prior to the first meeting of the AC included orientation materials, a draft of topics, tasks, rating scales, and demographic questions for the role delineation study survey, and a timeline for conducting the study. Background information was provided regarding both the role delineation process (and its relationship to the examination development process) and BCEN's role in the continuing development of the content validity of a certification examination. Seven major tasks were initiated during the AC meeting held in February 2017. These steps included: 1. Defining the target practitioner 2. Developing a sampling plan 3. Identifying topics and tasks for the survey instrument 4. Identifying major domains 5. Determining the rating scales 6. Determining the relevant demographic variables of interest 7. Integrating demographics, rating scales, topics, and tasks into a survey instrument A summary of each activity follows. 1. Defining the target practitioner For the purposes of this study, BCEN and the AC adopted the following target practitioner definition of a pediatric emergency nurse: A registered nurse who possesses advanced critical thinking and highly developed skills in providing emergency care to pediatric patients and their families. The emergency nurse is able to apply these skills autonomously, demonstrating the ability to assess, analyze, intervene, and evaluate ill or injured pediatric patients in the emergency setting. The role of the emergency nurse also includes communication, education, promotion of safety, and injury prevention. 2. Developing a sampling plan The AC considered various methods of identifying individuals who consider themselves to be pediatric emergency nurses, or who would be knowledgeable about the duties of pediatric emergency nurses. It was determined that invitation e-mails containing a link to the online role delineation study survey would be distributed to pediatric emergency nurses. 3. Identifying topics and tasks for the survey instrument After discussion, the AC decided that the test specifications would be topic-based, and that the primary tasks underlying all content would be the steps of the nursing process: assessment, planning, intervention, and evaluation. Therefore, a linkage between the areas of knowledge (topics) would be established by the development of the survey in this manner. The draft list was thoroughly discussed during the meeting held in February 2017. Topics and tasks representing individual job responsibilities were modified, added, and removed. All topics and tasks were verified as being appropriately linked to the associated domain. At the conclusion of this meeting, a draft list that included 130 topics and 81 tasks was developed by the AC. After ~ 2 ~
the review of the draft list, the AC authorized development of the final survey. The final survey included 130 topics and 81 tasks. 4. Identifying major domains The committee identified 8 major domains of practice, under which topics were categorized into subcategories. The AC unanimously agreed on the linkage of each topic to the respective domain. The domains were as follows: I. Clinical Practice: Triage Process and Assessment II. III. IV. Clinical Practice: Respiratory and Cardiovascular Emergencies Clinical Practice: Gastrointestinal and Genitourinary Emergencies Clinical Practice: Environmental and Toxicology Emergencies, and Communicable Diseases V. Clinical Practice: Neurological, Medical, and Neonatal Emergencies VI. VII. VIII. Clinical Practice: Maxillofacial, Ocular, Musculoskeletal, Integumentary Emergencies Clinical Practice: Behavioral and Maltreatment Emergencies Legal and Professional Practice 5. Determining the rating scales The committee discussed the advantages and disadvantages of various rating scales that could be used in responding to the topics and tasks. PSI suggested the use of a single significance scale. This single scale is intended to solicit judgments that combine the importance of a topic/task with the frequency with which it is addressed in practice, after first considering the extent to which it is necessary to the performance in practice. The significance scale adopted by the AC is shown below. For professional topics: How significant is this topic to your emergency nursing practice? For patient conditions: How significant to your practice is the ability to assess, analyze, intervene, and evaluate patients with the following conditions? 0 = Not part of my practice 1 = Minor 2 = Moderate 3 = Critical ~ 3 ~
6. Determining the relevant demographic variables of interest The committee identified 16 relevant and important demographic survey variables. Since this was a national study, it was important to identify the respondents' geographic regions of employment. Other demographic questions were written to assess characteristics of the representativeness of the respondents, including primary practice setting, facility type, primary position, years as pediatric emergency nurse, gender, age, and racial and ethnic background. 7. Integrating demographics, rating scales, topics, and tasks into a survey instrument After the first meeting, all components of the survey (demographics, rating scales, topics, and tasks) were combined and designed into a draft survey instrument. As a pilot test, this draft was distributed to the AC and other individual content experts via an e-mail message, which included a link to the pilot survey. Following a review of the comments, the final survey was prepared and distributed via an e-mail invitation. ~ 4 ~
Results The survey was accessible via the Internet through the response deadline of May 12, 2017. Of the 6,168 e-mail invitations distributed (among which 356 were undeliverable and 134 opted out of the study), a total of 3,492 respondents accessed the survey. After further reducing the sample size by removing participants who did not give any responses (n=2846) and who completed less than 25% of the knowledge statements (n=66) a total of 580 responses were considered to be valid responses, for a usable response rate of 10% (580/5678). Demographic Information Summaries of the respondents responses to the demographic questions are shown in the figures and tables below. Based on discussion with the AC, the demographic data were as expected, and judged to be representative of the profession. In addition to ensuring that the respondent group was representative, it was important to evaluate whether responses were received in appropriate numbers from relevant subgroups. The AC determined that sufficient responses were received from relevant subgroups for subsequent analysis. Figure 1. Respondent Distribution in the United States Survey respondents were asked to indicate in what country they worked, and if United States, what state. As shown in Figure 1, respondents were distributed throughout the U.S. Further grouping into geographic regions is shown in Figure 2. The largest group of respondents (35.9%) came from the South. ~ 5 ~
Figure 2. Region Northeast: CT, DC, MA, MD, ME, NH, NJ, NY, PA, RI,VT South: AL, AR, DC, DE, FL, GA, KY, LA, MS, NC, OK, SC, TN, TX, WV, VA Midwest: IA, IL, IN, KS, MI, MN, MO, ND, NE, OH, SD, WI, WY West: AK, AZ, CA, CO, HI, ID, MT, NM, NV, OR, UT, WA Figure 3 illustrates the primary practice setting of respondents. A small group of respondents worked in other settings (8%), while the two largest groups consisted of those who practiced in a pediatric ED in a pediatric hospital (34%) and an all age ED in an all age hospital (35%). These were followed by those who worked in a pediatric ED in an all age hospital (22%). Figure 3. Primary Practice Setting ~ 6 ~
Figure 4. Facility Type Figure 4 depicts the facility type of respondents. About one third indicated they worked in community hospitals (35%) and another third indicated they worked in an academic-based facility, followed by critical access facilities (25%) and other facilities (8%). Figure 5. Primary Position As shown in Figure 5, 69% of respondents indicated their primary position was staff or charge nurse, followed by 12% other, 10% managers, and 10% educators. ~ 7 ~
Figure 6. Years as Emergency Nurse Caring for Pediatric Patients As shown in Figure 6, respondents had various years of experience caring for pediatric patients as an emergency nurse. The average length of time in providing trauma nursing care is 14 years. Table 2. Holds CPEN Credential Frequency Percent CPEN 507 87.4 No 73 12.6 Total 580 100 Table 2 shows that 87% of respondents hold the CPEN credential. ~ 8 ~
Figure 7. Highest Degree in Nursing Completed Figure 7 shows that 57% of respondents reported a Bachelor s Degree as the highest nursing degree they had completed. Figure 8. Annual Pediatric Patient Census in Emergency Facility As shown in Figure 8, the largest group of respondents indicated the census in their emergency facility was under 20,000 (28%), while the smallest group of respondents worked in an emergency facility with more than 100,000 (6%). ~ 9 ~
Adequacy of the Instrument Approximately 99% of those responding to the question shown in Figure 9, which appeared at the end of the survey, felt that the role delineation study survey at least adequately addressed the responsibilities of trauma nurses. Thirty-three survey respondents did not answer this question. Another aspect of the adequacy of the instrument relates to its reliability. Figure 9. Rating of Survey Coverage ~ 10 ~
Examination Specifications In developing Examination Specifications (or a Detailed Content Outline), committee judgment must be used in interpreting the data gathered through the role delineation study. For purposes of this report, the Examination Specifications will be defined as the confidential document that is used to guide the examination development process, and includes sufficient detail to ensure the development of comparable examination forms. The Detailed Content Outline can be defined as a subset of the Examination Specifications; it is a document that includes a detailed listing of content available in outline form for candidates and item writers. Every examination item must be linked to the Detailed Content Outline as a first step in meeting the Examination Specifications during the examination development process. Of particular importance to a national certification examination program is that the Examination Specifications must appropriately reflect the responsibilities of all groups who will participate in the certification program. Therefore, it is important to ensure that neither the Examination Specifications nor the resulting examinations include topics/tasks that are not considered to be important responsibilities of the individuals for whom the examination is intended. Several decision rules for each topic and task were proposed for consideration by the AC in determining criteria by which they should be considered ineligible for assessment, and therefore excluded from the Detailed Content Outline. Applying the decision rules ensures that the resulting examination reflects the responsibilities of pediatric emergency nurses, as judged by a demographically representative group of pediatric emergency nurses. The general areas for consideration were discussed by the AC during a face-to-face meeting held in July 2017, and were based on a variety of the demographic characteristics included in the survey. The first decision rule helped ensure the content outline would only reflect topics and tasks that were part of practice; any that received a high percentage of respondents providing a 0 rating (Not part of my practice) were eliminated. The second decision rule, which was applied to both topics and tasks as well, established a threshold for the mean significance rating for the overall respondent group, ensuring that what remained on the Detailed Content Outline was clearly significant to practice. Finally, eight additional decision rules were adopted for topics and tasks based on subgroup analyses, to ensure that the remaining topics and tasks were significant to practice across different demographics. As a result, application of these decision rules eliminated 10 topics out of 130 and 7 tasks out of 81 from the Detailed Content Outline. In addition to applying decision rules, the AC examined the respondents comments and any additional topics and tasks that respondents had listed. Based on this review, the AC decided that two additional topics were needed. Other revisions are detailed in the full report. Development of Final Detailed Content Outline and Examination Specifications The AC determined that the Examination Specifications would more effectively reflect pediatric emergency nursing practice if based on topics rather than tasks, and that the tasks underlying all content should be the four steps of the nursing process: assessment, analysis, implementation, and evaluation. Therefore, the survey was designed with this linkage between the areas of knowledge (topics) and tasks in mind. The AC determined that clinical tasks would be classified as requiring assessment, analysis, intervention, or evaluation on the part of the candidate. ~ 11 ~
Detailed Content Outline and Examination Specifications ~ 12 ~
~ 13 ~
~ 14 ~
~ 15 ~
~ 16 ~