Test Analysis Report

Transcription

1 Test Analysis Report IBLCE Certification Examination 2013 Administrations Prepared for the: International Board of Lactation Consultant Examiners (IBLCE) Prepared by: Lawrence J. Fabrey, PhD Senior Vice President, Psychometrics Division and Lily Chuang, MS Research Associate, Psychometrics Department IBLCE. All rights reserved. ~ i ~

2 Test Analysis Report International Board of Lactation Consultant Examiners (IBLCE) IBLCE Certification Examination 2013 Administrations Overview The International Board of Lactation Consultant Examiners (IBLCE) certification examination 2013 test form was administered in 925 locations across 56 countries and territories in 17 languages: English (British), Croatian, Danish, Dutch, French, German, Greek, Hungarian, Indonesian, Italian, Japanese, Korean, Polish, Portuguese, Slovenian, Spanish, and Taiwanese. A total of 3,660 candidates sat for the examination in July Of all candidates, 3,025 (83%) candidates tested through computer-based test (CBT), and 635 (17%) candidates tested through paper-pencil test. Data from CBT and paper-pencil administrations were combined for analysis in this Test Analysis Report. ~ 1 ~

3 Test Construction The IBLCE certification examination is based on a 2-dimensional content outline, or test blueprint. This document was derived from a practice analysis by the Board, in conjunction with its Examination Committee members and a Representative Panel of Experts (RPE). On the basis of this study, the Board arranged the examination content according to scientific disciplines and developmental stages, which has since evolved into chronological periods. The examination blueprint appears in its entirety on the IBLCE website ( The examination is scored as a single integrated test. However, it has two multiple-choice item formats: text-based and image-based multiple-choice. The images usually present a color photograph depicting an aspect of breastfeeding, or breast anatomy or pathology that the candidates must resolve. These test items have a particularly high degree of clinical relevance, and because most of the item is contained in the graphic, translation is facilitated. Of the 175 items comprising the test, 100 reference images. Following standard operating procedures, the 2013 examination was developed by an Examination Subcommittee that prepared, reviewed, edited, and selected test items, and that participated in a cut score study. The Committee includes broad geographic and practice setting representation of subject matter experts. A preliminary item analysis was conducted after test administrations using all candidate score data (n = 3,660). Psychometrics staff at Applied Measurement Professionals, Inc. (AMP) reviewed the preliminary item analysis with members of the Examination Committee to confirm the appropriateness of potentially problematic items (i.e., those with slightly irregular statistics or adverse candidate comments). Appropriate item scoring changes were made as needed. In addition, an analysis of item performance by the various linguistic versions of the examination was conducted, and items with potential issues were reviewed by the Examination Committee, and the comparability of the items in the various languages was confirmed. After adjusting for the scoring changes, the result of the cut score study using a modified Nedelsky procedure, a judgmental criterion-referenced technique (Nedelsky, 1954; Gross, 1985), suggested a raw score of 122 (out of 175) as indicative of professional knowledge. Designated representatives from IBLCE and the Committee reviewed the results and unanimously approved the implementation of 122 as the raw score passing point for 2013 test form of the IBLCE certification examination. The standard setting process is documented in a separate report. ~ 2 ~

4 Test Scoring These statistics are computed for the first-time candidate population of 2,406 candidates, based on the combined text-based and image-based portions. The statistics are presented for the overall test rather than for the two subtests because the examination was designed and intended to be one comprehensive, integrated test. Figure 1. Distribution of Raw Scores for First-time Candidates ~ 3 ~

5 The performance for each content discipline and chronological period is shown in Tables 1 and 2. These data indicate total test performance as well as content discipline and chronological period performance for all and first-time candidates. Table1. Candidate Performance Summary for All Candidates by Discipline (n = 3,660, % passing = 84.78) DISCIPLINES n of Items Mean A. Maternal and Infant Anatomy B. Maternal and Infant Normal Physiology and Endocrinology C. Maternal and Infant Normal Nutrition and Biochemistry D. Maternal and Infant Immunology and Infectious Disease E. Maternal and Infant Pathology F. Maternal and Infant Pharmacology and Toxicology G. Psychology, Sociology, and Anthropology H. Growth Parameters and Developmental Milestones I. Interpretation of Research J. Ethical and Legal Issues K. Breastfeeding Equipment and Technology L. Techniques M. Public Health Total ~ 4 ~

6 Table 2. Candidate Performance Summary for All Candidates by Chronological Period (n = 3,660, % passing = 84.78) CHRONOLOGICAL PERIODS n of Items Mean 1. Preconception Prenatal Labor/birth (Perinatal) Prematurity days days days months months months Beyond 12 months General principles Total ~ 5 ~

7 Test Results A passing point of 122 raw score units was applied to the 2013 test form based on the modified Nedelsky procedures mentioned previously. Final results for the examination forms is shown in Table 3, displaying the mean and pass summary by certification status. Table3. Mean Score and Pass/Fail Summary by Certification Status Certification Status N Mean % Passing Initial Certification 2, at 5 Years at 10 Years at 15 Years at 20 Years at 25 Years Total 3, ~ 6 ~

8