Critical Thinking Skills in Health Care Professional Students: A Systematic Review

- - method/model presentation - - - Critical Thinking Skills in Health Care Professional Students: A Systematic Review Tracy J. Brudvig, PT, DPT, PhD, OCS, Angelique Dirkes, PT, DPT, MS, Priyanka Dutta, PT, MS, and Kalpita Rane, PT, MS Background and Purpose. The purpose of this systematic review is to determine if critical-thinking skills in health care professional students change as a result of participating in a health care professional education program, and to assess the quality of the evidence evaluating this change. Method/Model Description and Evaluation. The study was a systematic review. Six electronic databases were searched. Articles were graded using the Evaluation Guidelines for Rating the Quality of an Intervention Study. Outcomes. Eighteen articles were identified as meeting the inclusion criteria. Ten articles were identified in nursing, 5 in physical therapy and occupational therapy, 2 in pharmacy, and in medicine. The scores on the Evaluation Guidelines for Rating the Quality of an Intervention Study ranged from 7 to 26 out of 48. The intraclass correlation of the 2 raters on the Evaluation Guidelines for Rating the Quality of an Intervention Study was.87. Tracy Brudvig is a clinical associate professor in the Department of Physical Therapy in the MGH Institute of Health Professions, 36 st Avenue, Boston, MA 229 (tbrudvig@mghihp. edu). Please address all correspondence to Tracy Brudvig. Angelique Dirkes is a clinical instructor in the Department of Physical Therapy in the MGH Institute of Health Professions, Boston, Massachusetts. Priyanka Dutta was a student in the Master of Science in Physical Therapy program at the MGH Institute of Health Professions, Boston, Massachusetts, when this study was conducted. Kalpita Rane was a student in the Master of Science in Physical Therapy program at the MGH Institute of Health Professions, Boston, Massachusetts, when this study was conducted. The authors declare no conflict of interest. Received January 8, 22, and accepted August 9, 22. Eight studies in nursing, 2 studies in physical therapy, study each in occupational therapy, pharmacy, and medicine showed statistically significant change in criticalthinking skills. Discussion and Conclusion. The evidence measuring the acquisition of critical-thinking skills in the fields of physical therapy, occupational therapy, pharmacy, and medicine is scarce. In nursing, although the majority of the studies support the acquisition of critical- thinking skills, none of the studies established appropriate sample size and statistical power analysis, and the cross-sectional studies did not perform group matching beyond sex and age. In physical therapy, the results are mixed and the studies lack adequate statistical power. In occupational therapy, pharmacy, and medicine, the studies that showed a statistically significant increase in critical-thinking skills had relatively small sample sizes and weak statistical power. There is a need for additional welled studies looking at the acquisition of critical-thinking skills in all health care fields. Key words: Critical thinking, Clinical decision making, Health care professional students. INTRODUCTION Agreement on the meaning of critical thinking came about in the 99s through a crossdisciplinary consensus project coordinated by the American Philosophical Association. Critical thinking is defined as the purposeful, self regulatory judgment that results in interpretation, analysis, and inference, as well as explanation of the evidential, conceptual, methodological, criteriological, or contextual considerations upon which that judgment is based. (p2) Analysis, judgment, inference, reasoning, reflection, and synthesis are com- mon attributes that define the concept of critical thinking. Terms like clinical decision making, critical judgment, and diagnostic judgment are surrogate terms often used interchangeably with each other and the term critical thinking. 2 A comprehensive knowledge base is a prerequisite of critical thinking and forms the foundation on which the cognitive process builds. 3 In addition, the affective disposition, which manifests as the inclination to utilize one s knowledge base and their critical-thinking skills, is a very important element integral to critical thinking. 3 In the current rapidly changing health care environment, critical thinking is an essential skill that all health care professionals must have in order to make sound clinical decisions. Health care professionals are trained to make life-changing decisions in the most challenging of situations that affect an individual s physical, psychological, and social well-being. It is essential to promote the development of critical thinking in health care professional students so that they emerge as sound clinicians and deliver competent care. Educators are being invariably challenged to curricula and develop teaching strategies that cultivate and promote criticalthinking skills in students. 4 Critical thinking is one of the essential core competencies for nurses in the 2st century as defined by the National League for Nursing Accrediting Commission 5 and the American Association of the College of Nursing. 6 The American Physical Therapy Association (APTA) emphasizes the need for cognitive, psychomotor, and affective behaviors in students, including critical-thinking skills. 7,8 The goal and vision of APTA is to graduate students who will possess clinical decision-making skills (including clinical reasoning, clinical judgment, and reflective practice), engage in critical inquiry, and apply their knowledge in order to make decisions in various aspects of patient care. 7,8 APTA s Vision 22 states: By 22, physical therapy will be provided by physical therapists who are doctors of 2 Journal of Physical Therapy Education Vol 27, No 3, Fall 23

physical therapy, recognized by consumers and other health care professionals as the practitioners of choice to whom consumers have direct access for the diagnosis of, interventions for, and prevention of impairments, activity limitations, participation restrictions, and environmental barriers related to movement, function, and health. 9 Among the 5 key areas of Vision 22, direct access and evidence-based practice require criticalthinking skills. In order to achieve this vision, physical therapy schools are making concerted efforts to enhance students critical-thinking skills through their curricula. Although APTA publishes documents that encourage the development of critical-thinking skills in students, few studies have specifically examined the development of these skills in physical therapist students. Occupational therapy educators also have emphasized the development of critical-thinking skills in students. A common assumption made by many health care professional education programs is that students critical-thinking skills improve as a result of completing their respective training programs. However, there is little evidence that programs measure changes in criticalthinking skills. Several studies have investigated the development of critical-thinking skills in different health care professional students. The majority of studies have been conducted in nursing, while only a few studies have measured the acquisition of critical-thinking skills in the fields of physical therapy, occupational therapy, pharmacy, and medicine. These studies have used different tools to assess critical-thinking skills in students. These tools include the California Critical Thinking Skills Test (CCTST), the Watson-Glaser Critical Thinking Appraisal (WGCTA), the California Critical Thinking Disposition Inventory (CCTDI), the Critical Thinking Scale (CTS), interviewing, critical case analysis, and reflective writing. Several studies have shown improvement in critical-thinking skills among students due to various interventions (curricula, clinical internship, and patient simulation models), while others have shown no statistically significant improvement in critical-thinking skills. There is no published systematic review to date that has looked at the acquisition of critical-thinking skills in health care professional students. The purposes of this study are to: () identify articles that measure the acquisition of critical-thinking skills in health care professional students as a result of completing a health care profession education program (or some aspect of it); (2) perform a systematic review to assess the quality of the methods and research of the studies measuring the acquisition of critical-thinking skills in various health care professional programs; and (3) determine if students critical-thinking skills change as a result of participating in a health care professional education program. METHODS Search Strategy The study is a systematic review. The following electronic databases were searched: OVID MEDLINE 996 to present with Daily Update, OVID Nursing database, PubMed, Cumulative Index to Nursing and Allied Health Literature (CINAHL), Google Scholar, WorldCat, and Academic. The bibliographies of the identified articles were also reviewed to identify appropriate studies. The search terms used were: critical thinking, nursing, physical therapy, occupational therapy, pharmacy, medicine, dentistry, dietetics, and assessment of critical thinking. These keywords were searched independently and in combination. Detailed search terms and the combination of these terms are listed in Appendix A. In the first step of the search process,,33 articles were identified. The list was further narrowed by adding additional search terms. In the final step, the inclusion and exclusion criteria were applied and 8 articles qualified for inclusion. Detailed steps of Figure. Search Strategy Database: Ovid MEDLINE+ Ovid Nursing Database + Pub Med + CINAHL + Google Scholar + Academic One File + Northeastern University Database Keyword: Critical Thinking Number of Studies Found: 33 Keywords: Critical thinking in nursing/physical therapy/occupational therapy/pharmacy/ dentistry/medicine/ dietetics Number of Studies Found: 56 Review of titles and abstracts Number of Articles Shortlisted: 55 the search process and the subsequent number of articles identified are listed in Figure. Inclusion and Exclusion Criteria Inclusion criteria included: empirically ed studies published through March 2; studies performed in countries where English is the primary language; studies performed on professional (entry-level) students from nursing, physical therapy, occupational therapy, dentistry, medicine, pharmacy, and dietetics; studies using the CCTST and the WGCTA as the outcome measures; and studies using the defined health care professional educational curricula as the intervention. Both the CCTST and the WGCTA has been found to be reliable and valid assessment tools in health professional students. -3 Research performed on the psychometric properties of the CCTST shows that the internal consistency ranges from.68 to.7. 4 The reliability of the total score of the WGC- TA is.77 and the split half reliability is.69 to.85. 5 The concurrent validity between the CCTST and the WGCTA has also been established (r =.45, P <.). 6 Exclusion criteria included: theoretical studies exploring critical-thinking skills, studies performed in countries where English in not the primary language, critical-thinking studies performed on students enrolled in a curriculum other Keywords: Assessment of critical thinking in nursing/ physical therapy/ occupational therapy/ pharmacy/dentistry/ medicine/dietetics Number of Studies Found: 88 Matched with inclusion & exclusion criteria Number of Articles Finalized: 8 Vol 27, No 3, Fall 23 Journal of Physical Therapy Education 3

than the defined health care professional curricula, studies using interventions other than their respective health care professional curricula, and studies using any measurement tool other than the CCTST and the WGCTA. Although the Cornell Critical Thinking Test and the Ennis-Weir Critical Thinking Essay Test have been widely used in educational research, 7 the authors found no studies in any of the health care professions using these measures. Studies using the Critical Thinking Scale (a teacher-accessible tool developed for evaluating students reflective writing) were excluded because there was only moderate reliability with the use of this tool. 8 Methodological Quality Evaluation The Evaluation Guidelines for Rating the Quality of an Intervention Study developed by MacDermid 9 was used to evaluate the methodological quality of all selected studies. This tool has been used in a number of systematic reviews involving rehabilitation and has shown high reliability. 9-23 The evaluation criteria in the Evaluation Guidelines for Rating the Quality of an Intervention Study are explicit and clear. The scale consists of 24 items divided into 7 domains (Appendix B). These domains include study question, study, subject selection, intervention outcomes, analysis, and study recommendations. The scoring system is comprised of a 2 point scale for each question and the total maximum score is 48. A range of to 2 allows for allocating a point for criteria that are partially complete or partially described. Many critical appraisal tools, such as the Jadad or Pedro scales 24,25 have few items and focus on randomized controlled trials and sometimes lack relevance to educationally based research. The following ranges were used to qualify the methodological quality: a score of 36 48 points (> 75%) was a high-quality study, 25 35 points (52% 74%) was a moderate-quality study, and 24 points (% 5%) was a low-quality study. It has been suggested that a methodological score of 7% or greater is needed for a study to be of sound methodological quality. 26 Prior to the quality assessment of the included studies, all 4 authors met to discuss the scale and the specific grading criteria and independently scored article to check interrater reliability. The studies that met the inclusion criteria were independently assessed by 2 of the authors (P. D. and K. R.). Any discrepancies were resolved by the other 2 authors (T. B. and A. D.). The preconsensus interrater reliability for total scores was excellent with an intraclass correlation coefficient (ICC) of.87. The ICC was calculated using SPSS, version 8.. 27 RESULTS Eighteen articles met the inclusion criteria. Ten studies investigated critical-thinking skills in nursing, 28-37 5 studies in physical therapy and occupational therapy, 38-42 2 studies in pharmacy, 43,44 and study in medicine. 45 No articles were found in the fields of dietetics and dentistry. Of the 8 articles, 4 were pretest-posttest s 28-3,35-45 and 4 were cross-sectional s. 3-34 On the Evaluation Guidelines for Rating the Quality of an Intervention Study, study 45 scored low (7 points) while 7 studies scored within the moderate range. The individual study scores are found in Table. Convenient sampling was used in all the studies. The student population varied greatly among the studies with respect to age range. Education programs studied were: bachelor s degree programs in nursing, doctoral degree programs in pharmacy, bachelor s and master s degree programs in physical therapy, master s degree programs in occupational therapy, and undergraduate medicine. The phases in the curricula during in which the CCTST and the WGCTA were administered to the students were different and were Table. Summary of Methodological Quality of Studies Nursing Study not standardized among the studies. Seven studies 28,3,32,35,39,4,45 used the WGCTA to evaluate critical-thinking skills while studies 29,3,33,34,36-38,4,42-44 used the CCTST. Nursing Adams et al 28 performed a longitudinal pretest posttest study to measure change in critical- thinking skills of 23 baccalaureate nursing students (female = 85, male = 8) ranging in age from 2-48 years (mean age = 23 years). The WGCTA Form A was administered during the second semester of the sophomore year and the WGCTA Form B was administered when they completed their preceptorship during the last semester of the final year of the curriculum. Change in the WGCTA score between sophomore year and senior year was not statistically significant (t =.954). Beckie et al 29 compared nursing students scores on the CCTST Form A in a pretestposttest. The experimental group had a revised curriculum and the control group had the previously established curriculum. Cohort (n = 55) was the class before the curriculum revision, cohort 2 (n = 55) and Score on the Evaluation Guidelines for Rating the Quality of an Intervention Study Spelic et al 36 26 Adams et al 28 25 Beckie et al 29 25 McCarthy et al 36 24 Brown et al 3 2 Collucielo 3 2 Frye et al 32 2 Profetto-McGrath 34 2 Saucier 35 2 Thompson and Rebeschi 37 2 Physical and Occupational Therapy Vogel et al 4 27 Slaughter et al 39 25 Bartlett and Cox 38 24 Vendrely 4 24 Zettegren 42 24 Pharmacy Miller 44 24 Cisneros 43 2 Medicine Scott et al 45 7 4 Journal of Physical Therapy Education Vol 27, No 3, Fall 23

cohort 3 (n = 73) were the first 2 classes with the revised curriculum. Cohort 2 had statistically significant higher total CCTST scores on the posttest (F = 8.58, P <.). Cohorts and 3 did not show any significant improvement on the total score. Change in scores within a subject over time showed a statistically significant increase in total scores (F = 4.8, P <.9) in all cohorts. In the study by Brown et al, 3 a pretestposttest was used to study 3 groups of nursing students (N = 23). The groups consisted of traditional nursing students in a 32-month program (n = 45), Registered Nurse Bachelor of Science in Nursing students (RN BSN) entered with associate degree (n = 35), and accelerated BSN students in an8-month program (n = 43). The WGCTA was used as the outcome measure and was administered to all students at the beginning and at the end of their nursing curriculum. Results showed a statistically significant increase in the posttest scores in the traditional nursing students (t = -2.84, P =.7) and RN BSN students (t = -2.28, P =.3). Although there was an increase in scores in the accelerated RN BSN group, the results were not statistically significant. Colucciello 3 performed a cross-sectional study on 328 baccalaureate nursing students representing 5 academic levels (Sophomore II = 94; Junior I = 65; Junior II = 64; Senior I = 59; Senior II = 46). The CCTST was used as the outcome measure. An analysis of variance (ANOVA) showed the scores of Junior I, Senior I, and Senior II were statistically significantly higher than Sophomore II levels. Also, the scores of the Junior II level were statistically significantly different from the Junior I level. Frye et al 32 performed a cross-sectional study comparing the WGCTA Form A scores of 32 freshmen and 77 senior baccalaureate nursing students. They also examined critical-thinking skills of 27 students as freshman and as seniors longitudinally. There was a statistically significant (P =.5) increase in the scores of freshmen and seniors on an unpaired t test (cross-sectional analysis). The paired t test for the longitudinal data revealed no statistically significant difference. McCarthy et al 33 performed a cross-sectional study on 24 baccalaureate nursing students including 56 sophomores (females = 3, males = 43; mean age = 24. years) and 85 seniors. The CCTST was used as the outcome measure. Data were collected from sophomore and senior classes in the spring quarter. The senior class scored statistically significantly higher (t = 4., P <.) on the total score on the CCTST (t = 2.5, P <.). Profetto-McGrath 34 performed a crosssectional study on a group of 228 (female = 24) baccalaureate nursing students (year = 38; year 2 = 53; year 3 = 57; year 4 = 8). The CCTST was used as the outcomes measure. There was an increase in the mean CCTST score from year to year 4, but the increase was not statistically significant. The year 3 students did not show any change in mean scores. Saucier 35 evaluated whether students scored higher on the WGCTA upon completion of a baccalaureate nursing program as compared to scores at time of entry. Students from the classes of 988 to 992 were included in this study. Only students in the class of 99 (n = 27) showed a statistically significant increase in the WGCTA scores upon completion (P =.53). Spelic et al 36 measured critical thinking in 36 nursing students in 3 tracks: traditional nursing (4-year program = 5), Linking Education And Practice (LEAP) students ( calendar year program = 7), and accelerated nursing curriculum students ( calendar year program = 5). A pretest-posttest was used and the CCTST was administered at the beginning and the end of each program. The comparison of the total scores at entry and exit were statistically significant for all tracks (P <.). Thompson and Rebeschi 37 conducted a longitudinal study using a pretest-posttest on 38 baccalaureate nursing students. The CCTST Form A was used as the outcome measure. The pretest was administered at the end of the first semester and the posttest was conducted 2 months before the end of the final semester. The results showed a statistically significant increase in CCTST scores (5.97, 7.68, P =.6). Scores on the Evaluation Guidelines for Rating the Quality of an Intervention Study for all nursing studies are listed in Table. The study characteristics are listed in Table 2. Physical Therapy and Occupational Therapy Bartlett and Cox 38 performed a -group, repeated-measures (pretest-posttest) on 28 middle-year baccalaureate physical therapist students (female = 26, male = 2; mean age = 22 years) to determine critical-thinking skills. The CCTST was administered before and after the academic year and after clinical placements. The effect size was.64. A -way repeated-measures ANOVA and Tukey post hoc tests were calculated for total scores, and a multiple regression analysis was performed for correlation of descriptive variables and total CCTST scores. Results showed statistically significant gains in the total scores. Slaughter et al 39 performed a pretestposttest to evaluate the effectiveness of a model for teaching by measuring critical thinking using the WGCTA. First year baccalaureate physical therapist students (n = 3) were administered the WGCTA. Of the 3 students, 6 were in the experimental group and 4 in the control group. Different 4-week clerkships were provided to both groups. A 2-way ANOVA was used for data analysis. There was no statistically significant difference in the scores between the 2 groups (F,32 ) =.5, P >.5). Interaction between groups and performance was also not statistically significant (F,43 = 4.92, P >.5). Vendrely 4 conducted a longitudinal study with a pretest-posttest on 42 students in a Master of Physical Therapy program. The CCTST was administered to members of each incoming cohort before the beginning of their first semester and again during the last week of the program. There was no statistically significant change in the CCTST total or subscale scores. Vogel et al 4 evaluated critical-thinking skills in 5 occupational therapists (n = 3) and physical therapist students (n = 37) in a master s degree program using a pretestposttest. The WGCTA short form was administered at the beginning and the WGCTA B was administered after 2 months in the academic phase. The Box test, -way repeated-measures ANOVA, and post hoc tests were performed. A significant time X group interaction (F,48 = 4.69, P =.4) and main effect for time (F,48 = 4.857, P =.3) were observed. The post hoc tests showed that the increase from pretest to posttest was statistically significant for the occupational therapy group (t 2 = 3.26, P =.7) but not statistically significant for the physical therapy group (t 2 =.34, P =.97). In the study by Zettegren, 42 a cross-sectional was used to look at the development of critical-thinking skills in the Master of Physical Therapy degree students. There were 3 student groups: third-year (n = 86), fourth-year (n = 53), and fifth-year (n = 6) students. The CCTST was used as the outcome measure and was administered during the fall semester. Along with their traditional classroom instructions, students had blocks of clinical internships: The first was an 8-week internship after completing the fourth year and three 8-week internships after completing the fifth year of the program. Results showed a statistically significant difference in scores between the third- and fifth-year groups (M = 2.59, P =.) and between the fourth-year and fifth-year groups (M =.8, P =.5). No statistically significant differences were noted between the third-year and fourth-year Vol 27, No 3, Fall 23 Journal of Physical Therapy Education 5

Table 2. Study Characteristics: Nursing Author/ Design Population Outcomes Measure Results Adams et al, 28 longitudinal pretest-posttest 23 RN BSN students (F = 85, M = 8) WGCTA Form A & Form B WGCTA score difference not significant (t =.954). Beckie et al, 29 pretestposttest nonequivalent control group Baccalaureate nursing students: cohort (n = 55), class before curriculum revision; cohort 2 (n = 55); and cohort 3 (n = 73), first 2 classes with revised curriculum CCTST Form A Cohort 2 had statistically significant higher total CCTST score (F = 8.58, P <.). Cohort 3 did not show any significantly higher score. Change in score within a subject over time showed significant increase in total score (F = 4.8, P <.9). Brown et al, 3 pretestposttest Nursing students (N = 23): traditional (n = 45), RN BSN (n = 35), and accelerated (n = 43) WGTA Significant increase in post- WGCTA score in traditional (t = -2.84, P =.7) and RN- BSN (t = -2.28, P =.3). No significant increase in score of accelerated students. Colucciello, 3 cross-sectional study 328 baccalaureate nursing students: sophomore II = 94, junior I = 65, junior II = 64, senior I = 59, senior II = 46 CCTST Junior II and sophomore II were not significantly different. Junior I and senior I and II students were significantly differently from sophomore II levels. Frye et al, 32 longitudinal pretest-posttest For cross-sectional analysis, 32 freshmen and 77 senior; for longitudinal analysis, 27 students WGCTA Form A For unpaired t test, significant difference. For paired t test, nonsignificant difference. McCarthy et al, 33 crosssectional 24 nursing students: 56 sophomores (female = 3, male = 43) and 85 seniors (female = 67, male = 8) CCTST Form A Seniors CCTST scores significantly higher (t = 4., P <.) Profetto-McGrath, 34 crosssectional 228 baccalaureate nursing students: year = 38, year 2 = 53, year 3 = 57, year 4 = 8 CCTST Mean score of CCTST increased from year to year 4 with an exception of year 3; however, no statistical significance among groups. Saucier, 35 longitudinal pretest-posttest Students from classes of 988 to 992 WGCTA Students from class of 99 showed a significant increase in the WGCTA scores on completion. (P =.53). Spelic et al, 36 longitudinal pretest-posttest N = 36: traditional students (n = 5), LEAP (n = 7), ANC (n = 5). Male =, female = 25. Age range = 9 to 46 years. CCTST Statistically significant at entry and exit (P.), except for the analysis subscale for LEAP students (P =.6). Thompson and Rebeschi, 37 pretest-posttest 38 baccalaureate nursing students CCTST Form A Significant increase in CCTST score entry mean to exit mean (P =.6). groups. Scores on the Evaluation Guidelines for Rating the Quality of an Intervention Study for the physical therapy and occupational therapy studies are listed in Table. The summary of the study characteristics for these studies is listed in Table 3. Pharmacy Cisneros 43 conducted a pretest-posttest study on 37 Doctor of Pharmacy students. The CCTST was administered to first-, second-, third-, and fourth-year students at the beginning and the end of academic year. Results showed no statistically significant difference in beginning and final total scores. Miller 44 conducted an exploratory study using a pretest-posttest to measure the acquisition of critical-thinking skills in doctor of pharmacy students in a 4-year professional curriculum. Results showed a statistically significant increase in CCTST scores each year (total of 4 % increase, P <., n = 2). 6 Journal of Physical Therapy Education Vol 27, No 3, Fall 23

Table 3. Study Characteristics: Physical Therapy and Occupational Therapy Author/ Design Population Outcomes Measure Results Bartlett and Cox, 38 longitudinal pretestposttest N = 28 students: female = 26, male = 2; age range = 9.5 to 25.5 years; average age = 22 years. CCTST Statistically significant gains in subscales and total scores Slaughter et al, 39 longitudinal pretestposttest N = 3: experimental = 6, control = 4; female = 3, male = ; age range = 2 to 36 years. WGCTA No significant difference in the scores between the 2 groups (F,32 =.5, P >.5) Vendrely, 4 longitudinal pretest- posttest 42 graduates of a Master of Physical Therapy program CCTST No significant change in CCTST total scores Vogel et al, 4 longitudinal pretest- posttest 5 occupational and physical therapy students (OT = 3, PT = 37). Female: OT = 3, PT = 27. Male: OT =, PT =. Age range = 2 to 43 years. WGCTA Short Form & Form B Significant increase posttest in OT students (t 2 = 3.26, P =.7) No significant change in PT students (t 2 =.34, P =.97) Zettegren, 42 cross- sectional 2 Master of Physical Therapy students (year 3 = 86, year 4 = 53, and year 5 = 6). CCTST Significant score difference between year 3 and year 5 groups (mean = 2.59, P =.) and between year 4 and year 5 groups (mean =.8, P =.5). No significant difference between year 3 and year 4 groups. Table 4. Study Characteristics: Pharmacy Author/Design Population Outcomes Measure Results Cisneros, 43 pretest-posttest Miller, 44 longitudinal pretestposttest Year to year 4 pharmacy students CCTST No significant difference in beginning and final total scores. Students from classes of 988 to 992 CCTST Increase in scores each year and total score from year to year 4 (Total of 4% increase, P <.). Table 5. Study Characteristics: Medicine Author/Design Population Outcomes Measure Results Scott et al, 45 longitudinal pretestposttest 68 for change in scores from entry to exit, 82 for correlation in entry scores and clerkship performance, and 79 for correlation between year 3 scores and clerkship performance. Female = 4; male = 4. WGCTA Mean score increased significantly (P =.3). Scores on the Evaluation Guidelines for Rating the Quality of an Intervention Study for the pharmacy studies are listed in Table. The summary of the study characteristics is listed in Table 4. Medicine Scott et al 45 administered the WGCTA to a group of 68 medical students. The WGCTA was administered at the beginning of year and near the end of year 3. They also correlated this increase in score with clinical clerkship grades, which were completed in year 2 of the curriculum. The mean WGCTA score significantly increased (P =.3). The correlation between the increase in the WGCTA scores and the clerkship grades was low. The score on the Evaluation Guidelines for Rating the Quality of an Intervention Study for the medicine study are listed in Table. The summary of the study characteristics is listed in Table 5. DISCUSSION This systematic review is based on the highest quality literature available to date. The analysis of these 8 articles revealed mixed results. There are limitations to these studies and the results must be interpreted with caution. More than half of the studies (58%) Vol 27, No 3, Fall 23 Journal of Physical Therapy Education 7

examined the acquisition of critical-thinking skills in nursing students, 5 studies (26%) measured the acquisition of critical-thinking skills in physical therapist and occupational therapist students, 2 studies (%) examined the acquisition of critical-thinking skills in pharmacy students, and only study (5%) measured critical-thinking skills in medical students. Out of the studies examining the acquisition of critical-thinking skills in nursing students, 8 studies 29-33,35-37 showed a statistically significant increase in critical-thinking skills as students progressed through the curriculum, while 2 studies 28,34 showed no statistically significant increase in critical-thinking skills. Out of the 5 studies in physical therapy and occupational therapy, 2 studies 38,42 showed a statistically significant increase in critical-thinking skills, while 3 studies 39-4 showed no statistically significant increase in critical-thinking skills. Of the 2 studies in pharmacy, study 44 showed a statistically significant increase in critical-thinking skills, while the other study 4 reported no statistically significant results. The only study in medicine 45 showed a statistically significant increase in critical-thinking scores. Nursing The longitudinal pretest-posttest study by Adams et al 28 showed no statistically significant change in critical-thinking skills. The WGCTA was administered to the students in their sophomore year. However, the students may have undergone an increase in critical-thinking skills during their freshman year. This may have produced a ceiling effect and contributed to the lack of significant change in critical-thinking skills. In the cross-sectional study by Profetto-McGrath 34 the results showed no statistically significant increase in scores on the CCTST. The defined groups were not matched for age, sex, or baseline academic level, the sample size across the groups (sophomores = 56; seniors = 85) varied, and there was no sample size calculation or statistical power analysis performed. Lack of matching across groups in their baseline academic level and the difference in number of students in the groups might have led to the lack of statistically significant results. In addition, the lack of sample size calculation and statistical power analysis reduced the external validity of the study. Six studies 29-3,35-37 used a longitudinal pretest-posttest and showed statistically significant increases in critical-thinking skills. However, none of these longitudinal studies performed statistical power analysis, and there was a possibility of carry over effect because they used the same version of the CCTST or the WGCTA for the posttest. A carry-over effect can lead to a false significant increase in the posttest and the lack of statistical power analysis reduces the external validity of these studies. Two cross-sectional studies 33,34 that showed significant changes in critical-thinking skills had lower quality scores as compared to the longitudinal studies on the Evaluation Guidelines for Rating the Quality of an Intervention Study. These 2 cross-sectional studies 33,34 did not match groups for age, sex, or preadmission criteria (eg, preadmission education level or grade point average); no statistical power analysis was performed; and their sample size varied. The lack of group matching and statistical power analysis reduces the confidence in interpreting the results of these studies. The results of the longitudinal studies are more generalizable, as they measured the change in critical-thinking skills of the same group over time and control the effect of confounding factors like age, sex, and baseline academic level. Overall, the available evidence in the field of nursing does support the acquisition of critical-thinking skills as students progress through their programs; however, these results need to be interpreted cautiously because of flaws, which may have influenced the results. Physical Therapy and Occupational Therapy Three studies 39-4 with a pretest-posttest showed no statistically significant increase in critical-thinking scores. Slaughter et al 38 measured the change in critical-thinking skills in physical therapist students before and after a 4-week clinical clerkship of first-year physical therapist students. Both the defined control group and the experimental group completed an identical 4-week clerkship, with the experimental group participating using a problem-solving model during their clerkship. A 4-week clinical clerkship is a very short period of time to show measureable improvement in critical-thinking skills and may have contributed to the lack of a statistically significant increase in critical-thinking skills. In addition, the sample size was small (N = 3). Vogel et al 4 measured the acquisition of critical-thinking skills in physical therapist students across the 2 months of academic work in a pretest-posttest. Results showed no statistically significant increase in the acquisition of critical-thinking skills, however, a small sample size and the fact that the posttest occurred prior to the clinical phase of the program may have contributed to the lack of significant changes. Although Vendrely s pretest-posttest longitudinal study 4 found an increase in critical-thinking skills, this increase was not statistically significant. Zettergren 42 performed a crosssectional study that yielded a statistically significant increase in critical-thinking skills; however, these results must be interpreted cautiously. This study had a medium statistical power of.27, unmatched groups and variability in the sample sizes (third year = 86; fourth year = 53; fifth year = 6). Bartlett and Cox 38 performed a longitudinal study that showed a statistically significant increase in critical- thinking skills. This study had a small sample size (N = 28), a medium effect size (.64), and no power calculation, posing a threat to external validity. Vogel et al 39 studied the change in criticalthinking skills in occupational therapist students using a pretest-posttest. Results showed a statistically significant increase in critical-thinking scores. However, these results need to be interpreted with caution because the sample size was small (N = 3) and the statistical power was low. Currently, the majority of physical therapist education programs confer a Doctor of Physical Therapy (DPT) degree. With the development of DPT programs, major changes have occurred in curricula with an emphasis on critical-thinking skills. To date, no studies have examined the acquisition of criticalthinking skills in students completing these curricula. * Pharmacy Cisneros 43 performed a pretest-posttest longitudinal study on Doctor of Pharmacy students. The results showed no statistically significant increase in the CCTST scores. The time between the pretest and posttest was relatively short (pretest at the beginning of fall 25 and the posttest at the end of spring 26). Two semesters might not be a sufficient amount of time for students to show an increase in their critical-thinking skills. Miller 44 performed a longitudinal study on Doctor of Pharmacy students that showed statistically significant increase in criticalthinking skills. There was a 5% attrition rate over the 4- year testing period (pretest = 2; posttest = 2). This attrition resulted from students dropping out of the program, students dropping back in the program, as well as students not being available for the posttest. The students who dropped out of the program and the students who dropped * Editors note: This issue of the Journal of Physical Therapy Education includes an article by Huhn et al, Tracking Change in Critical-Thinking Skills, which measures change in critical thinking in students enrolled in 2 DPT programs using the Health Sciences Reasoning Test. 8 Journal of Physical Therapy Education Vol 27, No 3, Fall 23

back in the program might have had poorer academic performances, influencing their acquisition of critical-thinking skills as they progressed through the curriculum. This attrition might have skewed the results, reducing the external validity of the study. In addition, both assessments 43,44 had a small sample size. The results of the literature in pharmacy are mixed and do not provide conclusive evidence with respect to the acquisition of critical-thinking skills in pharmacy students. Medicine Scott et al 45 performed a pretest-posttest (pretest at the beginning of the first year and posttest at the end of the third year) longitudinal study on medical students. The results showed a statistically significant increase in critical-thinking skills. The small sample size (N = 68) and the lack of statistical power analysis reduced the confidence in the generalizability of the result. With only study with a small sample size on medical students, it is difficult to draw a definitive conclusion. All the studies included in this systematic review lacked randomization and blinding. Although randomized controlled trials (RCTs) are the highest level of evidence for intervention studies, randomization is not a practical approach in the case of measuring the acquisition of critical-thinking skills because the intervention (the curriculum) could not be excluded for the control group. The studies in this review were either a pretest-posttest or cross-sectional. The time between the pretest and posttest varied among studies with some studies measuring the acquisition of critical-thinking skills over the course of academic year, while other studies measured the acquisition of criticalthinking skills over the course of the entire program. Cross-sectional studies included in this review lacked detailed matching between groups and measured the acquisition of critical-thinking skills at different points in programs. Each study included in this systematic review had a convenience sample group. There was a large variability of age across samples. In the study by Spelic et al, 36 the age ranged from 9 to 45 years, while in the study by Bartlett and Cox 38 the age ranged from 9 to 25.5 years. Age may influence the acquisition of critical-thinking skills. Barlett and Cox 38 found age to be negatively correlated to changes on the CCTST. There was variability in education level prior to the beginning of their respective health care education programs. These variables may have acted as confounding factors. Small sample sizes were also an important factor. Only 3 studies 38,4,42 reported an effect size and power analysis. A smaller effect size can jeopardize external validity and may contribute to a lack of statistically significant change. Curricula were used as the intervention in all identified studies. None of the studies specifically described the curriculum being used. Variability in curricula from one study to another certainly could have had an impact on the results. The outcome measures used in the studies were the CCTST (6%) and the WGCTA (39%). Some of the studies also used the CCTDI as a secondary outcome measure and found correlation between the CCTST/ WGCTA and the CCTDI scores. Adams et al, 28 Vogel et al, 4 and Slaughter et al 39 used the WGCTA as their outcomes measure and showed no statistically significant increase in critical-thinking scores. These authors question whether this tool is sensitive enough to measure the acquisition of critical-thinking skills in health care personnel. They recommended a more specific tool for health care professionals. Beckie et al 29 found changes in the CCTST scores in the group who completed the revised nursing curriculum (experimental group) but they also recommended a more specific tool for health care professionals. In 26, Facione and Facione developed the Health Sciences Reasoning Test (HSRT). 46 The test items focus on clinical and professional practice in health care professionals. Test items are set in clinical and professional practice context but do not require any specialty knowledge in health care. This tool consists of 33 multiple-choice questions. This tool is presently being used in some ongoing research studies; however, results using the HSRT have not been published (see footnote on page 8). CONCLUSION Critical thinking is a vital skill needed in health care professionals in order to make sound clinical decisions and deliver competent patient care. Professionals are challenged by the complexities of the health care system to make good clinical judgments. Health care professional curricula need to facilitate the development of critical-thinking skills in students. This systematic review shows that there are mixed results with respect to the acquisition of critical-thinking skills in health care professional students as measured by the CCTST and the WGCTA. There are a limited number of moderate quality studies in the fields of occupational therapy, physical therapy, pharmacy, and medicine. Although there are studies identified in the field of nursing, the results are mixed. No studies were found measuring critical-thinking skills in Doctor of Physical Therapy, dietetics, or dentistry students (see footnote on page XX). Additional well-ed studies are needed in health care professional students in order to be able to make more definitive conclusions across various health care professional programs. RECOMMENDATIONS Critical thinking is an abstract construct. Measuring the improvement in the acquisition of critical-thinking skills is a challenge in health care professional education programs. This systematic review suggests additional high-quality studies are needed in this area. Although RCTs are the highest level of evidence, it is not a practical approach in measuring the acquisition of critical-thinking skills using curriculum as the independent variable. Administering the critical-thinking measure at the beginning and end of a health care professional curricula will give a clearer understanding of the effect of the entire curriculum on the acquisition of critical-thinking skills in students. In the future, studies should control for the variability in the preadmission educational level, as this can have an impact on the acquisition of critical-thinking skills by skewing scores. Future studies need to include larger sample sizes as well as power analyses. Since several of the authors have suggested that the CCTST and the WGCTA may not be sensitive enough to changes in critical-thinking skills in these populations, the HSRT may be a good alternative to consider, since the questions are set in clinical and professional practice context and may be more sensitive to change in these populations. In addition, qualitative evaluation through portfolio review may contribute to better understanding of the development of critical-thinking skills of students progressing through a health professions curriculum. Because the tools currently used to measure critical thinking are not discipline-specific, the profession of physical therapy may benefit from the creation of a critical-thinking tool specific to the discipline. In addition, studies in physical therapy need to focus on the development of critical-thinking skills throughout an entire program to include clinical education. By measuring across the entire curriculum, we may be able to determine where in the curriculum students critical-thinking skills improve the most. This would minimally involve measurement at baseline or entry into the program, measuring at the end of the academic portion of the curriculum, and again at the end of the students clinical experiences. In addition, studies in physical therapy also need to compare the acquisition of critical-thinking skills in Vol 27, No 3, Fall 23 Journal of Physical Therapy Education 9

students in different programs, such as freshman professional programs verses postgraduate professional programs. REFERENCES. Facione PA. Critical Thinking: A Statement of Expert Consensus for Purposes of Educational Assessment and Instruction. Newark, DE: American Philosophical Association; 99. ERIC Document Reproduction Service, No. ED 35-423. 2. Phyllis T. Critical thinking in nursing education and practice as defined in the literature. Nurs Educ Perspect. 25;26(5):272-277. 3. Chen FC, Lin MC. Effects of a nursing literature reading course on promoting critical thinking in two-year nursing program students. J Nurs Res. 23;(2):37-47. 4. Bell ML, Heye ML, Campion L, Hendricks PB, Owens BL, Schoonover J. Evaluation of a process-focused learning strategy to promote critical thinking. J Nurs Educ. 22;4(4):75-77. 5. Accreditation Manual with Interpretive Guidelines. New York, NY: National League for Nursing Accrediting Commission. 25. 6. Recommended Baccalaureate Competencies and Curricular Guidelines to Assure High Quality and Safe Patient Care. Washington, DC: American Association of Colleges of Nursing; 26. 7. O Dell B, Mai J, Thiele A, Priest A, Salamon, K. The hot seat: challenging critical thinking and problem solving skills in physical therapist students. The Internet J Allied Health Sci Pract. 29;7(). 8. American Physical Therapy Association. Guide to Physical Therapist Practice. Alexandria, VA: American Physical Therapy Association; 2. 9. American Physical Therapy Association. Vision 22. http://www.apta.org/vision22. Accessed November 2.. Velde BP, Wittman TT, Vos P. Development of critical thinking skills in OT students. Occup Ther Int. 26;3():49-6.. Facione N, Blohm S, Howard K, Giancarlo C. California Critical Thinking Skills Test Manual. Millbrae, CA: California Academic Press; 998. 2. Gadzella BM, Hogan L, Masten W, Stacks J, Stephens R, Zascavage V. Reliability and validity of the Watson-Glaser Critical Thinking Appraisal-Forms for different academic groups. J Instructional Psychol. 26;33:4-43. 3. Gadzella BM, Stacks J, Stephens RC. Watson- Glaser Critical Thinking Appraisals, Form-S for education majors. J Instructional Psychol. 25:32();9-2. 4. Facione NC, Facione PA. Externalizing the critical thinking in knowledge development and clinical judgment. Nurs Outlook. 996;44(3):29-36. 5. Watson G, Glaser EM. Critical Thinking Appraisal: Manual. San Antonio, TX: Psychological Corp; 98. 6. Facione N, Blohm S, Howard K, Giancarlo C. California Critical Thinking Skills Test Form A & B: Test Manual. Millbrae, CA: California Academic Press; 998. 7. Brunt BA. Models, measurement, and strategies in developing critical thinking skills. J Continuing Educ Nurs. 25;36(6):255-262. 8. Kennison MM. The evaluation of students reflective writing for evidence of critical thinking. Nurs Educ Perspect. 26; 27(5):269-273. 9. MacDermid JC. An introduction to evidencebased practice for hand therapists. J Hand Ther. 24;7(2):5-7. 2. Iversen MD, Choudhary VR, Oatel SC. Therapeutic exercise and manual therapy for persons with lumbar stenosis. Int J Clin Rheum. 2;594:425-437. 2. Muller M, Tsui D, Biddurph-Deisroth L, Hard J, MacDermid JC. Effectiveness of hand interventions in primary management of carpal tunnel syndrome: a systematic review. J Hand Ther. 24;7(2):2-228. 22. Williams RM, Schmuck G, MacDermid JC. Effectiveness of workplace rehabilitation intervention in the treatment of work-related upper extremity disorders: a systematic review. J Hand Ther. 24;7(2);267-273. 23. Brudvig TJ, Kulkarni H, Shah S. The effects of therapeutic exercise and mobilization on patients with joint dysfunction: a systematic review with meta-analysis. JOSPT. 2;4():734-748. 24. The Pedro scale. http://www.predro.org.au. Accessed May 28, 23. 25. Jadad AR, Moore RA, Carroll D. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Controlled Clin Trials.996;7:-2. 26. Labelle H, Guibert R, Joncas J, Newman N, Fallaha M, Rivard CH. Lack of scientific evidence for the treatment of lateral epicondylitis of the elbow: an attempted meta-analysis. J Bone Joint Surg Br. 992;74: 646-65. 27. SPSS (computer program). Version 8. Armonk, NY: IBM; 2. 28. Adams MH, Stover LM, Whitlow JF. A longitudinal evaluation of baccalaureate nursing students critical thinking abilities. J Nurs Educ. 999;38(3):39-4. 29. Beckie TM, Lowry LW, Barnett S. Assessing critical thinking in baccalaureate nursing students: a longitudinal study. Holist Nurs Pract. 2;5(3):8-26. 3. Brown JM, Alverson EM, Pepa CA. The influence of a baccalaureate program on traditional, RN-BSN, and accelerated students critical thinking abilities. Holist Nurs Pract. 2;5(3):4. 3. Colucciello ML. Critical thinking skills and dispositions of baccalaureate nursing students- -a conceptual model for evaluation. J Professional Nurs. 997;3(4): 236-245. 32. Frye B, Alfred N, Campbell M. Use of the Watson-Glaser Critical Thinking Appraisal with BSN students. Nurs Health Care Perspect. 999; 2(5):253-255. 33. McCarthy P, Schuster P, Zehr P, McDougal D. Evaluation of critical thinking in a baccalaureate nursing program. J Nurs Educ. 999;38(3):42-44. 34. Profetto-McGrath J. The relationship of critical thinking skills and critical thinking dispositions of baccalaureate nursing students. J Adv Nurs. 23;43(6):569-577. 35. Saucier BL. Critical thinking skills of baccalaureate nursing students. J Professional Nurs. 995;(6):35-357. 36. Spelic SS, Parsons M, Hercinger M, Andrews A, Parks J, Norris J. Evaluation of critical thinking outcomes of a BSN program. Holist Nurs Pract. 2;5(3): 27. 37. Thompson C, Rebeschi LM. Critical thinking skills of baccalaureate nursing students at program entry and exit. Nurs Health Care Perspect. 999;2(5):248-252. 38. Bartlett DJ, Cox PD. Measuring change in students critical thinking ability: implications for health care education. J Allied Health. 22;3(2):64-69. 39. Slaughter DS, Brown DS, Gardner DL, Perritt LJ. Improving physical therapy students clinical problem-solving skills: an analytical questioning model. Phys Ther. 989;69(6):44. 4. Vendrely A. Critical thinking skills during a physical therapist professional education program. J Phys Ther Educ. 24;9:55-59. 4. Vogel KA, Geelhoed M, Grice KO, Murphy D. Do occupational therapy and physical therapy curricula teach critical thinking skills? J Allied Health. 29;38(3):52-57. 42. Zettergren KK. Changes in critical thinking scores: an examination of one group of physical therapist students. J Phys Ther Educ. 24;8(2):73-79. 43. Cisneros RM. Assessment of critical thinking in pharmacy students. Am J Pharm Educ. 29;73(4):-7. 44. Miller DR. Longitudinal assessment of critical thinking in pharmacy students. Am J Pharm Educ. 23;67(4):2. 45. Scott JN, Markert RJ, Dunn MM. Critical thinking: change during medical school and relationship to performance in clinical clerkships. Med Educ. 998;32():48. 46. Health Sciences Reasoning Test (HSRT). http://www.insightassessment.com/9test-hsrt. html. Accessed December 5, 2. 2 Journal of Physical Therapy Education Vol 27, No 3, Fall 23