Jane Carpenter PhD, MSN, RN Clinical Teaching Institute July 22, 2016
Discuss construction of test items Discuss alternate format types of questions Review of test questions
Are you testing preparation for class? Are you testing content mastery? Assignment of grades or points (low-stakes or high-stakes) Program Assessment Measurement of student learning.
Before starting ask yourself these questions: When will tests be scheduled for the semester? What content areas are to be covered per exam? How many items included per content area? (Consider time spent on content area as a guide) What are the total number of items for the exam and how much time will be given to test? How will the test be delivered?
The use of a test grid can help guide test construction as you begin to develop or redo your exam items: Consider including the following: Course objectives Nursing process Client need category from NCLEX detailed test blueprint Concepts important for them to know Cognitive Level (Bloom s)
Test banks can be a guide need to rework. Use grid and course objectives as you develop your items. What level of Bloom s do you want the item to be? Writing teams Always have your items reviewed by other faculty
http://pcs2ndgrade.pbworks.com/w/page/46897760/revised%20b loom's%20taxonomy
Recall of facts or details Can be used to review basic information. Ex. Normal lab values, name of a medication, diagnostic test What does the information or facts mean? REMBERING UNDERSTANDING
APPLICATION LEVEL Involves application of knowledge related to situations. Used to perform a task, solve a problem, or obtain a particular result. Simulates a real world situation in nursing. Can be written to various levels of difficulty. Longo, B. Improving Test Evaluation in Nursing programs. Retrieved at: Presentation retrieved at: www.mynursingcommunity.com/ uploads/2012/03/improving-test-evaluation-in- Nursing-Programs.pdf (slide 27) ANALYSIS LEVEL Analysis breaks information into parts, detects relationship of those parts and the way they are organized. Used to dissect and analyze information & distinguish between critical and noncritical care. Two or more pieces of data are gathered and interpreted before deciding a course of action. Tends to be a more difficult level of question.
Who is the Subject? (nurse, client, family) What is the action or verb? (ex. Assess, teach) What descriptors are needed? What is the condition? What is your question? (What do you want to know) Be sure questions are aligned with your curriculum and your course objectives. Sanders, S. Test Construction & Item-Writing. Presentation retrieved at: www.ateaonline.org/resources/documents/atea Item Writing Workshop 2013 handouts.pdf Williams, N., & Reichman, R. (2015). NCLEX Regional Workshop for Educators. Presented in Topeka, KS on November 6 th
What are the etiology and/or risk factors for this health problem? What are the associated nursing assessment and lab findings for this health problem? What nursing care and teaching are associated with the diagnosis studies for this health problem? What are the appropriate nursing diagnosis and goals of care for this health problem? Longo, B. Improving Test Evaluation in Nursing programs. Retrieved at: Presentation retrieved at: www.mynursingcommunity.com/uploads/2012/03/improving-test-evaluation-in- Nursing-Programs.pdf (slide 6)
Start by first writing the answer Validate your answer (know the rationale) Should be similar in length (ascending or descending by size) Should be plausible Keep it simple Avoid use of always and never Those who are not familiar with the content will be more likely to choose a distractor.
Multiple Choice Most commonly used type of question. Usually 3 distractors plus one correct answer Write your correct answer first. Be sure that your distracters are feasible.
Describe a problem instead of naming it. Provide the clinical manifestations instead of stating the disease. Multiple disease processes involved. Integrated concepts. Prioritization. (which should you do first) More than one concept into the stem. Which statement is the best? Sanders, S. Test Construction & Item-Writing. Presentation retrieved at: www.ateaonline.org/resources/documents/atea Item Writing Workshop 2013 handouts.pdf
Question followed by 5-6 answer choices Each answer is independent of the other answer choices. Should be able to answer as Yes or No OR True or false Minimum of two correct answers Never all answers correct. What percentage of this type of question are you including on your exams? How are you grading?
Used most frequently for math questions Space is for numbers only (not unit of measure) NCLEX specifies decimal point required (ex. Round to the hundredth) If using for other than math be clear in your directions if spelling must be correct.
Used on procedures or prioritization. What do you do 1 st, 2 nd, 3 rd, etc. Students are asked to put items in order. All in the correct order to receive credit.
Usually a picture or a chart. Great choice for anatomy. Student moves cursor or places mark over the spot discussed (need to be sure of your parameters for grading). Ex. Marking the correct dose on a syringe. Where to assess a particular heart sound.
Use of several charts or exhibits for students to view in order to make an answer decision. Can answer without looking at the exhibits but will be difficulty without the information. Ex. Set of health care provider prescriptions, lab results, medication list.
A tape that students must listen to in order to answer the question. NCLEX prompts student if they try and answer the question without listening to the audio. Need to provide headphones if administering the test in a group setting.
Student is provided a picture than asked to identify an item. Pictures might be listed as answer choices. Picture may be part of the question stem. NCLEX tutorial of alternate format questions is available for students: www.pearsonvue.com/nclex
No names Eliminate scenarios Distracters should be feasible and homogenous Alternate location of your correct answer Avoid all of the above & none of the above Written for an entry level nurse No product names (ex. Coke)
To help eliminate bias avoid the following unless is pertinent to the question: Religion Race Gender Age Marital Status Sexual Orientation
Client Healthcare provider Unlicensed assistive personnel Prescription Will not see All of the above or None of the above Medications listed only as generic.
Grammatial Clues Logical Clues Word Repeats Greater detail in correct option Lost sequence in data Absolute terms Vague terms Negative stem Implausible distractors Unfocused stem No correct or greater than one correct response Unnecessary information in stem All of the above None of the above Fill in the blank (blank in the middle of stem) Tarrant, M., & Ware, J. (2008). Impact of item-writing flaws in multiple-choice questions on student achievement in high stakes nursing assessments. Medical Education, 42, 206.
Test blueprint important first step. Item analysis is critical. Rework of items based on data. Review of test items with students Possibility of re-teach for items students scored poorly on (was it the question or does the class not understand the concept). Exam performance goes hand-in-hand with study skills. Goal for the student should be Quality & Safety when providing patient care.
References Black, Pam. (2014). Structuring Test Items to the NCLEX Format. Retrieved from: http://www.toadn.org/presentations/2014-02_structuring_your_test_to_the_nclex_test_plan_handout.pdf Campbell, D. E. (2011). How to write good multiple-choice questions. Journal of Paediatrics and Child Health, 47, 322-325. Cleary, M., & Horsfall, J. (2012). Developing and using multiple-choice tests in clinical settings. The Journal of Continuing Education in Nursing, 43(8), 344-345. Garner, S. Critical Thinking &Test Item Writing, ACG: Academic Consulting Group. Retrieved at: https://www.midwestern.edu/documents/chs%20dean/2-26-1 5%20Critical%20Thinking%20PowerPoint.pdf Killingsworth, E., Kimble, L., & Sudia, T. (2015). What goes into a decision? How nursing faculty decide which best practices to use for classroom testing. Nursing Education Perspectives, 36(4), 220-225. Doi:10.5480/14-1492. Longo, B. Improving Test Evaluation in Nursing Programs Test Construction, Alternate Item Format, and NCLEX Preparation. Presentation retrieved at: www.mynursingcommunity.com/uploads/2012/03/improving-test-evaluation-in- Nursing-Programs.pdf Moore, B., & Clark, M. (2016). The role of linguistic modification in nursing education. Journal of Nursing Education, 55(6), 309-315. Morrison, S., & Walsh Free, K., (2001). Writing multiple-choice test items that promote and measure critical thinking. Journal of Nursing Education, 40, 17-24. Penn, B. (2011). Creating and Analyzing Test Items. Retrieved online at www.aacn.nche.edu/membership/members-only/presentations/2011/11facdev/penn.pdf
References continued Reiter-Palmon, R. (University of Nebraska at Omaha). Test Development & Item Writing Guidelines. Retrieved at: app1.unmc.edu/nursing/conweb/test_development_and_item_writing_guidelines.pdf Sanders, S. Test Construction & Item-Writing. Presentation retrieved at: www.ateaonline.org/resources/documents/atea Item Writing Workshop 2013 handouts.pdf Tarrant, M. & Ware, J. (2008). Impact of item-writing flaws in multiple-choice questions on student achievement in high-stakes nursing assessments. Medical Education, 28, 128-206. Williams, N., & Reichman, R. (2015). NCLEX Regional Workshop for Educators. Presented in Topeka, KS on November 6 th.