An Internationally Consented Standard for Nursing Process-Clinical Decision Support Systems in Electronic Health Records

Transcription

1 An Internationally Consented Standard for Nursing Process-Clinical Decision Support Systems in Electronic Health Records Maria Müller-Staub, PhD, EdN, RN, FEANS, Helen de Graaf-Waar, RN, MSc, Wolter Paans, PhD, RN Nurses are accountable to apply the nursing process, which is key for patient care: It is a problem-solving process providing the structure for care plans and documentation. The state-of-the art nursing process is based on classifications that contain standardized concepts, and therefore, it is named Advanced Nursing Process. It contains valid assessments, nursing diagnoses, interventions, and nursing-sensitive patient outcomes. Electronic decision support systems can assist nurses to apply the Advanced Nursing Process. However, nursing decision support systems are missing, and no gold standard is available. The study aim is to develop a valid Nursing Process-Clinical Decision Support System Standard to guide future developments of clinical decision support systems. In a multistep approach, a Nursing Process- Clinical Decision Support System Standard with 28 criteria was developed. After pilot testing (N = 29 nurses), the criteria were reduced to 25. The Nursing Process-Clinical Decision Support System Standard was then presented to eight internationally known experts, who performed qualitative interviews according to Mayring. Fourteen categories demonstrate expert consensus on the Nursing Process-Clinical Decision Support System Standard and its content validity. All experts agreed the Advanced Nursing Process should be the centerpiece for the Nursing Process-Clinical Decision Support System and should suggest research-based, predefined nursing diagnoses and correct linkages between diagnoses, evidencebased interventions, and patient outcomes. KEY WORDS: Clinical decision support system, D-catch, Documentation, Electronic nursing record, Q-DIO, Standard validation Nursing documentation is essential for safe and efficient patient care 1,2 ; missing documentation endangers care continuity, and the lack of standardized language Author Affiliations: Stadtspital Waid, Zürich, Switzerland; Pflege PBS, Switzerland; Lectoraat Nursing Diagnostics, Hanze University of Applied Sciences, Groningen, the Netherlands (Dr Müller-Staub); Erasmus Medical University Center, Rotterdam, the Netherlands (Ms de Graaf-Waar); and Lectoraat Nursing Diagnostics, Hanze University of Applied Sciences, Groningen, the Netherlands (Dr Paans). The first development stage (literature reviews) of this standard was supported by start-up grant from the European Academy of Nursing Science. The authors have disclosed that they have no significant relationships with, or financial interest in, any commercial companies pertaining to this article. Corresponding author: Maria Müller-Staub, PhD, EdN, RN, FEANS, Lectoraat Nursing Diagnostics, Hanze University of Applied Sciences, Eyssoniusplein 18, Groningen, the Netherlands (muellerstaub@me.com). hampers good documentation. 2 The nursing process is key to ensure patient safety as it provides the structure for care plans and documentation. 3 Nurses are accountable for the nursing process, and health law requires its documentation. 3 However, applying the nursing process is demanding, and nurses would profit from computer systems providing clinical decision support to its use. 3 Clinical decision support systems (CDSSs) are designed to assist physicians and nurses in determining diagnoses based on accurate information. 4,5 Hunt et al 6 defined CDSS as any software designed to directly aid in clinical decision-making in which characteristics of individual patients are matched to a computerized knowledge base for the purpose of generating patient-specific assessments or recommendations that are (pp ) then presented to clinicians for consideration. The study aim is to produce a standard that guides the development of a CDSS addressing the nursing process. The nursing process comprises nursing assessments, nursing diagnoses, planning outcomes, implementing interventions, and evaluating the effectiveness of interventions and nursing-sensitive patient outcomes. 7 Nursing diagnoses are a core element of this process because they are the basis for selecting effective interventions. 7,8 This process guides communication at handovers, whereby the oncoming shift nurse receives information regarding patients nursing diagnoses (eg, nausea), planned outcomes (eg, well-being), and interventions (eg, nausea care). Studies have shown that handover effectiveness, interdisciplinary communication, and intrainstitutional and interinstitutional patient information rely on comprehensive and accurate nursing process documentation, 9,10 which not only must be accurate and complete but also legible regardless of whether it is handwritten or typed. 11 According to the current literature, the nursing process is based on nursing classifications and is composed of valid, standardized concepts. Standardized concepts such as nursing diagnoses, interventions, and patient outcomes are referred as Standardized Nursing Language (SNL). 12 A classification of an SNL that is applied in the nursing process is an Advanced Nursing Process, 8 which is defined as The advanced nursing process consists of defined, validated concepts. It includes assessments, nursing diagnoses, nursing interventions, and nursing outcomes that are rooted in scientifically based Volume 34 Number 11 CIN: Computers, Informatics, Nursing 493

2 nursing classifications. 3(p13) Nurses experience difficulties in accurately applying the Advanced Nursing Process because it requires knowledge of standardized concepts (eg, standardized descriptions of nausea, anxiety, and risk for bleeding) and clinical decision-making skills. 3,13,14 Standardized Nursing Language is not only used to define knowledge that is ordered in classifications, it also is a prerequisite to construct electronic health records (EHRs) and for implementing ehealth strategies. It is essential for data retrieval, statistical analyses, and benchmarking 10 and also guarantees clearer, more comprehensive, and faster data entry and retrieval. 10,15 It is important that EHRs represent the Advanced Nursing Process by using SNL as EHRs per se do not guarantee care quality. 16 Nurses require support by an SNL-based Nursing Process-CDSS (NP-CDSS) to effectively apply and document the Advanced Nursing Process. 10 Evaluations of this process among settings are needed to ensure care continuity and quality when employing ehealth. When patients transfer from hospitals to home care, nurses require information on actual nursing diagnoses (eg, risk for falls, impaired self-care), planned interventions (fall prevention care, self-care support), and desired patient outcomes (fall prevention, no injuries, enhanced self-care). Research revealed inaccurate and redundant nursing documentation, including lack of information 13,14,17 as a reason for inadequate patient care. 1,16,17 Applying the Advanced Nursing Process by using SNL would lead to increased accuracy of nursing process documentation and improve the quality of electronic documentation. 16,18,19 In a literature review, Piscotty and Kalisch 20 concluded that CDSSs are not actually used in nursing. Studies focused primarily on physicians use of CDSS, and research supporting CDSS integration into nursing practice is limited. 20 Piscotty and Kalisch 20 reported that nurses have minimal knowledge of CDSS developments and about the knowledge base of clinical decision making to electronically support nurses workflow. Findings also demonstrated limited research on relationships between nurses experience and their use of CDSS. 21,22 In summary, the literature describes three major issues: 1. Nurses documentation in the patient record is often narrative, unstructured, and incomplete. It contains redundancies and does not represent the Advanced Nursing Process. 2. Standardized Nursing Language offers possibilities to precisely title and code nursing diagnoses, interventions, and outcomes; coded concepts allow performing evaluations. However, SNL is not yet fully implemented into practice and EHRs. 3. Nurses have difficulties in linking their professional knowledge with data obtained in patient assessments and transferring this knowledge into SNLs that incisively define nursing diagnoses, interventions, and outcomes. The majority of EHRs do not contain Advanced Nursing Process decision support based on SNL. Nursing-specific CDSSs do not exist, and no gold standard is available for accurate nursing process documentation. 23 Future EHRs should contain NP-CDSSs that provide standardized, valid nursing diagnoses based on patient assessment data. They should also propose evidence-based nursing interventions to enhance patient safety and high-quality outcomes. STUDY AIM The aim of this study is to develop an internationally accepted, reliable, and valid NP-CDSS Standard to guide the development of future electronic documentation systems by exploring previous instruments, applying Advanced Nursing Process criteria, and utilizing expert consensus. The standard must describe NP-CDSS criteria and must be applicable for developing and evaluating EHRs. METHODS The study included three steps. First, a literature review was performed to determine previous instruments and criteria for an NP-CDSS Standard. Second, the NP-CDSS Standard was developed and subjected to a pilot validation by 27 Dutch nurses who assessed its applicability and face validity. Third, the NP-CDSS Standard was refined and sent to internationally known experts. Semistructured qualitative interviews were performed with eight international experts to reveal categories and establish the NP-CDSS Standards content validity according to the method described by Waltz et al. 24 Previous instruments that laid the foundation for the NP-- CDSS Standard are presented in the next section, followed by the specific methods used in the second and third steps of this study. Previous Instruments and Nursing Process-Clinical Decision Support System Development The literature revealed seven instruments including criteria for Advanced Nursing Process documentation. 11,25 Previous instruments and their main characteristics with foci of measurement are presented in Table 1. All seven instruments were clinically applied and psychometrically tested, however, primarily in national contexts. The Quality of Diagnoses, Interventions, and Outcomes (Q-DIO) was the only SNL-based instrument that measured coherent, accurate linkages between assessments, diagnoses, interventions, and outcomes. It is a criterion-referenced instrument with 29 items. It has been translated into seven languages and psychometrically tested in several settings across continents. 26 Based on study results, the Q-DIO was suggested as functioning as a basis for EHR developments. 27,28 The D-Catch measures the accuracy of the record structure, 494 CIN: Computers, Informatics, Nursing November 2016

3 Table 1. Previous Instruments and Their Main Characteristics or Foci of Measurement 11,25,26,28 Instrument Name Ziegler Criteria for Evaluating the Quality of the Nursing Process (ZCEQNP) NoGA Degrees of Accuracy in Nursing Diagnoses Scale (Lunney Scale) Cat-ch-Ing Main Characteristics/Focus of Instrument Measuring the structural aspects of nursing process documentation Measuring the content and structure of nursing documentation Measuring the accuracy of nursing diagnoses based on defining characteristics and their relevancy with respect to patient assessment data Measuring the quantity and quality of the nursing process documentation based on Swedish law Measuring the accuracy of nursing diagnoses Quality of Nursing Diagnosis (QOD) Q-DIO Measuring the quality, accuracy, and correctness of assessment data, nursing diagnoses, interventions, nursing outcomes, and their coherent internal, logical relations, or linkages 25,26,28 D-Catch This instrument bases on the Cat-ch-Ing and on the Lunney Scale for Degrees of Accuracy. 28 It measures the accuracy and quality of the nursing process. 11 with 10 items addressing admission data, nursing diagnoses, nursing interventions, progress notes and outcome evaluations, and legibility of nursing documentation. 11 None of the instruments were developed to measure the accuracy of the Advanced Nursing Process documentation in EHRs, and only the Q-DIO was applied in a multicenter study. 26 On the basis of these results, the authors developed an NP-CDSS Standard. Nursing Process-Clinical Decision Support System Standard Development In the second step, the NP-CDSS Standard containing 28 criteria was developed. A group of 27 Dutch nurses performed a pilot validation to assess its face validity, completeness, and clinical relevance. The inclusion criteria for nurses included being a clinical nurse or nurse educator with knowledge and expertise of the Advanced Nursing Process and being involved in EHR developments. Each nurse judged the NP-CDSS Standard on a 3-point scale: The criterion is essential (2 points); the criterion is important but not essential (1 point); or the criterion is not necessary (0 points). Based on this pilot validation, 29 only a small number of criteria were reformulated, and three that were redundant were omitted. To assess the content validity of the NP-CDSS Standard with its remaining 25 criteria (Table 2), an international expert validation study was performed. Content validity is defined as describing the degree to which the criteria/ items of a standard or instrument adequately represent the universe of content for the concept being measured. 30(p723) Sample To establish international consensus, international SNL experts were subsequently invited to validate the NP-CDSS Standard. Inclusion criteria for the experts included the following: A doctoral or master s degree in nursing science with postgraduate specialization or a doctoral degree in nursing informatics; Demonstration of expertise by published research on the Advanced Nursing Process and its application and evaluation in practice/teaching SNLs developments, implementations, and/or evaluations of the nursing process in electronic systems (eg, EHR developers). Qualitative Expert Interviews The aim of the interviews was to obtain experts feedback, critique, and opinions on the NP-CDSS Standard. The first and last authors developed an interview guideline with direct verbal questions according to the 25 criteria. To exclude researcher bias, a research assistant performed the interviews and data analyses. In a first Skype meeting (duration maximum, 10 minutes), the experts received the study information, were asked for informed consent and a confidentiality statement, and obtained the NP-CDSS Standard. The experts then had at least 2 days to review the NP-CDSS Standard and to prepare for the semistructured interviews, 31 which were performed in a second Skype meeting. In addition to the interview guideline, the experts were asked to offer additional, qualitative comments on each criterion; for example, what is your opinion of this criterion? Do you think this criterion should be part of the NP-CDSS Standard? Did you ever consider an argument, opinion, or point of critique? Do you find the criterion clear and unambiguous? Do you suggest a change or addition to this criterion? Each interview was audio recorded and transcribed verbatim. Data Analysis For data analysis, Mayring sapproach 32 was applied by using QSR NVivo 10 (QSR International, Doncaster, Victoria, Australia). This approach includes (1) defining evaluative categories, (2) coding text segments, and (3) performing descriptive and statistical analyses. 32(p13ff) Mayring s basic principles (eg, units of analysis, step models, working with categories) were applied by employing the guideline. First, the experts answers to the two main dimensions of Criteria for Nursing Volume 34 Number 11 CIN: Computers, Informatics, Nursing 495

4 Table 2. NP-CDSS Standard NP-CDSS standard 1. The NP-CDSS offers the Nursing Process as the CENTERPIECE of nursing information and documentation, eg, thenursing process is the backbone for nursing care planning and documentation which is presumed to be nursing-process centered planning and documentation. 2. The NP-CDSS supports the nurse by presenting all phases of the Advanced Nursing Process: Assessment, nursing diagnoses, nursing interventions, planning, nursing-sensitive patient outcomes, and evaluations that are based on state-of-the-art (evidence-based/knowledge-based) Advanced Nursing Process and nursing classifications/taxonomy literature. 3,8,10,11,17,34 3. The NP-CDSS supports the nurse by containing SNL presenting knowledge (defined concepts) for each phase of the Advanced Nursing Process. 4. The NP-CDSS supports the nurse with possibilities for free text entries (besides standardized terminology) and open documentation (ie, specific judgments, rapports, protocol adaptations related to the patient situation, operationalization of unclassified diagnoses, goals, interventions, and outcomes and evaluations). 5. The NP-CDSS supports the nurse by containing relevant, holistic nursing documentation regarding patient information on physical aspects, psychosocial aspects, functional aspects, and environmental aspects. 6. The NP-CDSS supports the nurse by presenting prestructured, standardized, logical, and coherent (knowledge-based) linkages a between all phases of the nursing process: It links assessments with diagnoses, interventions and nursing-sensitive patient outcomes. c 7. The NP-CDSS supports the nurse by offering nursing diagnoses (to choose from) when defining characteristics/problem statements are entered. 8. The NP-CDSS supports the nurse by offering nursing diagnoses when related causes/etiologic factors/problem statements are entered. 9. The NP-CDSS supports the nurse by offering risk diagnoses when risk factors are entered. 10. The NP-CDSS supports the nurse by offering health promotion diagnoses when health readiness descriptions are entered. 11. The NP-CDSS supports the nurse by offering standardized, knowledge-based, effective nursing actions/interventions when nursing diagnoses are entered/selected. 12. The NP-CDSS supports the nurse by offering standardized, knowledge-based, meaningful nursing-sensitive patient outcomes when nursing diagnoses or interventions are entered/selected. 13. The NP-CDSS supports the nurse by offering evaluation criteria or outcome indicators related to the nursing diagnoses and nursing goals. 14. The NP-CDSS supports the nurse by connecting nursing data with medical, allied healthcare, and interdisciplinary diagnostics, interventions, and outcome data. 15. The NP-CDSS supports the nurse by connecting the results of measurement instruments such as VAS-pain scores, delirium scores, ulcer scores, or (basic) biomedical scores to the documentation of nursing diagnoses, interventions, and outcomes. Criteria related to meaningful use for data retrieval and further evaluations 16. The NP-CDSS supports the nurse by pre-structured, taxonomically ordered information as presented in internationally acknowledged classification systems (ie, CCC, ICNP, Omaha, NANDA-I NIC, and NOC) for nursing diagnoses, interventions, and outcomes. 17. The NP-CDSS is contained in SNOMED and/or Nursing Minimum Data Sets to support aggregation of output data for use and evaluation in national/international healthcare strategy planning. 18. The NP-CDSS contains a defined transparent taxonomic structure including coded concepts with synonymous text representations and hierarchical or definitional linkages among concepts to be used for data warehouses, data mining, and meaningful statistical analyses. b 19. The NP-CDSS supports measurements of nursing workload (eg, by containing allotted time weights/doses for nursing interventions) to be retrieved and used for nurse staffing and workload analyses. 20. The NP-CDSS describes and links with nursing staff qualification or educational levels that are needed to perform the nursing interventions contained in the NP-CDSS. 21. The NP-CDSS supports the nurse for modifying interdisciplinary care plans or clinical pathways. 22. The NP-CDSS alerts the nurse when (biomedical) results are below or above norm values and if essential information is lacking (eg, when Nursing Process entries are missing, respectively, if measurement scores are not entered into the system). 23. The NP-CDSS alerts the nurse about conflicting data entries/information (ie, diagnoses, interventions and/or outcomes that are not coherently fitting with each other or with measurement scores). 24. The NP-CDSS supports the nurse in documenting the nursing process efficiently, eg, direct linkages prevent leaving the nursing process screen and opening separate forms, as direct linkages omit redundant documentation. (continues) 496 CIN: Computers, Informatics, Nursing November 2016

5 Table 2. NP-CDSS Standard, Continued NP-CDSS standard 25. The NP-CDSS allows adequate mutations when new evidence was demonstrated, eg, changes in guidelines/protocols such as relevant nursing intervention standards that have to be integrated into the nursing process documentation. Abbreviations: CCC, clinical care classification; ICNP, international classification for nursing practice; NANDA-I, international nursing diagnoses classification; NIC, nursing interventions classification; NOC, nursing outcomes classification; VAS, visual analog scales. a Linkages describe correct and logical relationships among diagnoses, interventions, and outcomes as defined in SNL classifications. b Criterion 18: This means the classification is knowledge driven and coherently organizes several taxonomic levels: Each subordinate concept is distinguished by at least one characteristic from its superordinate concept (eg, domain, class). Therefore, it allows concept-driven data retrieval for meaningful statistical analyses, for example, clustering data to perform prevalence and patient population studies of nursing diagnoses and for effectiveness intervention and outcome research. c Criterion 6: An example further explains this criterion; for example, if pain is entered/clicked on in a patient assessment, the nursing diagnoses pain, pain management interventions, and related outcomes are suggested by the NP-CDSS. Process-Clinical Decision Support Systems and Criteria related to meaningful use for data retrieval and further evaluations were analyzed as primary units of analysis. Second, all answers to the 25 criteria were subjected to content analyses. Third, inductive development of categories and deductive application of categories which are the central procedures of qualitative content analysis were derived by constructing core ideas and building categories to describe consistencies between experts (cross analysis). Crossanalyses were performed by analyzing the findings of two researchers to capture the experts validation with respect to their accordance to the NP-CDSS Standard. For each of the 25 criteria, agreements and arguments of the experts were judged as no agreement, agreement, and strong agreement. Each transcribed interview was first analyzed by the research assistant, followed by a second analysis by two other researchers. Next, differences were discussed to reach an agreement, and finally, the findings were validated within the research group. No statistical procedures were applied because the expert validation was best reflected by the qualitative approach that includes providing direct citations of interviewee statements. 31,32 Ethical Considerations The participants were informed regarding the study aims, questions, and design. They were instructed that all information would be anonymized and that excerpts of direct comments would be cited according to guidelines for qualitative interview reports. All study participants gave their informed consent for inclusion and were notified to obtain early access to the results. RESULTS The NP-CDSS standard including its 25 criteria is presented in Table 2. The findings representing the content validity of the NP-CDSS Standard based on the qualitative interviews are presented in categories followed by directly cited expert statements. Content Validity Findings Ten experts were invited to participate, of whom eight completed the process. These experts represented Brazil (n = 1), Canada (n = 1), Finland (n = 1), Spain (n = 1), Switzerland (n = 1), Sweden (n = 2), and the United States (n = 1). All experts satisfied the inclusion criteria; all have expertise with published research on the topic; seven are employed in nursing informatics in academic settings; and all with the exception of an information technology (IT) developer work in universities. Interviews occurred between November 2013 and February 2014 (duration mean, 72 mean; range, minutes). The qualitative analyses revealed 14 categories, each validating one to four criteria of the NP-CDSS Standard. The categories are presented as headings followed by descriptions and explained by citations of experts key answers. 31 Only a small number of experts provided comments on the order of the NP-CDSS criteria, on their dissemination to software developers/vendors, and on nurses need for education. As these comments did not provide a category, they were added at the end of the findings. The Nursing Process Is Key to Support the Nurse in a Nursing Process-Clinical Decision Support System Encompassing the entire nursing process in an NP-CDSS as presented in criteria 1, 2, 6, and 24 was strongly supported, and all experts agreed that the nursing process represents the centerpiece of an NP-CDSS. Two experts noted redundancies in the four criteria addressing the nursing process; however, all stated that these criteria are the most significant for the standard: Nr 1 is the backbone. The only way of having the nursing process in clinical practice is by having it in an electronic system. It s crucial. The most important thing is, that the system should provide possibilities for linkages between the phases of the nursing process. Criterion 24, which addresses that the nurse must be supported throughout the nursing process and must omit redundant information, was considered vital for clinical practice: We have to look at the nursing work flow. In this way you can easily capture the information that the nurse needs. Volume 34 Number 11 CIN: Computers, Informatics, Nursing 497

6 Contain Standardized Nursing Language All experts agreed on the inclusion of SNL in an NP-CDSS as described in Criteria 3 and 16. An SNL would contribute to a more concrete knowledge base since nurses and IT developers are struggling in constructing IT systems without SNL. One respondent perceived this as being the reason that IT systems do not produce meaningful outcome results. Valid and agreed upon SNL was supported as being vital for developing an NP-CDSS. SNL was considered as a basic foundation of decision support systems and such systems must be able to interchange data and to compile data for research and for the development of practice on a European and international basis. Allow Additional Free Text All experts agreed that free-text entries should be made possible in addition to SNL as stated in Criterion 4. A number of experts had previously worked with systems that can analyze free-text, while others had not. The experts agreed that free text should be limited in systems that cannot aggregate/evaluate freetext data. Support Holistic Nursing Documentation All experts found it very important that holistic nursing documentation as described in Criterion 5 must be supported by an NP-CDSS. However, it was questioned, to a certain degree, how to achieve this. The majority of experts wanted spiritual aspects added as an extra point; however, one respondent stated: If you use NANDA-I, 52% of the diagnoses are on psycho-social issues if you accept going to use an SNL that is full, complete and comprehensive, you have already achieved that. Offer Accurate, Evidence Based Nursing Diagnoses All of the experts were positive in regard to Criteria 7, 8, 9, and 10, which address the offering of different types of nursing diagnoses that are based on inputs of actual patient data into an NP-CDSS. All experts considered these criteria as being very important, but some made considerations: If the nurse does not know about nursing diagnoses, she doesn t know how to use them (in the system). We might miss something the computer misses than I will miss it, too. The idea that a system could actually do this really well is very difficult until defining characteristics are coded there is redundancy of meaning and redundancy of terms. It works well at the label level, but is very difficult at the discrete level and I have always been amazed how everybody wanted to catch up all the details. One respondent remarked that arriving at a nursing diagnosis with the etiologic factors is difficult because nurses are not educatedinthiscontextandwillnotknowwhythehypothetical nursing diagnosis suddenly appears. Other experts stated that this works perfectly when nurses know the nursing diagnoses. One respondent perceived a risk for clinical nurses in becoming overly dependent on IT systems. This expert suggested that nurses should obtain background information only if needed, and this should depend on the nurses knowledge level and setting. For instance, an intensive care nurse might not need to see the definitions of nursing diagnoses such as acute pain or ineffective airway clearance because he/she knows these diagnoses without the assistance of an NP-CDSS. Offer Effective, Evidence-based Nursing Interventions All experts strongly supported the suggestion of nursing interventions by an NP-CDSS as described in Criterion 11 and considered this as being equally important as the proposal of nursing diagnoses. One respondent was uncertain as to how a system could accomplish this and mentioned the risk of nurses just clicking. Some considerations were made about refining this criterion: We would not be able to define the state of the art evidence-based knowledge for every intervention that we perform. Offer Standardized, Knowledge Based Outcome-indicators Criterion 12 and 13, which address nursing-sensitive patient outcome-indicators and evaluation criteria, were also judged as significant. The experts substantiated their opinions with the fact that nurses and nursing science are diligently searching for criteria concerning the quality of care that nurses provide. However, one expert added that interrater reliability could possibly be an issue: You and I are going to rate a patient using these indicators that come up with a score. But without these indicators, you don t get a reliable rate at all. Contain Connections Between Results of Measurement Instruments and Nursing Diagnoses, Interventions, and Outcomes All experts agreed that the NP-CDSS should connect measurement results (eg, pain scores of visual analog scales) with nursing diagnoses (ie, with the nursing diagnosis pain) and with nursing interventions and outcomes. It was also argued that these measurement instruments should be part of the outcome indicators, for example, pressure ulcer scores describing ulcer stages. One respondent expressed concern that health authorities are selecting such features: One system might offer one set of indicators, and another system a different set. 498 CIN: Computers, Informatics, Nursing November 2016

7 Contain Coded, Standardized Concepts to Allow Data Gathering, Sharing, and Research Criteria 17 and 18 address the use of coded concepts to support the aggregation of patient output data and coded hierarchical linkages to be employed for data gathering and research. All experts agreed that this would contribute to nursing research and evidence and that this feature should be in a system although not necessarily visible for clinical nurses on the interface screen. One respondent stated that she had never had heard or seen such a thing and doubted whether this could be possible, while others mentioned using such systems already in practice and research. Contain Connections Between Nursing Data and Medical, Interdisciplinary Diagnostics, Interventions, and Outcome Data The respondents considered the connection of data with/ from other healthcare workers as stated in Criterion 14 as very important. All agreed that this criterion is a necessity since healthcare is not represented only by nurses. The linkage or the connection should be directional and complete nursing data should be connected to medical data, and medical data should be connected to nursing data, It should be a multi-professional record on the patient, not of specific professional groups, and Nurses are a group which is integrated in all of this different health care workers. Link Nursing Process Elements With Staffing Levels and Workload Workload measurements to advise nurse managers on staffing requirements and qualifications or competency levels to be combined with nursing intervention performance are stated in Criteria 19 and 20. The majority of the experts agreed on these criteria and stated that such linkages would support practice. Interconnections also called linkages between nursing diagnoses and interventions were supported as important information for nursing management: It is good for planning mainly, and if these criteria are adhered, then the data can be retrieved. I think the criteria are perfectly well defined and in the right place. If it is behind the scene, then when you pick up interventions automatically, the systems shows the work load and that is why I would like to have this in the system. While all agreed on the benefits of such a function, the experts without experience on this topic questioned how this could be achieved in an NP-CDSS. Nursing Process-Clinical Decision Support System Alerts are Helpful if Not Constantly Alarming Criteria 22 and 23 address alerts in the event of deviating patient results or when a nurse enters conflicting data. Despite the fact that all but one expert agreed that an NP-CDSS should incorporate this feature, uncertainties regarding constant pop-ups on the screen were expressed. The experts stated that required alert functions depend on the respective setting and on nurses knowledge. One respondent suggested reformulating the two criteria into one. Support Modifying Care Plans and Produce New Evidence Criteria 21 and 25 address modifying care plans/pathways andmutationsbasedonnewevidencegainedbyannp-cdss. The experts strongly supported this; however, one remarked on it as being good but not very necessary. Not all experts completely understood Criterion 25 because it does not explain the source of new evidence. This Nursing Process-Clinical Decision Support System Standard Is Needed All experts supported the need for this NP-CDSS Standard. A minority reported the existence of NP-CDSS and addressed further need for future developments. The experts' experience and opinions on NP-CDSS were influenced by the EHRs used in their countries and settings: This is part of my direction of research so it s a concept that is well known to me and I find it important for the development of better quality in clinical information of nursing care. It (the NP-CDSS) is already present in our setting. We are using such a system for more than eight years, but not all IT developers can provide such systems. Others mentioned not being very familiar with such systems: I believe in this work for nursing, but for instance in my country this is very complicated because we don t have many EHRs in hospitals and clinics. Order of the Nursing Process-Clinical Decision Support System Criteria, Dissemination to Software Developers, Vendors, and Nursing Education A number of experts suggested rearranging the criteria, and one proposed ordering them into (1) Theoretical background, (2) Management information, and (3) Criteria addressing NP-CDSS for nurses/practice. The experts suggested that software companies should utilize the NP-CDSS Standard for EHR developments and for certification. One respondent with extensive experience as a nursing EHR developer stated that the criteria must be concisely explained to system developers. Otherwise, developers without a nursing background could feasibly misunderstand this standard. This respondent also indicated that Volume 34 Number 11 CIN: Computers, Informatics, Nursing 499

8 this NP-CDSS Standard represents the most significant level of decision support. Furthermore, several experts argued that a system is only as beneficial as the nurse who works with it: If a nurse is not well trained regarding the Advanced Nursing Process, the individual will probably not recognize the knowledge base presented by an NP-CDSS. As SNL provides the basis for evidence-based nursing diagnoses, interventions, and outcomes, nurses must be trained on these subjects. DISCUSSION All experts were positive in regard to the NP-CDSS criteria and considered its development as a necessity. None of the experts regarded it as being overly futuristic. The considerable agreement on the 25 criteria and the fact that no new criteria were suggested can be perceived as its strength. The 14 emerging categories demonstrate consensus and support the content validity 31,32 of this NP-CDSS Standard. The experts judged the Advanced Nursing Process to be the centerpiece for an NP-CDSS, and the literature supports this finding: The nursing process was described as being a common thread among nurses worldwide. 33 As the experts opinions depend on experiences in using SNL, and all countries are not yet using the same generations of EHRs and SNLs, 20 the substantial support for the NP-CDSS criteria can be judged as a valuable content validity result, which was also supported by the literature. 9,10 All experts agreed that an NP-CDSS must contain research-based assessment cues, predefined nursing diagnoses, and correct associations/linkages among diagnoses, interventions, and patient outcomes and that such a system can enhance nursing care quality. The literature on CDSSs supports this finding. 6,21 The experts also agreed on the importance of the Advanced Nursing Process despite it not yet being applied in all countries nor is it widely included in EHRs. However, the Advanced Nursing Process is strongly supported in the literature. 3,8,10,16,17,34 Several experts feared that nurses could possibly begin disregarding a patient as an individual when using SNL and that nurses would just click and agree on what an NP-- CDSS suggests without considering the correctness of the suggestions. The authors agree that an NP-CDSS cannot replace nurses critical thinking nor can a system gather genuine patient data. However, NP-CDSSs are built to make suggestions based on patient data entered by a nurse. The nurse must then accept or reject the system s suggestions. Clinical decision support systems were developed to foster clinical decision making by matching individual patient data or characteristics with a knowledge-base that is compiled in a computer. 5,6 Clinical decision support systems generate patient-specific recommendations and suggest these to clinicians for consideration in their own clinical judgments. This means that nurses are always asked by the NP-CDSS to verify the suggested nursing diagnoses, to validate diagnostic cues, and to select the most suitable interventions. 18,19,27,35 This standard describes criteria to assure that nurses perform the Advanced Nursing Process founded on evidence-based, accurate, and valid diagnoses; patient-centered interventions; and outcome indicators. The fear of diminishing individual caregiving is a commonly ascertained argument from nurses that do not have experience with NP-CDSS including evidence-based knowledge and decision support. The authors are convinced that nurses must be educated in nursing diagnostics, clinical reasoning, and critical thinking. These are prerequisites to apply the Advanced Nursing Process and for meaningful use of an NP-CDSS. 10,15 The literature supports this finding. A recent study 27 demonstrated that nurses competencies in clinical reasoning and their knowledge on the Advanced Nursing Process and not only the EHR per se assisted in formulating accurate nursing diagnoses. These nurses also chose more effective interventions, which led to highly significant and clinically relevant improved patient outcomes. 14,27,36 The experts indicated that an NP-CDSS affords opportunities for future big data research. Big data research demonstrates that employing interoperable data is key for future meta-analyses and for comparing outcomes between settings and countries. 10,12 A pain study in end-of-life (EOL) patients demonstrates big data research : Longitudinal data were aggregated over 24 months from eight diverse medicalsurgical units in US hospitals. 37 The findings in the EHR of 1425 EOL care episodes indicated that 596 (41.3%) patients had a nursing diagnosis of pain. Significant differences were determined for EOL patients suffering from pain between units (P <.001), and pain was lower (27.7%) in patients with acute confusion. Severe or significant pain was reported in 30% of EOL patients at their death or at discharge to a hospice, and fewer than half of the patients (42.7%) actually achieved the expected pain outcome ratings. This study is an impressive example for demonstrating how nursing outcome indicators that were daily and electronically documented by clinical nurses can be used for research. The study detected a gap between pain science and clinical practice, and its findings are assisting in enhancing pain care. 37 By using an NP-- CDSS with SNL, future research allows evaluating and comparing nursing diagnostic accuracy, intervention effectiveness, and patient outcomes among wards and/or institutions. Findings of such studies can positively influence outcome quality and patient safety. Concrete research on the Advanced Nursing Process and its documentation are unachievable until a valid, standard-based NP-CDSS becomes available. 1 This NP-CDSS Standard closes a research gap and provides a guideline for future EHR developments. 500 CIN: Computers, Informatics, Nursing November 2016

9 Only a small number of experts considered reordering the criteria and addressed criterion redundancy. As the findings did not reveal consensus on these issues, the numbers and order of the criteria will be addressed in future studies. No expert representing eastern countries was included in this study. Qualitative research criteria such as consensus validation, transferability, and credibility were satisfied by using a three-step approach. However, the last stage suggested by Mayring statistical analyses was not performed due to the fact that the literature describes qualitative validation as sufficient for establishing content validity. 24,30 This NP-CDSS Standard will be further tested in larger samples by factor and item analyses as well as in clinical validations in order to offer a gold standard for NP-CDSS. 24,38 By adding a measurement scale, it will become an internationally applicable instrument. 24 CONCLUSIONS The Advanced Nursing Process is composed of valid and accurate nursing assessments, nursing diagnoses, interventions, and outcomes that are described by SNL in scientifically based classifications. The use of SNL in this process provides defined, coded concepts and allows clear, accurate, and reliable communication between health professionals. Clear communication and accurate documentation are essential for ensuring patient safety. Nurses require electronic decision support to apply the Advanced Nursing Process; however, NP-CDSSs are unavailable in many settings. In this study, an NP-CDSS Standard was developed, and its content validity was established. The application of this standard fosters the development of EHRs to establish the ability to share patient data between settings and to assure care continuity and quality by using ehealth strategies. Nursing Process-Clinical Decision Support Systems are key to ascertain interoperable, safe data production, and exchange for big data research. This NP-CDSS Standard has been suggested for application and testing in EHRs in several countries. It guides highquality documentation of the Advanced Nursing Process and can promote effective communication in and between healthcare teams to facilitate continuity and individuality of care. References 1. Wang N, Hailey D, Yu P. Quality of nursing documentation and approaches to its evaluation: a mixed-method systematic review. J Adv Nurs. 2011;67(9): World Alliance for Patient Safety. Guidelines for Safe Surgery. Geneva, Switzerland: WHO Press; Müller-Staub M, Abt J, Brenner A, Hofer B. Expert Report on Nurses Responsibility. Bern, Switzerland: Swiss Nursing Science Association (ANS); Berner ES. Clinical Decision Support Systems: Theory and Practice. 2nd ed. New York: Springer; OpenClinical. Decision support systems. Advanced Computation Laboratory of the Imperial Cancer Research Fund. July 8, openclinical.org/dss.html. Accessed January 7, Hunt DL, Haynes RB, Hanna SE, Smith K. Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review. JAMA. 1998;280(15): American Nurses Association. Nursing s Social Policy Statement: The Essence of the Profession. 5th ed. Sliver Spring, MD: American Nurses Association; Ackley BJ, Ladwig GB. Nursing Diagnosis Handbook: An Evidence-Based GuidetoPlanningCare.10 ed. St Louis, MO: Mosby/Elsevier; Ammenwerth E, Rauchegger F, Ehlers F, Hirsch B, Schaubmayr C. Effect of a nursing information system on the quality of information processing in nursing: an evaluation study using the HIS-monitor instrument. Int J Med Inform. Jan 2011;80(1): Keenan G, Yakel E, Dunn Lopez K, Tschannen D, Ford YB. Challenges to nurses efforts of retrieving, documenting, and communicating patient care information. JAmMedInformAssoc.Mar-Apr 2013;20(2): Paans W, Sermeus W, Nieweg RM, van der Schans CP. D-Catch instrument: development and psychometric testing of a measurement instrument for nursing documentation in hospitals. J Adv Nurs. 2010;66(6): Jones D, Lunney M, Keenan G, Moorhead S. Standardized nursing languages: essential for the nursing workforce. Ann Rev Nurs Res. 2010;28: Müller-Staub M, Lavin MA, Needham I, van Achterberg T. Nursing diagnoses, interventions and outcomes application and impact on nursing practice: a systematic literature review. J Adv Nurs. 2006;56(5): Paans W, Nieweg R, van der Schans CP, Sermeus W. What factors influence the prevalence and accuracy of nursing diagnoses documentation in clinical practice? A systematic literature review. J Clin Nurs. 2011;20(17 18): Institute of Medicine. Keeping Patients Safe. Washington, DC: National Academy Press; Hayrinen K, Saranto K. Patients needs assessment documentation in multidisciplinary electronic health records. Stud Health Technol Inform. 2010;160(pt 1): Saranto K, Kinnunen UM. Evaluating nursing documentation research designs and methods: systematic review. JAdvNurs.2009;65(3): Odenbreit M. Electronically supported nursing documentation. Paper presented at: Nanda International 2008 Conference: Capturing the Expert Knowledge of Nursing; November 13 15, 2008; Miami, FL. 19. Odenbreit M. Entwicklung und Implementierung der elektronischen Pflegedokumentation der Solothurner Spitäler AG: Eine Erfolgsstory. Swiss Med Inform. 2010;69(2): Piscotty R, Kalisch B. Nurses use of clinical decision support: a literature review. Comput Inform Nurs. 2014;32(12): Dowding D. Using computerised decision-support systems. Nurs Times. 2013;109(36): Dowding D, Randell R, Mitchell N, et al. Experience and nurses use of computerised decision support systems. Stud Health Technol Inform. 2009;146: Müller Staub M, de Graaf-Waar H, Paans W. Standard for Nursing Process Clinical Decision Support Systems (NP-CDSS) in EHRs. ACENDIO. Bern, Switzerland: Association for Common European Nursing Diagnoses, Intervention and Outcomes; Waltz CF, Strickland OL, Lenz ER. Measurement in Nursing and Health Care Research. 4th ed. New York: Springer; Müller-Staub M, Lunney M, Odenbreit M, Needham I, Lavin MA, van Achterberg T. Development of an instrument to measure the quality of documented nursing diagnoses, interventions and outcomes: the Q-DIO. JClinNurs.2009;18(7): Linch GF, Rabelo-Silva ER, Keenan GM, Moraes MA, Stifter J, Muller-Staub M. Validation of the Quality of Diagnoses, Interventions, and Outcomes (Q-DIO) instrument for use in Brazil and the United States. Int J Nurs Knowl. 2015; 26(1): Volume 34 Number 11 CIN: Computers, Informatics, Nursing 501

10 27. Bruylands M, Paans W, Hediger H, Muller-Staub M. Effects on the quality of the nursing care process through an educational program and the use of electronic nursing documentation. Int J Nurs Knowl. 2013;24(3): Müller-Staub M, Lunney M, Lavin MA, Needham I, Odenbreit M, van Achterberg T. Testing the Q-DIO as an instrument to measure the documented quality of nursing diagnoses, interventions, and outcomes. Int J Nurs Terminol Classif. 2008;19(1): Paans W, Müller-Staub M, Odenbreit M. Developing an international nursing documentation audit instrument: Expert consensus study. ACENDIO Conference proceedings. Dublin, Ireland: ACENDIO; Polit DF, Tatano Beck C. Nursing Research: Generating and Assessing Evidence for Nursing Practice. 9th ed. Philadelphia, PA: Wolters Kluwer/ Lippincott, Williams & Wilkins; Hennink M, Hutter I, Bailey A. Qualitative Research Methods. London: Sage; Mayring P. Qualitative Inhaltsanalyse: Grundlagen und Techniken. 11th ed. Beltz: Weinheim, Germany; American Nurses Association. Nursing: Scope and Standards of Practice. Sliver Spring, MD: American Nurses Association; Moorhead S, Johnson M, Maas M, Swanson E. Nursing Outcomes Classification (NO(3). 5th ed. St Louis, MO: Elsevier; Paans W, Müller-Staub M. Standardized Nursing Language in Intelligent Electronic Healthcare Documentation. Chicago, IL: HIMMS Europe; Technology update. 36. Paans W, Sermeus W, Nieweg RM, Krijnen WP, van der Schans CP. Do knowledge, knowledge sources and reasoning skills affect the accuracy of nursing diagnoses?: a randomised study. BMC Nurs. 2012;11: Yao Y, Keenan G, Al-Masalha F, et al. Current state of pain care for hospitalized patients at end of life. Am J Hospice Palliat Care. 2013;30(2): Bortz J. Statistik für Human-und Sozialwissenschaftler. Vol. 6. Heidelberg, Germany: Springer; CIN: Computers, Informatics, Nursing November 2016