Appendix 1. Better Allocation for Better Health and Healthcare: The First Annual Population Value Review

Transcription

1 Appendix 1. Better Allocation for Better Health and Healthcare: The First Annual Population Value Review

2 Better Allocation for Better Health and Healthcare: The First Annual Population Value Review Peter Brambleby, Andrew Jackson and J. A. Muir Gray Foreword by: Duncan Selbie, Commissioning Director NHS National Knowledge Service Commissioning Directorate Department of Health 1

3 Contents Foreword Programme Budgeting for commissioners: in brief What is the aim of commissioning? What is the aim of the annual population value review? Why have an annual population value review? How can we improve healthcare locally through the annual population value review? How do I conduct an annual population value review? What is the right answer? What will be the outcome of this annual population value review? Section 1: Taking the plunge into Programme Budgeting and Marginal Analysis What is Programme Budgeting and Marginal Analysis (PBMA)? What data and resources are available to undertake Programme Budgeting? Data on expenditure Data on outcomes How is the information collected? What do I do now? Additional resources to support commissioners Section 2: What tasks do I need to do for the general assessment of the whole budget? Proformas 2.1 and 2.2 Section 3: What tasks do I need to do for the specific assessment of a programme budget for a single disease category? Section 4: The future What further information will be available to increase the usefulness of Programme Budgeting? More detailed programme and sub-programme financial data More reliable data More outcome and output data Marginal analysis Further reading 2

4 Annex 1: Background to Programme Budgeting What is the National Programme Budgeting Project? Why is Programme Budgeting important now? What is the history of Programme Budgeting? When was Programme Budgeting introduced into the NHS? The National Programme Budget Project Board What are the benefits of Programme Budgeting for the NHS? What are the advantages of Programme Budgeting to individual NHS organisations? How will Programme Budgeting be used in the long term? Annex 2: The Annual Review of Evidence, Process, Outcomes and the Configuration of the 50 major health problems (EPOC) Programme 3

5 Dear Colleagues, Foreword Welcome to the first Annual Population Value Review. One of the core functions of commissioners is resource allocation to different population groups based on an analysis of need. Excellent work has been done by the Programme Budgeting Team within the Department of Health, and clear information about spend on each patient group can now be provided for each PCT. In the first instance, this will raise questions about the accuracy of the data, however, this will improve with use. Before the start of the next financial year ( ), each primary care trust (PCT) might want to reflect upon the distribution of resources among the different groups and how the distribution compares with the distribution of other commissioners, particularly commissioners who are serving populations that have a similar socio-economic profile. Some of you will already be experienced in the technique of Programme Budgeting, but for those of you who are not we have been fortunate in obtaining the skill and experience of both Peter Brambleby, lately Director of Public Health at Norwich PCT, and Andrew Jackson from the DH Finance Directorate to help promote and disseminate this work. For those to whom this technique is new, the main objective this year is to reflect upon the distribution of resources across and within programme categories, and to identify areas that would reward further investigation and analysis. For those who have already carried out programme budgeting before, data can offer the opportunity for more detailed analysis and action. It is our intention to produce guidance on marginal analysis at a later date. For many of you, this will be of most use during the second annual value population review, which will be based on data available from November Throughout 2007, more detailed work on individual programme budgets relating to disease groups and then specific conditions within each disease group will take place as part of the work for the National Knowledge weeks, introduced by the National Knowledge Service with the focus on individual conditions. I commend this work to you and look forward to seeing its impact on the ground. Duncan Selbie Commissioning Director 4

6 Programme Budgeting for commissioners: in brief Over the last year, there has been an acceleration of the commissioning function in the NHS. In July 2006, the Department of Health published the Commissioning Framework for Hospital Services and the complementary Commissioning Framework for Health and Well-being was published in December In November 2006, Richard Douglas wrote to the NHS explaining that data on programme budgets and outcomes was available to commissioners. What is the aim of commissioning? When commissioning health services, the aim is to maximise value for the population served from the resources invested on the population s behalf. This can be done in two ways: directly and indirectly. The commissioner can maximise value directly by allocating the resources they control among the patient and population groups within the population to best effect. The resources can be invested with specific health objectives, as follows: to reduce health inequalities and improve equity; to promote and improve the health of the population; to improve the value and quality of healthcare. Value derived from investment in healthcare is maximised when it is impossible to reallocate resources from one patient or population group to another to achieve more benefit or less harm. This is sometimes referred to as allocative efficiency, a task increasingly shared with local authorities as service providers. The commissioner can maximise value indirectly by using their contracting power to help providers of health services improve the quality and safety of the healthcare they provide while minimising its cost. This is sometimes referred to as technical efficiency. What is the aim of the annual population value review? The aim of the annual population value review is to improve healthcare for the local people for whom the commissioner is responsible by clarifying the allocation decisions that have been made, both explicitly and implicitly. In order to do this we need to initiate NHS learning from expenditure and outcomes data that will enable us to improve the planning and delivery of services based on need rather than traditional patterns of investment. 5

7 Why have an annual population value review? The annual population value review for 2007 will be the first in a series. The first annual population value review will give commissioners an opportunity to reflect on the distribution of resources among different programmes according to the programme classification developed by the Department of Health from the International Classification of Diseases version 10 (ICD10) Classification. It will also enable commissioners in newly created primary care trusts (PCTs) to reflect on the pattern of investment in health services that they have inherited following the recent re-organisation. The second Annual Population Value Review will be launched in December However, it will be preceded by training opportunities to improve commissioners skills in marginal analysis How can we improve healthcare locally through the annual population value review? The first steps that need to be taken before we can improve healthcare locally are to identify: the distribution of resources between different population groups; how the amount invested in each major health category compares with those for the national average; how the amount invested in each major health category compares with the amounts invested by the commissioners serving populations that have a similar socio-economic profile. This gives us the foundation from which we can assess the appropriateness and effectiveness of these investments in terms of the health outcomes achieved for the local population. How do I conduct an annual population value review? The first phase in conducting an annual population value review is to compile a summary of expenditure in the programme budget categories. Having done this, circulate the data widely with appropriate explanations and possible reasons for high and low levels of spending. The audience for this information should include: patient forums; practice-based commissioners; clinicians in hospital and community settings; specialist commissioning groups; local authority personnel, especially those responsible for local area agreements (LAAs); local strategic partnerships; 6

8 overview and scrutiny committees; local media. What is the right answer? There is no right answer for this one. Richard Gleave As commissioners have to match local need to local resources, the situation for each PCT will be different for each population in England. However, it is essential that commissioners are aware of any variations in expenditure, and are able to explain the reasons for outliers on particular programmes of spend, and have a plan to bring them into line if there is no plausible explanation for the variation seen. What will be the outcome of this annual population value review? For the first year, the main outcomes will be: a better understanding among the newly created commissioners of the distribution of resources they have inherited; stimulation of discussions on each programme s position with respect to PCTs serving similar populations; identification of opportunities to improve health and healthcare locally; initiation of discussions with local authorities on matching health needs and investment in health services for the local population. Topics for discussion about individual programme budgets could include: programme objectives; programme budgets ( inputs ); programme activities ( outputs ); programme achievements ( outcomes ). Ultimately, the focus of discussion for the Annual Population Value Review is the redeployment of resources to achieve better value outcomes to meet objectives. Information that can be used to support detailed discussion about individual programmes will be provided during the National Knowledge Weeks, which are being organised as part of the EPOC Programme (see Annex 2). The focus of the National Knowledge Weeks will be the principal health issues within each programme budget, providing: up-to-date evidence about relevant health problems; detailed information about prescribing and hospital utilisation trends and variations. Given the recent innovation of programme data collection, there may be variation in the data. If there is lack of confidence in the financial estimate for a programme budget, the discussion should still go ahead but be framed in terms of: 7

9 admissions to hospital; prescriptions; staff; beds and bed-days; theatre sessions; similar tangible resources. If nothing else, discuss programme objectives, such as meeting un-met needs and tackling inequalities. 8

10 Section 1: Taking the plunge into Programme Budgeting and Marginal Analysis (PBMA) What is Programme Budgeting and Marginal Analysis (PBMA)? Programme budgeting is a technique that enables personnel in a health service, and those who use the health service, to identify how much money has been invested in major health programmes, with a view to influencing future investment. Marginal analysis is an economic appraisal technique that evaluates incremental changes in costs and benefits when resources in programmes are increased, decreased or deployed in different ways. PBMA can be used within programmes of care or across services and programmes within a health organisation. The technique provides users with the capacity to identify: Where resources are currently being invested; The level of effectiveness of those investments; The most effective way of investing in health services in future in relation to the needs of the population for which services are being commissioned. Programme budgeting provides a rich source of information that will enable managers and clinicians to work together to improve healthcare for the local population (see box below). Although the best way to use the information generated by programme budgeting is through a structured analytical technique particularly marginal analysis, information on programme budgets can be used by anyone it is not necessary to have skills in economic appraisal. Delivery of efficient (and in the case of the NHS, equitable) health care requires doctors to take responsibility for resources and to consider the needs of populations while managers need to become more outcome and patient centred. programme budgeting and marginal analysis has the potential to align the goals of doctors and managers and create common ground between them. Ruta et al. (2005) 1 1 Ruta, D., Mitton, C., Bate, A. et al. (2005) Programme budgeting and marginal analysis: bridging the divide between doctors and managers. British Medical Journal 330:

11 What data and resources are available to undertake Programme Budgeting? As part of the National Programme Budgeting Project (see Annex 1), there are two sets of data available: 1. Data on expenditure; 2. Data on outcomes. Data on expenditure Programme budgeting expenditure data for the financial years , and are published as per the previous configuration of 303 PCTs, and are available as a spreadsheet (together with a range of supporting material) on the Department of Health website, available at: The programme budgeting spreadsheet presents PCT (and SHA) level expenditure data, programme by programme, in a variety of ways, as follows. The PCT s per-capita spend (using raw or weighted populations) in each of the programme budget categories. A comparison of the PCT s spend in any programme budget category with that of: o a cluster of similar PCTs; o the SHA average; o the England average. Changes in expenditure over the 3 years. Spider web charts, which show the programmes (and sub-programmes) around the circumference of the web, the block of cluster PCTs as a shaded area in the centre and superimposed on that a line representing the PCT in question this format enables any discordant areas of spend between the PCT in question and comparable PCTs from the same cluster to be identified. Expenditure distribution charts of each Programme Budget category, which show the PCT position in the distribution as a gold square, other PCTs in the cluster (i.e. comparable PCTs) as pink dots and all other PCTs as blue dots this format enables a PCT to identify whether it is an outlier for particular programmes of spend. Cancer networks, built up from their constituent PCTs, showing per capita Cancer and other programme expenditure per network, between-years changes in investment at network level and the change the cancer programme s total share of the total budget. Cardiovascular disease networks, built up from their constituent PCTs, showing per capita Cardiovascular and other programme expenditure per network, between-years changes in investment at network level and the change the cardiovascular programme s total share of the total budget. 10

12 Data on outcomes The Department of Health has worked closely with the National Centre for Health Outcomes Development (NCHOD) at the London School of Hygiene and Tropical Medicine to link the programme budgeting expenditure data (described above) with NCHOD s compendium of health outcomes indicators. To date, linked expenditure and outcomes data are available for 15 of the 23 Programmes and allow users to compare at PCT level: cardiovascular disease expenditure and mortality from cardiovascular disease; cancer expenditure and cancer incidence. The linked expenditure and outcomes data are available on the NCHOD website, but only to users of the NHS net (note the NHS web address with nww prefix): The data on the NCHOD website are presented using an interactive map base software, and users may have to download the SGV viewer to view this analysis instructions on how to do this are available on the NCHOD website. To access the programme budgeting data, after logging onto the NCHOD website: click on Compendium Indicators (on the left hand side); then click on Programme Budgeting Atlas. A beige box will appear, which for 15 of the 23 programmes, offers analysis using either the nested rate or correlation plots. The nested rate plot presents analysis for a single indicator (i.e. expenditure or outcomes) as a single map of England. The software also presents a distribution plot of the chosen indicator, assigning each PCT to the relevant Office of National Statistics (ONS) cluster. The correlation plot presents analysis using two indicators, for example, expenditure and mortality. The software presents a double map, and draws a correlation plot of the two indicators, thereby allowing users, for example, to examine the relationship between expenditure and outcomes. Individual PCTs can thus determine if their (e.g. high) expenditure in a particular programme is accompanied by either high or low mortality. DH and NCHOD will be continually updating the Programme Budgeting Atlases. The next data release is due to include data (per programme per PCT) on: admissions (elective and non-elective); average length of stay; bed-days; Family Health Service data on the number of prescriptions dispensed and prescription expenditure; 11

13 Quality and Outcomes Framework data (including disease prevalence). How is the information collected? Information for the National Programme Budgeting Project is collected from two different sources by two separate groups. Information on expenditure is collected through the Financial Returns Exercise as part of the statutory accounts process. Although these data are not currently subject to audit, there are plans to do so in due course; Information on outcomes is collected by the National Centre for Health Outcomes Development (NCHOD), led by Professor Azim Lakhani. What do I do now? Once you have accessed the data, there are two main types of assessment you need to perform: 1. General, painting the big picture of the PCT s investment across all the major programmes in 2005/06 and comparing them with those of the cluster of comparable PCTs and those of the average for England see Section 2; 2. Specific, programme by programme, showing how the pattern of your PCT s expenditure across the individual programme budget categories compares with the average for that of England and the average for that of PCTs in the same cluster see Section 3. Additional resources to support commissioners There is a range of other tools and techniques that can be used to support commissioner decisions, produced as part of the Commissioner Development Programme. Some commissioners are already modelling possible cost pressures in all 23 programme budget areas, which can then be used to discuss potential trade-offs, both between programme budgets and within a single programme budget, for example, considering shifting resources from acute care to health promotion and prevention. 12

14 Section 2 What tasks do I need to do for the general assessment of the whole budget? For the first task in the general assessment, use the proforma 2.1. For each programme budget category, insert the following information into the proforma: Spend per 100,000 weighted population for your PCT (or its constituent PCTs if you are in a newly created PCT) during 2005/06; The average spend per 100,000 weighted population for the cluster to which your PCT belongs during 2005/06 choose Group from the expenditure spreadsheet; The average spend per 100,000 weighted population for England during 2005/06. Interpreting the results what does this tell us? This will help you to identify whether overall expenditure, and the spend by programme budgeting category, for your PCT is similar to, or greater or less, than the spend for the average for your cluster and the average for England. (You can note this in the comments column on the far right-hand side of the proforma.) For the second task in the general assessment, identify the main providers of services to residents in your PCT during 2005/06. For instance, this list will probably include: all local hospitals, including tertiary care; the mental health trust; the community services; the GP prescribing budget; the ambulance trust; the voluntary sector; practice-based commissioning (PBC) provider services. Then list the providers in order of spend. Interpreting the results what does this tell us? This list will disclose the relative size of each provider s contribution to health services for your population. For the third task in the general assessment, identify the top 10 health programmes in each of the major providers in the list you have just compiled. Look at the differences. For example the cancer programme is likely to be near the top of the list in the district general hospital, in the middle of the list for community services but not even in the top 10 in the GP prescribing budget. Circulatory disorders may be high on all three lists. The pattern for mental health will be different again. Discuss and understand these differences. Try and identify the networks and pathways these patterns reveal.. 13

15 Interpreting the results what does this tell us? This shows on which programme categories you should focus your attention. For the fourth task, identify the age breakdown as it applies to programme budget categories. As this information is not required as part of the return to the centre yet, you will need to look at local data, and there will be gaps. It should be possible to use the number of PCT residents admitted to any hospital during 2005/06 for each programme budgeting category. Using the proforma set out at Proforma 2.2, fill in the cells in the matrix with financial information if you have it, but also make notes. For example, the first cell is about infection in the pre-school population. Is that population set to rise? What are your immunisation uptake rates? What notifications of disease have you seen? Which infections take these children into hospital? Interpreting the results what does this tell us? Once the matrix is annotated: 1. look across the rows to see what the issues are for each programme; 2. look down the columns to see what the issues are for each age-group. Then consider what you know about demographic projections, for instance, if an agegroup is projected to rise, the programme expenditures will need to be anticipated and planned in. To help you with an analysis of quality outcomes, and to examine the interaction between spend and outcome, use the matrix shown in Figure 2.1, which helps you to identify programmes where there are: Good outcomes for low spend; Good outcomes for high spend; Poor outcomes for low spend; Poor outcomes for high spend. 14

16 Programme Budget category 1. Infectious diseases 2. Cancers & tumours 3. Blood disorders 4. Endocrine, nutritional & metabolic disorders 5. Mental health problems 6. Learning disability problems 7. Neurological system problems 8. Eye/vision problems 9. Hearing problems 10. Circulation problems 11. Respiratory system problems 12. Dental problems 13. Gastrointestinal system problems 14. Skin problems 15. Musculoskeletal system problems 16. Trauma & injuries 17. Genitourinary system disorders 18. Maternity & reproductive health 19. Neonate conditions 20. Poisoning and adverse events 21. Healthy individuals 22. Social care needs 23. Other areas of spend/conditions Total Spend per 100,000 weighted population for your PCT Proforma 2.1 Spend per 100,000 weighted population for cluster average Spend per 100,000 weighted population for England average Comments 15

17 Programme Budget category 1. Infectious diseases 2. Cancers & tumours 3. Blood disorders 4. Endocrine, nutritional & metabolic disorders 5. Mental health problems 6. Learning disability problems 7. Neurological system problems 8. Eye/vision problems 9. Hearing problems 10. Circulation problems 11. Respiratory system problems 12. Dental problems 13. Gastrointestinal system problems 14. Skin problems 15. Musculoskeletal system problems 16. Trauma & injuries 17. Genitourinary system disorders 18. Maternity & reproductive health 19. Neonate conditions 20. Poisoning 21. Healthy individuals 22. Social care needs 23. Other areas of spend/conditions Issues affecting age-group 0-4 years 5-14 years Proforma years years years years 85+ years Age-related issues for programme 16

18 Figure 2.1 High High outcome /low spend High outcome/high spend Outcome Low outcome/low spend Low outcome/low spend Low Low Spend High 17

19 Section 3 What tasks do I need to do for the specific assessment of a programme budget for a single disease category? Assess the spend for each programme budget category in relation to: the aims of the programme locally, including the aims within the main stages of the care pathway prevention, diagnosis and assessment, treatment, rehabilitation and continuing care and terminal care; look at NICE guidance, targets, National Service Frameworks, local patient feedback, and other guidance and feedback. any other health issues relevant to the programme locally. Consider convening a marginal analysis advisory group for programme or subprogramme budgets of particular interest or concern. The main steps for such an initiative 2 are: Determine the aim and scope of the priority-setting exercise, for example, the whole of the musculoskeletal programme or rheumatoid arthritis alone Compile the most comprehensive programme budget (or sub-programme budget) information you can Set up a marginal analysis advisory panel, for example, comprising managers, clinicians from primary and secondary care, service users and an economist Determine the decision-making criteria, using evidence where available but group opinion where it is lacking Identify options for redeploying resources by expanding some areas and reducing others Appraise the costs and benefits of the options Consult, decide, and make it happen! Recent NHS reforms give doctors increased responsibility for the efficient and fair use of resources. Programme budgeting and marginal analysis is one way to ensure the views of all stakeholders are properly represented. Ruta et al. (2005) 3 2 Adapted from Peacock, S., Richardson, J., Carter, R. and Edwards, E. (2007) Priority setting in health care using multi-attribute utility theory and programme budgeting and marginal analysis (PBMA). Social Science and Medicine 64: Ruta, D., Mitton, C., Bate, A. et al. (2005) Programme budgeting and marginal analysis: bridging the divide between doctors and managers. British Medical Journal 330:

20 There will be the opportunity to consider further detail later this year when knowledge relevant to the principal health problems, organised at the level of the individual disease within the ICD grouping (such as rheumatoid arthritis within the musculoskeletal disease programme) will be sent to each PCT during the relevant national Knowledge Week see Annex 2 for information about the Annual Review of Evidence, Process, Outcome and Configuration (EPOC) Programme. 19

21 Section 4 The future What further information will be available to increase the usefulness of Programme Budgeting? More detailed programme and sub-programme financial data The 2006/07 returns will contain more detail in some of the programmes, for example the cancer programme will have around 10 sub-programmes based on the main cancer body sites. More reliable data It is anticipated that as pricing mechanisms such as payment by results mature, and as people use the data more to support decisions, the quality of data returns will improve. More outcome and output data The programme budget atlas produced jointly by NCHOD and DH will contain more data next year. Marginal analysis The rigour with which marginal analysis is conducted, within programmes and between programmes, should improve with familiarity and as more experience is published. 20

22 Further reading Brambleby, P. (1993) A purchaser s guide to purchasing healthcare. Clinician in Management 2(6): 3-6. This paper is 14 years old. It called for programme budgeting as a framework for commissioning, setting out the authority s view on fair shares between programmes and giving its officers operating parameters within which to pursue efficiency. Brambleby, P. (1995) A survivor s guide to programme budgeting. Health Policy 3: An account of experience in Hastings Health Authority. Brambleby, P. (2004) The quiet revolution. ACCA, Health Service Review, pp This paper was commissioned by ACCA and aimed chiefly at a finance audience. Brambleby, P. (2005) Finance. [Letter] Health Service Journal, 7 April 2005, p. 20. Brambleby, P. and Dixon, J. (2005) The HSJ Debate: Programme budgeting is better for the health service than payment by results. Health Service Journal, 21 July 2005, pp Clapperton, A. (2004) The road to success, Healthcare Finance, February 2004, pp Enthoven, A. (1999) In pursuit of an improving National Health Service. Nuffield Trust, London. pp Gray, J.A.M. (2006) Better Value Healthcare. Offox Press, Oxford. Kings Fund (2006) Local variations in NHS spending priorities. King s Fund Briefing. King s Fund, London. This recent paper, reporting the emerging findings of the English NHS programme budget work, was widely reported in the national media. Mitton, C. and Donaldson, D. (2001) Twenty-five years of programme budgeting and marginal analysis in the health sector, Journal of Health Services Research and Policy 6: Mitton, C. and Donaldson, C. (2004) Priority Setting Toolkit: a guide to the use of economics in healthcare decision making. BMJ Books/Blackwell, London. 21

23 Peacock, S., Richardson, J., Carter, R., and Edwards, E. (2007) Priority setting in health care using multi-attribute utility theory and programme budgeting and marginal analysis (PBMA). Social Science and Medicine 64: Ruta, D., Mitton, C., Bate, A. and Donaldson, C. (2005) Programme budgeting and marginal analysis: bridging the divide between doctors and managers. British Medical Journal 330: Smith, P. (2003) Clarity begins at home. Health Service Journal, 20 November 2003, pp Brambleby, P. (2003) Collective good. [Letter] Health Service Journal, 18 December 2003, p. 26. A response to Smith (2003) Thalange, N., Gardner, C. and Reading, R. (2004) How is money spent on children s services? Child: Care, Health and Development 30(5): This paper is based on experience in Norfolk. 22

24 Annex 1 Background to Programme Budgeting What is the National Programme Budgeting Project? The aims of the National Programme Budgeting Project in the NHS are: to develop a primary source of information, which can be used by all bodies, to give a greater understanding of what we are getting for the money we invest in the NHS ; to map all NHS expenditure, including primary care services, to programmes of care based on medical conditions. The categories currently being used for Programme Budgeting in the NHS are shown in Box A1. The first 20 categories correspond to chapters in the World Health Organization s International Classification of Disease (Version 10). In addition, there are three further categories that have been introduced to capture remaining areas of spend that do not fit easily into the first 20 categories. In future, these three categories may be absorbed into the first 20. Box A1: Programme Budget categories in the National Programme Budgeting Project 1. Infectious diseases 2. Cancers and tumours 3. Blood disorders 4. Endocrine, nutritional and metabolic problems 5. Mental Health problems 6. Learning disability problems 7. Neurological system problems 8. Eye/vision problems 9. Hearing problems 10. Circulation problems 11. Respiratory system problems 12. Dental problems 13. Gastrointestinal system problems 14. Skin problems 15. Musculoskeletal system problems (excludes trauma) 16. Trauma and injuries (includes burns) 17. Genitourinary system disorders (except infertility) 18. Maternity and reproductive health 19. Neonate conditions 20. Poisoning 21. Healthy individuals 22. Social care needs 23. Other areas of spend/conditions 23

25 Why is Programme Budgeting important now? Two important changes will begin to affect and influence the way healthcare is procured and provided in the NHS: 1. The introduction of commissioning; 2. The end of a period of significant regular increase in NHS funding. As the combination of these two changes alters the operating context for the provision of health services in England, the relevance and usefulness of programme budgeting and marginal analysis will be brought into focus. As the NHS moves towards financial balance and attention is taken from issues such as waiting times, programme budgeting and marginal analysis will play an increasing important role in maximising value for the resources invested. Traditionally, in the NHS, expenditure has been reported on the basis of inputs using budget categories such as GP prescribing, hospital services and community services. However, the classification of spend according to these categories has meant that it is not possible to assess the outputs of the health service against financial investment. Although reference costs, which focus on secondary and tertiary care, can help in part to assess this, they do not cover primary care services upon which significant NHS resources are expended. As the focus of the NHS has been changing over the last 5-10 years with the introduction of Local Delivery Plans, National Service Frameworks (NSFs) and Long Term Service Agreements (LTSAs), it is important to concentrate on health service outcomes. Programme Budgeting gives government, commissioners and providers the capacity to generate information appropriate to and necessary for commissioning and managing health services in this new paradigm. The analysis of expenditure across primary, secondary and community care will provide the capacity: To identify current patterns of service delivery for groups of patients and to identify variations between providers and access to services by geographical area To map care pathways To agree the desired patterns of commissioning To monitor NHS expenditure against NSFs To identify different expenditure patterns within the country Change in productivity = change in outputs/change in inputs Change in efficiency = change in outcomes/change in inputs 24

26 What is the history of Programme Budgeting? Alain Enthoven, an American academic and economist, implemented Programme Budgeting when he was United States Assistant Secretary of Defence. He then moved into healthcare management, and later was a proponent of an internal market (i.e. commissioning ) for the NHS. He maintains that effective commissioners of health services need Programme Budgeting and Marginal Analysis. 4 However, one of the authors (AJ) has tracked down a Department of Health Report from 1972, entitled Planning Programme Budgeting System for the health and personal social services, in which it states: the main recommendations in our first report (April 1971) were that a programme budget for health and personal social services should be developed mainly on the basis of client groups and that its prime aims should be to assist with determining priorities and with achieving the most cost-effective use of resources. 5 When was Programme Budgeting introduced into the NHS? In 2002, the Secretary of State for Health requested the collection of financial information that identifies all primary care trust expenditure, including primary care services, to programmes of care based on medical condition. For the financial years , and , all PCTs in England submitted a return to the Department of Health recording how their total expenditure was deployed across the 23 Programme Budget categories shown in Box A1. For the financial year , the data on expenditure were also linked to data on activity and outcomes in an interactive atlas produced in collaboration with NCHOD. The National Programme Budget Project Board The Programme Budgeting Project Board was originally set up in 2003 to coordinate and facilitate the development of the Programme Budgeting Project. In 2005, the Board was re-formed and re-named the National Programme Budget Project Board. The objectives of the Project Board are shown in Box A2. 4 Enthoven, A. (1999) In pursuit of an improving National Health Service. Nuffield Trust, London. 5 Department of Health (1972) Planning Programme Budgeting System for the Health and Personal Soial Services. Second Report by the Project Team. 25

27 Box A2: Objectives of the National Programme Budget Project Board To oversee the production of annual Programme Budget data and DH Resource Accounts (Schedule 5) and ensure it meets National Audit Office (NAO), Treasury and DH requirements To ensure the methodology of the project takes account of other DH initiatives including Financial Flows and Patient Choice To recommend areas where existing data collection exercises could be reduced To ensure all stakeholders are consulted and kept fully informed of the progress of this project To oversee the development of the Programme Budget Project for future collection exercises, including: o Non-admitted patient care reporting; o Admitted patient care reporting integration with system users service; o Dis-aggregation of data by age and primary/secondary care; o SHA expenditure apportionment, e.g. WDC spend; o Linking Programme Budgeting data to health outcomes; o Engagement of specialist tsars, e.g. mental health or diabetes, with Programme Budgeting Project Key stakeholders of the National Programme Budget Board include: Department of Health National Health Service However, other Government departments and agencies will also be interested in the results of programme budgeting and marginal analysis in the NHS, including: National Audit Office Audit Commission Her Majesty s Treasury Healthcare Commission What are the benefits of Programme Budgeting for the NHS? The National Audit Office and the Audit Commission have identified the following overall benefits of Programme Budgeting for the NHS: Showing where total NHS funds have been spent in a way that is useful and interesting to taxpayers; Enabling expenditure on particular conditions to be assessed against National Service Frameworks and health outcomes; Providing consistent data to compare one NHS organisation s expenditure with that of another; Assisting primary care trusts (PCTs) in planning the provision of services, thereby providing support for more effective budgeting and commissioning; Increasing transparency about the performance of PCTs and Strategic Health authorities (SHAs) 26

28 What are the advantages of Programme Budgeting to individual NHS organisations? In November 2006, Richard Douglas, the Director of Finance at the Department of Health highlighted that, by focussing on medical conditions rather than on input costs, it is possible to make a direct link between the object of expenditure and the patient care it delivers. Once expenditure is analysed in this way, it is then possible for PCTs to investigate the level of health gain that can be obtained for the amount of money invested, which will help to inform understanding about equity and how patterns of expenditure map to the epidemiology of the local population. 6 The Director of Finance concludes that taking into account epidemiological factors and differing local priorities, PCTs should be able: To understand their expenditure; To question their expenditure; To explore the possibilities of changing spending patterns among different programmes of care. How will Programme Budgeting be used in the long term? A database of NHS spend will be compiled over time which will give a detailed analysis of PCT spending patterns. This database will be used to help: PCTs assess the health gain obtained from investment; PCTs address issues of equity in relation to patterns of expenditure and the epidemiology of the local population; Healthcare Commission and other regulatory bodies in their scrutiny of the NHS. 6 Letter from Richard Douglas, Director of Finance, DH, to all SHAs. 8 December

29 Annex 2 The EPOC Programme 1. The EPOC Project annual review of Evidence, Process, Outcome and Configuration relating to the 50 biggest health conditions and problems that cause 80% of the burden of disease: Stomach cancer Bowel cancer Heart failure Chest pain & Coronary disease Incontinence Failure to cope in old age Alzheimer's disease Inflammatory bowel disease Hepatitis and Liver failure Disability COPD (Bronchitis) HIV & AIDS Stroke Skin cancer Psoriasis Eczema Dysmenorrhoea and Menorrhagia Depression Schizophrenia Drug and alcohol dependence Breast cancer Osteoporosis Allergy Renal failure Antenatal and pregnancy care Children s problems Oral health Deafness and tinnitus Asthma TB Lung Cancer Glaucoma Visual failure Deafness Stroke Trauma Headache Spinal disease Parkinson s Epilepsy Prostate hypertrophy and cancer Diabetes 28

30 Project description Review packs will be put together for each Primary Care Trust (PCT) covering the 50 major causes of disease burden. These packs will comprise: o new knowledge that has been published in the last year, co-ordinated by the National Library for Health specialist libraries; o prescribing data; o hospital admission data; o operating data where relevant; o outcome data where available, for example, through audit projects; o outcome tools and measures that could be used if none are being used at present. PCTs will be required to carry out a formal review if more than two standard deviations above or below the mean. They could be expected to give consideration if they were one standard deviation above or below the mean, but a well-organised PCT should be able to review each of these topics every year, because the mean may not be representative of best value. Deliverables The focus of this review is to identify opportunities for increased value. By separating the review from the financial year, clinicians, patients and commissioners will focus on the steps that could be taken within existing resources to increase value, for example by identifying interventions of high value that should be increased, and interventions of low value that should be decreased. The identification of these interventions could, of course, be used to bid for more resources or as a focus for service reduction, depending upon the availability of resources, but the main focus is on value improvement by change in service. At the end of the review, commissioners and providers, with patient support, will have a view of the activities that need to be increased or decreased to achieve better outcome. At the end of the review, it would also be appropriate to identify outcomes of interest to patients and to consider ways in which those could be measured, even on a sample of patients, until Connecting for Health has delivered the Secondary Uses Service. In addition, commissioners will be asked to describe the network of existing services and communication between different parts of the service, for example between optometrists and Ophthalmology Departments, or between one acute Trust and another, with a view to laying the foundations for network strengthening or development. 29

31 Crown Copyright Produced by the NHS National Knowledge Service February 2007 The text of this document may be reproduced without formal permission or charge for personal or in-house use. First published February If you require further copies of this title, contact: J A Muir Gray Badenoch Building University of Oxford Old Road Campus Headington Oxford OX3 7LF rosemary.lees@dphpc.ox.ac.uk 30

32 Appendix 2. Priority setting in health care using multi-attribute utility theory and programme budgeting and marginal analysis (PBMA)

33 ARTICLE IN PRESS Social Science & Medicine 64 (2007) Priority setting in health care using multi-attribute utility theory and programme budgeting and marginal analysis (PBMA) Stuart J. Peacock a,b,,1, Jeff R.J. Richardson c, Rob Carter d, Diana Edwards e a Centre for Health Economics Research in Cancer, Cancer Control Research, BC Cancer Agency, 675 West 10th Avenue, Vancouver, BC, Canada V5Z 1L3 b Department of Health Care and Epidemiology, University of British Columbia, Canada c Centre for Health Economics, Monash University, Australia d Program Evaluation Unit, University of Melbourne, Australia e Pfizer Australia, NSW, Australia Available online 4 December 2006 Abstract Programme budgeting and marginal analysis (PBMA) is becoming an increasingly popular tool in setting health service priorities. This paper presents a novel multi-attribute utility (MAU) approach to setting health service priorities using PBMA. This approach includes identifying the attributes of the MAU function; describing and scaling attributes; quantifying trade-offs between attributes; and combining single conditional utility functions into the MAU function. We illustrate the MAU approach using a PBMA case study in mental health services from the Community Health Sector in metropolitan South Australia. r 2006 Elsevier Ltd. All rights reserved. Keywords: Priority setting; Programme budgeting; Marginal analysis; Decision analysis; Multi-attribute utility theory; Mental health; Australia Introduction The development of frameworks to allow decision-makers to manage scarce resources is one of the most important challenges facing health services. Managing resource scarcity involves making choices about which types and amounts of health Corresponding author. Tel.: ; fax: addresses: speacock@bccrc.ca (S.J. Peacock), jeff.richardson@buseco.monash.edu.au (J.R.J. Richardson), r.carter@unimelb.edu.au (R. Carter), Diana.Edwards@pfizer.com (D. Edwards). 1 Stuart Peacock is a Michael Smith Foundation for Health Research Scholar. care to provide for different individuals and populations from a given budget. Decision-makers managers and clinicians therefore face the challenge of prioritising between competing claims on scarce health service resources and implementing their choices. In recent literature, economic approaches to priority setting have been criticised on the grounds that they have only had limited success in practice (Alban, 1994; Carter, 2001; Hoffmann & von der Schulenburg, 2000; MacDonald, 2002). This has been, at least in part, due to their failure to adequately capture the complex and multifaceted nature of both objectives and constraints in health service decision-making (Carter, 2001; Jan, 2000; /$ - see front matter r 2006 Elsevier Ltd. All rights reserved. doi: /j.socscimed

34 898 ARTICLE IN PRESS S.J. Peacock et al. / Social Science & Medicine 64 (2007) MacDonald, 2002). In response to this criticism economic approaches to priority setting are giving greater attention to understanding the inherent complexity of the health services decision-making environment (Peacock et al., 2006). In particular, developments in the programme budgeting and marginal analysis (PBMA) framework have focussed on better understanding decision-makers objectives and utility functions, balancing improved rigour against user friendliness, the importance of due process, and the political and institutional constraints that decision-makers face in setting priorities (Carter, 2001; Jan, 2000; Peacock, 1998; Mitton, Peacock, Donaldson, & Bate, 2003). In this paper, we focus on eliciting and valuing attributes in decision-makers utility functions using a novel multi-attribute utility (MAU) approach to priority setting in health services. We present an MAU approach to identifying, measuring and valuing multiple attributes in health service decision-makers utility functions, which is used to inform the marginal analysis phase of PBMA. The PBMA framework PBMA is a practical tool to assist decisionmakers in setting priorities in health care and has been used in over 70 priority setting exercises in countries such as Australia, Canada, New Zealand and the UK (Mitton, Peacock, Donaldson, & Bate, 2003). It offers a pragmatic framework to aid health service decision-makers in setting priorities based on three pivotal concepts, namely: the need to consider the opportunity cost of activities; the need for an analysis of marginal and not average costs and benefits; and, the existence of a fixed budget which implies a need to contract some services if others are to be expanded (Mooney, Gerard, Donaldson, & Farrar, 1992). The seven stages in a PBMA study are summarised in Table 1. PBMA requires information about the benefits from the services being studied. However, highquality quantitative evidence is often not available (for an exception see Donaldson and Farrar (1993)). Reflecting a common view in the PBMA literature, Cohen (1995) and Peacock (1998) argue that much can be achieved even when only crude data are available. Consistent with this, some PBMA studies have used an advisory panel to provide best estimates of the benefits from services based on knowledge of available quantitative and qualitative evidence and expert judgement. This approach has Table 1 Stages in a PBMA priority setting exercise 1. Determine the aim and scope of the priority-setting exercise Determine whether PBMA will be used to examine changes in services within a given programme (micro/ within programme study design) or between programmes (macro/between programme study design). 2. Compile a program budget The resources and costs of programs may need to be identified and quantified, which, when combined with activity information, is the programme budget. 3. Form a marginal analysis advisory panel The panel is made up of key stakeholders (managers, clinicians, consumers etc.) in the priority setting process. 4. Determine locally relevant decision-making criteria To be elicited from the advisory panel (e.g. maximising benefits, improving access and equity, reducing waiting times etc.), with reference to national, regional and local objectives, and specified objectives of the health system and the community. 5. Identify options for (a) service growth (b) resource release from gains in operational efficiency (c) resource release from scaling back or ceasing some services The programme budget, along with information on decision-making objectives, evidence on benefits from service, changes in local health care needs, and policy guidance, are used to highlight options for investment and disinvestment. 6. Evaluate investments and disinvestments Evaluate in terms of costs and benefits and make recommendations for (a) funding growth areas with new resources (b) moving resources from 5(b) and 5(c) to 5 (a). 7. Validate results and reallocate resources Re-examine and validate evidence and judgements used in the process and reallocate resources according to cost-benefit ratios and other decision-making criteria. Source: adapted from Mitton and Donaldson (2004). been described as options appraisal (Cohen, 1995). While the judgement of an advisory panel may be criticised on the grounds that it provides less reliable evidence than a randomised controlled trial or meta-analysis, it may be justified by the dictum perfection is the enemy of the good. We would argue that the structured framework for decisionmaking provided by PBMA is better than arbitrary processes and judgements, which may be the default option. The use of PBMA advisory panels has several other advantages. First, there may be different types of benefits from health programmes, such as health

35 ARTICLE IN PRESS S.J. Peacock et al. / Social Science & Medicine 64 (2007) gains for patients, reductions in health inequalities, and improvements in community empowerment. That is, benefits may consist of multiple attributes. The panel can define these attributes, which are used to evaluate programmes included in the PBMA analysis. Second, the panel can determine the relative importance of different attributes. If a single index of benefit is to be calculated for each programme, it is necessary to combine information on how well each programme performs under each attribute. This requires that each programme is given a score under each attribute, and that attribute scores are weighted for their relative importance when they are combined. The resulting measure of benefit can be used to calculate costbenefit ratios. Third, the panel can make judgements, which are well structured, explicit and transparent, allowing panel members and (potentially) third parties the chance to scrutinise decisions and how they were reached. Methods for including an MAU assessment of benefits in PBMA The MAU approach we now describe uses decision analytic techniques to model and quantify attributes in a decision-maker s utility function (Kenney & Raiffa, 1993). 2 The MAU model combines multiple attributes, such as health, equity and empowerment, by eliciting importance (trade-off) weights for attributes from members of the PBMA advisory panel, and combining attribute scores to calculate a single index of benefit. This approach involves the following methodological steps. Methodological steps in the MAU approach (I) Identifying attributes in the MAU function. The advisory panel defines attributes, which are relevant to the organisational context and the objectives of decision-makers. The panel should be allowed full discussion of the meaning of different attributes in order to gain a clear and shared understanding of the component parts of benefit in their decisionmaking context. (II) Describing attributes. The panel constructs measurement scales for each attribute, against 2 Following Edwards (1977) we use the term utility function to include measures of benefits both under certainty and uncertainty. which programmes are to be evaluated. To do so, the panel constructs descriptions for the best and worst, and a number of intermediate, levels for each attribute. For example, the best and worst levels for health might be excellent health and death. (III) Scaling attribute levels. The different levels within attributes are scaled by the panel; that is their relative importance is determined on a scale. For example, on a health scale excellent health may receive a score of 100, death 0, and intermediate levels a score between these. Scaling should use a recognised scaling instrument such as a rating scale or a time trade-off instrument. In the decision analysis literature the resulting measurement scales are referred to as single conditional utility functions. (IV) Quantifying trade-offs between attributes. The advisory panel assesses the relative importance of each attribute, as they are unlikely to be of equal importance. Theoretically valid and reliable techniques should be used to derive meaningful estimates of each attribute s relative importance. (V) Evaluating programmes. The panel evaluates each programme in terms of how well it performs with respect to each attribute, by scoring each programme using the attribute measurement scales. Scores should be based on available quantitative and qualitative evidence and technical judgements (where necessary) from the panel. The degree to which judgement should be employed will vary between contexts depending on the availability of relevant evidence, and the quality of that evidence. (VI) Combining attribute scores. Panel scores for each attribute are combined using the MAU model to calculate the combined benefit score of a programme. Benefit scores should be measured on a scale, which is easy to understand, and be intuitive and plausible. The programmes are then ranked according to their combined benefit and the panel validates results through discussion and deliberation. If the panel is dissatisfied with the implied ranking each stage of the process is re-evaluated until they are satisfied with the integrity of the results. Benefits from the MAU model are used to construct costbenefit ratios, with higher priority given to services

36 900 ARTICLE IN PRESS S.J. Peacock et al. / Social Science & Medicine 64 (2007) with a lower cost-benefit ratio. Resources should be allocated to competing services based on the costbenefit ranking from the most efficient service until resources are exhausted. Choosing a functional form for the MAU model The importance of each attribute in the MAU function may vary for one of two reasons, due to: the size of the importance weight elicited from the panel; and, the type of function (e.g. additive, multiplicative etc.) which is selected for the MAU model. We now focus on different types of MAU functions and their implications for estimating benefits, an issue which has received relatively little explicit attention in the literature. Any type of MAU or social welfare function (SWF) imposes particular (algebraic) properties on the utility function (Dolan, 1998; von Winterfeldt & Edwards, 1986). In the MAU quality of life literature, the use of additive, multiplicative and multi-linear functions has been discussed by Feeny, Torrance, and Furlong (1996) and Richardson, Day, Peacock, and Iezzi (2004). In the SWF literature, Wagstaff (1991) explored the properties of the linear exponential SWF and its ability to capture different distributive objectives. This form of the SWF contains parameters reflecting aversion to inequality in the distribution of health and the relative weight given to the treatment of different individuals. Dolan (1998) presented a more flexible SWF, which permits a wider range of prescriptions concerning the distribution of health. The properties of these different types of functions may or may not be appropriate for different PBMA priority setting contexts. The advisory panel may elect to use an importance-weighted average of the attributes (an additive function). However, this imposes particular, and restrictive, properties on the computation of benefit: the effect of attributes is unchanging as the overall level of benefit changes; weights must be relatively small; and, they must sum to unity. For example, in the context of health state utilities, the 15-dimensional (15D) MAU quality of life instrument uses such an approach (Sintonen 2001). The average weight of each attribute in the model is 1/15, and no one attribute can independently impact catastrophically on overall quality of life. The result is that, for example, severe depression in isolation cannot reduce quality of life to a level corresponding with a near suicidal health state. Equally, the independent effect of extreme pain on quality of life is small. Other examples of additive functions in the quality of life literature include the Quality of Well-Being Scale, the EQ-5D and the SF-6D (Furlong, Barr, Feeny, & Yandow, 2005). MAU theory suggests that when two or more attributes can independently have a large impact on overall benefit a multiplicative or multi-linear model should be used (von Winterfeldt & Edwards, 1986). These models allow extreme pain and severe depression, for example, to independently reduce quality of life to zero. Examples of MAU quality of life instruments that use multiplicative functions include the health utilities index (HUI) (Feeny, Torrance, & Furlong, 1996) and the assessment of quality of life (AQoL) (Richardson, Day, Peacock, & Iezzi, 2004). When one attribute (or set of attributes) is considered to be of primary importance, and other attributes of secondary importance, theory suggests that the MAU function should reflect the underlying objectives hierarchy (von Winterfeldt & Edwards, 1986). Objectives hierarchies have received little attention in the health literature. However, empirical evidence supporting the existence of hierarchical preferences for environmental goods is mounting (Spash, 2000). In such cases, it may be appropriate to adopt a functional form, which permits only one attribute (or set of attributes) to independently have a large impact on overall benefit. For example, the community may consider that improving the health of individuals is the primary objective of health services, and that providing programmes that are culturally appropriate is a secondary objective. A programme that provides a large health gain may independently produce a large impact on overall benefit. However, a programme that is highly culturally appropriate must also provide a positive health gain to impact on overall benefit. The MAU model should also have the property that preferences for health are not lexicographic (Tversky & Sattah, 1979). Similarly, alternative models from the SWF literature may or may not be appropriate in different priority setting contexts. For a SWF model to be appropriate, the panel must consider that health gain and the distribution of health are the only two objectives relevant to their decisionmaking context. If this condition is met, the panel must then estimate of the aversion to inequality parameter, which may take a wide range of values

37 ARTICLE IN PRESS S.J. Peacock et al. / Social Science & Medicine 64 (2007) reflecting a range of utilitarian and non-utilitarian perspectives. The critical task is to select an MAU function that allows attributes to be combined in a way, which is consistent with the panel s stated preferences (which should, in turn, reflect community preferences). The need for such a choice is universal: a combination rule must have an algebraic form (even the simple utilitarian SWF has a particular algebraic property, namely the additivity of utilities) and this does not depend upon the adoption of either welfarist or extra-welfarist objectives. Both Wagstaff (1991) and Feeny, Torrance, & Furlong (1996) explicitly claim to be measuring utility albeit with a more complex equation for combining the attributes. The choice of an appropriate MAU function is essential if PBMA is to be consistent with local decision-making objectives and organisational context. Proper consideration of local decision-making objectives and objectives hierarchies is critical to the success of a PBMA study (Peacock et al., 2006). However, for this reason, the MAU function developed for one context may not be generalisable to other contexts. Methods for constructing a Community Health MAU model We now turn to methods for constructing an MAU model in the context of South Australian Community Health Services (CHS). Before implementing the steps described above two methodological decisions had to be made. The first concerned the form of the MAU function which was consistent with the local CHS decision-making context. This context proved to be atypical in one important respect. The advisory panel for the PBMA project considered that the primary objective of South Australian Community Health programmes is to improve individuals health, with a secondary purpose to do so in a way that achieves other stated objectives relating to equity and community health. As discussed above, many additive and multiplicative MAU functions do not incorporate this property, and SWF functions are not applicable because they only allow health and equity objectives to be considered. For this reason, and in consultation with the advisory panel, a second model was adopted which is additive with respect to attributes other than health, but incorporates a multiplicative relationship between health and other attributes (which we call the multiplicative weights for health model ). For a given service, this model takes the score for individual health gain and weights it according to the scores for all other attributes: U ¼ U H ½1 þ W 1 D 1 þ; ; þw n D n Š, (1) where U is the combined utility (or benefit) score for a given service, U H is the utility for individual health, D 1,y, D n are scores for the n other attributes on their respective measurement scales, and W 1,y, W n are importance weights for those n attributes. This model has the property that if individual health gain is zero, U is also zero. A service that provides large individual health gains may independently produce a large impact on U, but a service that scores highly on other attributes must also provide improvements in individual health to impact on U. The term U H is given by U H ¼ MH, (2) where (for a given service) H is the score for individual health on its attribute measurement scale and M is the importance weight for individual health. This model can be expanded to be used directly with data elicited from an advisory panel as shown in Eq. (3). In this equation, U lies between 0 (worst utility) and 100 (best utility). 3 U ¼ H 100 M W 1 D 1 M þ þ; ; 100 þ 100 M P W i P W i 100 W n D n. ð3þ 100 The panel determines the relative importance of each attribute, yielding the values M and W i, and the scores for each attribute for each service being considered (H and D i ). The second methodological decision concerned the method for deriving the importance weights for attributes. A handful of published studies have considered weighting multiple attributes within PBMA (e.g. Cohen, 1995). These studies have used an options appraisal approach where weights are typically assessed by asking respondents to distribute 100 points over attributes to reflect their relative importance (a constant-sum variant of ratio estimation). However, there are two potentially serious problems with this approach. Constant-sum ratio estimation procedures typically produce flatter importance weight distributions 3 The [(100 M)/ P W i ] term rescales weighted attribute scores so that U is measured on a scale (Kenney & Raiffa, 1993).

38 902 ARTICLE IN PRESS S.J. Peacock et al. / Social Science & Medicine 64 (2007) than ratio estimation (von Winterfeldt & Edwards, 1986) which may lead to systematic under-estimation of trade-offs between attributes. And, direct judgments of importance may be insensitive to the ranges of measurement scales used in assessing the performance of alternative programmes (Gabrielli & von Winterfeldt, 1978). Following common practice in the decision analysis literature a swing weights method was selected for weighting attributes. This method overcomes many of the problems of constant-sum ratio estimation; is relatively simple, transparent and easy to use; and produces weights which are practically indistinguishable from indifference methods (von Winterfeldt & Edwards, 1986). In this approach respondents are asked how much an attribute contributes to overall utility relative to other attributes by comparing hypothetical health programmes that swing between the worst and best levels in each attribute. They then estimate the change in utility that would result from changing each attribute from its worst to best level using a rating scale which has endpoints of 100 (all attributes at their best level) and 0 (all attributes at their worst level) (von Winterfeldt & Edwards, 1986). In this study two forms of the swing weights method were used, termed the bottom up method and the top down method. The bottom up method first asked each respondent to consider a state of the world in which no health programme exists for a hypothetical group of individuals. This state is represented by 0 (all attributes at their worst level) in Fig. 1. Each respondent is then asked to consider the introduction of a hypothetical new health programme, which provides the best possible level of individual health but no improvements in community health or equity. That is, the new programme represents a state in which the individual health attribute is at its best level, but community health and equity attributes remain at their worst level. Using Fig. 1, this results in a swing up the scale from 0 to the point M (arrow A). The value on the rating scale of this new state indicates the relative importance of the individual health attribute. As discussed above, it was agreed that services, which did not provide any health gain at the individual level should receive an overall benefit score of zero. Consequently, for attributes other than individual health, the starting state for the (Best) 100 Individual Health at best Community Health at worst Equity at best Individual Health at best Community Health at best Equity at worst C B Individual Health at best Community Health at worst Equity at worst M M Individual Health at best Community Health at worst Equity at worst 20 A (Worst) 10 0 Individual Health at worst Community Health at worst Equity at worst Fig. 1. Weighting attributes bottom up method.

39 ARTICLE IN PRESS S.J. Peacock et al. / Social Science & Medicine 64 (2007) bottom up method is not 0, but the state where individual health is at its best level and all other attributes are at their worst level. That is, each respondent is asked to use their value for M in Fig. 1 as the starting point for assessing the relative importance of community health and equity attributes. For the community health attribute, each respondent is asked to consider the introduction of new health programme that represents a state in which both individual health and community health attributes are at their best level, but equity is at its worst level. In Fig. 1, this is the swing up the scale from the point M shown by arrow B, where the distance of the swing up the scale from M indicates the relative importance of the community health attribute. Similarly for the equity attribute, each respondent is asked to consider a new health programme that represents a state in which both the individual health and equity attributes are at their best level, but community health is at its worst level. This is the swing up the scale from the point M shown by arrow C, where the distance of the swing indicates the relative importance of the equity attribute. The top down method first asked each respondent to consider a health programme that resulted in a state where all attributes are at their best level, represented by the best possible state (100) in Fig. 2. Each respondent is then asked to consider the replacement (or contraction) of the best possible programme with a new programme, which provides the best levels of both individual health and equity, but the worst level of community health. That is, each respondent is asked to consider a swing down the scale (arrow D in Fig. 2), where the distance of the swing indicates the importance of the community health attribute. The swing down the scale is a measure of the disutility each respondent places on a move from the community health attribute at its best level to its worst level. Similarly for equity, each respondent is asked to consider the replacement (or contraction) of the best possible programme with a new programme, which provides the best levels of individual health and community health, but the worst level of equity. That is, each respondent is asked to consider a swing down the scale (arrow E in Fig. 2), where the distance of the swing indicates the importance of the equity attribute. Individual Health at best Community Health at best Equity at best (Best) Individual Health at best Community Health at best Equity at best E D Individual Health at best Community Health at best Equity at worst Individual Health at best Community Health at worst Equity at best (Worst) Fig. 2. Weighting attributes top down method.

40 904 ARTICLE IN PRESS S.J. Peacock et al. / Social Science & Medicine 64 (2007) Results from a mental health case study In this section, we present results from the mental health PBMA study in South Australian CHS as a case study application of our methods (see Peacock and Edwards (1997) and Peacock, Richardson, and Carter (1997)). The advisory panel consisted of CHS mental health service managers and providers from four South Australian CHS regions, service providers from other primary and mental health care agencies, finance and data officers from each region, and representatives from the South Australian Health Commission and the community (a total of 19 panel members). South Australian CHS mental health services place significant emphasis on the promotion of mental health, the prevention of mental health problems, early and crisis intervention for mental health problems, advocacy, and community participation, as well as an emphasis upon community support and increasing access for marginalised groups. The marginal analysis exercise took place over two consecutive days. An independent facilitator was engaged for the exercise, assisted by health economists, prior to which the panel received training covering the background, aims, and methods of PBMA. Identifying attributes in the MAU function The first step in the marginal analysis exercise was to define benefits from mental health services in the context of the study (the attributes of the MAU function) using a process of semi-structured group discussion and deliberation. In defining attributes panel members were asked to consider local and regional/ national level policy objectives, published evidence, and local expert knowledge. The facilitators guided debate to help ensure that two key assumptions of MAU theory were adhered to: attributes were mutually exclusive (orthogonal) to avoid double counting of the same attribute; and attributes were collectively comprehensive to ensure all relevant aspects of the decision-makers utility function were captured. This task proved to be the most challenging part of marginal analysis. The exercise commenced with a general discussion, which identified three potential attributes: individual health, community health, and equity. The panel was split into three groups (of six, six, and seven), with each group defining the meaning of a single attribute and reporting back. The panel then debated the definitions and clarified distinctions between attributes, in particular between the notions of health gain for individuals and community health. The latter was defined in terms of community ownership and control of a health programme and the sustainability of benefits beyond the lifetime that programme. Equity was defined in terms of the extent to which a health programme is culturally appropriate, available, perceived to be accessible and addresses the needs of disadvantaged groups (see Table 2). Describing attributes Descriptions for each attribute s best and worst levels, as well as two or three intermediate levels, were then constructed. Since the advisory panel would later be asked to score services in terms of the three attributes, it was vital that each panel member had a clear and shared understanding of each attribute level. The panel deliberated over and refined the descriptions into final attribute levels (see Table 2). Scaling attribute levels Attribute worst and attribute best levels were placed on a rating scale at 0 and 100 to define the endpoints of the measurement scale for each attribute. The concepts of interval-scale properties were explained and members asked to place each intermediate level on the scale between 0 and 100. The average of panel members responses for each intermediate level was calculated and used to calibrate the scale for each attribute (see Table 2). These measurement scales form the basis of single conditional utility functions, which are combined in the MAU function. The descriptions for each attribute demonstrate a further important point about our approach. The individual health attribute seeks to measure the direct health benefits that clients derive from the use of health services, i.e. benefit is measured at the individual level. However, whilst individuals benefit if a service is highly equitable or provides ongoing community structures for health maintenance and promotion, benefits from equity and community health are measured at the service level, i.e. the impact the service has on specified disadvantaged groups or the community. Identifying, describing and scaling attributes took a full day of discussion and deliberation. Quantifying trade-offs between attributes The relative importance of each attribute in the MAU function was assessed using the methods

41 ARTICLE IN PRESS S.J. Peacock et al. / Social Science & Medicine 64 (2007) Table 2 Attribute levels and measurement scales Individual health Position on Community health Position on Equity Position on rating scale a rating scale a rating scale a Individual has sense of independence and wellbeing; belonging/ acceptance; acquired useful skills; able to achieve their potential; aware of social context and of own needs; responds positively to life s challenges (i.e. personal, interpersonal and environmental); able to influence events around them. 100 Community knowledge, skills, and support to take action to deal with health issues and their collective health; to select, manage and control services and planning/funding system; benefit is sustained and operates over long term. 100 Highly culturally appropriate, physically available, perceived to be accessible, free at point of use and addresses circumstances which assist disadvantaged groups. 100 Individual has sense of belonging/acceptance; occasionally feels distressed/unable to resolve issues alone, but recognises need for help and able to gain assistance. Able to respond to life s challenges but less able to influence events around them. 76 Community in real partnership with planning/funding system; with real input to decision-making but no control. Benefits go beyond life of immediate project, but not sustained. 63 Culturally appropriate, available, perceived to be accessible, and addresses some circumstances which assist disadvantaged groups. 74 Individual has sense of distress and isolation, recognises need for help and able to seek help. Unable to respond to life s challenges in a positive way without assistance. 43 Community participation, but no ownership or control; limited influence on decision-making. Benefits limited to life of the project. 36 Culturally appropriate, available perceived to be accessible, but only for relatively advantaged groups. 29 Individual has sense of distress and isolation and recognises need for help. Unable to respond to life s challenges in a positive way without assistance, and is unable to access help. 20 Token community participation; no impact on decisionmaking; no influence; few choices. Benefits dissolve during life of project. 14 Individual in crisis and unable to respond. At risk to self/others; isolated and unable to access help, yet has acute mental health needs. 0 Community with no participation in planning/provision; no ownership and control; plus divisive and disenfranchised. 0 Culturally inappropriate, only available and accessible to relatively advantaged groups. Does not consider special needs and presents barriers to disadvantaged groups. 0 a Average of all panel member s responses to rating scale questions. described above, where each respondent considered the swings A (individual health), B (community health) and C (equity) for the bottom up method (Fig. 1), and D (community health) and E (equity) for the top down method (Fig. 2). These swings represent changes in utility

42 906 ARTICLE IN PRESS S.J. Peacock et al. / Social Science & Medicine 64 (2007) associated with moves between attributes worst and best levels. The mean of panel member s responses for A was used to calculate the importance weight (M ¼ 23:84) for individual health in the MAU model. Mean responses for B and C were used to calculate the bottom up importance values for community health and equity attributes, respectively. Similarly, mean responses for D and E were used to calculate the top down importance values for community health and equity. Descriptive statistics are shown in Table 3. To adjust for the potential impact of reference point effects, the mean of the importance values from the bottom up and top down methods were used to calculate the importance weights (W i ) for community health and equity attributes in the MAU model (see Table 4). Following discussion and deliberation over the results from both swing weights methods the panel agreed that final weights Table 3 Descriptive statistics for bottom up and top down methods Attribute n Mean s.d. Min Max Bottom up method Individual health (swing A, Fig. 1) Community health (swing B, Fig. 1) Equity (swing C, Fig. 1) Top down method Individual health n.a. n.a. n.a. n.a. n.a. Community health (swing D, Fig. 2) Equity (swing E, Fig. 2) Table 4 Mean responses for attribute weights Attribute Bottom up method a Top down method a Attribute weight b Individual health n.a Community health Equity a Entries in columns are the mean of panel member s responses for the swings A E in Figs. 1 and 2 where swings represent changes in utility associated with moves between attribute worst and best levels. b Attribute weight for Individual Health is M in Eq. (3) from the bottom up method. Attribute weights for community health and equity (W i in Eq. (3)) are the mean of their bottom up and top down scores. were intuitive, plausible and valid for the PBMA study. Some care is needed to correctly compare results from the top down and bottom up methods. Using Fig. 2, D measures the change in utility resulting from a move from the best possible state (1 0 0) to the state individual health at best, community health at worst, equity at best. Inspection of Fig. 1 shows that the change in utility resulting from a move from the worst state (0) to the state individual health at best, community health at worst, equity at best is the sum of the two moves A and C. From Table 3, the mean values were for D, for A and for C. The sum of A and C is which is slightly larger than the value for D. Similarly, E in Fig. 2 measures the change in utility resulting from a move from the best possible state (1 0 0) to the state individual health at best, community health at best, equity at worst. In Fig. 1 the change in utility resulting from a move from the worst state (0) to the state individual health at best, community health at best, equity at worst is the sum of the two moves A and B. Using Table 3, the mean values were for E, for A and for B. The sum of A and B is 50.10, which is slightly lower than the value for E. Evaluating programmes Panel members scored each service being evaluated using the attribute measurement scales. Members were not confined to choosing one of the attribute levels described in Table 2: they were told that descriptions were markers on the scale to aid them in evaluating services (to maintain the interval property of the conditional utility functions). Combining attribute scores For the mental health study, Eq. (3) simplified to: U ¼ H M W C C 100 M W E E M þ þ ð4þ W C þ W E 100 W C þ W E 100 where the attribute scores (on a scale) for each service being evaluated are represented by H (individual health), C (community health), and E (equity); and importance the weights for community health and equity are represented by W C and W E, respectively. Entering the values for M, W C and W E

43 ARTICLE IN PRESS S.J. Peacock et al. / Social Science & Medicine 64 (2007) from Table 4 yields the calibrated mental health MAU model: U ¼ H C 23:84 þ 33: þ 43:03 E. (5) 100 A worked example of calibrating the MAU model and calculating utility scores for services is given in Appendix A. Services in the case study were ranked according to their utility score and following discussion the panel was satisfied with the consistency of the ranking. Discussion The MAU approach to marginal analysis described in this paper has extended the options appraisal methodology of earlier PBMA studies. Our MAU approach is based on the notion that benefits from health programmes are multifaceted, and extend beyond health gains for individuals. The challenge for the analyst who wishes to estimate cost-benefit ratios for programme options is how to combine preferences for multiple types of benefits into a single (cardinal) utility measure in a systematic and transparent manner. We believe our MAU approach offers some insights on how to meet this challenge and raises a number of issues for discussion and future research. The initial task of eliciting attributes from advisory panels is demanding, but this is to be expected. Defining measures of health outcomes is a complex and demanding task in its own right. Defining measures of other types of benefits from health programmes (such as improving access to health care for disadvantaged groups) places further demands on panels. However, the process of deliberation and values clarification undertaken in eliciting attributes in the MAU approach, we would argue, provides valuable insights for the prioritysetting process, precisely because panel members are challenged to explicitly define (and defend) concepts of benefit. Of course, some decision-makers might prefer a less-demanding process. However, there is likely to be a trade-off between the demands of the task and increased potential for measures of benefits to be poorly specified and/or understood. Some PBMA studies have failed to elicit importance weights for attributes, and have assigned unitary weights in the absence of data. Such an approach is questionable. To avoid this problem, the importance of attribute weighting was outlined at a workshop prior to marginal analysis, in the preliminary questionnaire, and at the 2-day exercise. Our MAU approach uses a swing weights method to explicitly estimate the relative importance of the different attributes, adjusted for potential reference point effects. A prerequisite for most prioritysetting exercises is that the methods can be used in a timely manner for assisting decision-makers. To this end swing weights appear to represent a robust and relatively easy to use methodology, which can yield immediate results. There are four caveats. First, two PBMA panels in later studies reported having some difficulty following the logic of swing weights and chose to use weights based on simple ratio estimation techniques instead (Edwards, Peacock, & Carter, 1998). Second, it is not yet clear whether decision-makers would favour more technically complex, time intensive, weighting methods, such as willingness to pay and discrete choice experiments. Deciding the optimal method for attribute weighting represents an important research topic in the economics of priority setting. Third, we found evidence of reference point effects. The disutility of a move from equity at its best to its worst level was greater than the utility of a move in the opposite direction: respondents placed greater importance on equity in the context of reductions to services than in the context of expansions to services. This suggests that decision-makers may have different attribute weights for service reductions and expansions. This study is the first to report such a finding, which may explain why some PBMA panels have refused to consider service reductions. Fourth, resource allocation recommendations are based on the cost-benefit ratios, but the use of ratios involves implicit assumptions about perfect divisibilities for all programmes and constant returns to scale (Gafni & Birch, 2006). Decisionmakers are dealing with increments, not margins, since most programmes cannot be designed to provide individuals with a single unit of improvement in benefit. The measurement of MAU may be carried out in one of two ways. In the holistic or composite approach, vignettes are used to describe a set of hypothetical scenarios in terms of different levels of all attributes, and utilities for scenarios are elicited using a scaling instrument. Typically, this approach then requires estimation of regression model to obtain the parameters for the MAU model. The

44 908 ARTICLE IN PRESS S.J. Peacock et al. / Social Science & Medicine 64 (2007) second, decomposed, approach was adopted in this paper. It requires that utilities are estimated for each attribute in isolation, and overall utility is calculated by scaling an appropriate MAU function. Both approaches have strengths and weaknesses. Holistic measurement permits a description, which is tailored to a scenario (e.g. a specific health service). Unique aspects of the service, its context, its consequences, and the process of health care delivery may all be included in the vignette. However, in the MAU quality of life literature, the validation of vignettes has seldom (if ever) been carried out, and regression modelling requires data from large surveys. In contrast, the decomposed MAU approach may be unable to capture some of the nuances of alternative services. However, in this paper, following common practice in decision analysis, validity and consistency of responses was established through panel discussion and deliberation. Further, the advisory panel consisted of 19 members, which was too small to allow the use of regression techniques. Combining utility functions for single attributes into the MAU function as described here takes account of the fundamental importance of the improvements in health for individuals in most (if not all) health programmes. The multiplicative weights for the MAU functional form avoids a major weakness of the simple additive MAU function: that programmes which do not offer improvements in health for individuals can score highly if they perform well in terms of other attributes. This problem appears to have gone unnoticed in the PBMA literature. Critically, the multiplicative weights for health model was not imposed on the analysis by researchers. Rather, the panel selected this function when they were offered it as an option in preference to other functional forms. We are not suggesting that the functional form we have proposed is the (universally applicable) MAU functional form for marginal analysis. Legitimate questions about the choice of function might include: what is the effect of assuming that other attributes are included using a multiplicative interaction with individual health gain compared to other approaches? And, does this assumption under or over-state benefits from other attributes? In the absence of a gold standard against which MAU models can be judged, these questions are difficult to answer. The important point is that MAU theory and methods used here have been largely overlooked in the literature. We believe that they should be part of the ongoing priority-setting research agenda. Acknowledgements The research presented in this paper was funded by a grant from the Faculty of Business and Economics, Monash University and by the South Australian Health Commission (for the mental health PBMA case study). We would like to thank the South Australian PBMA committees for their invaluable efforts, and Gavin Mooney, Colin Green, Steve Birch and two anonymous referees for their very useful comments on drafts of this paper. The views expressed in this paper are those of the authors, and not the funding agencies or South Australian health services. Appendix A. A worked example of calibrating the MAU model and calculating utility scores A. Calibrating the mental health MAU model The mental health MAU model (Eq. (4)) is given by U ¼ H 100 M W C C 100 M W E E M þ þ, 100 W C þ W E 100 W C þ W E 100 where the utility score for a given service is represented by U; the importance weights for individual health, community health and equity are given by M, W C and W E, respectively; and the attribute scores for each service being evaluated are represented by H (individual health), C (community health), and E (equity). (i) Calculate the constant [(100 M)/(W C +W E )] from the attribute weights in Table 4. ½ð100 MÞ=ðW C þ W E ÞŠ ¼ ð100 23:84Þ=ð38:26 þ 49:71Þ ¼ 0:8657. (ii) Multiply the attribute weights for community health and equity by this constant. ½ð100 MÞ=ðW C þ W E ÞŠ W C ¼ 0: :26 ¼ 33:12, ½ð100 MÞ=ðW C þ W E ÞŠ W E ¼ 0: :71 ¼ 43:03.

45 ARTICLE IN PRESS S.J. Peacock et al. / Social Science & Medicine 64 (2007) (iii) This gives the calibrated mental health MAU model (where utilities lie in the range). U ¼ H C 23:84 þ 33: þ 43:03 E. 100 B. Evaluating services and calculating utility scores Two services (X and Y) are evaluated as part of a mental health priority setting exercise. X provides relatively small health gains for individuals who have limited ownership and control of the service, but it targets disadvantaged groups. Y provides large health gains for individuals who have greater ownership and control of the service, but it targets relatively advantaged groups. (i) Elicit the advisory panel s scores for how well X and Y perform under each attribute using the measurement scales in Table 2. For example: Attribute scores Service X Service Y Individual health (H) Community health (C) Equity (E) Enter the attribute scores for X and Y into the (ii) calibrated MAU model to obtain the overall utility (or benefit) scores. UðXÞ ¼ :84 þ 33:12 þ 43: ¼ 21:45, UðYÞ ¼ :84 þ 33:12 þ 43:03 ¼ 66: (iii) Combine the overall scores for X and Y with cost data to produce cost-benefit ratios for use in PBMA. References Alban, A. (1994). The role of economic appraisal in Denmark. Social Science & Medicine, 38, Carter, R. (2001). Priority setting in health: Processes and mechanisms. Expert paper prepared on invitation from the Commonwealth Secretariat for the 13th Commonwealth health ministers meeting, Christchurch, New Zealand, November Cohen, D. (1995). Messages from Mid Glamorgan: A multiprogramme experiment with marginal analysis. Health Policy, 33, Dolan, P. (1998). The measurement of individual utility and social welfare. Journal of Health Economics, 17, Donaldson, C., & Farrar, S. (1993). Needs assessment Developing an economics approach. Health Policy, 25, Edwards, W. (1977). How to use multiattribute utility measurement for social decision-making. IEEE Transactions on Systems, Man, and Cybernetics, SMC-7, Edwards, D., Peacock, S., & Carter, R. (1998). Setting priorities in South Australian community health III: Regional applications of program budgeting and marginal analysis. Centre for Health Program Evaluation, Monash University. Feeny, D., Torrance, G., & Furlong, W. (1996). Health utilities index. In B. Spilker (Ed.), Quality of life and pharmacoeconomics in clinical trials. Philadelphia Lippencott Raven Publishers. Furlong, W., Barr, R. D., Feeny, D., & Yandow, S. (2005). Patient-focused measures of functional health status and health-related quality of life in pediatric orthopedics: A case study in measurement selection. Health and Quality of Life Outcomes, 3, 3. Gabrielli, W., & von Winterfeldt, D. (1978). Are importance weights sensitive to the range of alternatives in multiattribute utility measurement? Research report 78 6, University of Southern California, Social Science Research Institute. Gafni, A., & Birch, S. (2006). Incremental cost-effectiveness ratios (ICERs): The silence of the lambda. Social Science & Medicine, 62, Hoffmann, C., & von der Schulenburg, J. M. G. (2000). The influence of economic evaluation studies on decision making. A European survey. Health Policy, 52, Jan, S. (2000). Institutional considerations in priority setting: Transactions costs perspective on PBMA. Health Economics, 9, Kenney, R. L., & Raiffa, H. (1993). Decisions with multiple objectives. Cambridge: Cambridge University Press. MacDonald, R. (2002). Using health economics in health services: Rationing rationally?. Houston: Open University Press. Mitton, C., & Donaldson, C. (2004). Priority setting Toolkit: A guide to the use of economics in healthcare decision making. London: BMJ Books. Mitton, C., Peacock, S., Donaldson, C., & Bate, A. (2003). Using PBMA in health care priority setting: Description, challenges and experience. Applied Health Economics and Health Policy, 2, Mooney, G., Gerard, K., Donaldson, C., & Farrar, S. (1992). Priority setting in purchasing: Some practical guidelines (Research Paper 6). Scotland: National Association of Health Authorities and Trusts. Peacock, S. (1998). An evaluation of programme budgeting and marginal analysis applied in South Australian hospitals. Centre for Health Programme Evaluation, Monash University. Peacock, S., Edwards, D. (1997). Setting priorities in South Australian community health I: The mental health program budget. Centre for Health Program Evaluation, Monash University. Peacock, S., Richardson, J., & Carter, R. (1997). Setting priorities in South Australian community health II: Marginal analysis of mental health services. Centre for Health Programme Evaluation, Monash University.

46 910 ARTICLE IN PRESS S.J. Peacock et al. / Social Science & Medicine 64 (2007) Peacock, S., Ruta, D., Mitton, C., Donaldson, C., Bate, A., & Murtagh, M. (2006). Using economics to set pragmatic and ethical priorities. British Medical Journal, 332, Richardson, J., Day, N., Peacock, S., & Iezzi, A. (2004). Measurement of quality of life for economic evaluation and the assessment of quality of life (AQoL) mark II instrument. Australian Economic Review, 37, Sintonen, H. (2001). The 15D instrument of health-related quality of life: Properties and applications. Annals of Medicine, 33(5), Spash, C. (2000). Ecosystems, contingent valuation and ethics: The case of wetland re-creation. Ecological Economics, 34, Tversky, A., & Sattah, S. (1979). Preference trees. Psychological Review, 86, von Winterfeldt, D., & Edwards, W. (1986). Decision analysis and behavioural research. New York: Cambridge University Press. Wagstaff, A. (1991). QALYs and the equity efficiency trade-off. Journal of Health Economics, 10(1),

47 Appendix 3. Programme Budgeting and Marginal Analysis: Bridging the divide between doctors and managers

48 Downloaded from bmj.com on 15 November 2007 Programme budgeting and marginal analysis: bridging the divide between doctors and managers Danny Ruta, Craig Mitton, Angela Bate and Cam Donaldson BMJ 2005;330; doi: /bmj Updated information and services can be found at: References Rapid responses alerting service These include: This article cites 9 articles, 3 of which can be accessed free at: 3 online articles that cite this article can be accessed at: 3 rapid responses have been posted to this article, which you can access for free at: You can respond to this article at: Receive free alerts when new articles cite this article - sign up in the box at the top left of the article Topic collections Articles on similar topics can be found in the following collections Health Economics (388 articles) Resource allocation (including rationing) (309 articles) Notes To order reprints follow the "Request Permissions" link in the navigation box To subscribe to BMJ go to:

49 Downloaded from bmj.com on 15 November 2007 Education and debate Programme budgeting and marginal analysis: bridging the divide between doctors and managers Danny Ruta, Craig Mitton, Angela Bate, Cam Donaldson Recent NHS reforms give doctors increased responsibility for efficient and fair use of resources. Programme budgeting and marginal analysis is one way to ensure the views of all stakeholders are properly represented Tensions between doctors and managers and the differences between medical and managerial cultures have existed since the earliest provision of organised health care. 1 In a resource allocation context, doctors are caricatured as taking the role of patient advocate while managers take the corporate, strategic view. Delivery of efficient (and in the case of the NHS, equitable) health care requires doctors to take responsibility for resources and to consider the needs of populations while managers need to become more outcome and patient centred. One economic approach, called programme budgeting and marginal analysis, has the potential to align the goals of doctors and managers and create common ground between them. We describe how the approach works and why it should be more widely used. Economic principles Programme budgeting and marginal analysis is an approach to commissioning and redesign of services that can accommodate both medical and managerial cultures and the widest constituency of professional, patient, and public values within a single decision making framework. It allows for the complexities of health care while adhering to the two key economic concepts of opportunity cost and the margin. When having to make choices within limited resources, certain opportunities will be taken up while others must be forgone. The benefits associated with forgone opportunities are opportunity costs. Thus, we need to know the costs and benefits of various healthcare activities, and this is best examined at the margin that is, the benefit gained from an extra unit of resources or benefit lost from having one unit less. If the marginal benefit per pound spent from programme A is greater than that for B, resources should be taken from B and given to A. This process of reallocation should continue until the ratios of marginal benefit to marginal cost for the programmes are equal, maximising total patient benefit across the two programmes. The opportunity cost of funding one more hip replacement, for example, could be the benefit forgone by not using that resource to fund renal dialysis. Thus, the application of economics becomes about the balance of services, not introduction or elimination of services. Such marginal analysis is central in making the most of resources available. Five questions The approach starts by examining how resources are currently spent before focusing on benefits and costs of changes to the spending pattern. 2 It can be used at micro levels (within programmes of care) or at a macro level (across services and programmes within a single health organisation). At its core, the approach can be operationalised by asking five questions about resource use (box 1). The first two questions relate to programme budgeting, and the other three to marginal analysis. The underlying premise of programme budgeting is that we cannot know where we are going if we do not know where we are. All primary care trusts now have to collect programme budgeting information as part of the statutory accounts process. What they are not yet required to do is proceed with the marginal analysis. An advisory panel is usually formed to examine the costs and benefits of proposed changes in services and use this information to improve benefit overall. The School of Population and Health Sciences, University of Newcastle, Newcastle upon Tyne NE2 4AA Danny Ruta senior lecturer in epidemiology and public health Angela Bate research associate in health economics Cam Donaldson health foundation chair in health economics Centre for Healthcare Innovation and Improvement, Research Institute for Children s and Women s Health, University of British Columbia, Vancouver, Canada V6H 3V4 Craig Mitton assistant professor Correspondence to: D Ruta danny.ruta@ ncl.ac.uk BMJ 2005;330: BMJ VOLUME JUNE 2005 bmj.com 1501

50 Education and debate Downloaded from bmj.com on 15 November 2007 Box 1: Five questions about resource use What are the total resources available? On which services are these resources currently spent? What services are candidates for receiving more or new resources (and what are the costs and potential benefits of putting resources into such growth areas)? Can any existing services be provided as effectively, but with fewer resources, so releasing resources to fund items on the growth list? If some growth areas still cannot be funded, are there any services which should receive fewer resources, or even be stopped, because greater benefit would be reached by funding the growth option as opposed to the existing service? panel is charged with making recommendations in line with predefined criteria. If the budget is fixed, opportunity cost is accounted for by recognising that the items for service growth can be funded only by taking resources from elsewhere. Resources can be obtained from elsewhere by being more technically efficient (changing practice to achieve the same health outcome at less cost) or more allocatively efficient (treating entirely different conditions to achieve a greater health outcome at the same cost). The analysis can be done at the margin by considering the amounts of different services provided. Although in reality quantitative data on marginal benefits are often lacking, it is the way of thinking underpinning the framework that is important. Of course, governments tend to add real resources to health organisation budgets each year. But the increased funds are unlikely to cover all proposed growth areas. Scarcity still exists, and the principles of programme budgeting and marginal analysis still apply. In effect, although sounding extreme, the whole budget is available for consideration for re-allocation. Who decides and how? Careful consideration must be given to the make up of the advisory panel and to the various stakeholder groups whose views and advice will be sought. The key is to obtain representation 3 5 without rendering the process unmanageable. 6 The composition of the panel will depend on the questions under consideration and the scope of the exercise, but it is likely to comprise a mix of clinical staff and managers and perhaps patients or members of the public. Information analysts and financial staff are also key resources to provide support. Whenever possible, local knowledge should be supplemented with evidence from sources such as economic evaluations, effectiveness studies, needs assessments, national and local policy documents, and surveys of healthcare professionals and the public. 5 6 In the end, however, it is the members of the advisory panel who decide whether to recommend that resources should be shifted. When evidence is lacking, group members may base recommendations on their expert opinion. 7 It is also important to conduct a final round of consultations with a wider group of relevant stakeholders. This tests the validity of the recommendations and makes it more likely that they will be accepted. Box 2 outlines the formal stages of the process. A practical toolkit is now available describing these stages in more detail. 8 Barriers to use Although programme budgeting and marginal analysis is not without challenges, 9 the framework has been used in over 60 health organisations in Australia, New Zealand, Canada, and the United Kingdom. A systematic review showed that use was sustained in over half of the 80 cases where the approach has been implemented. Given this, it is hard to understand why greater use has not occurred, particularly in bodies like NHS primary care trusts. Perhaps its use has been discouraged by the poor uptake of results of traditional economic evaluations at the local level in the NHS. 10 However, programme budgeting and marginal analysis is different from economic evaluation. Although based on the same principles, it uses these principles to create a management process into which results from standard economic evaluations and other evidence can be incorporated. Indeed, such a process could be seen as the missing piece in the jigsaw of reform, providing defensible mechanisms to help primary care trusts remain within budget while prioritising between national guidance and local needs. The approach requires an acceptance of resource scarcity and the need to manage it. Another important barrier to its effective use may stem from reluctance by doctors to accept loss of funding if their services are judged to have lower marginal benefit. Financial incentives, whereby clinicians are empowered to reinvest a portion of resources released directly back into their services, have been shown to encourage participation. 4 Box 2: Seven stages in setting priorities Determine the aim and scope of the exercise Compile a programme budget (map of current activity and expenditure) Form marginal analysis advisory panel and stakeholder advisory groups Determine locally relevant decision making criteria with input from decision makers and stakeholders (eg service providers, patients, public) Advisory panel identifies options in terms of: Areas for service growth Areas for resource release through producing same level of output (or outcomes) but with fewer resources Areas for resource release through scaling back or stopping some services Advisory panel makes recommendations in terms of: Funding growth areas with new resources Decisions to move resources released through increased productivity to areas of growth Trade-off decisions to move resources from one service to another if relative value is deemed greater Validity checks with additional stakeholders and final decisions to inform budget planning process 1502 BMJ VOLUME JUNE 2005 bmj.com

51 Downloaded from bmj.com on 15 November 2007 Education and debate Improving the doctor-manager relationship A successful partnership between medicine and management is widely believed to require joint leadership and alignment of goals. To accomplish a convergence of cultures Ham suggests we need to Harness the energies of clinicians and reformers in the quest for improvements in performance that benefit patients. 11 The programme budgeting and marginal analysis process has the potential to do this, providing a practical framework to facilitate joint working in several ways. The approach requires and values equally the contributions of doctors and managers. For example, different models of medical practice 1 each play a legitimate part at different stages in the process. A reflective model, drawing on tacit knowledge borne of individual clinical experience, is invaluable in formulating the criteria for assessing candidates for increased and decreased funding, to identify these candidates, and to assess subjectively the benefits gained or lost from proposed shifts in resource allocation. At the same time, doctors bring essential critical appraisal skills to the evaluation of investment and disinvestment options and for integration of clinical evidence from systematic reviews. Managers, in addition to providing more obvious organisational, operational, financial, and strategic management skills, can ensure success at critical stages of the process through cooperation, negotiation, delegation, teamwork, and persuasion. 12 Managers will also ensure that the local and national policies exert an appropriate level of influence on final priorities. Consideration of policies is no less important than clinical evidence if the process is to lead to real change in delivery of services. Other advantages of programme budgeting and marginal analysis include transparency and inclusivity. Contextual information, evidence, and subjective judgment are explicitly presented, evaluated, and recorded. This makes it more difficult for any professional group to defend (or reject) a stance simply through obfuscation or unsubstantiated assertion. It is also likely to minimise legal intrusion into public policy making. 13 In addition, the perspectives of doctors and managers are both mediated and illuminated by a range of other viewpoints garnered from patient, public, and professional groups. Perhaps the most important benefits for the doctor-manager relationship would come through interdisciplinary education. Joint participation at each stage of the process has the potential to lead to a shared understanding of each other s cultures. The net result may be a shared appreciation of opportunity cost, the need to focus on resources and health outcomes and to balance clinical autonomy with financial responsibility. Sustainable publicly funded healthcare systems depend on a mature recognition of the need to manage scarcity. Programme budgeting and marginal analysis can help achieve this precisely because it bridges clinical and strategic decision making. Contributors and sources: CD is an experienced health economist with a long research interest in priority setting. DR and CM have extensive experience implementing programme budgeting and marginal analysis (PBMA) as health services Summary points Programme budgeting and marginal analysis has the potential to align the goals of doctors and managers It is an economic approach to priority setting that adheres to the two key economic concepts of opportunity cost and the margin The method requires and values equally the contributions of doctors and managers Contextual information, evidence, and subjective judgment are explicitly presented, evaluated, and recorded The approach fosters a shared appreciation of the need to focus on resources and health outcomes and the need to balance clinical autonomy with financial responsibility researchers. Both CD and CM have recently completed a major review of PBMA studies over the last 25 years. This article is a synthesis of their personal experiences, the results of the review, and the preliminary findings of a PhD study by AB. Competing interests: None declared. 1 Davies HT, Harrison S. Trends in doctor-manager relationships. BMJ 2003;326: Donaldson C, Farrar S. Needs assessment: developing an economic approach. Health Policy 1993;25: Cohen D. Marginal analysis in practice: an alternative to needs assessment for contracting health care. BMJ 1994;309: Lockett T, Rafferty J, Richards J. The strengths and limitations of programme budgeting. In: Lockett T, ed. Priority setting in action. Oxford: Radcliffe Medical Press, Ruta DA, Donaldson C, Gilray I. Economics, public health and health care purchasing: the Tayside experience of programme budgeting and marginal analysis. J Health Services Res Policy 1996;1: Craig N, Parkin D, Gerard K. Clearing the fog on the Tyne: programme budgeting in Newcastle and North Tyneside Health Authority. Health Policy 1995;33: Peacock S. Program budgeting and marginal analysis: options for health sector reform. Melbourne: Centre for Health Program Evaluation, Monash University, Donaldson C, Mitton C. Priority setting toolkit: a guide to the use of economics in health care decision making. London: BMJ Books, Mitton C, Peacock S, Donaldson C, Bate A. Using PBMA in health care priority setting: description, challenges and experience. Appl Health Econ Health Policy 2003;2: McDonald R. Using health economics in health services: rationing rationally? Buckingham: Open University Press, Ham C. Improving the performance of health services: the role of clinical leadership. Lancet 2003;361: Crosson FJ. Improving the doctor-manager relationship. Kaiser Permanente: a propensity for partnership. BMJ 2003;326: Greschner D, Lewis S. Autonomy and evidence-based decision-making: Medicare in the courts. Canadian Bar Rev 2003;82: (Accepted 6 May 2005) Endpiece Discovery Discovery consists of seeing what everybody has seen and thinking what nobody has thought. Albert von Szent-Gyorgy ( ), Hungarian born American biochemist (Nobel prize 1937) Fred Charatan, retired geriatric physician, Florida BMJ VOLUME JUNE 2005 bmj.com 1503

52 Appendix 4. Manager s Faculty Annual Survey (2006)

53 Walker & Cate (2006) Manager s Faculty Annual Survey (2006) Sarah Walker & Tim Cate Sarah Walker, Trainee Clinical Psychologist, University of Newcastle Upon Tyne Tim Cate, Associate Director of Psychology, Tees, Esk & Wear Valleys NHS Trust Introduction The DCP Manager s Faculty Survey has been conducted annually, with this year being its fifth year of assessing psychological services within the NHS. The survey was again distributed to members of the DCP Managers Faculty and was conducted electronically. The survey assessed various areas of services including service configuration, vacancy factors for Clinical Psychology posts, distribution of CP posts across the Agenda for Change bandings, waiting times, and benchmarking of services against the criteria set out in the DoH report Organising and Developing Psychological Therapies (2004). Results were compared to previous year s results (Walker & Cate, 2006; Walker et al, 2005) in order to gauge any changes that may have taken place within the last twelve months. Respondents The number of respondents had again increased slightly to 86 services (83 in 2005, 71 in 2004, 68 in 2003, 60 in 2002), with 51 of the respondents indicating that they had completed the survey for their trust in Respondents were asked to indicate which specialities their service encompassed, which indicated that the survey was representative of clinicians across a wide range of specialities within psychological services in the NHS. Table 1: Type of Services Responding Type of Service No. of Services Adult Mental Health 30 Child & Family 27 Forensic 11 Health &/or Medical 23 Learning Disabilities 28 Older Adults Mental Health 22 Addictive Difficulties 4 Eating Disorders 4 Neuropsychology 19 Primary Care Mental Health 17 Psychodynamic Psychotherapy 15 CBT 15 Miscellaneous 2 1

54 Walker & Cate (2006) Vacancy Factor Respondents were asked to indicate the number of WTE psychology posts that were currently occupied, vacant, frozen or filled by short-term temps. In addition, respondents were asked to record their position 12 months ago, and to predict how their service may look in 12 months time. Out of the 64 services that provided responses to this section, only 19 services were able to complete all sections. The figures shown in brackets in Table 2 indicate the number of services completing that column. Table 2: Summary of Vacancy Factor data (n=64) Total WTE Psychology Posts WTE Vacant Posts WTE Frozen Posts Filled by ST Assistants Total WTE Posts Needed Sum % Mean Range Last Year (74) Present (76) Next Year (68) Last Year (58) Present (64) Next Year (48) Last Year (48) Present (55) Next Year (42) Last Year (42) Present (48) Next Year (40) Last Year (40) Present (48) Next Year (39) The figures in Table 2 indicate that (for the services completing this section) the number of WTE psychology staff has increased when compared to their position 12 months ago, by an average of 0.9 WTE (7.2 to 8.1). It is predicted that, on average, this will be maintained in the next 12 months (8.7). The percentage of WTE posts vacant in these services appears to have increased from 4.7% to 8.8%, however the forecast for the coming year is that this will drop, with a figure of 1.2% predicted for the number of posts expected to be vacant in 12 months time. This is in contrast to the predictions made by last year s respondents who anticipated a vacancy factor of only 0.01% by this point in time (Walker & Cate, 2005). The percentage of posts frozen has increased slightly within these services from the figures they report from the previous year (2.0% to 4.6%), however, this is expected to decrease again over the next 12 months (1.1%). The number of posts filled by short-term assistants has also increased somewhat, from 1.8% to 3.3%, although predictions for the coming 12 months indicate that services expect this to decrease again (1.1%). The total number of WTE posts needed has increased slightly over the last 12 months, with the average number of posts needed rising from 1.8 to 2.2. Somewhat surprisingly, the overall picture does not appear to have changed dramatically from the results of previous years surveys, despite the major changes which have taken place and indeed are still ongoing in the NHS. 2

55 Walker & Cate (2006) Agenda for Change Respondents were asked to indicate the WTE staff in each of 9 categories of staff within their service. Table 3: Number of WTE staff in each profession Profession No. of Staff % Clinical Psychologist Counselling Psychologist Health Psychologist Forensic Psychologist Adult Psychotherapist Child Psychotherapist Counsellors Assistant Psychologist Others* * Including Occupational Therapists, Graduate Mental Health Workers, Associate Psychologists and Psychological Therapists A total of 15 forecast retirements were indicated over the next 12 months. 51 out of 86 services indicated that they expect changes to their service configuration to take place over the next year. Many such changes were related to proposed service reconfigurations, often as a result of trust mergers and changes in commissioning of services. Eight services indicated that they expected to lose clinical psychology posts as a result of the forthcoming changes, however, ten services indicated that they expected at least one post to be created to accommodate changes. Table 4: Percentage of staff in each AfC banding for each profession Agenda for Change Banding Profession a 8b 8c 8d 9 Clinical Counselling Health Forensic Adult Psychotherapist Child Psychotherapist Counsellors Assistant Others* Of the 86 services included in this year s survey, 23 services had at least one member of staff in the process of appealing against their Agenda for Change banding. Six services reported that they had not yet been fully assimilated to the AfC system. At the same point last year, 46 out of 83 services had not been fully assimilated, and 43 services had staff appealing against their banding. Clearly a large amount of work has been completed in the last twelve months to reduce these figures, and it is hoped that the remaining appeals will be resolved before next year s survey. 3

56 Walker & Cate (2006) Waiting Times Respondents were asked to provide figures for the average waiting time for the different services within their department. Estimates were given for the wait from referral to initial assessment, and from initial assessment to intervention. Table 5: Waiting time in weeks for psychological services from referral to initial assessment, and from initial assessment to intervention Service Type n WT (weeks) Referral to Assessment WT (weeks): Assessment to Intervention Range Average Range Average Primary Care * Adult Mental Health Older Adult MH Child & Family Learning Disabilities CAMHS Psychodynamic Psychotherapy CBT Neuropsychology Addictive Difficulties Eating Disorders Health Psychology Forensic Psychology Other *Would be 13.2 without service reporting There appears to be a large variation between services in terms of waiting times for assessment and treatment. Although it is pleasing and encouraging to see that some services are able to see patients within a week of referral, such services are the exception rather than the norm, as is reflected in the average waiting times shown. Many services are still exceeding the target of 18 weeks from referral to intervention. This may reflect the strain being put on services by an increasing amount of referrals to psychological services, and further highlights the need for services to be given the capacity to expand in order to tackle waiting list problems effectively. 4

57 Walker & Cate (2006) Table 6: Average waiting times in 2004, 2005 & 2006 Service Type Average WT (weeks) Referral to Assessment Average WT (weeks): Assessment to Intervention Primary Care Adult Mental Health Older Adult MH Child & Family Learning Disabilities CAMHS Psychodynamic Psychotherapy Neuropsychology Health Psychology Forensic Psychology When compared to previous year s figures, it appears that on average waiting times from referral to assessment have continued to decrease, or at least maintain previous decreases in most services. Only Neuropsychology services have failed to decrease waiting times over the last three years, reflecting the constant demands on this limited service. Average waiting times from assessment to intervention also appear to becoming more acceptable, with decreases on last years long waiting times in Adult Mental Health, Older Adults, and Child & Family services. Only Primary Care services show an increase in waiting times. However, it must be noted that waiting times for psychodynamic psychotherapy, although less than those reported in 2005, are still just over one year from assessment to intervention. Overall, it appears that patients are waiting longer for assessment than the wait from assessment to intervention, and that the overall waiting time continues to drop. There is, however, still work to be done to reduce this further and keep waiting times to a minimum. Although the national picture on average is encouraging, it must be noted that some services are still reporting waiting times of up to 156 weeks in Adult Mental Health, 100 weeks in Primary Care and 52 weeks in Learning Disabilities and Child & Family services. These services are the exception however, and many services are reporting waiting times of less than one month across the specialities, which must be commended. Progress to reduce waiting times is slow, but it is happening and further initiatives are required for all services to be able to meet national targets. 5

58 Walker & Cate (2006) Benchmarking Respondents were asked to grade their service against the ten benchmarks set out in the DoH report (2004) using the following levels: Level 1: Not present at all Level 2: Minimal evidence of the feature Level 3: Making significant progress Level 4: Present to a notable degree Level 5: Present to an extent that warrants beacon status Table 7: Percentages of respondents scoring at each level for each benchmark (n=59) Level 1 Level 2 Level 3 Level 4 Level 5 Average Targeting those with greatest need Multidisciplinary & Multi professional Clear training strategy Single point of entry Involve Service users & carers Tiered psych network Manage WL effectively Routine audit & research Clear & defined leadership structure Well integrated into NHS The data gathered indicates that the majority of services are meeting standards in terms of being multidisciplinary & multi-professional, training strategies, leadership structure and integration. There appears to be wide variation in the ability of services to meet criteria for having a single point of entry and effectively managing waiting lists. This may also account for the variations in waiting times demonstrated earlier, and reflects the disparity in service configurations both across and within specialities. 6

59 Walker & Cate (2006) Table 8: Comparing average results from 2004 (n=59), 2005 (n=59) and 2006 (n=73) Targeting those with greatest need Multidisciplinary & multi professional Clear training strategy Single point of entry Involve Service users & carers Tiered psych network Manage WL effectively Routine audit & research Clear & defined leadership structure Well integrated into NHS As there have been no major fluctuations in average scores reported, it appears that standards achieved in previous years are being maintained, an important point as it shows that efforts to achieve the standards set out by the DoH (2004) have not diminished in the last year. With the exception of involvement of services users and carers, all the standards investigated are achieving an average rating above 3, indicating that in general, services are progressing satisfactorily towards achieving the standards set out in Organising and Delivering Psychological Therapies. Involving service users and carers has consistently been difficult to achieve with fewer than 10% of services reporting their involvement in management level service decisions and input into improving access to services. Services which were achieving these high levels of involvement came from a range of specialities including learning disabilities, addictive difficulties, forensic, health and CAMHS, indicating that service user involvement is possible regardless of the speciality. On a positive note, almost 65% of services are reporting that service users are involved in at least an advisory capacity, a notable achievement. Further work may need to be done to look at why services fail to regularly achieve the higher levels in this category and make services aware of the strategies used by those services that are able to achieve level 4 or 5 in this standard. 7

60 Walker & Cate (2006) Summary Of particular interest in this year s survey was the impact Agenda for Change and reconfiguration of services in the NHS has had on Clinical Psychology. In terms of jobs, it was somewhat surprising and pleasing to report that the number of CP posts is not starting to diminish; in contrast services were reporting on average an increase in the number of CP posts vacant, and although there had been an increase in the number of posts frozen, this was expected to decrease again over the next twelve months. Many services forecast service reconfigurations over the coming year, and although some predicted the loss of CP posts, slightly more services predicted the development of new posts or growth in existing ones. Most services are now fully assimilated to the AfC payscales, with a marked reduction in the number of outstanding appeals since the 2005 report. When taken collectively, these results paint a reassuring picture for clinical psychology for the coming months. It appears that work continues to progress towards meeting DoH targets for services, such as waiting times, gains have been made which should be commended, but equally there is still a lot of work to be done in terms of reducing waiting times and meeting other criteria for services such as the inclusion of service users and carers in service development. Perhaps services which are able to meet level five on the benchmarking section of the survey should be encouraged to use their beacon status to share their ideas and successes with other services in order to help all services achieve the DoH targets for psychological therapies (2004). The broad range of services covered in this year s survey suggests a representative sample of psychology in the NHS currently has been provided and examined. As the survey continues to grow year on year, it is hoped that more and more respondents will provide information, in order to encompass a greater number of trusts and services. We would like to thank all those who took part in this year s survey, and encourage you to continue to contribute in future years. 8

61 Walker & Cate (2006) References Department of Health (2004). Organising and Delivering Psychological Therapies. London: DoH. Walker, S., Cate, T., Gray, I., Paxton, R. & Ball, T. (2005). Organising and Delivering Psychological Therapies: How great is the challenge? Clinical Psychology, 50, Walker, S. & Cate, T. (2006). Managers Faculty annual survey Clinical Psychology Forum, 168,

62 Appendix 5. Mental Health Observatory Brief: Psychological therapy staff

63 Mental Health Observatory Briefs April 2007 Issue 1 Psychological therapy staff Key resource Wide variations in availability. Gyles Glover, Rebecca Lee and Richard Dean Introduction Psychological therapies in mental health care are effective, popular with service users and, for some types of mental illness, have been shown to have more enduring benefits than drug treatments. Recent work by the Office for National Statistics has shown that between 1993 and 2000, the proportion of people suffering with neurotic disorders receiving any drug treatment doubled from 10% to 20%, and the proportion receiving antidepressants nearly trebled (from 6% to 16%). However the proportions of those receiving psychological therapies remained constant at around 7% (3% receiving psychotherapy, 3% counselling and 1% behavioural and cognitive therapies) (Brugha et al 2004). Ten years ago, Parry (1996) showed that provision varied considerably around the country. The National Service Framework for Mental Health (DH 1999) drew attention to evidence that members of ethnic minorities had less access. In his update on the NSF Five Years On, the National Director for Mental Health announced a new programme to address the long waiting lists existing in many places for psychological therapies (Appleby 2004). This brief presents the most recent data (March 2006) from mental health service mapping about the availability of staff to provide psychological treatments for working-age adults with mental health problems. The aim is to show the extent to which this varies around England. National and new strategic health authority figures are shown here; an accompanying spreadsheet details the pattern in each region. What are psychological therapies for? Modern mental health services cannot function without adequate psychological therapy resources. Recent NICE guidelines (NICE ) identify these as key elements of treatment for anxiety and panic disorders, depression, eating disorders, obsessivecompulsive disorder, post-traumatic stress disorder and schizophrenia. For mild depression, anxiety and panic disorders and most eating disorders, psychological therapies are the principal recommended treatments. They also have a role in the treatment of bipolar disorders. For commoner conditions, simpler psychological therapies are appropriate. In some cases Cognitive Behaviour Therapy (CBT) for less complicated depression and anxiety disorders can be provided by a computer instead of a therapist. For more severe disorders, a range of specialised techniques is required. The importance and cost effectiveness of CBT specifically for depression and anxiety disorders has recently been highlighted by Layard (2004). These disorders are common and frequently impede people s ability to function at home or work. They are particularly amenable to this simple and comparatively cheap treatment. Total Primary Secondary Tertiary Other Qualified Clinical Psychologists % 69% 17% 11% Psychotherapists 544 4% 36% 55% 5% Counsellors % 54% 5% 14% Assistant Psychologists 313 4% 47% 31% 19% Graduate Workers % 19% 2% 1% Total 5116 Table 1. National numbers of staff (whole time equivalents - WTEs) providing psychological therapies, and proportions working in different types of NHS setting. How many therapists are needed? Lavender and Paxton estimated the numbers of clinical psychologists needed for local services on the basis of recommended service models and provision in a number of well functioning services. Their estimates covered primary, local secondary care, including some services that would probably often be shared between neighbouring districts, and national specialist services. Omitting a few components not relating to services for mentally-ill people, their work suggests that, for an average district, 18.8 clinical psychologists and 4.4 assistant psychologists would be required per 100,000 working age adults. Socially deprived areas might be expected to need more while more comfortable areas would need less. Assistant psychologists are commonly employed in secondary care settings. Not formally qualified in clinical psychology, they are often used to provide simple therapies under supervision and to assist service users with the operation of computer-administered CBT. In 2000, the government set up a scheme to train 1000 graduate primary care mental health workers to provide similar functions in primary care settings. In addition to this other groups, including, psychotherapists and counsellors are needed. What do the figures show? In the adult mental health service mapping of March 2006, a total of 5116 whole time equivalent (wte) staff involved in psychological therapies were reported working in units run by the NHS. Table 1 shows the numbers by broad staff group and the level of the service in which they were working. The number of qualified clinical psychologists is below half the estimated requirement. They work mainly in secondary care settings (for example community mental health teams). Graduate workers are mainly found in primary care (at GP health centres) and psychotherapists in specialist psychotherapy units. The majority of 1. A full listing of the types of service included in each of these four groups can be found on the accompanying website. 2. Population figures have been weighted (increased or decreased) to allow for the fact that mental health problems are commoner in some areas using was the AREA index. This is the system currently used by the Department of Health for allocating money for mental health care.

64 Strategic Health Authority Qualified clinical Psychologists Psychotherapists Counsellors Assistant Psychologists Graduate Workers Key questions: East of England East Midlands Yorkshire and the Humber West Midlands South East Coast North East North West South Central South West London England Table 2. Numbers of psychological therapy staff by discipline, per 100k population weighted for need (ordered by their total). counsellors reported appeared to be in secondary care settings. This is surprising as they are more usually associated with primary care work, where the numbers look implausibly low. Where counsellors are employed directly by GP practices they may be less reliably reported. About 10% of the total workforce identified, mostly clinical psychologists and counsellors, were operating in a range of other settings, from inpatient wards and acute home treatment teams to mother and baby facilities and homeless mental health services. Substantial numbers of both counsellors and psychotherapists work outside the NHS, funded through charitable sources, directly by clients or in other ways. Mapping of these is very incomplete and is therefore not reported here. Table 2 and figures 1 to 5 shows staff numbers in relation to population size. In each case the range between new strategic health authorities is considerable with the best provided areas having 35% more counsellors, 45% more qualified psychologists and 50% more psychotherapists than the national total. The least provided have 39%, 32% and 44% below respectively. There is some suggestion of substitution. London has the highest provision of qualified psychologists and more than average psychotherapists, but relatively few graduate workers. The north-west, with the highest provision of graduate workers, has the lowest number of clinical psychologists and fewer than average psychotherapists. Discussion The total number of staff available appears to be well below what is required. The variation between regions, particularly in clinical psychologist provision, is so large as to indicate that major interventions to expand the numbers of available qualified staff would be needed to achieve anything like uniform national access to adequate levels of services. In any area, it is important to provide both sufficient numbers and an appropriate mix of skill levels. Graduate workers and assistant psychologists are considerably cheaper than clinical and consultant psychologists, however they require supervision and are not capable of providing for people with more complex problems. Are the figures accurate? Data come from the annual reporting system set up to monitor the National Service Framework. Reporting requirements have been broadly stable for five years and the figures were signed-off as accurate by local NHS Trust chief executives. However, we have anecdotal reports that some services have been missed. It seems likely that this reflects limited understanding and awareness of these services by Local Implementation Team managers, itself a Service users and representative groups: Are our services adequate for our local population and how do they compare with others? Are services provided in pleasant and convenient locations? Commissioners: Is the number and range of therapists we are providing adequate to service the requirements of NICE treatment guidelines? Do we have an appropriate quota of graduate workers for an efficient skill mix, and are supervision arrangements adequate for them? Providers: Is the skill mix right? Are services provided in a range of nonstigmatising locations convenient for people's homes? Strategic Health Authorities: Do the figures suggest staff availability for our region is adequate, and -what strategic steps are we taking to ensure it is? problem. If you think the figures do not represent your area accurately, look at the detail on the website. At present this is at Select option 4, Reporting autumn 2004 and spring 2006 and look for the tables with names starting DMHB1, selecting the year Spring Click on the button to get the report and look at the row covering your area. Click on the area's name and a more detailed presentation will appear. Repeat this until you see the individual services. If the service is missing, or inaccurately reported, tell its manager and ask them to make sure it is corrected in this year's mapping which should be happening through April and May More local figures can be found on our website. In reading these, remember that some services are shared between neighbouring districts. Mental Health Observatory Wolfson Research Institute, Durham University, Queens Campus, University Boulevard, Stockton on Tees, TS17 6BH 2

65 Figures 1 to 5. Difference between regional and overall national staff provision. Bars are shown above and below the national figure. (Strategic health authorities are ranked by total provision.) 1. Qualified clinical psychologists (7.0 per 100k working age adults nationally) London South West South Central North West North East South East Coast West Midlands Yorkshire and the Humber East Midlands East of England wte staff per 100k need weighted-population 4. Assistant Psychologists (1.05 per 100k working age adults nationally) London South West South Central North West North East South East Coast West Midlands Yorkshire and the Humber East Midlands East of England wte staff per 100k need weighted-population 2. Psychotherapists (1.8 per 100k working age adults nationally) 5. Graduate Workers (1.3 per 100k working age adults nationally) London South West South Central North West North East South East Coast West Midlands Yorkshire and the Humber East Midlands East of England wte staff per 100k need weighted-population London South West South Central North West North East South East Coast West Midlands Yorkshire and the Humber East Midlands East of England wte staff per 100k need weighted-population 3. Counsellors (4.5 per 100k working age adults nationally) London South West South Central North West North East South East Coast West Midlands Yorkshire and the Humber East Midlands East of England wte staff per 100k need weighted-population For more detailed information, visit the Mental Health Observatory website References Appleby, L. (2004). The National Service Framework for Mental Health-five years on., Department of Health, London. Brugha TS ; Bebbington PE ; Singleton N ; Melzer D ; Jenkins R ; Lewis G; Farrell M ; Bhugra D ; Lee A ; Meltzer H. (2004) Trends in service use and treatment for mental disorders in adults throughout Great Britain. British Journal of Psychiatry 185: DH (1999). National service framework for mental health: modern standards and service models. London, National Health Service Executive, Department of Health. Lavender, T. and Paxton, R. (2004). Estimating the Applied Psychology Demand in Adult Mental Health. British Psychological Society. St. Andrew's House, 48 Princess Road East, Leicester. ISBN Layard, R. E. (2004). The case for psychological treatment centres., London School of Economics. NICE All the guidance quoted in this paragraph can be found online at: Parry, G. D. (1996). NHS psychotherapy services in England: A strategic review., Department of Health, London. Acknowledgements: The authors are grateful to Di Barnes and Steve Bradley of the Durham Mapping service for access to the data, and to Mike Slade and James Seward for comments. 3

66 Annex 1. Mapping service types included in the categories Primary, Secondary and Tertiary care. Table 1 distinguishes three care levels at which psychological therapies would normally be found. The categorisation of service types for these levels is set out below, showing the total number of whole time staff reported nationally. Primary care. These are services which would normally be located in the GP surgery and which, while often funded through the local PCT, can be accessed without referral through the local psychiatric services. Service Type Qualified clinical psychologists Psychotherapists Counsellors Assistant psychologists Graduate workers Primary care mental health service GP Counselling Service Graduate Primary Care Worker Secondary care. These are services to which individuals are normally referred by their GP or a social worker. Some have self referral routes, but in any case an initial assessment is usually undertaken to determine whether this level of mental health care is needed. These services would be provided in every National Service Framework local implementation team (LIT). Service Type Qualified clinical psychologists Psychotherapists Counsellors Assistant psychologists Graduate workers Psychological Therapies and Counselling Services (Statutory sector) Community Mental Health Teams Early Intervention in Psychosis Service (commonly shared between several LITs) Psychiatric Outpatient Care Mental Health Crisis Intervention Service (direct access psychological therapy services available in a few places, not to be confused with crisis resolution/home treatment teams for acute psychiatric illness) Tertiary care. These provide more specialised care, usually on the basis of referral from a secondary mental health services. Some of these services are commonly shared between two or more LITs. In all cases, psychological interventions are key components of the role. Service Type Qualified clinical psychologists Psychotherapists Counsellors Assistant psychologists Graduate workers Specialist Psychotherapy Service Other community and/or hospital professional team/specialist Assertive Outreach Team Regional medium secure unit Local Medium Secure Service Local Low Secure Service High Dependency Unit Personality Disorder Service Local Psychiatric Intensive Care Unit NHS 24-hour nurse staffed care Other settings Psychological therapy staff are also found in a wide range of other settings. All of the following included psychological therapy staff in least one location. They have been grouped together here as they are not places to which individuals would normally be referred for psychological therapies. Service Type Qualified clinical psychologists Psychotherapists Counsellors Assistant psychologists Graduate workers Acute Inpatient Unit/ Ward Advice and Information Service Residential Rehabilitation Unit Rehabilitation or Continuing Care Team Crisis Resolution Team Home/Community Support Service Self-help and Mutual Aid Group Carers Support Service NHS Day Care Facility Staff-facilitated Support Group Community Forensic Services Psychiatric liaison service Prison Psychiatric Inreach Service Mental Health Promotion Initiative Criminal Justice Liaison and Diversion Service Mother and Baby Facility Gateway Worker Carers Support Group Service User Group/Forum Advocacy Service Short-term Breaks / Respite Care Service Homeless Mental Health Service Peri-natal Mental Illness Service Crisis Accommodation

67 Appendix 6. Service change proforma

68 Appendix 6. Service change proforma

69 Appendix 7. Functional Somatic Symptoms in A and E (example of bid)

70 Executive Summary Functional Somatic Symptoms in A&E Due to the Governments recent focus on targets within A&E departments, greater scrutiny has taken place as to the type of patient who uses this NHS service. It has been concluded that a small proportion of patients attending A&E account for a large percentage of healthcare utilisation within the A&E department. This is also the case for other NHS services such as outpatient clinics and GP surgeries. Included within these figures are repeat attenders who have no medical diagnosis or physical disease. These patients make catastrophic misinterpretations of their bodily sensations, believing that they indicate a physical disease, and therefore need investigating by their GP, A&E or other hospital clinics. Such patients are referred to as having Functional Somatic Symptoms (FSS), somatisization, hypochondriasis or health anxiety. In order to determine the extent of the problem within the Hillingdon borough, the Research and Development department at the Hospital funded an A&E based study. Results showed that over a six month period, 100 patients with no medical diagnosis attended A&E 595 times. 60% of patients were admitted to hospital at least once, for an average of 3 days, with a total of 439 days spent in hospital. Over 80% of patients breached the A&E 4 hour waiting target. Attendance at GP surgeries was not routinely monitored in the study, but other studies have shown that such patients attend their GP s on average 11 times a year. This indicates in the region of 2,200 visits were made by patients with no medical diagnosis to Hillingdon GPs over the last year. The annual cost to the Hospital to assess, investigate and admit patients with FSS totals 189,854. The annual cost to the GP to assess and investigate patients with FSS totals 44,000. The total annual cost of FSS to both PCT and Hospital is 233,854 The cost effectiveness of cognitive behavioural therapy provided by clinical psychology has been proven. It is proposed that a full time clinical psychologist be employed specifically for assessing and treating patients with FSS, at a cost of 39,097 pa. Presented by: Dr Simon Dupont, Consultant Clinical Psychologist

71 Introduction By far the majority of patients attending their GP s have a diagnosable condition which is treated successfully within the practice. A minority need secondary health care due to their illness, and are referred to hospital for either in- or out-patient treatment. A third group of patients have no medical diagnosable condition yet utilise the health care system repeatedly and are referred to as having Functional Somatic Symptoms (FSS). Functional Somatic Symptoms (FSS) are bodily sensations which do not result from physical disease but which patients respond to as if they do (Sharpe et al. 1992). FSS are typically characterized as unexplained pain, non-cardiac chest pain or abdominal disturbances and are associated with high rates of physical and psychiatric illness as well as social problems (Karlsson et al. 1997). FSS are generally understood to be caused by a catastrophic misinterpretation of bodily sensations (Mayou et al. 1995). GP surgeries, A&E departments and outpatient clinics have on average 40% of patients who have FSS (Mayou & Sharpe 1997, Williams et al. 2001, Murphy et al. 1999). As a group they are heavy users of the health service and cost the NHS millions of pound every year. Recent studies have shown the efficacy of psychological treatments amongst patients with FSS, with a resultant reduction in the patients distress, health care attendance record and associated costs (Mayou & Sharpe 1997, Lidbeck 1997, Hellman et al. 1990, Warwick et al. 1996, Clark et al, 1998, Bourman & Visser 1998, Speckens et al.1995). Before any such treatment could be implemented within Hillingdon, it was important to obtain detailed information as to the extend of the FSS situation in the Borough. Present situation In order to determine the extent of the problem in Hillingdon Hospital, a study was carried out specifically within the A&E department. All patients who had attended A&E four or more times in six months were investigated to see whether a diagnosis was made, and if not what care-pathway such patients followed. Results from the study show that in the 6 month period (Oct 02 to Apr 03), 26,967 patients attended A&E, of which 350 patients attended A&E four or more times. The majority of patients (37%) were medically diagnosed, and others were excluded from the study due to; a psychiatric history (13%); a drug / alcohol problem (4.5%); pregnancy (5.5%); NFA (3%) or; had died (8%). This resulted in 100 patients who had attended A&E four or more times in six months who had not received a medical diagnosis, and who could be classified has having FSS. It is important to keep in mind that this group excluded patients with an overt psychiatric illness / psychiatric history.

72 Care pathways for patients with FSS On average, the 100 patients with FSS made 6 visits to A&E. The total number of visits during the 6 months was 595. The results of this were: 60 % of patients were admitted to hospital at least once 119 visits lead to hospital admission 449 visits lead to discharge home / GP/ did not wait 27 visits lead to outpatient appointment 83 patients breeched the 4 hour A&E waiting time target Each hospital admission lasted on average 3 days with a total of 439 days spent in hospital. In summary, during a 12 month period, 1190 A&E visits were made by 200 patients with no diagnosable condition, of whom 60% were admitted to hospital. Over of patients breeched the 4 hour A&E waiting target. Financial Implications for a full year 1190 visits to A&E, costing a minimum of 37 each 44, visits resulted in outpatient appointment, costing 56 each 3, visits resulted in hospital admission for approximately three days, costing 200 per day 142,800 Total Cost to Hospital of patients with FSS for one year 189,854 Impact of patients with FSS on GP practices Previous studies have showed that frequent attenders in A&E visit their GP on average 11 times during a year (Williams et al. 2001). Extrapolating from this data it can be assumed that in the Hillingdon borough 2,200 visits to the GP were made by patients with FSS. Using GP unit costs of 20 per consultation (PSSRU 2003), this equates to 44,000 per year excluding investigation costs. Total annual cost of FSS to both PCT and Hospital 233,854 If patients with FSS could be identified early in their trajectory through the health care system, they could be targeted for intervention to avert excessive costs and maladaptive utilisation which otherwise ensue.

73 Cost effectiveness of treatments The most effective treatment used for patients with FSS is Cognitive Behaviour Therapy (CBT) which aims to address patients thoughts, beliefs and underlying assumptions. This is particularly pertinent as FSS results from catastrophic misinterpretation of bodily sensations. Several randomised trials have shown CBT is both acceptable to patients with FSS and more effective than conventional medical care (Mayou & Sharpe 1997, Lidbeck 1997, Hellman et al. 1990, Warwick et al. 1996, Clark et al, 1998, Bourman & Visser 1998, Speckens et al.1995). CBT is a brief, structured psychological treatment offered either in a group or individual setting. On average, 8 to 10 sessions are needed, although interventions as minimal as 5 sessions are effective, with benefits sustained at 12 months (Kroenke & Swindle 2000). Smith et al. (1995) found a reduction in medical costs of over 30% if patients with medically unexplained symptoms were dealt with using more psychologically orientated treatments. Matalon et al. (2002) found the average yearly costs of frequent attenders was $4035 per person which reduced to $1161 following short term intervention. These costs included reductions in emergency room visits, hospital admissions and decreasing the number of tests performed. Clinical Psychology Proposal As part of the above study, a number of patients with FSS were invited to discuss their situation with a clinical psychologist. The majority of interviewees acknowledged their mental rather than physical state had the greater influence over their health seeking behaviour. When asked directly, 80% stated that a psychological treatment would reduce their attendance at A&E. It is proposed that a clinical psychology service be set up to assess and treat patients attending A&E with FSS. The most effective system for such a service would be in close cooperation with the health care staff at A&E. Following exclusion of other diagnoses, A&E staff could inform the proposed clinical psychologist, who could participate in a joint discussion with the patient and medical staff. This would allay any fears regarding a physical cause, and enable the patient to make a smooth transition from a physical to a psychological focus.

74 Based on the annual numbers of patients with FSS at Hillingdon Hospital, the following service would be needed: Assessment 4 patients (on average) per week: 2 hours per week Individual Treatment 200 patients per year 9 sessions per patient (on average) 1800 hours per year: 36 hours per week Taking into account the necessary liaison with A&E staff, the mental health liaison service and GP s, along with supervision and research, the total input required would amount to one full time A Grade clinical psychologist (at spine point 35) Total costs (including on-costs) 39,097 References Bouman TK, & Visser S (1998): Cognitive and behavioural treatment of hypochondriasis. Psychother Psychosom, 67: Clark DM, Salkovskis PM, Hackman A, Wells A, Fennel (1998): Two psychological treatments for hypochondriasis. A randomised controlled trial. British Journal of Psychiatry;173: Hellman CJC, Budd M, Borysenko J, McClelland DC, Benson H (1990): A study of the effectiveness of two group behavioral medicine interventions for patients with psychosomatic complaints. Behav Med, 16: Karlsson, H., Lehtinen, V. & Joukamaa, M. (1997). Frequent attender profiles: different clinical subgroups among frequent attender patients in primary care. Journal of Psychosomatic Research, Vol 42, Kroenke, K and Swindle, R (2000). Cognitive-behavioural therapy for somatization and symptom syndromes: a critical review of controlled clinical trials. Psychotherapy and Psychosomatics 69(4): Lidbeck J (1997): Group therapy for somatization disorders in general practice: Effectiveness of a short cognitive-behavioural treatment model. Acta Psychiatr Scand 96: Matalon,A., Nahmani,T., Rabin,S. and Hart,J (2002) A short-term intervention in a multidisciplinary referral clinic for primary care frequent attenders. Family Practice. Vol. 19 (3). pp Mayou, R., Bass.,C. and Sharpe. M. (1995). Treatment of functional somatic symptoms. Oxford: Oxford University Press. Mayou, R. and Sharpe, M. (1997). Treating medically unexplained physical symptoms: Effective interventions are available. British Medical Journal. Vol pp

75 Murphy, A.W., Leonard, C., Plunkett, P.K., Brazier, H., Conroy, R., Lynam, F & Bury, G. (1999). Characteristics of attenders and their attendances at an urban accident and emergency department over a one year period. Journal of Accident & Emergency Medicine, Vol (16), Personal Social Services Research Unit (2003). Unit Costs of Health and Social Care. Smith, G., Rost, K, and Kashner, T. (1995). A trial of the effect of a standardised psychiatric consultation on health outcomes and costs in somatizing patients. Archives of General Psychiatry. Vol 52. pp Sharpe, M., Peveler, R. & Mayou, R. (1992). The psychological treatment of patients with functional somatic symptoms: A practical guide. Journal of Psychosomatic Research, Vol 36 (6), Speckens, A., Van Hemert,A., Spinhoven, P., and Hawton, K (1995). Cognitive behavioural therapy for medically unexplained physical symptoms:a randomised controlled trial. British Medical Journal. Vol pp Warwick MCH, Clark DM, Cobb AM, Salkovskis PM (1996): A controlled trial of cognitive behavioural treatment of hypochondriasis. Brithish Journal of Psychiatry 169: Williams, E.R.L, Guthrie, E., Mackway-Jones, K., James, M., Tomenson, B., Eastham, J & McNally, D. (2001). Psychiatric status, somatisation, and health care utilization of frequent attenders at the emergency department. A comparison with routine attenders. Journal of Psychosomatic Research, Vol (50)

76 Appendix 8. Handling uncertainty in economic evaluations of healthcare interventions

77 Downloaded from bmj.com on 15 November 2007 Methods in health service research: Handling uncertainty in economic evaluations of healthcare interventions Andrew H Briggs and Alastair M Gray BMJ 1999;319; Updated information and services can be found at: References Rapid responses alerting service These include: This article cites 9 articles, 2 of which can be accessed free at: 8 online articles that cite this article can be accessed at: You can respond to this article at: Receive free alerts when new articles cite this article - sign up in the box at the top left of the article Notes To order reprints follow the "Request Permissions" link in the navigation box To subscribe to BMJ go to:

78 Downloaded from bmj.com on 15 November 2007 Education and debate 24 Hansson L, Lindholm LH, Niskanen L, Lanke J, Hedner T, Niklason A, et al. Effect of angiotensin-converting enzyme inhibition compared with conventional therapy on cardiovascular morbidity and mortality in hypertension: the captopril prevention project (CAPPP). Lancet 1999; 353: SHEP Cooperative Research Group. Prevention of stroke by antihypertensive drug treatment in older persons with isolated systolic hypertension. Final results of the systolic hypertension in the elderly program (SHEP). JAMA 1991;265: Thijs L, Fagard R, Lijnen P, Staessen J, Van Hoof R, Amery A. A meta-analysis of outcome trials in elderly hypertensives. J Hypertens 1992;10: Messerli FH, Grossman E, Goldbourt U. Are ß-blockers efficacious as first-line therapy for hypertension in the elderly? A systematic review. JAMA 1998;279: Medical Research Council s General Practice Research Framework. Thrombosis prevention trial: randomised trial of low-intensity oral anticoagulation with warfarin and low-dose aspirin in the primary prevention of ischaemic heart disease in men at increased risk. Lancet 1998;351: Downs JR, Clearfield M, Weis S, Whitney E, Shapiro DR, Beere PA, et al. Primary prevention of acute coronary events with lovastatin in men and women with average cholesterol levels. Results of AFCAPS/TexCAPS. JAMA 1998;279: (Accepted 11 August 1999) Appendix Material for patients x Patient information booklet: Understanding High Blood Pressure x Fact sheets: Selfhelp measures Antihypertensive drugs Blood pressure measurement Reducing dietary salt Blood pressure and kidney disease x Diet sheet: Healthy Eating Available from the British Hypertension Society Information Service, Blood Pressure Unit, St George s Hospital Medical School, Cranmer Terrace, London SW17 0RE (tel: ; fax: ; (for information service); website: Material for doctors x Blood Pressure Measurement Recommendations of the British Hypertension Society. 3rd edition, (Edited by E O Brien et al; price 4.95.) x BHS/BMJ. Recommendations for Blood Pressure Measurement. CD Rom, price Available from BMJ Publications or the BMJ Bookshop, BMA House, London WC1H 9JR (tel: ; fax: ; orders@bmjbookshop.com). x The Joint British Societies Cardiac Risk Assessor computer program and copies of the Joint British Societies coronary heart disease risk assessment chart can be downloaded from the British Hypertension Society website ( Methods in health service research Handling uncertainty in economic evaluations of healthcare interventions Andrew H Briggs, Alastair M Gray Summary points Economic evaluations are beset by uncertainty concerning methodology and data A review of 492 articles published up to December 1996 found that a fifth did not attempt any analysis to examine uncertainty Only 5% of these studies reported some measure of cost variance Closer adherence to published guidelines would greatly improve the current position Use of a methodological reference case will improve comparability The constant introduction of new health technologies, coupled with limited healthcare resources, has engendered a growing interest in economic evaluation as a way of guiding decision makers towards interventions that are likely to offer maximum health gain. In particular, cost effectiveness analyses which compare interventions in terms of the extra or incremental cost per unit of health outcome obtained have become increasingly familiar in many medical and health service journals. Considerable uncertainty exists in regard to valid economic evaluations. Firstly, several aspects of the underlying methodological framework are still being debated among health economists. Secondly, there is often considerable uncertainty surrounding the data, the assumptions that may have been used, and how to handle and express this uncertainty. In the absence of data at the patient level sensitivity analysis is commonly used; however, a number of alternative methods of sensitivity analysis exist, with different implications for the interval estimates generated (see box). Finally, there is a substantial amount of subjectivity in presenting and interpreting the results of economic evaluations. The aim of this paper is to give an overview of the handling of uncertainty in economic evaluations of healthcare interventions. 3 It examines how analysts have handled uncertainty in economic evaluation, assembled data on the distribution and variance of healthcare costs, and proposed guidelines to improve current practice. It is intended as a contribution towards the development of agreed guidelines for analysts, reviewers, editors, and decision makers. 4-7 This is the last of four articles Health Economics Research Centre, Institute of Health Sciences, University of Oxford, Oxford OX3 7LF Andrew H Briggs training fellow Alastair M Gray reader Correspondence to: A H Briggs andrew.briggs@his. ox.ac.uk Series editor: Nick Black BMJ 1999;319:635 8 BMJ VOLUME SEPTEMBER

79 Education and debate Downloaded from bmj.com on 15 November 2007 Box 1: Sensitivity analysis Sensitivity analysis involves systematically examining the influence of uncertainties in the variables and assumptions employed in an evaluation on the estimated results. It encompasses at least three alternative approaches. 1 One way sensitivity analysis systematically examines the impact of each variable in the study by varying it across a plausible range of values while holding all other variables in the analysis constant at their best estimate or baseline value. Extreme scenario analysis involves setting each variable to simultaneously take the most optimistic (pessimistic) value from the point of view of the intervention under evaluation in order to generate a best (worst) case scenario. Of course, in real life the components of an evaluation do not vary in isolation nor are they perfectly correlated, hence it is likely that one way sensitivity analysis will underestimate, and extreme scenario analysis overestimate, the uncertainty associated with the results of economic evaluation. Probabilistic sensitivity analysis, which is based on a large number of Monte Carlo simulations, examines the effect on the results of an evaluation when the underlying variables are allowed to vary simultaneously across a plausible range according to predefined distributions. These probabilistic analyses are likely to produce results that lie between the ranges implied by one way sensitivity analysis and extreme scenario analysis, and therefore may produce a more realistic estimate of uncertainty. 2 Nature of the evidence A structured review examined the methods used to handle uncertainty in the empirical literature, and this was supplemented by a review of methodological articles on the specific topic of confidence interval estimation for cost effectiveness ratios. The first step in the empirical review was a search of the literature to identify published economic evaluations that reported results in terms of cost per life year or cost per quality adjusted life year (QALY). This form of study was chosen as the results of these studies are commonly considered to be sufficiently comparable to be grouped together and reported in cost effectiveness league tables. Searches were conducted for all such studies published up to the end of 1996 using Medline, CINAHL, Econlit, Embase, the Social Science Citation Index, and the economic evaluation databases of the Centre for Reviews and Dissemination at York University and the Office of Health Economics and International Federation of Pharmaceutical Manufacturers Association. Articles identified as meeting the search criteria were reviewed by using a form designed to collect summary information on each study, including the disease area, type of intervention, nature of the data, nature of the results, study design, and the methods used to handle uncertainty. This information was entered as keywords into a database to allow interrogation and cross referencing of the database by category. This overall dataset was then used to focus on two specific areas of interest, using subsets of articles to perform more detailed reviews. Firstly, all British studies were identified and reviewed in detail, and information on the baseline results, the methods underlying those results, the range of results representing uncertainty, and the number of previously published results quoted for purposes of comparison were entered on to a relational database. By matching results by the methods used in a retrospective application of a methodological reference case (box), 5 a subset of results with improved comparability was identified, and a rank ordering of these results was then attempted. Where a range of values accompanied the baseline results, the implications of this uncertainty for the rank ordering was also examined. Secondly, all studies that reported cost data at the patient level were identified and reviewed in detail with respect to how they had reported the distribution and variance of healthcare costs. Thirdly, and in parallel with the structured review, five datasets of patient level cost data were obtained and examined to show how the healthcare costs in those data were distributed and to elucidate issues surrounding the analysis and presentation of differences in healthcare cost. Economic analyses are not simply concerned with costs, but also with effects, with the incremental cost effectiveness ratio being the outcome of interest in most economic evaluations. Unfortunately, ratio statistics pose particular problems for standard statistical methods. The review examines a number of proposed methods that have appeared in the recent literature for estimating confidence limits for cost effectiveness ratios (when patient level data are available). Findings Trends in economic evaluations A total of 492 articles published up to December 1996 were found to match the search criteria and were fully reviewed. The review found an exponential rate of increase in published economic evaluations over time and an increasing proportion reporting cost per QALY results. Analysis of the articles in terms of the method used by analysts to handle uncertainty shows that the vast majority of studies (just over 70%) used one way sensitivity analysis methods to quantify uncertainty (see box 1). Of some concern is that almost 20% of studies did not attempt any analysis to examine uncertainty, although there is weak evidence to show that this situation has improved over time. The reference case The Panel on Cost-Effectiveness in Health and Medicine, an expert committee convened by the US Public Health Service in 1993, proposed that all published cost effectiveness studies contain at least one set of results based on a standardised act of methods and conventions a reference case analysis which would aid comparability between studies. The features of this reference case were set out in detail in the panel s report. 5 The current review used this concept retrospectively, selecting for comparison a subset of results which conformed to the following conditions: An incremental analysis was undertaken; A health service perspective was employed; and Both costs and health outcomes were discounted at the UK Treasury approved rate of 6% per annum. 636 BMJ VOLUME SEPTEMBER

80 Downloaded from bmj.com on 15 November 2007 Education and debate Cost effectiveness ( ) Cost effectiveness ( ) Fig 1 Alternative rank orderings of 61 British cost effectiveness results by baseline value (above) and highest sensitivity analysis value (below) Handling of uncertainty Of the 492 studies, 60 reported results for the United Kingdom. From these, 548 baseline results were extracted for different subgroups. The importance of separate baselines for different subgroups of patients is shown in the results of an evaluation of an implantable cardioverter defibrillator where the average cost per life year saved across the whole patient group masks important differences between patients with different clinical characteristics. 8 For patients with a low ejection fraction and inducible arrhythmia that is not controlled by drugs, the cost effectiveness of the device is per year of life saved. By contrast, the use of the device in patients with high ejection fraction and inducible arrhythmia that is controlled by drugs is associated with an incremental cost effectiveness of around per year of life saved. The 548 baseline results used no fewer than 106 different methodological scenarios, and consequently a reference case methodological scenario was applied retrospectively to each article; this resulted in a total of 333 methodologically comparable baseline results. These results were converted to a common cost base year and ranked to give a comprehensive league table of results for the United Kingdom. Of the 333 results, 61 reported an associated range of high and low values to represent uncertainty. Alternative rankings based on the high or low values from this range showed that there could be considerable disruption to the ranked order based on the baseline point estimates only. This is illustrated by figure 1, which shows the rank ordering of these 61 results by their baseline values and by the highest value from their range. This analysis of UK studies reporting the ranges of sensitivity analyses raises the further concern that the median number of variables included in the sensitivity analysis was just two. Therefore, the ranges of Rank order values shown in figure 1 are likely to be less than if a comprehensive analysis of all uncertain variables had been conducted. Clearly, this would further increase the potential for the rank order to vary depending on the value chosen from the overall range. Cost data at patient level Of the 492 studies on the database, only 53 had patient level cost data and just 25 of these reported some measure of cost variance. Eleven reported only ranges, which are of limited usefulness in quantifying variance. Five articles gave a standard error, seven a standard deviation, and only four studies ( < 1%) had calculated 95% confidence intervals for cost. In the five datasets of cost at the patient level, analysis indicated that many cost data were substantially skewed in their distribution. This may cause problems for parametric statistical tests for the equality of two means. One method for dealing with this is to transform the data to an alternative scale of measurement for example by means of log, square root, or reciprocal transformations. However, our analysis of these data indicated that although a transformation may modestly improve the statistical significance of observed cost differences or may reduce the sample size requirements to detect a specified difference, it is difficult to give the results of a transformed or back transformed scale a meaningful economic interpretation, especially if we intend to use the cost information as part of a cost effectiveness ratio. It would be appropriate to use non-parametric bootstrapping to test whether the sample size of a study s cost data is sufficient for the central limit theorem to hold, and to base analyses on mean values from untransformed data. Estimating confidence intervals for cost effectiveness ratios Finally, our review identified a number of different methods for estimating confidence intervals for cost effectiveness ratios that have appeared in the recent literature, 9-14 and we applied each of these methods to one of the five datasets listed above. 15 These different methods produced very different intervals. Examination of their statistical properties and evidence from recent Monte Carlo simulation studies suggests that many of these methods may not perform well in some circumstances. The parametric method based on Fieller s theorem and the non-parametric approach of 95% confidence interval 3 Initial cost effectiveness and cost utility studies: 368 Studies reporting patient-level cost data: 41 Studies reporting some measure of cost variance: 20 Standard error 3 Standard deviation Range 6 Interquartile range 2 Fig 2 The handling of cost variance by studies reporting patient level cost data BMJ VOLUME SEPTEMBER

81 Education and debate Downloaded from bmj.com on 15 November 2007 bootstrapping have been shown to produce consistently the best results in terms of the number of times, in repeated sampling, the true population parameter is contained within the interval. Recommendations Uncertainty in economic evaluation is often handled inconsistently and unsatisfactorily. Recently published guidelines should improve this situation, but we emphasise the following: x Ensure that the potential implications of uncertainty for the results are considered in all analyses; x When reporting cost and cost effectiveness information, make more use of descriptive statistics. Interval estimates should accompany each point estimate presented; x Sensitivity analyses should be comprehensive in their inclusion of all variables; x Cost and cost effectiveness data are often skewed. Significance tests may be more powerful on a transformed scale, but confidence interval should be reported on the original scale. Even when data are skewed, economic analyses should be based on means of distributions; x Where patient level data on both cost and effect are available, the parametric approach based on Fieller s theorem or the non-parametric approach of bootstrapping should be used to estimate a confidence interval for the cost effectiveness ratio; x When comparing results between studies, ensure that they are representative; x Using a methodological reference case when presenting results will increase the comparability of results between studies. This article is adapted from Health Services Research Methods: A Guide to Best Practice, edited by Nick Black, John Brazier, Ray Fitzpatrick, and Barnaby Reeves, published by BMJ Books. Competing interests: None declared. 1 Briggs AH. Handling uncertainty in the results of economic evaluation. London: Office of Health Economics,1995. (OHE briefing paper No 32.) 2 Manning WG, Fryback DG, Weinstein MC. Reflecting uncertainty in costeffectiveness analysis. In: Gold MR, Siegel JE, Russell LB, Weinstein MC, eds. Cost-effectiveness in health and medicine. New York: Oxford University Press, 1996: Briggs AH, Gray AM. Handling uncertainty when performing economic evaluations of health care interventions: a systematic review with special reference to the variance and distributional form of cost data. Health Technol Assess 1999;3(2). 4 Drummond MF, Jefferson TO. Guidelines for authors and peer reviewers of economic submissions to the BMJ. BMJ 1996;313: Gold MR, Siegel JE, Russell LB, Weinstein MC, eds. Cost-effectiveness in health and medicine. New York: Oxford University Press, Canadian Coordinating Office for Health Technology Assessment. Guidelines for the economic evaluation of pharmaceuticals: Canada. 2nd ed. Ottawa: CCOHTA, Drummond MF, O Brien B, Stoddart GL, Torrance GW. Methods for the economic evaluation of health care programmes. 2nd ed. Oxford: Oxford University Press, Anderson MH, Camm AJ. Implications for present and future applications of the implantable cardioverter-defibrillator resulting from the use of a simple model of cost efficacy. Br Heart J 1993;69: O Brien BJ, Drummond MF, Labelle RJ, Willan A. In search of power and significance: issues in the design and analysis of stochastic costeffectiveness studies in health care. Med Care 1994;32: Wakker P, Klaassen M. Confidence intervals for cost-effectiveness ratios. Health Econ 1995;4: Van Hout BA, Al MJ, Gordon GS, Rutten FF. Costs, effects and C/E-ratios alongside a clinical trial. Health Econ 1994;3: Chaudhary MA, Stearns SC. Estimating confidence intervals for cost-effectiveness ratios: an example from a randomized trial. Stat Med 1996;15: Briggs AH, Wonderling DE, Mooney CZ. Pulling cost-effectiveness analysis up by its bootstraps: a non-parametric approach to confidence interval estimation. Health Econ 1997;6: Polsky D, Glick HA, Willke R, Schulman K. Confidence intervals for costeffectiveness ratios: a comparison of four methods. Health Econ 1997;6: Fenn P, McGuire A, Phillips V, Backhouse M, Jones D. The analysis of censored treatment cost data in economic evaluation. Med Care 1995;33: Briggs AH, Mooney CZ, Wonderling DE. Constructing confidence intervals around cost-effectiveness ratios: an evaluation of parametric and non-parametric methods using Monte Carlo simulation. Stat Med (in press). How the defibrillator saved a patient s life Initially it was quite a struggle just getting the partners to agree that purchasing a defibrillator would benefit the practice. We did not even have to pay as the Friends of the Health Centre kindly raised the money. The equipment was installed in the nurses treatment room and gradually gathered dust. Does the defibrillator work? and I bet the batteries aren t charged were some of the jocular comments from the partners. We had a couple of attempts at resuscitation, the equipment worked well, but unfortunately the patient did not survive. It was decided to hold a training day on resuscitation for the nurses. The alarms sounded, I rushed to the treatment room only to find that it was a mock emergency. In the middle of a busy afternoon surgery the same day the alarm went off again and there was an urgent telephone call. When I arrived several partners and nursing staff were in the middle of full cardiopulmonary resuscitation. The patient had been sent down from the doctor s surgery to the treatment room for an electrocardiogram as he had chest pain and had collapsed. The tracing showed ventricular fibrillation. Bring out the defibrillator! Charge to 200 deliver shock! It s just like ER! Unfortunately, the patient was unstable; there were further episodes of ventricular fibrillation and further defibrillation. As a former medical registrar it started to flood back. We need lignocaine, but what is the dose? It was like the blind leading the blind. Four cardioversions later the ambulance arrived. Was he stable enough to transfer to our local hospital? It was decided that I should accompany the patient in the ambulance; this was just as well as he had two further arrests in the ambulance requiring defibrillation. An emergency stop as a bus pulled out in front of us hurled the patient forward into my lap. But he survived, and as he was only 40 with two children he was eternally grateful. What have we learnt? Clearly, we need more training in resuscitation. We now have a very persuasive argument for the partner who said that we did not need a defibrillator as the ambulance always carries one. Our Friends of the Health Centre are now saving to buy us a better model that can record the cardiac rhythm through the paddles. Alexander Williams, general practitioner, Exeter We welcome articles up to 600 words on topics such as A memorable patient, A paper that changed my practice, My most unfortunate mistake, or any other piece conveying instruction, pathos, or humour. If possible the article should be supplied on a disk. Permission is needed from the patient or a relative if an identifiable patient is referred to. We also welcome contributions for Endpieces, consisting of quotations of up to 80 words (but most are considerably shorter) from any source, ancient or modern, which have appealed to the reader. 638 BMJ VOLUME SEPTEMBER

82 Appendix 9. Examples of Excellence, achievement and innovation in the psychology services of NHS Argyll and Clyde

83 Examples of Excellence, Achievement and Innovation in the Psychology Services of NHS Argyll & Clyde Services to Adults with Mental Health Problems Dissemination of Psychological Knowledge and Skills Senior psychologists in Renfrewshire have been co-founders, with other professional colleagues, of the now Argyll and Clyde wide Psychological Therapies and Interventions Group (PTIG). This is a multidisciplinary group of senior mental health specialists in psychological therapies. The group s remit is to support the development and clinical governance of enhanced skills in psychological therapies and psychosocial interventions among other staff. There have been a number of initiatives that psychologists in Renfrewshire have led, or co-led with other colleagues, aimed at disseminating knowledge of psychological therapies among other mental health disciplines. For instance, for several years the Adult Psychology Department in Renfrewshire has worked closely with colleagues from the Psychotherapy Department at Dykebar Hospital to run a multi-disciplinary psychological therapies journal club/ case discussion forum on a monthly basis. These two services have also jointly staged a number of training seminars for a range of disciplines, including junior and consultant psychiatrists. Moreover, the departments also co-operate closely in the provision of supervision in psychological therapies for SHOs and SPRs in psychiatry. More recently, the Inverclyde Psychology Department has started hosting a regular Psychological Therapy Journal Club since April This has become an effective multi-professional forum for discussions on issues related to psychological interventions and therapies. Contributions to Multi-Disciplinary Working in AMH Over the past four years the consultant clinical psychologist in the Inverclyde Community Mental Health and Resource Team has made important contributions to the management of the Team as well as to its clinical services. Besides playing a key role as a senior member of the Team Management Co-ordinating Group, she has overall responsibility for the groupwork of the Team. Argyll psychologists have also, over time, played leading roles in facilitating multidisciplinary service initiatives. These have included an art therapy project, an improved care pathway for mothers with post-natal depression and new approaches to the delivery of psychotherapies across Argyll and Bute. Innovative Practices in AMH Clinical Work An example of innovative clinical practice is the work of the Inverclyde CMHRT psychologist in developing and auditing a psychosocial family intervention group

84 programme for patients suffering from schizophrenia. This ongoing programme has proved to be popular with families and has become an important resource for the training of team members in psychological approaches. In Argyll, the nature of the extensive rural geography has led clinical psychologists to alter their practice by incorporating occasional telemedicine exchanges with patients on the islands. More commonly, they have employed extended telephone follow-up patient consultations to monitor patient progress, ensure adherence to treatment plans and resolve difficulties with the latter. Services to People with Learning Disabilities Service-Related Establishment of Autistic Spectrum Disorders (ASD) clinical psychology service extending between child and adult services and including adults with autism and Asperger s syndrome. As well as facilitating more integrative approaches with mainstream services, this innovative service refocused attention upon specialist services that extend through the Learning Disability boundary to include non-ld patients. Set up NHS Argyll & Clyde wide LD clinical psychology community forensic service. This enabled specialist service input to this group, particularly in the area of risk assessment and court advisement. It simultaneously took a considerable pressure off the generic services, which previously dealt with these referrals. The Head of LD Psychology undertook a lead role in piloting the initial joint Community Learning Disability Team between NHS and Renfrewshire Social Work and thereafter in establishing the three current CLDTs. This included a research project that led to a publication comparing the community services in Renfrewshire with those in Lothian. Clinical Developed Self-Report Depression Scale for adults with mild and moderate learning disability that is now in use internationally. Completed assessments of the Quality of Life of long stay LD, MH and Elderly inpatients across Renfrewshire and Inverclyde. This major clinical project, undertaken in 2003, prepared the way for improved care standards for inpatients that could also be implemented in a cost-effective way.

85 Services to Children & Adolescents In the late 1990s Child and Adolescent Psychology Services in Renfrewshire and Inverclyde were at the forefront of developing the Autistic Spectrum Disorder Assessment Service. Until recently this was a uniquely psychology service but has now become a multi-disciplinary service that psychology continues to contribute to as a key discipline. In recent years C&A Psychology staff have established close liaison links with paediatric services and psychology continues to be the only mental health discipline to provide regular input to these services. In particular, staff have regular ongoing involvement with the Cystic Fibrosis Team and the Diabetes Team. In 2001 clinical psychologists became the first staff in the CAMHS to gain a recognised qualification in family therapy and subsequently developed a Family Therapy assessment and treatment service that continues to this day. In association with this, the lead consultant psychologist offers clinical supervision to trainees and to those from other disciplines who wish to develop their skills in this area of clinical work. Neuropsychological Services Inverclyde Over the past 4 years, colleagues in Inverclyde have successfully developed Ravenscraig Neuropsychology into a well-respected specialist service in high demand locally. Neuropsychology input to both Adult Mental Health, as well as Older People Mental Health, has proven to be invaluable to these two specialties. Medical and psychiatric colleagues enormously value the expert diagnostic skills of the local consultant clinical neuropsychologist, particularly in relation to assessment of very complex neurological or neuropsychiatric cases. Another aspect of the achievement of the Inverclyde Neuropsychology Service is the major research study conducted by the consultant neuropsychologist into the population of head-injured patients in Inverclyde. This ongoing project has yielded very significant results which are likely to have important implications for service development, not only locally but also area-wide and possibly nationally. Lomond and Argyll Another consultant clinical psychologist specialising in neuropsychology from the Lomond Psychology Department has played a leading role in establishing and developing the Lomond and West Dunbartonshire Brain Injury Team. This service was nominated for, and reached the finals of, the NHS/Daily Record Scottish Health Awards of This is a Joint Futures service that is jointly funded by NHSGG, NHSA&C, West Dunbartonshire Council and Argyll & Bute Council. Related to the above, a new post of assistant psychologist, to provide a variety of functions (assessment, designing interventions, research and training) within the areas of traumatic brain injury and alcohol related brain damage, has just been established. This unique post will be hosted within Social Services and will

86 provide a stepping stone for psychology graduates seeking entry into doctoral courses in clinical psychology. The consultant clinical neuropsychologist from the Lomond Department enjoyed recent success in obtaining, jointly with a consultant psychiatrist colleague, a CSO research grant to conduct a RCT investigating the value of aromatherapy in the management of disruptive behaviour among elderly patients with dementia. The results indicated significant effects and these were published in two articles in respected international peer reviewed journals, one of which is cited in the Cochrane Review. Outline permission has also been given for further funding for a multi-centre trial. Physical Health Psychology A unique initiative within NHS A&C that has the potential to positively influence CHD healthcare across NHS Scotland is the Heart in Mind project, led by a clinical psychologist attached to the Cardiac Rehabilitation team in the Royal Alexandra Hospital, Paisley. This project is jointly sponsored and supported by Health Scotland and Have-a-Heart Paisley. It entails the development and evaluation of a training programme and associated training materials to teach all professional groups with a role in cardiac care to incorporate psychological understanding, assessment and interventions in their day-to-day work. One of the Argyll psychologists has also recently been involved in an innovative project providing psychology input to a Rehabilitation Group for patients with Chronic Obstructive Pulmonary Disease (COPD). A physiotherapist and nurse specialist set up this currently time-limited service. Funding is being sought to continue to provide the service.

87 Appendix 10. Briefing about the economic impact of Early Intervention Services Worcestershire Early Intervention Service Evaluation Summary

88 Worcestershire Early Intervention Service Evaluation Summary October 2006 The early intervention service in Worcestershire provides support and care for 14 to 35 year olds who have experienced a first episode of psychosis within the past year and who live within the County of Worcestershire. The service also carefully monitors those with suspected psychosis. The service offers open access. Individuals or family members can self refer for advice and support. Inappropriate referrals are signposted on to other more appropriate agencies. The service offers recovery based interventions to individuals and families in their home or a place of their choice. Support may include information about psychosis, medication and specialist psychological support and counselling for the young person and their family. The service also offers regular fun social and leisure activities to boost confidence and encourage individuals to mix and socialise. The service links with a range of statutory and non statutory agencies including Connexions, youth services, drug and alcohol services, housing, local schools and colleges, and work projects to assist the individual in returning to education or work and in addressing other needs. The service works closely with their three local CAMHS teams, has a strong transition protocol and has a jointly appointed dedicated worker to support the under 18 years. Since 1 st October 2006, the team also links with a Countywide research driven early detection team (EDIE2) offering Cognitive Behavioural Therapy intervention to individuals who are deemed either at high risk or who are suspected first episode cases. The team receives regular positive feedback from individuals and families via annual EI service satisfaction audit questionnaire sent to individuals and their carers. Recent comments include: The support, care, help, guidance and counselling has been excellent for my son, my wife and myself in understanding this health condition We feel that the care (our relative) has received from the EIS has been excellent and we can find no fault at all in it, only professional, quality care from everyone The service has been absolutely brilliant- a life saver for both of us.the team is wonderful and absolutely essential I am highly impressed and extremely grateful for the treatment and care my son has received from the EIS. Their care has been second to none The service has been successful in engaging individuals with the service and in reducing the need for hospital admission, use of the Mental Health Act, substance use and suicide attempts while also helping many individuals return to school, college or work or pursue plans to travel or move to independent accommodation. Formal evaluation data from service audit of case notes and medical records and routine user and carer satisfaction returns are reported below together with appropriate comparative data where this is available.

89 Impact on service delivery Total number of patients contributing to data Total never having a hospital admission Percentage admitted to hospital for first episode Percentage admitted for first episode using MHA Hospital admissions for further episodes Percentage admissions using MHA for further episodes Percentage engaged with services Percentage with case manager allocated Percentage aware of crisis contact details / arrangements National Available research and audit data Worcestershire Pre EIS Baseline Audit Data 2000 (based on 3 year cohort data from clients 15 carers Worcestershire EIS Audit Data 2006 (based on 3 year cohort data from ) 77 20% 13% 42.7% (n = 75) 80% 73% 41.4% (Prior to or as entry point to EIS, n = 75) 50% 59% 27% (Prior to or as entry point to EIS, n = 74) N/A 50% 30% 2 episodes 20% 3+ episodes 27.6% 6.6% 2 admissions 3.9% 3 admissions (n = 76) N/A N/A 9.5% (n = 74) 50% (at 12 months) 97% 100% Mean engagement rating 39 (scores above 33 indicate good engagement) 79.2% well engaged 20.8% poorly engaged (n = 77) N/A 33% 100% (n = 77) N/A 63% 90% service users 91.7% carers (n= 22)

90 Care planning Family involvement Medication and treatment Information about psychosis 20% aware of CPA 13% had seen copy of their care plan 60% involved in planning care 49% carers given management advice 56% satisfied with help received 62% felt listened to and understood 40% given information about medication and 50% about potential side effects 35% involved in decisions about medication 67% reported given little or no information about psychosis 100% users and carers aware of CPA 100% users and 91.7% relatives had seen copy of care plan 80% users and 66.7% carers involved in planning care (n = 22) 100% offered family support or formal family intervention 71.4% satisfied with help received 91.7% felt case manager had discussed and understood their views (n = 22) 90% given information about medication and 100% about potential side effects 85.7% involved in decisions about medication (n = 22) 70% users and 75% carers reported having had enough information about psychosis (n =22)

91 DUP Social activities Mean range DUP: months 52% satisfied with social activities organised by the service Median DUP 2004: 5 months 2005: 6 months 2006: TBC 83.3% users and 87.5% relatives satisfied with social activities provided by EI (n = 22) Impact on outcomes National audit data Worcs EIS 2006 Unemployed 45.7% (n = 70) Employed 8-18% (rates for people with SMI-all people with MH problems) 25.7% (n = 70) Education 28.6% (n = 70) Satisfaction with help to find/return to education/employment 10% satisfied with help in finding employment (pre EIS audit) 71.4% users and 77.8% carers satisfied with help offered (n = 22) Completed suicides 10% 0% (n = 77) Attempted suicides 48% 21.9% (n = 73) Homelessness 13.3% 86.6% had no days homeless (n = 70) Cannabis Usage On entry to EIS: 49.3% none 20% daily Overall service satisfaction 55% users and 56% carers mostly/extremely satisfied (Worcs pre EIS audit data 2000): Current usage: 64.3% none 8.6% daily (n = 70) 100% users and carers mostly/extremely satisfied (n = 22)

92 Discharge outcomes (n=19 to date) Discharge destination % GP 36.8% CMHT 10.5% Recovery service 5.2% AO 10.5% Private hospital 5.2% Moved out of area (eg. returned to University or family home) 31.5% Evaluation summary To date, the impressive reduction in hospital admissions (first and subsequent), use of the Mental Health Act, suicide rates and cannabis usage, the high levels of engagement (with 0% of users lost to follow up) combined with high levels of user and carer satisfaction with the care and interventions received and 54.3% employment / occupation rates for service users demonstrates considerable value for money. An annual audit cycle has been set in place to continually listen to and learn from users and carers of the service to address unmet needs and identify more effective ways of helping to deliver early intervention and further improve the service Acknowledgements Thanks are due to Liam Atwal, Faiza Anwar and Helen Lowe, Psychology assistants and Emma Cotes, Clinical Psychologist who collected and analysed the data that formed the basis for this evaluation summary and all the Early Intervention staff team for the support and information they provided in the course of data collection. Jo Smith, Worcestershire Early Intervention Clinical Development Lead Tony Gillam, Early intervention Team Manager October 2006 Contact details: Jo Smith, Early Intervention Clinical Development Lead ( joda@lineone.net) or Tony Gillam, Early Intervention Team Manager ( tony.gillam@worcs-mht.nhs.uk) Worcestershire Early Intervention Service 1, Britannia Court, Moor Street, Worcester, Worcestershire WR1 3DB Tel:

93 Appendix 11. What added value can clinical psychology bring to a specialist mental health trust?

94 Subject: Added Value of Clinical Psychology doc. Dear Tim A number of clinical psychology colleagues and I have put together a document (attached) to help inform our efforts to influence and enable senior managers and commissioners to utilise clinical psychology skills more effectively. The main impetus for this initiative has, of course, been the significant challenges facing the profession both locally and nationally. I am aware that the Division of Clinical Psychology is currently developing a marketing strategy and I hope our local document may be of some interest in this regard. Although the contents of the document are largely structured around the local FT business plan, I suspect that the arguments made are of relevance to other mental health trusts. Please feel free to use any aspect of the document as part of the national initiative to "sell" clinical psychology skills. Best wishes Gary L. Sidley Professional Lead for Psychology (Salford) Bolton, Salford & Trafford NHS Mental Health Trust

95 WHAT ADDED VALUE CAN CLINICAL PSYCHOLOGY BRING TO A SPECIALIST MENTAL HEALTH TRUST? June

96 1. Purpose of this document? The primary aim of this document is to increase awareness among senior managers and commissioners of the range of contributions clinical psychologists can make towards delivery of a high quality, specialist mental health service. It is hoped the information will be helpful and timely, particularly in light of the imminent transition to Foundation Trust status. 2. What is a clinical psychologist? A clinical psychologist is an expert in applying the science of psychology to health and social care settings. Psychology focuses on the factors that influence how people (service-users, staff, carers) feel, think and behave. As such, it is a very versatile science that is ideally suited to addressing the wide range of people problems faced by a large mental health Trust, with regards to both service delivery and organisational issues. The core training of a clinical psychologist is lengthy and intensive. A newly qualified clinical psychologist will have successfully completed a psychology degree (3 years), undertaken relevant experience as an assistant psychologist (typically at least 2 years), and then gone on to achieve a post graduate degree in clinical psychology (3 years) incorporating both intensive academic study and a wide range of supervised clinical placements. Thus, the minimum period required to achieve the core qualification, a Doctorate, is 8 years of full time study and training, an entry route that is comparable in length and depth to that of medical practitioners. 3. Current context? Despite a range of national policy directives indicating the need for expanding the availability of psychological interventions (e.g. Mental Health Ten Years On: Progress on Mental Health Care Reform, Appleby, 2007), the clinical psychology profession within the Trust has experienced significant loss of resource over the last couple of years. Furthermore, ongoing pressures continue in the form of year-on-year Cost Improvement Programmes. Recently the Agenda for Change (AfC) process, although broadly re-affirming the status quo for clinical psychology salaries, has resulted in the perception that the going rate for clinical psychology is relatively expensive (particularly as medical colleagues were excluded from the AfC review). Collectively these influences may result in further reductions in clinical psychology posts being seen as convenient and effective short-term cost-saving measures. Under these circumstances it is important and timely to highlight the likely longer-term detrimental effects upon the quality and productivity of mental health service provision of further erosion of the specialist clinical psychology resource. What follows is a summary of the main distinctive added value contributions that clinical psychology can make within a specialist mental health Trust. To aid clarity, it will be structured around the stated corporate objectives and intentions. 3

97 WHAT DOES CLINICAL PSYCHOLOGY ADD TO MENTAL HEALTH SERVICES? Theme Corporate objective/intention Clinical Psychology skills that will help meet this objective/intention Relevant examples of application within a specialist mental health service Existing examples of relevant Clinical Psychology-led innovation within the Trust Risks of not having sufficient clinical psychology expertise within the Trust C O M P L E X I T Y To focus core business on COMPLEX MENTAL HEALTH NEEDS (Foundation Trust Integrated Business Plan, 2007) To emphasise EXPERTISE AND CLINICAL CAPABILTIY as basis for market expansion (Foundation Trust Integrated Business Plan, 2007) To comply with PERSONALITY DISORDERS NO LONGER A DIAGNOSIS OF EXCLUSION (NIMHE, 2003) Formulation of complex problems by drawing flexibly on a range of psychological models and theories (an independent & comprehensive review of clinical psychology services in Britain, [MAS, 1988] concluded that clinical psychologists were the only professional group able to perform this role) 1. Individual therapy with complex/ hard to treat presentations/ability to work off- piste with problems that typically do not respond to firstline medical and talking therapy intervention; 2. Providing framework for guiding multidisciplinary team communication and corresponding intervention /management; 3. Providing framework for positive risk-taking/ risk-management; 4. Neuropsychological assessment. RAPID PSYCHOLOGICAL ASSESSMENT & FORMULATION CLINIC WITHIN SALFORD S COMMUNITY MENTAL HEALTH TEAMS EFFECTIVE WORKING WITH CLIENTS WITH SIGNIFICANT LEVELS OF PSYCHOLOGICAL, PHYSICAL AND SOCIAL CO- MORBIDITY (SALFORD OLDER PEOPLE PRIMARY CARE PSYCHOLOGY) BOLTON S 174 DAY SERVICE FOR PEOPLE WITH PERSONALITY DISORDERS 1. Ineffective services due to over-reliance on firstline interventions; 2. Failure to win contracts for working with severe/complex metal health problems; 3. Litigation on basis of clinical governance deficiencies.

98 Theme Corporate objective/intention Clinical Psychology skills that will help meet this objective/intention Relevant examples of application within a specialist mental health service Existing examples of relevant Clinical Psychology-led innovation within the Trust Risks of not having sufficient clinical psychology expertise within the Trust H U M A N I S I N G To promote RECOVERY, SOCIAL INCLUSION & REDUCE INEQUALITIES (A strategic intention within the Trust s Annual Report, 2006; Mental Health & Social Exclusion, Social Exclusion Unit 2004); BST Care Programme Approach, 2006; Everybody s Business, CSIP, 2005) Normalising framework for embedding people s problems (psychology is fundamentally about the science of normal behaviour and emotion as such there is an inherent expectation that everyone can change, and that the mechanisms for change for people with mental health problems will be no different from those that influence each and every human being) Depth and breadth of knowledge/expertise to challenge traditional, overly biological models of servicedelivery (typically clinical psychologists are the lone voice in raising concerns about such practices) Understanding of interpersonal dynamics and the inherent power imbalance in service-user/mental health worker interactions 1. Non-stigmatising service provision; 2. Empowering service-users; 3. A service that does not exclude people on the basis of dubious and unreliable psychiatric diagnoses; 4. Minimising prejudice/ promoting equity through compassionate formulations of people s problems; POSITIVE PSYCHOLOGY & MENTAL HEALTH WORKSHOPS IN COLLABORATION WITH BOLTON S PATIENT S COUNCIL (funding acquired from the PCT to support 10 further workshops) THE IMPACT & EDIT SERVICES (precursors to the early intervention service) OFFERING NORMALISING WAYS OF MAKING SENSE OF PSYCHOTIC EXPERIENCES THE RECOVERY ORIENTATED MODEL OF CARE WITHIN THE HIGH DEPENDENCY UNIT (A DIRECT CONSEQUENCE OF A CLINICAL PSYCHOLOGY- FACILITATED AWAY DAY) 1. A less humanistic service treating service-users as illnesses rather than as people. 2. Arbitrary exclusion of some high risk groups with complex problems. 3. Services that stigmatise rather than enable. 5

99 Theme Corporate objective/intention Clinical Psychology skills that will help meet this objective/intention Relevant examples of application within a specialist mental health service Existing examples of relevant Clinical Psychology-led innovation within the Trust Risks of not having sufficient clinical psychology expertise within the Trust I N N O V A T I O N Income generation from SERVICE EXPANSION/ DEVELOPMENT Continuous SERVICE IMPROVEMENT A FOCUS ON QUALITY as much as quantity Income generation through HIGH PROFILE RESEARCHERS (All highlighted within the Foundation Trust Integrated Business Plan, 2007) Research expertise The core training uniquely equips all clinical psychologists with a research doctorate 1. Service-innovation; 2. Service evaluation and the development of measurable outcomes; 3. Evidence-based practice THE IMPACT TEAMS (WITHIN TRAFFORD & SALFORD) AS PRECURSORS TO EARLY INTERVENTION SERVICES [commended by the Health Care Commission] CLINICAL AUDIT OF OLDER PEOPLE S PRIMARY CARE PSYCHOLOGY SERVICE SERVICE-USER/ EMPLOYEE RESEARCHER ROLES LINKED WITH THE EARLY INTERVENTION SERVICE A sterile service, lacking innovatory ideas and therefore unable to compete within the mental health market THE DEVELOPMENT OF THE EDIT SERVICE FOLLOWING RESEARCH TO DEMONSTRATE THAT PSYCHOLOGICAL INTERVENTION CAN PREVENT SOME HIGH RISK INDIVIDUALS 6

100 I N N O V A T I O N (Cont d) FROM BECOMING PSYCHOTIC THE DEVELOPMENT AND RAPID EXPANSION OF THE PRIMARY CARE PSYCHOLOGY SERVICES IN SALFORD AND TRAFFORD (services that closely resemble those currently being recommended by Lord Layard THE COGNTIVE THERAPY TRAINING CENTRE (SALFORD) (led by a clinical psychologist and potentially a major source of Trust income) PERINATAL PROJECT (SALFORD PRIMARY CARE PSYCHOLOGY) recently achieved award for Innovative Service Delivery 7

101 Theme Corporate objective/intention Clinical Psychology skills that will help meet this objective/intention Relevant examples of application within a specialist mental health service Existing examples of relevant Clinical Psychology-led innovation within the Trust Risks of not having sufficient clinical psychology expertise within the Trust S K I L L I N G O T H E R S To improve ACCESS TO PSYCHOLOGICAL THERAPIES (A commissioning priority, Organising and Delivering Psychological Therapies, 2004; We Need to Talk, 2006) Expert therapists/ changeagents 1. Supervision of a range of other staff groups in the appropriate provision of psychological interventions; 2. In-house training of a range of other staff in appropriate levels of psychological intervention; 3. Developing, supporting and informing steppedcare models of provision to ensure compliance with governance and effectiveness agendas ASYLUM SEEKERS SERVICE in collaboration with the Horizon Centre GP practice. A NOVEL TRAINING PROGRAMME FOR MULTI-DISCIPLINARY STAFF WITHIN REHABILITATION SERVICES INCORPORATING MOTIVATIONAL APPROACHES 1. Little or no access to specialist psychological services; 2. Psychological interventions offered by inadequately trained/supervised staff leading to ineffective delivery and risk of litigation 8

102 Theme Corporate objective/intention Clinical Psychology skills that will help meet this objective/intention Relevant examples of application within a specialist mental health service Existing examples of relevant Clinical Psychology-led innovation within the Trust Risks of not having sufficient clinical psychology expertise within the Trust V E R S A T I L I T Y Competitive advantage from FLEXIBILITY OF SKILLS (Foundation Trust Integrated Business Plan, 2007) Unique core competency to work with any age group or speciality 1. Ability to work (on basis of core training) across traditional service boundaries IMPACT SERVICE/ EARLY INTERVENTION SERVICES WORKING WITH YEARS AGE GROUP CLINICAL PSYCHOLOGISTS HOLDING JOINT POSTS ACROSS PRIMARY AND SECONDARY CARE CLINICAL PSYCHOLOGISTS HOLDING JOINT POSTS ACROSS ADULTS OF WORKING AGE AND OLDER ADULTS 1. Lack of joined-up services with impermeable interfaces; 2. Unhelpful competitiveness between specialities. 9

103 P S Y C H O L O G I C A L L Y Theme I N F O R M E D S E R V I C E S Corporate objective/intention Each service user to expect that EVERY INTERACTION IS THERAPEUTIC (Strategic intention within the Trust s Annual Report, 2006) Clinical Psychology skills that will help meet this objective/intention Expertise in non-drug interventions as a result of extensive/comprehensive core training, allowing the flexible use of a range of models of human development and social psychology to inform the moment by moment management and interactions with service users. Relevant examples of application within a specialist mental health service 1. Clinical leadership of multi-disciplinary teams; 2. Formulation sharing with other professions (see above); 3. Clinical supervision of other staff (see above); 4. In-house training of other staff; Existing examples of relevant Clinical Psychology-led innovation within the Trust CLINICAL SUPERVISION TO STAFF ON THE CONDITIONS MANAGEMENT PROGRAMME (RECENT SUCCESSFUL BID IN PARTNERSHIP WITH THE DEPARTMENT OF WORKS & PENSIONS) PROMOTING PSYCHOLOGICAL APPROACHES TO THE FEAR OF FALLING IN A PHYSICAL HEALTH CARE TEAM (PSIGE Newsletter, No. 96, November 2006) CLINICAL SUPERVISION OF NON-PSYCHOLOGISTS WITHIN SALFORD S COMMUNITY MENTAL HEALTH TEAMS Risks of not having sufficient clinical psychology expertise within the Trust 1. Residential/inpatient settings that provide little more than custodial care; 2. Poor quality milieus that lack therapeutic sophistication, dominated by exclusively medication and support models of service provision; 3. Inconsistent practices in the management of service-users. 10

104 Theme Corporate objective/intention Clinical Psychology skills that will help meet this objective/intention Relevant examples of application within a specialist mental health service Existing examples of relevant Clinical Psychology-led innovation within the Trust Risks of not having sufficient clinical psychology expertise within the Trust M O T I V A T I O N To nurture and maintain STAFF MOTIVATION (Strategic intention within the Trust s Annual Report, 2006) Expert skill/knowledge around the factors that motivate people High level communication skills so as to be able to match the style of communication to the target group Expertise in group dynamics Expertise as change enablers 1. Team-building initiatives; 2. Conflict resolution; 3. Provision of staff support groups. 4. Provision of stress management programmes (for teams and individuals). NORTH WEST MULTI- PROFESSIONAL SHA MENTOR SCHEME NORTH WEST CLINICAL PSYCHOLOGY CPD SCHEME COMMUNITY GROUP FACILITATORS SUPERVISION FORUM WITHIN THE HIGH DEPENDENCY SERVICE 1. High levels of staff sickness and absenteeism; 2. High staff turn-over, with associated high recruitment costs; 3. An unhappy, unproductive workforce; 4. High cost of commissioning external experts to provide stress management courses. TRAINING INITIATIVES FOR CPNs/ OTs/SUPPORT WORKERS WITHIN SALFORD SERVICES EXPLICIT PROCESS FOR STAFF SUPPORT FOLLOWING SERIOUS INCIDENTS ON THE MEADOWBROOK INPATIENT UNIT 11

105 Appendix 12. Psychological services in the NHS

106 Psychological Services in the NHS 1. The extent of need for psychological services? 9% NHS expenditure mental health problems. 4-13% population suffers from mental health problems at any point in time. 10% of people with mental health problems suffer from psychotic disorders, another 10% suffer from dementia. The remaining 80% suffer from anxiety, depression and other acute or chronic problems. This group includes personality disorders, eating disorders, harmful drinking and drug taking. In a substantial number of cases psychological problems are in the context of physical disorders, requiring liaison work with physicians and surgeons. On average GPs detect 6 out of 10 cases of mental health problems and 1 in 10 of these are referred to specialist services. Following major disasters incidence of severe psychological morbidity increases sharply for at least 2 years. The need for psychological services need to take into account sociodemographic features including: o o Unemployment rates Number of single parent families Commissioners and service planners need to consider the contribution psychological services can make to: o o o o o Crisis intervention and suicide prevention Management of major behavioural problems, including depression, anxiety, obsessions, eating problems and alcohol/drug abuse Reactions to trauma, chronic health problems or disability Relationship problems, including sexual problems and family adjustment to major mental issues Need for mental health promotion and prevention. 2. The contribution of Clinical and Counselling Psychologists to organising and delivering psychological services Clinical and counselling psychologists are trained to apply scientific knowledge about human behaviour to alleviate psychological problems. As well as offering psychological interventions of proven efficacy, they are committed to enhancing the work of other professionals, developing new services and evaluating objectively interventions.

107 Specialist skills include: o o o Direct intervention in the form of assessment and treatment for individuals, families and groups. Clinical consultancy in terms of training and supervision. Psychological consultancy in developing and evaluating services, teaching, prevention and health promotion as well as research. 3. Context of Psychologists' work Clinical Psychologists work largely in health and social care settings including hospitals, health centres, community mental health and learning disability teams, child and adolescent mental health services and social services. Clinical Psychologists usually work as part of a team with, for example, social workers, medical practitioners and other health professionals. Most clinical psychologists work in the National Health Service, which has a clearly defined career structure. 4. Qualifications and Training of Clinical Psychologists Obtain Graduate Basis for Registration (GBR) by completing an accredited degree course in psychology. Complete a minimum of 1-2 years relevant work experience, usually as an assistant psychologist within the NHS, however care assistant and related research experience is acceptable. Complete professional training via the Doctorate in Clinical Psychology funded by the NHS. Post graduate training takes 3 years full time. Ivan Burchess January

108 Appendix 13. National Advisory Group on Mental Health, Safety & Well-Being towards Proactive Policy: Five Universal Psychological Principles

109 National Advisory Group 1 on Mental Health, Safety & Well-Being Towards Proactive Policy: Five Universal Psychological Principles Summary of the Five Principles 1. Human well-being depends on universal psychological needs as well as physical and social needs 2. A psychologically-informed generic policy framework is badly needed to promote psychologically safe mental health services (over and above the growing guidance on specific psychological treatments) 3. Attachment theory provides a universal evidence-base that has not yet been harnessed. 4. In mental health it is primarily relationships that can kill and cure, so all our staff must be trained and supported psychologically to promote therapeutic responses to service users 5. Choosing between medical and psychological approaches is a false choice. Mental distress always has a psychological meaning and context. Any response to distress (including medical) conveys a psychological attitude and impact. Psychological standards for all interventions are therefore required. Principle One: Human health and well-being depends on psychological (mind) as well as biological (body) and social factors. Fundamental psychological needs are: (a) To have a secure and stable attachment to at least one significant other person who knows us well and on whom we can trust and depend (ATTACHMENT & TRUST) (b) To have our attempts to communicate recognised and attended to by at least one other available person who is motivated to understand the meaning behind them (EMPATHIC COMMUNICATION & RELATIONSHIP) (c) To belong to a family or other care-giving social group/system and to have a recognised and respected identity and position within that group (IDENTITY & BELONGING) (d) To have secure, clear and consistent social boundaries and rules within which to live which are enforced in a firm, fair and containing manner (CONTAINMENT, SECURITY & DISCIPLINE) (e) To have a sense of hope, belief, meaning, value and purposeful occupation in relationship to self and others (ESTEEM, BELIEF & PURPOSE) (f) To develop understanding and influence over ourselves and the environment in which we live (SELF-DETERMINATION) (g) To develop our capacity to tolerate frustration and fully experience pleasure (RESILIENCE & HAPPINESS) (h) To learn reciprocal respect, regard and responsibility towards others (RESPECT & RESPONSIBILITY) These needs can be observed and extracted as themes from a wide spectrum of psychosocial research, knowledge, writing and experience over the last five decades and more. Any family, school, nursery, hospital, clinic, organisation, institution, community or society can be measured in terms of how far it meets or fails to meet these standards. These standards could be said to provide a psychosocial definition and measure of civilisation and humanity itself. Within public mental health and care services, no single profession, brand of therapy or care model has any monopoly on supplying these psychological needs. Within society as a whole, the increasing fragmentation of family life and the demise of a coherent religious or spiritual framework for community life have led to a net decline in meeting some of these standards despite an increase in economic prosperity. The holy trinity of health (bio-), wealth (social) and happiness (psycho-) is in fact a bio-psycho-social trinity. As our democratic society evolves and matures as a secular entity, it is beginning to refocus more on the psychological and spiritual dimension, the happiness and well-being dimension. The above psychological principles provide a scientific basis for secular care services that also connects 1 National Advisory Group are Susie Orbach, Andrew Samuels, Valerie Sinason, Lucy Johnstone, Martin Seager, Glenda Fredman, Ross Hughes, James Antrican, Margaret Wilkinson and Peter Kinderman assisted by Tanya Woolf and David Spektor 1

110 with the golden rule of all formal world religions, namely do unto others as you would have them do unto you Principle Two: Generic mental health (and learning disability) services therefore need an overarching psychologically-informed framework, model or culture in which all staff can operate If we accept the core psychological needs described above under Principle One, it follows that our public health and care services require operational policies and frameworks that are informed and guided by these psychological principles. Without such an overarching framework, there will be considerably less psychological coherence or safety in the systemic culture of care in which all staff and service users are expected to live and breathe. These standards must apply not just to service users but also to care staff, administrative and reception staff, managers and care institutions as a whole if they are to become effective professional families. Policy developments to date (eg. national service frameworks, clinical governance, standards for better health, NICE guidelines, the Ten Essential Capabilities, New Ways of Working) have so far lacked this broader psychological dimension but could quite easily be expanded and amplified. Principle Three: Attachment Theory is the most universal and most rigorously tested of all psychological theories relating to mental health and well-being. To date it provides the soundest scientific evidence-base around which to design and measure mental health (and learning disability) services. However, it has yet to be truly utilised by policy makers. Research universally shows that those in our society with the most severe mental health problems also have the most severely disturbed or disrupted attachments to care-givers over the developmental years. Severe mental health problems are consistently linked to histories of abuse (physical, sexual, emotional), trauma, neglect, abandonment and discontinuity of attachment. Research also consistently shows that in our attempts to help those with severe mental health problems it is the quality of our attachments and relationships with them that is the single most effective therapeutic ingredient, regardless of any particular therapy brand, profession, model or approach. Attachment underpins and links all core psychological therapies (CBT, psychodynamic and systemic) and the commonality between them in this respect is far more critical than any differences in technique or brand. If we truly want evidence-based mental health and learning disability care practices, then the global lessons of attachment theory, tested over decades and universally accepted across the three fields of biological, social and psychological research, need to be built into the fabric of operational policy and training. This could vastly improve the safety and the effectiveness of the overall culture of care. It could significantly improve the way in which all staff are trained, supported and managed. It could significantly improve the ways in which services operate, for example how they conceptualise and handle referrals, waiting lists, appointments, caseloads, discharges, transfers of care, follow-ups and outcome measurement. Principle Four: In mental health, relationships are both cause and cure. Relationships are the baby, not the bathwater. This means that our individual staff members as people are more crucial even than the particular treatments they provide. For each service user, the relationship that they form with the care staff and the care system IS the treatment! We cannot therefore properly care for service users except by also caring for our staff. In mental health it is primarily failures and problems of attachment and relationship that damage the developing child and his or her capacity to become a happy and productive citizen. Our unhappiest citizens have been abused, traumatised, neglected, abandoned and/or passed between care-givers during their developing years. All too often they then reexperience themselves as unwanted children in our public mental health and care institutions, being passed from one service provider to another without continuity or stability 2

111 of attachment. In public services the pressure to discharge people from care can often be experienced as another rejection rather than as genuine progress. This revolving door treatment culture can be depersonalising and costly for our society both in human and financial terms. It can escalate rather than reduce problems of low self-esteem, alienation, self-harm and destructive behaviour. Staff too in our public services are also frequently moved around and reorganised so that it is hard for staff to form meaningful attachments to each other, to their units, to their clients and to a model of care. It will be far cheaper and much more effective in the long run therefore to develop psychologically-informed policies and practices that enshrine the personal value of care staff (in terms of the 8 core needs above) individually and as a professional family. Such an attitude amongst staff will then be passed on to the service users. In other words our society is still too often placing its most damaged and vulnerable citizens into care environments where those that are expected to care for them are not fully trained, nurtured, valued, contained and supported psychologically. According to the golden rule (above) we cannot expect staff to show a more caring attitude for service users (who will often by definition be very suspicious, hostile and disrespectful) if they themselves do not feel sufficiently respected and cared for by those that they depend on in turn. Our culture of care is therefore less safe, stable and healthy psychologically than it could be. Staff operating under psychologically-informed national policy and guidance could be happier, less stressed, more available to service users and more confident about how to help them. They could take less time off through sickness and they could have more sense of purpose in their own work. This would translate to service users with a reduction in human and financial cost measured for example by rates of re-referrals, self-harm, destructive behaviour and critical incidents. Principle Five: The concept of a treatment choice between a medical and a psychological approach is a false one. It is impossible to operate in a psychological vacuum without a psychological impact. Mental distress, however extreme, always has a psychological meaning and context. In trying to respond to mental distress there is always a relationship formed between user and provider which will exert a powerful influence on outcome, regardless of the techniques or treatments that are ultimately selected. Even apparently stand alone medical treatments cannot be extricated from their psychological context and impact. Psychological factors are always critical in understanding mental distress and any treatment response will always have a psychological impact for good or ill. There is no realistic option not to have a psychological impact on those who turn to us for help even when a formal psychological therapy is not being offered. Any attempt to respond to mental distress by staff in our public services automatically conveys a psychological attitude and stance (for good or ill) to the service user even if this is not intended. Vulnerable service users will also always bring their own attitudes into the relationship and this can create a positive or a negative interaction with carer attitudes. These psychological interactions are always present and are always critical to outcome regardless of the particular techniques, models or approaches that might be used. National policy and guidance could be expanded to spell out explicitly the desirable psychological attitudes and standards of communication and relationship that all mental health services should meet. Psychologically-minded policy would foster psychologicallyminded practice. By not being fully psychologically-minded in our national policy, there will be at best a failure to make the most of opportunities to improve the lives of vulnerable people and at worst the increased risk of repeating, entrenching and exacerbating past rejections and failures of relationship. 3

112 Seven Key Areas of Practical Application 1. Pilot Sites - The above principles could be inexpensively operationalised and tested by being used to retrain staff (in terms of the basic principles) and redesign the service model (including referrals, admissions, allocations, basic therapy model, discharge, caseloads, structure of time, team communications and support model, supervision, management, leadership and administration arrangements) in one or more experimental pilot sites as compared with control sites in which treatment as usual was provided. Potential pilot sites could include (1) a typical acute adult psychiatric ward (2) a typical care, nursing or residential home (3) a typical adult community mental health team (4) a typical primary care health centre or clinic. Outcome measures could include re-referral and re-admission rates, critical incident rates, risk measures, satisfaction measures (staff, service users and significant others), symptom measures and medication usage. 2. Existing policy standards and good practice guidelines (eg. The Ten Essential Capabilities, Clinical Governance or Standards for Better Health, Nice Guidelines) can be embellished and amplified to take account of these universal psychological principles. 3. The vitally important Improving Access to Psychological Therapies (IAPT) national project can be built upon further to incorporate the above principles, thus creating a broader, cultural or systemic dimension. This might be termed Improving Access to Psychologically-Minded Care 4. Current Core Guidance for all health and care professions on New Ways of Working could be amplified and embellished to take account of these vital principles. 5. Staff Training - these principles could be used to design basic generic (psychologically-informed) training courses and manuals for all staff, including managers and administrative staff 6. Evaluation measures and standard setting - these principles could be used to redesign key dimensions of measurement of the effectiveness of treatments and services in relation to how far they meet or satisfy universal psychological needs. They can also be used to define operationally the psychological standards that mental health services should meet. For example, using these principles a service user should be able to expect a secure and stable attachment with at least one professional and also that his basic life story should be recognised and remembered by all others involved with his or her care. 7. These principles could be used to improve the quality of existing service development and workforce planning. For example, using attachment theory would lead to a more radical and dynamic concept of what a caseload should look like. Under attachment theory, no mental health care worker or professional should have more attachments to service users than they can hold in mind at any one time. This limit might vary in different contexts but would be set according to psychological parameters of risk. To go beyond this limit would then be seen to constitute unsafe practice psychologically and creating a meaningful and practical definition of this limit would help both staff and service users to feel more contained. Martin Seager, May

113 Appendix 14. BPS / DCP Briefing paper 1: Clinical and other applied psychologists within the NHS

114 British Psychological Society Division of Clinical Psychology Briefing Paper 1 Clinical & other applied psychologists in the NHS 1 st May 2007 highlighting key issues in employing and managing psychologists in the NHS Key issues at a glance Clinical psychologists use psychological theory in order to offer a range of professional services to individuals, groups or organisations Their core skills include research and evidence based interventions applicable to service redesign. Psychologists have a distinctive psychological perspective on many issues in across a breadth of activity in healthcare. Psychologists are set to play key roles in the Mental Health Act and Mental Capacity Acts and the Improving Access to Psychological Therapies programme. Like many professionals in the NHS, clinical psychologists are working with the Care Services Improvement Partnership (CSIP) on New Ways of Working and developing leadership and professional management in NHS Trusts. Chartered psychologists There are approximately 17,000 applied psychologists working in public service in the UK (Department of Health, Home Office and British Psychological Society, 2004). Applied psychologists perform a wide range of roles, and consequently have different specific job titles: Clinical Psychologists, Neuropsychologists, Forensic Psychologists, Occupational Psychologists, Counselling Psychologists, Health Psychologists, Educational Psychologists and Sport & Exercise Psychologists. Since 2001 the number of clinical psychologists in the NHS has risen by 1,608 (29%) to 7,122 and the number of training places has increased by 157 (37%) to 582. The number of clinical psychologists in post has increased by over 3,000 since Core skills The core skills of a clinical psychologist are: Assessment; Formulation; Intervention; Evaluation and research; Communication. The information gained from assessment of psychological processes and behaviour is used by psychologists to develop formulations. These summarize and integrate this assessment information with psychological theory and research to provide a framework for describing a problem, how it developed and is being maintained. Because of their particular training in the linkage of theory to practice, psychologists draw on a number of different explanatory models and so a formulation may comprise a number of provisional hypotheses. What makes this activity unique to psychologists is the knowledge base and information on which they draw. The ability to access, review, critically evaluate, analyse and synthesise data and knowledge from a psychological perspective is one that is distinct to psychologists. Intervention, if appropriate, is based on the formulation. This approach clearly prioritizes an evidence-based approach to healthcare. Evaluation is a critical and integral part of the psychologist's work. Research includes the ongoing evaluation of assessment, formulation and intervention in relation to specific services provided. It also includes explorations of psychological processes and outcomes and the development and evaluation of specific psychological interventions. These competencies, built on the body of psychological theory and research and applied to helping people solve personal, family, group, work or organisational problems, makes psychology unique in health and social care. Source: Quality Assurance Agency for Higher Education (2004) Benchmark Statement: Health Care Programmes, Phase 2, Clinical psychology. Gloucester: Quality Assurance Agency for Higher Education. Contact details and websites The British Psychological Society St Andrews House, 48 Princess Road East, Leicester LE1 7DR Tel: enquiry@bps.org.uk British Psychological Society Division of Clinical Psychology: Lynn Hartshorn, Advisor Division of Clinical Psychology: Direct line: lynn.hartshorn@bps.org.uk

115 Training and qualifications All psychologists working in mental health teams have undergone in-depth and extensive training in the application of psychological science to mental disorders. Clinical psychology is the only healthcare profession where a doctorate is the minimum entry requirement to practice. After a minimum of 6 years university education, during which time they are awarded both undergraduate university degrees and their doctorates, psychologists qualify and specialise in specific areas of work such as adult mental health, working with children and young people, or forensic and prison settings. Psychologists are not normally considered for consultant level posts for 6 years post qualification, during which time are obliged to undertake continuing professional development, and many obtain further specialist postgraduate training. Only after some 12 years of education and training, therefore, are psychologists working in mental health likely to reach consultant level. Since 2001 the number of clinical psychologists in the NHS has risen by 1,608 (29%) to 7,122 and the number of training places has increased by 157 (37%) to 582. The number of clinical psychologists in post has increased by over 3,000 since Evidence based interventions Clinical psychologists are more than psychological therapists. While many do practise psychotherapy at a high level, this is not a skill distinct to clinical psychologists, nor should it be. The background and training of clinical psychologists is rooted in the science of psychology, and clinical psychology is one of the applications of psychological science to help address human problems. The ability to design and carry out innovative applied research is a key skill developed in clinical psychologists training and is important for the development and delivery of evidencebased practice. Psychologists are also, therefore, competent in the critical evaluation of research activity; for instance in the development and testing of new interventions and activities, based on psychological theory. Thus practice feeds and draws on research and theory that in turn influences practice. The role of psychologists in developing evidence-based clinical guidance can be seen in the work of the National Collaborating Centre for Mental Health (NMCCMH) The NMCCMH is a partnership between the British Psychological Society's Centre for Outcomes Research and Effectiveness (CORE) and the Royal College of Psychiatrists' Research Unit (CRU), and is one of seven collaborating centres established and funded by National Institute for Clinical Excellence (NICE) to develop clinical guidelines on their behalf. Sources: Quality Assurance Agency for Higher Education (2004) Benchmark Statement: Health Care Programmes, Phase 2, Clinical psychology. Gloucester: Quality Assurance Agency for Higher Education. British Psychological Society Division of Clinical Psychology. (2001) The core purpose and philosophy of the profession. Leicester: British Psychological Society. Service redesign Psychologists therefore have invaluable skills in problem solving and innovation; skills which can assist Trusts in service redesign. These are coupled with clinical and therapeutic skills, which can help retain a focus on service user needs and aspirations. Psychologists are research skilled and research literate they can understand, critically review and add to the evidence base not just for clinical practice but also for organisation development and change management. How to use a scarce resource Psychologists are a scarce resource and their skills need to be targeted to where they will be used most efficiently. Clinical and other applied psychologists have high expectations to meet. The Agenda for Change process has made psychologists more visible, it has clarified our competencies and our role. But it is sometimes unclear what we do or what should be expected of us. Given current financial constraints in the NHS, there are understandable pressures on managers to maximise their return on investment. This briefing paper aims to give guidance about our unique skills in mental health, physical and health and other areas of health and social care. We understand that NHS Trusts are independent organisations that are free to make their own decisions; nevertheless, we have a valid perspective on what constitutes appropriate organisational arrangements for the delivery of safe and effective psychological services to the public. We believe that service users and the wider public are protected through proper clinical governance, accountability, professional leadership and supervision. A psychological perspective A psychological perspective places psychological issues centrally in understanding mental disorder. This perspective has important implications for mental health service policy: i) Services should be planned on the basis of need and functional outcome rather than diagnostic categories, where residential care is necessary, a concept of hospital care should be avoided. ii) The current emphasis on specialist teams should continue, but the focus of these teams should be based on underlying psychological principles, iii) Services should fully embrace the recovery approach, iv) Services should facilitate genuine service user involvement, v) Access should be improved to psychological therapies based on individual case formulations and driven by recovery models, vi) Nurses should develop increasing competencies in psychosocial interventions, occupational therapists and social workers should see their roles develop, and vii) Psychologists should be prepared to offer consultation and clinical leadership. Kinderman, P. (2005) A psychological model of mental disorder. Harvard Review of Psychiatry; 13 : 1 16; Kinderman, P. & Tai, S. (2006) Clinical implications of a psychological model of mental disorder. Behavioural and Cognitive Psychotherapy, 35, Kinderman, P, Sellwood, W & Tai S (2007 in press) Policy implications of a psychological model of mental disorder. Journal of Mental Health.

116 Improving Access to Psychological Therapies Guidelines from NICE (the National Institute for Health and Clinical Excellence), the NSF for Mental Health and academic papers have repeatedly stressed the effectiveness and appropriateness of psychological therapies for a range of mental health problems. The Department of Health s and the Care Services Improvement Partnership s (CSIP) Programme: Improving Access to Psychological Therapies (IAPT) sets out a framework for action, including two national demonstration sites, to address these issues in England. Clearly, as experts in conducting and delivering psychological interventions, clinical and applied psychologists have a key role here. Psychologists have experience of working together with other psychological therapists, and in providing training in a range of therapies. We envisage the IAPT programme engaging a broad range of psychological therapists, and anticipate Trusts employing a range of professionals, including new graduates, to be trained in specific therapies such as CBT cognitive behavioural therapy. Psychologists are therefore also important as managers, supervisors and trainers of such psychological therapists. It will also be important that new services are evaluated with respect to clinical outcomes and users perspectives; psychologists have an important role in collecting such practice-based evidence. Finally, psychologists training enables them to formulate complex care planes where the application of manualised and deliverable packages of therapy can be integrated into more holistic care plans. Turpin, T., Hope, R., Duffy, R., Fossey M & Seward J. (2006). Improving access to psychological therapies: implications for the mental health workforce. Journal of Mental Health Workforce Development. 1(2):7-15. Leadership and professional management in Trusts Several Department of Health reports (NHS Psychotherapy Services in England: Review of Strategic Policy, 1996; Treatment Choice in Psychological Therapies and Counselling, 2001; have collated evidence for the effectiveness of psychological therapies and offered practical guidance about how to drive forward the evidence based practice agenda. These were supported in the NHS Plan (2000), the Priorities and Planning Framework (PPF) for and Organising and Delivering Psychological Therapies (Department of Health /NIMHE, 2004). The British Psychological Society further recommends: - Psychologists, by virtue of their training, competencies and experience, can lead and manage teams, and take 'clinical responsibility' while supervising more junior staff - There should be specific Board-level representation for the delivery of Psychological Services - Services must be aligned with the vision of future service delivery and the key external drivers for organising Psychological Services. This includes the need to consider how psychological therapies and approaches are organised and delivered in multiprofessional, multi-disciplinary context The overarching approach of applied psychologists is the application of psychology across whole of health and care system Source: British Psychological Society Division of Clinical Psychology. (2007) Leading psychological services. Leicester: British Psychological Society. Mental Heath Act & Mental Capacity Act The introduction of the Mental Capacity Act 2005 and the proposed changes to the 1983 Mental Health Act are significant for psychology. In the case of the Mental Capacity Act 2005, substantial legal responsibilities will fall to psychologists to clinical psychologists and neuropsychologists. These will be additional clinical roles, but also substantial legal responsibilities. Similarly, the proposals to replace the Responsible Medical Officer under the Mental Health Act 1983 with a Responsible Clinician who may be a psychologist is important for all employers. These changes are in keeping with best quality mental health care, and allow for proper multidisciplinary practice. Such an approach explicitly permits proper use of the skills and competencies of the workforce - including psychologists. A second controversial aspect of the proposed reforms Supervised Community Treatment Orders, permitting compulsory care outside of hospitals may also involved substantial changes in the roles of psychologists employed in health and social care Kinderman P (2007) Reforms to the Mental Health Act and implications for psychologists. Issues in Forensic Psychology No Breadth of activity Good practice in the management, training, access, choice, and supervision of psychological therapists includes: - Improved access to therapies to avoid long waiting time - Attention to the psychotherapeutic needs of different groups: for example, older people, people from minority groups - Involvement of users in choosing the most appropriate therapy for their condition and situation - Systematic training in psychological therapies for mental health professionals supported by specialist supervision once they return to the workplace - Clear leadership, both professionally and managerially, is best achieved through the development of an organisation wide body i.e. a Psychological Therapies Management Committee. Clinical and other applied psychologists work across a wide range of areas of key importance to health service targets in addition to their core activities in mental health. Within the NHS, psychologists work with individuals, couples, families, groups and at the organisational and community level. They work in a variety of settings, including hospital wards, day centres, Community Mental Health Teams, NHS Trusts, primary and social care contexts and forensic settings, and with all age groups from very young children to older people. They work with people with mild, moderate and severe mental health problems, developmental and learning disabilities, physical and sensory disability, and brain injury; people who have substance misuse problems and people with a range of physical health problems (e.g. HIV and AIDS, cancer, heart disease, pain, diabetes).

117 Psychologists work with people with long term neurological and other conditions conducting skilled neurological or other complex assessment, working to improve motivation and rehabilitation and coordinating pain management. Psychologists and psychological approaches are also important in helping combat anti-social behaviour; including the respect agenda in the community and helping address challenging behaviour in forensic and secure clinical settings. Health psychologists and specialist clinical psychologists also work in the public health arena and in assisting in health behaviour change. A key message for Department of Health policy leads and for senior NHS managers is to think beyond merely mental health and psychological therapies when considering the contribution of psychology. New Ways of Working The New Ways of Working Programme of the Department of Health, National Institute for Mental Health in England (NIMHE) and the Care Services Improvement Partnership (CSIP) is designed to help professions working in mental health to become more modern and flexible. As part of this programme, applied psychologists have an ambitious plan examining: - Models of organising, leading & managing psychological therapies services in trusts - Pathways and access to psychological therapies - The training model for applied psychologists - The undergraduate curriculum - Role of assistants/associates in the future workforce - Career pathways and roles - Applied psychology and multidisciplinary team working - Mental health legislation The New Ways of Working for Applied Psychologists group will report in spring of 2007 with a range of recommendations and plans for ongoing work. Flexibility and increasing access to psychological services will be key elements. The British Psychological Society The British Psychological Society (BPS) is the learned and professional body for psychologists in the United Kingdom. The BPS has a total membership of over 42,000 and is a registered charity. The profession (or professions) encompassed by the term psychologist is, however, not at present regulated by Statute. The BPS fully supports the principle of independent statutory regulation of psychologists and we welcome the imminent publication of a Government White Paper. The BPS is, however, authorised under its current Charter to maintain a Register of Chartered Psychologists, to hold a Code of Conduct and to have investigatory and disciplinary systems in place to consider complaints of professional misconduct relating to its members. This curious legal position is enshrined in the European Communities (Recognition of Professional Qualifications) (First General System) Regulations The BPS is also an examining body granting certificates and diplomas in specialist areas of professional applied psychology and has quality assurance programmes for accrediting both undergraduate and postgraduate university degree courses. In these activities, it works alongside Government Agencies such as the Quality Assurance Agency for Higher Education. Branches of psychology The Society recognizes several different branches of applied psychology, including - Clinical Psychology Neuropsychology Educational and Child Psychology Forensic Psychology Counselling Psychology Health Psychology Occupational Psychology (details of all these groups can be found at: The largest single group of applied psychologists are clinical psychologists. There are some 7000 currently employed in the NHS. Clinical psychology (according to the Government s Quality Assurance Agency for Higher Education, 2004) aims to reduce psychological distress and to enhance and promote psychological wellbeing by the systematic application of knowledge derived from psychological theory and research. The core skills of a clinical psychologist are defined as: assessment, formulation, intervention, evaluation and research, and communication. About 400 forensic psychologists work in the NHS, the Prison Service, in a range of Offender Management Services and in the special, high and medium secure hospitals. Their competencies include assessment, evaluation and riskassessment, but also psychological interventions and therapies aimed at reducing the risk of re-offending. In addition, there are some 300 counselling psychologists (whose competencies include the delivery of psychological therapies) as well as health psychologists (addressing the physical healthcare of people with mental health problems) and neuropsychologists working in health and social care. Sources: British Psychological Society, Department of Health & Home Office (2005). English survey of applied psychologists in health & social care and in the probation & prison service. British Psychological Society, Leicester; Lavender, T. & Paxton, R. (2004). Estimating the applied psychology demand in adult mental health. British Psychological Society, Leicester. Advertising and competencies There is overlap between the competencies of psychologists in these Divisions. In some cases, therefore, a particular job could be appropriately filled by a psychologist from more than one Division. Senior psychology managers should ensure that the competencies of each individual role are described and each post advertised appropriately. This may best be achieved through phrases such as Clinical or Counselling Psychologist or Forensic / Clinical Psychologist or Health or Clinical Psychologist. The National Assessors group of the British Psychological Society are experienced in assessing the match between and individual psychologist and the specific competencies for a particular post. The Department of Health recommends ( s/publicationspolicyandguidance/publicationspolicyandguida ncearticle/fs/en?content_id= &chk=8sfudx) that two National Assessors are involved in any appointments made to Agenda for Change bandings 8c and above.

118 Appendix 15. Potential contribution of applied psychologists to implementing NHS reform

119 POTENTIAL CONTRIBUTION OF APPLIED PSYCHOLOGISTS TO IMPLEMENTING NHS REFORM User Choice Knowledge of wide range of therapies and therapists Working with users to promote understanding of range of psychological approaches in order to achieve informed choice Promoting well-being, psychoeducation and self-help Providing individually tailored psychological formulations Championing a psychosocial understanding of mental health Supporting user agendas and promoting social inclusion Value for money Ability to assess and formulate complex problems Effective therapies and therapists Therapy innovations Service redesign Working through other staff (GWs, psychology assistants etc) to broaden delivery of therapies Disseminating psychological knowledge within teams Education and training Attending to the psychological needs of staff and organisations IMPROVED CLIENT EXPERIENCE Standards Knowledge of wide range of interventions and competencies Knowledge of professional accreditation Expertise around clinical governance and risk Promoting ethical practice Expertise around supervision and training supervisors Critical thinkers and problem solvers Ability to think organisationally and to support other staff Supporting Trust Boards in delivering business plans Ensuring that standards and knowledge are regularly updated through R&D Commissioning Advising commissioners on needs assessments and effective interventions Knowledge of a wide range of psychological interventions, not restricted to a single therapeutic modality Knowledge of psychological issues and disorders across the age range Integrating mental and physical health needs Providing a bio-psychosocial approach Addressing user perspectives, recovery and social inclusion

120 Appendix 16. Report of the High Level Group on Clinical Effectiveness

121 Report of the High Level Group on Clinical Effectiveness Chaired by Professor Sir John Tooke A report to Sir Liam Donaldson Chief Medical Officer

122 DH INFORMATION READER BOX Policy HR/Workforce Management Planning Clinical Document Purpose Estates Performance IM & T Finance Social Care/Partnership Working For information ROCR Ref: Gateway Ref: 8945 Title Author Report of the High Level Group on Clinical Effectiveness DH/CMO Publication Date 23 October 2007 Target Audience Circulation List PCT CEs, NHS Trust CEs, SHA CEs, Care Trust CEs, Foundation Trust CEs, Medical Directors, Directors of PH, Directors of Nursing, Local Authority CEs, Directors of Adult SSs, PCT PEC Chairs, NHS Trust Board Chairs, Special HA CEs, Allied Health Professionals, GPs, Emergency Care Leads, Directors of Children s SSs Description Cross Ref Superseded Docs Action Required Timing Contact Details For Recipient s Use The Government established a High Level Group (HLG) to report to the Chief Medical Officer on the scope for enhancing and incentivising more effective and efficient clinical care and to make recommendations for future action. This is the HLG s report. N/A N/A N/A N/A Deirdre Feehan Service Reviews Department of Health Wellington House Waterloo Road London SE1 8UG

123 Contents Foreword Open letter from the Chair of the High Level Group on Clinical Effectiveness 3 5 Part 1: Issues and Recommendations 7 Introduction Issues Recommendations High Level Group on Clinical Effectiveness Membership Part 2: Scoping Report 15 Summary 16 Section 1: Activities in the NHS related to supporting clinical effectiveness 17 Section 2: Section 3: How to approach the issues of clinical effectiveness and barriers to implementation of guidelines or effective practices Current knowledge on the effectiveness of strategies to promote the uptake of clinically effective practice Appendix 1: Activities in the NHS related to supporting clinical effectiveness 35 Appendix 2: Department of Health selection criteria for referral of topics to NICE Appendix 3: Intervention-based and clinical condition-based reviews and protocols in the Effective Practice and Organisation of Care Review Group of the Cochrane Library 45 47

124 Foreword The NHS, like many other healthcare systems, experiences inappropriate variation in treatments and treatment rates. Yet all citizens have the right to expect the same high standard of care and the same level of access to treatment and services. That was the case when the NHS was founded and remains so today. However, since 1948 when the NHS was set up, the range and complexity of available treatments have grown substantially, as has the public s demand for medical care. The cost of healthcare has also continued to rise. So it is imperative that the NHS makes the best use of its resources to provide high quality, equitable care for patients. That is why in 2005 I focused on the issue of variation in clinical practice in my Chief Medical Officer s Annual Report On the state of the public health. The lead chapter, Waste Not, Want Not, highlighted the adverse impact of inappropriate variation in clinical practice on the use of resources across the NHS as well as on individual patients. In 2006, therefore, the Government asked me to set up an independent High Level Group (HLG) to report to me on the scope for enhancing and incentivising more effective and efficient clinical care, with recommendations for a programme of action. I am grateful to have received the report and recommendations of the High Level Group on Clinical Effectiveness from Professor Sir John Tooke, the Group s Chair. I would like to thank Sir John and his HLG colleagues for the thoroughness of the report and the hard work that went into delivering this to me within a challenging timetable. The scoping report for the HLG from Professor Martin Eccles sets out a summary of existing activity on clinical effectiveness. It also shows that there is a great deal of work already being carried out in this area by a number of organisations and many examples of good practice. Nevertheless, it also identifies two major issues - a lack of overarching high level leadership and the need for more effective partnership between the NHS and higher education. I accept that there needs to be better alignment and coordination of activity in this field at a national level. I am encouraged that, through the HLG, NICE (the National Institute for Health and Clinical Excellence) and the NHS Institute for Innovation and Improvement have already agreed to work together to better align and integrate appropriate workstreams on clinical effectiveness. Since the HLG developed its recommendations, Lord Darzi has published his interim report on Our NHS, Our Future and announced the creation of a new Health Innovation Council (HIC). The Council, whose membership will include representatives from NICE and the NHS Institute for Innovation and Improvement, will have an oversight role for the whole innovation pathway and provide leadership in encouraging innovation at the local level as well as local clinical and management ownership. I am pleased to announce that the Department of Health will therefore take forward appropriate recommendations from the HLG through the Health Innovation Council. The HLG also identified the need to harness more effectively the capacities of academia. I am very pleased that at the same time as this report is being published, the Department of Health s Research and Development Directorate is announcing the establishment of NIHR (National Institute of Health Research) Academic Health Centres of the Future. These will develop innovative models for conducting applied health research and translating research findings into improved outcomes for patients, through partnerships between academia and the NHS across the health community covered by the Centre. I am also grateful to Sir John for suggesting practical ways forward on educational issues to key bodies such as the Medical Schools Council, ACCEA (the Advisory Committee on Clinical Excellence Awards) and the GMC (the General Medical Council). I look forward to their responses. 3

125 Foreword In addition, following my 2005 Annual Report, NICE has launched a new programme to assist the NHS in reducing spending on ineffective treatments. I also recommended varying tariff payments in the NHS so that less effective treatments would result in less money going to hospitals that persist in providing them. This issue will be examined by the recently established Clinical Advisory Panel. Part of the panel s remit is to examine the optimal use of the NHS Payment by Results policy as support for the delivery of high quality clinical care. The Department of Health, together with the British Association of Day Surgery, is also testing approaches to tariff setting so as to identify those which best reduce inappropriate variation in standards of patient care. I also called for more active knowledge management to improve access to quality assured clinical information. National Knowledge Weeks and the National Knowledge Board will also help to ensure that information and guidance produced nationally will be accessible locally. As the High Level Group has identified, there is no simple answer to improving clinical effectiveness. Clinical engagement is key and we need to harness the knowledge and expertise of clinicians to promote clinically effective practice. I commend this report and the initiatives arising from it to clinicians and managers across the NHS. Sir Liam Donaldson Chief Medical Officer, England 4

126 Open letter from the Chair of the High Level Group on Clinical Effectiveness Sir Liam Donaldson Chief Medical Officer, England Dear Sir Liam In October 2006, you asked me to chair the High Level Group on Clinical Effectiveness. I was delighted to accept your invitation to bring together an impressive team of experts to consider how clinical effectiveness in the NHS can be improved. I would like to thank the members of the High Level Group for sharing their expertise in this important area and I am very grateful to them for their enthusiasm and dedication to the Group s work. In the chapter Waste Not, Want Not in your 2005 report On the State of the Public Health, you focused on the issue of variation in clinical practice. This highlighted the need to reduce inappropriate variation in clinical practice to deliver therapeutic treatment equitably. It is vital to the delivery of effective healthcare for patients and effective use of resources that the NHS reduces the use of treatments of little therapeutic value, ensures the use of treatments of proven effectiveness and minimises the misuse of treatments. The High Level Group focused on the scope for enhancing and incentivising more effective and efficient clinical care. Patients have the right to expect the same high standard of care and treatment wherever they access NHS services. It is also important for the NHS to make best use of resources by adopting effective therapies and abandoning ineffective interventions. We commissioned a scoping report from Professor Martin Eccles, Professor of Primary Care Research at Newcastle University. His report established that there is a plethora of clinical effectiveness workstreams being undertaken by a range of organisations, and some fine examples of good practice in local organisations, but no overarching high-level leadership or capitalisation on the potential productive relationships between the NHS and academia. From our work, we have identified no single bullet to address the issue of clinical effectiveness. Instead, systematic, context-specific initiatives are needed, requiring local clinical engagement. Evidence-based medicine should be complemented by evidencebased implementation, demanding attention to education programmes from undergraduate studies onwards. Our report makes recommendations on: alignment of central activities and support promoting local ownership ensuring clinical engagement harnessing the capacities of academia the research agenda. We have worked closely with colleagues at NICE (the National Institute for Health and Clinical Excellence), the NHS Institute for Innovation and Improvement and the Healthcare Commission. They have agreed to support a cross-sectoral group to take forward the alignment agenda and propose mechanisms to ensure better integration of workstreams. NICE and the NHS Institute have also agreed to a joint appointment to support the cross-sectoral group. I have written to the Medical Schools Council to encourage them to review their curricula to ensure that the right foundations are being laid for clinically effective practice and to the GMC (the General Medical Council) to ensure such requirements are reflected in the imminent revision of Tomorrow s Doctors. It will also be important to encourage postgraduate training initiatives which help clinicians develop skills in improvement science and resource management. In this regard I would hope that the creation of a management track during core specialty training as outlined in my Modernising Medical Careers (MMC) Inquiry Interim Report would facilitate this agenda. 5

127 Open letter from the Chair of the High Level Group on Clinical Effectiveness A major educational event involving the Medical Schools Council, PMETB (Postgraduate Medical Education and Training Board), the GMC, the Academy of Medical Royal Colleges and the NHS Institute for Innovation and Improvement will be planned to coincide with the 150th anniversary of the Medical Act 1858 to focus on the needs of the whole educational continuum and the implications for professional practice. The Group has also identified the importance of partnership working between the NHS and higher education to deliver advances in medical practice. We believe that the health service can do more to harness more effectively the capacity of higher education to assist with this agenda. We have therefore made a key recommendation that new models of community wide academic health centres should be developed to encourage relevant research and engagement and embed a critical culture which is more receptive to change. I am also pleased that Professor Martin Eccles has agreed to chair an expert group to define the research agenda and inform the considerations of the National Institute of Health Research (NIHR) Service Delivery and Organisation Programme. All the stakeholders with whom we have engaged in the course of the High Level Group s work have eagerly supported the need to improve clinical effectiveness and promote clinical engagement. The High Level Group s work has been a vital initiative to address these key issues. Improving clinical effectiveness and clinical engagement will continue to be a key priority to improve patient care and make the best use of NHS resources. I commend to you this report of the High Level Group on Clinical Effectiveness and its recommendations. Professor Sir John Tooke Chairman High Level Group on Clinical Effectiveness 6

128 Part 1 Issues and Recommendations

129 Introduction 1.1 In October 2006 the Chief Medical Officer asked Professor Sir John Tooke to chair a High Level Group (HLG) on Clinical Effectiveness in response to the chapter Waste not, want not in his 2005 report On the State of the Public Health. The scope of the HLG s work was defined by the findings of this report: Variation in the provision of specific health services may be appropriate but it can also suggest waste or inequity within the NHS. Both under-use and over-use of medical interventions can be costly and expose patients to unnecessary risk. Variation that cannot be explained by the needs of patients may occur on the basis of geography ( postcode prescribing ), on account of the preferences and habits of clinical decision makers, or due to other factors, such as the socioeconomic status of patients. Variation is demonstrable in many areas of medical practice, including prescribing patterns, hysterectomy, treatment for people with coronary disease, and tonsillectomy among children. Inappropriate variation may be a function of poor knowledge, the flawed application of the correct knowledge, a lack of resources, or the inappropriate allocation of extant resources. The efforts of the National Institute for Health and Clinical Excellence (NICE) and NHS Connecting for Health in the effective dissemination of knowledge should be redoubled. Commissioners of health services should reaffirm their commitment to the NHS principle of equity, and techniques should be developed further to facilitate benchmarking of provision. NICE should be asked to issue guidance to the NHS on disinvestment, away from established interventions that are no longer appropriate or effective, or do not provide value for money The Group s terms of reference were: To identify illustrative and informative examples of major clinical significance drawn from major domains of clinical care in England where there are significant variations in (evidence-based) best practice and/or where efficiency and organisation of care fall short of international benchmarks. To review the evidence base for measures aimed at enhancing the effectiveness and efficiency of clinical care that might be applied to these domains. Drawing on these analyses, to recommend a programme of action to enhance the effectiveness and efficiency of clinical care, including specific incentives that might be employed. 1.3 High Level Group members are listed at page The HLG subsequently commissioned Professor Martin Eccles of the Institute of Health and Society at Newcastle University to summarise existing activity within government, the NHS, the Royal Colleges and professional and other relevant bodies, and set out current thinking on clinical effectiveness and barriers to implementation of guidelines and the boundaries of current knowledge. His work forms Part 2 of this publication. It shows that there is a wide range of activities aimed at supporting clinical effectiveness ongoing within the NHS, but the degree to which they are fully integrated and aware of each other s activities is not clear. Additionally, it is not clear which organisation has lead responsibility for the several dimensions of clinical effectiveness. The report also identifies the rapidly increasing volume of evidence available to inform promotion of clinical effectiveness. Professor Eccles proposed that a series of explicit frameworks would be useful in order to address systematically the promotion of clinical effectiveness and suggested 1 On the State of the Public Health, Annual report of the Chief Medical Officer Department of Health,

130 Introduction that such frameworks are not currently commonly used. 1.5 The Group considered this evidence and identified specific underlying issues, and has made recommendations to the Chief Medical Officer to address them. These include possible ways forward to improve clinical effectiveness in the NHS and promote clinical engagement to deliver this. 9

131 Issues 2.1 The HLG s view is that the NHS does not systematically address evidence-based clinical effectiveness (EBCE). There are a number of factors behind this: A policy climate that emphasises aspects of performance other than those that relate to EBCE. Whilst many of these may be important, they are prosecuted at the cost of the clinical effectiveness agenda (e.g. clinical governance versus clinical effectiveness). This produces disincentives for managers to engage with this as an issue. Whilst there is much activity that, in general, relates to EBCE, the coordination and alignment of these activities is unclear. EBCE is complex and requires a systematic, organisation-wide approach. The absence of systematic data on the quality of clinical care. Whilst there are some areas of data (e.g. some areas of the Quality and Outcomes Framework in primary care) there are many where this is not the case. There was a general feeling within the HLG that such data should be the subject of legitimate and routine discussion at Board level in healthcare organisations, and is not. Healthcare organisations have little or no financial resource to direct towards EBCE. This is a chicken and egg problem; where other things are important and urgent they will attract attention and funding. Until EBCE is important and urgent, it will not. Were resources to be available, most organisations do not have trained staff with the requisite skills to address EBCE. Novel skills would be likely to include: being able to utilise the evidence in Section 3 of Part 2 of this report; understanding the science of behaviour change at an individual, team and organisational level; understanding theoretical perspectives on human behaviour change; conducting diagnostic analyses; identifying and using explicit change models and frameworks. 2,3 Sadly, there is the issue of paradigm wars. The nature and credibility of evidence will be challenged from disciplinary, rather than scientific perspectives. Rather than concentrating on legitimate debates, such as how best to match study design to question posed, arguments will instead continue to be conducted about questions such as whether randomised controlled trials (RCTs) are ever relevant in quality of care or organisational research. What do we mean by evidence-based clinical effectiveness? 2.2 What are the core activities that comprise EBCE, not only for clinicians but for all staff working in the NHS? There are current definitions of clinical effectiveness (e.g. footnote 6 on page 17) but these do not automatically translate into the structure, staffing and skills that would be required. Defining these (by some means) would be a large step along the road to defining what a systematic approach to EBCE would look like. 2.3 This would be an important prerequisite to subsequent steps. 2 Sanson-Fisher RW, Grimshaw JM, Eccles MP. The science of changing provider s behaviour: the missing link in evidence-based practice. Medical Journal of Australia, 2004; 180: Grol R, Wensing M, Eccles M (Eds). Improving Patient Care: Implementing change in clinical practice. Oxford, Elsevier,

132 Issues Engaging constituencies 2.4 Fundamental to the HLG s approach is engagement of the constituency clinicians, managers and patients throughout the process. This needs to include a Clinical Reference Board of respected clinicians, currently not in management roles, as well as links with the professions through the Colleges and Academy of Royal Colleges. 2.5 Part of the process of engaging constituencies will involve engaging with those involved in the current activities listed in Appendix 1. Strong central clinical voices 2.6 As a new generation of medical leaders emerges through a variety of educational initiatives this need will begin to be addressed; but the voice for clinical quality needs to be heard more strongly at Strategic Health Authority level and professional bodies need to promote the need for evidencebased transformational change. What are the educational needs raised by this agenda? 2.7 Education spanning undergraduate selection through to Continuous Professional Development, revalidation and remediation needs to embrace not only the technical aspects of evidence-based practice and change management but also the inculcation of a new professionalism, which acknowledges the resource management and service responsibilities of the doctor and other clinical leaders. 2.8 Pursuing an EBCE agenda would rapidly identify a number of educational needs of which this list provides a starting point: There is a need for clinicians to critically appraise, understand and be able to use the results of evidence-based implementation. In general, evidence-based medicine skills (question recognition and formulation, searching, critical appraisal, interpretation and application of research evidence) are still absent or poorly developed in much of clinicians education, both undergraduate and postgraduate. These skills are an essential underpinning for a culture of clinical effectiveness, and national standards and training are needed. Diagnosis and treatment of clinical effectiveness problems, including understanding of how to use the science of behaviour change at an individual, team and organisational level; understanding theoretical perspectives on human behaviour change; conducting diagnostic analyses; identifying and using explicit change models and frameworks. Knowledge and skills in this area will draw heavily on the social sciences. 4 An understanding of the relationships between health policy, management and evidence. This will deal with issues such as the differing use of evidence by different groups Examples of two programmes from Canada that address these issues to some extent are the Alberta SEARCH Training Programme ( and the Canadian Health Services Research Foundation EXTRA Programme ( What are the research needs raised by this agenda? 2.10 The research agenda was not the main focus of the scoping document. However, a number of areas emerged from the content; there are doubtless many more. Secondary research 2.11 There is a need for continuing efforts to identify and synthesise the relevant literature and to continue to develop the methods of synthesis in this area. Primary research 2.12 There is a need for research at all points of the process of promoting effective practice: the nature and applicability of evidence; the choice and performance of various change models; the role of behavioural theory within this area; the development of methods of formative evaluation; the production of standard tools and instruments; 4 Wensing M, Wollersheim H, Grol R. Organizational interventions to implement improvements in patient care: a structured review of reviews. Implementation Science, 2006; 1: 2. 5 Eccles M. What is the role of research and evidence in policy making? In: Rawlins M, Littlejohns P (Eds). Delivering Quality in the NHS. Oxford, Radcliffe Medical Press,

133 Issues the systematic development and trialling of interventions across a range of conditions and NHS settings. What is the clinician focus of the High Level Group s work? 2.13 Nowadays many areas of clinical activity are multidisciplinary. All of the writings on improving quality of care identify the importance of appropriate involvement of all members of clinical teams It will be important to ensure appropriate engagement of both primary care (where over 90% of patient contacts occur) and public health. What are the available incentives? 2.15 Job planning needs to recognise the requirement for protected time to manage change and re-engineer services Reward systems need to reflect the value of adopting clinically effective practice. The development domain of the Clinical Excellence Award (CEA) system could be better defined to recognise such achievements Status may motivate, as well as financial reward, and consideration should be given to the creation of a virtual faculty (analogous to the National Institute for Health Research NHS research faculty) of drivers of implementation. Faculty membership could contribute to CEA recognition. 12

134 Recommendations Alignment of central activities and support 3.1 The need for central support to encourage clinically effective practice is widely recognised and there is currently a wealth of activity, delivered by a number of agencies. Recommendation 1 We recommend that there should be better alignment and coordination of such activities at a national level, in the first instance with the creation of a cross-sectoral group to take this agenda forward. Promoting local ownership 3.2 The analysis of initiatives seeking to improve clinical effectiveness reveals that there is no magic bullet and solutions are context specific, requiring local diagnosis of the cause and often systematic multi-layer interventions. This can only be achieved if the local providers of healthcare are appropriately empowered to respond to these challenges, a need that will be even greater with the decentralisation of NHS control and a multiplicity of providers emerging as part of the plurality agenda. Recommendation 2 We recommend a range of measures to promote local ownership of the clinical effectiveness agenda, including: the need for demonstration of Board-level commitment; the creation and alignment of individual, team and institutional incentives to engage in such activity; the recognition and celebration of strong local practice including nationally coordinated webbased exposure and virtual faculty membership. Ensuring clinical engagement 3.3 Integral to local ownership is the critical issue of clinical engagement. Clinicians are key determinants of resource utilisation. Furthermore meeting population health needs, just like meeting those of the individual patient, requires an appreciation of the context, the complexity of the factors at play and the evidence base for particular interventions; qualities that doctors in particular should be able to exhibit. Harnessing such potential requires a contemporary interpretation of professionalism on the part of the clinician, adequate education and training, and alignment of incentives. Recommendation 3 We recommend a range of measures to equip and encourage clinicians to aspire to take the lead in promoting clinically effective practice, including: a fundamental review of the curricula from undergraduate studies through to revalidation to ensure that relevant technical skills and knowledge of improvement science are taught, and that the developing professional identity embraces the need to combine patient advocacy with broader responsibilities to the population served; putting clinicians, working in partnership with health service managers, at the heart of the local initiatives referred to in recommendation 2 above. Harnessing the capacities of academia 3.4 Many key advances in medical practice have derived from the NHS and higher education working in close partnership. University hospitals outperform non-university hospitals on quality criteria in the most recent Annual Health Check conducted by the Healthcare Commission. Yet changes in NHS 13

135 Recommendations organisation and the commissioning of education and training threaten the advantages such partnership can bring. Educational outreach, academic detailing and peer leaders are helpful approaches in the pursuit of clinical effectiveness. Moves to provide more care in the community and the growing importance of public health and chronic disease management offer opportunities for new models of academic health centres to drive quality improvement. Recommendation 4 We recommend that the health service harnesses better the capacity of higher education to assist with this agenda through promoting the development of new models of community-wide academic health centres to encourage relevant research, engagement and population focus and embed a critical culture that is more receptive to change. Pursuing the research agenda 3.5 If the solutions were straightforward, they would have been discerned by now. The HLG has uncovered many areas where the evidence base is lacking. In the long run the adoption of a non-evidence-based approach undermines the very culture that is being promoted. Recommendation 5 We recommend that an expert group be formed to reveal the major evidence deficits in relation to the implementation of clinical effectiveness and highlight the issues of organisational receptiveness and culture and functional teams for particular attention. The outcome of such work should feed in to commissioning considerations of the NHS R&D Service Delivery and Organisation Programme. High Level Group on Clinical Effectiveness Membership Professor Sir John Tooke Sir Iain Chalmers Professor Bernard Crump Ian Dodge Professor Shah Ebrahim Professor Martin Eccles Professor Paul Glasziou Sir Muir Gray Chairman, and Dean, Peninsula Medical School Editor, James Lind Library Chief Executive Officer, NHS Institute for Innovation and Improvement Reform Implementation, Department of Health Professor of Public Health, London School of Hygiene and Tropical Medicine Professor of Primary Care Research, Newcastle University Director, Centre for Evidencebased Medicine, University of Oxford Director of Clinical Knowledge, Process and Safety, NHS Connecting for Health Dr Gillian Leng Professor Stuart Logan Implementation Systems Director, National Institute for Health and Clinical Excellence Director, Health and Social Care, Peninsula Medical School Professor Chairman, National Institute for Sir Michael Rawlins Health and Clinical Excellence Hugh Rogers NHS Institute for Innovation and Improvement Dr Sheila Shribman National Clinical Director for Children, Young People and Maternity Services, Department of Health Dr Jenny Simpson Chief Executive, British Association of Medical Managers John Bromley Deirdre Feehan Anne Moger Secretariat, Department of Health Secretariat, Department of Health Secretariat, Department of Health 14

136 Part 2 Scoping Report for the High Level Group on Clinical Effectiveness by Professor Martin Eccles Paula Whitty (Section 1) Institute of Health and Society Newcastle University

137 Summary 4.1 This scoping report has three sections. 4.2 Section 1 presents a descriptive background of who is currently doing what. It covers initiatives within the Department of Health (DH), the National Institute for Health and Clinical Excellence (NICE), the NHS Institute for Innovation and Improvement, and the Healthcare Commission; ongoing workstreams within the Royal Colleges and other professional bodies, and local initiatives in the NHS. There is a wide range of national-level activities aimed at supporting clinical effectiveness ongoing within the NHS. The degree to which activities are fully integrated, and participants are aware of each other s activities, is not clear. It is not clear whether any single organisation has lead responsibility for the several dimensions of clinical effectiveness. It is much less certain what is happening at a local level; activities appear to be much less well documented. We are not aware of local organisations with the capability to carry out detailed diagnostic analysis, intervention development, or comprehensive monitoring of key clinical outcomes or proxy outcomes. 4.3 Section 2 presents current thinking on how to approach the issues of clinical effectiveness and barriers to implementation of guidelines or effective practices. 4.4 When it comes to promoting clinically effective practice, everyone is sure that they know what needs to be done. Their solutions usually reflect discipline, areas of expertise or interest; they seldom agree, and are more likely to be belief- than evidence-based. Different players in healthcare use different approaches to changing clinical practice; most of these approaches are more based on beliefs than on scientific evidence. Implementing such changes seldom entails a single action; it usually demands good planning and a combination of different interventions. Before a strategy to implement change is selected, the obstacles to change should be identified. Evidence-based medicine should be complemented by evidence-based implementation. 4.5 This section presents a series of frameworks that offer a structure for thinking about improving clinical effectiveness in the context of the NHS in England. The frameworks reflect: levels of organisation within healthcare; priorities for choosing topics for attention; a model for diagnosing and treating problems of clinical effectiveness; available empirical treatments for clinical effectiveness problems. 4.6 Section 3 presents an overview of current knowledge on the effectiveness of strategies to promote the uptake of clinically effective practice. Evidence of what to do (effectiveness of interventions) to promote clinical effectiveness should come from (systematic reviews of) randomised controlled trials (RCTs). Evidence of how to do it should come from formative evaluations, ideally conducted within or alongside RCTs. There is a large and increasing amount of evidence to inform the uptake of clinically effective practice. While there may be methodological issues within this literature, it represents a valuable resource to underpin evidence-based clinical effectiveness. 16

138 Section 1 Activities in the NHS related to supporting clinical effectiveness Summary There is a wide range of national-level activities aimed at supporting clinical effectiveness ongoing within the NHS. The degree to which activities are fully integrated, and participants are aware of each other s activities, is not clear. It is not clear whether any single organisation has lead responsibility for the several dimensions of clinical effectiveness. It is much less certain what is happening at a local level; activities appear to be much less well documented. We are not aware of local organisations with the capability to carry out detailed diagnostic analysis, intervention development, or comprehensive monitoring of key clinical outcomes or proxy outcomes. Introduction 5.1 This section provides an overview of current activity on clinical effectiveness in the English NHS; detail is provided in the tables in Appendix 1. It covers initiatives within DH, NICE, the NHS Institute for Innovation and Improvement, and the Healthcare Commission; ongoing workstreams within the Royal Colleges and other professional bodies, and an overview of local initiatives in the NHS. 5.2 Note that we have attempted to give a reasonably comprehensive overview of relevant activities, regardless of whether or not there is an evidence-base for the effectiveness of the initiatives described. We have also focused on the activities and not on the outcomes of those activities. 5.3 The inclusion of activities as relevant is informed by the NHSE 1996 definition of clinical effectiveness 6 and the domain outcome in DH Standards for Better Health. 7 However, while some of the activities described do have a bearing on cost-effectiveness/efficiency, the focus here is mainly on clinical effectiveness. We have also omitted activities related to the basic underpinning of clinically effective care such as new evidence generation through research, and undergraduate or other basic clinical training. 5.4 To ensure reasonable coverage of relevant activities, they were initially mapped and are categorised in Appendix 1 using the following framework: 8 setting evidence-based standards; supporting delivery of evidence-based standards/effective care; monitoring the delivery of effective care. 5.5 Activities deemed to be supporting delivery of effective care are further sub-divided as: inputs (knowledge, skills, other resources); processes that support delivery; activities that focus on outcomes. 5.6 A draft of this section and Appendix 1 has been rapidly reviewed for omissions by colleagues leading on clinical effectiveness in two acute Trusts and two Primary Care Trusts (PCTs). 5.7 The detailed list of relevant activities is provided in Appendix 1. The remainder of this section of the report provides an overview, and also highlights obvious gaps in current provision. 6 The extent to which specific clinical interventions, when deployed in the field for a particular patient or population, do what they are intended to do i.e. maintain and improve health and secure the greatest possible health gain from the available resources. Promoting Clinical Effectiveness A framework for action in and through the NHS. NHSE, Patients achieve health benefits that meet their individual needs through health care decisions and services based on what assessed research evidence has shown provides effective clinical outcomes. In: Standards for Better Health. Department of Health, July Adapted from A First Class Service. Department of Health,

139 Section 1: Activities in the NHS related to supporting clinical effectiveness National-level activities 5.8 National activities fall, broadly, into government- and non government-funded initiatives. Of the government-funded initiatives, there are core clinical effectiveness organisations or programmes (e.g. NICE) and supportive programmes (such as the National Programme for IT (NPFIT)/NHS Connecting for Health (CfH)). Non governmentfunded initiatives are chiefly those funded and provided by professional bodies, although there are some charitable foundations operating in the broader quality field (e.g. the Health Foundation). 5.9 There is clear responsibility for setting evidencebased clinical standards through NICE. Currently NICE produces guidance in four areas of health: Technology appraisals guidance on the use of new and existing medicines and treatments within the NHS in England and Wales. Clinical guidelines guidance on the appropriate treatment and care of people with specific diseases and conditions within the NHS in England and Wales. Interventional procedures guidance on whether interventional procedures used for diagnosis or treatment are safe enough and work well enough for routine use in England, Wales, Scotland and Northern Ireland. Public health intervention and programme guidance National Service Frameworks (NSFs) also set standards at an organisational level. There will always be an issue of coverage, particularly in keeping up with technology appraisal of new treatments. NICE has also recently gained responsibility for a workstream to identify inappropriate interventions The Healthcare Commission has clear responsibility for assessing performance against Standards for Better Health, and new and existing national targets, in individual healthcare organisations. While the Commission s Annual Health Check currently covers Trust structures, policies and procedures relevant to clinical effectiveness reasonably well, evidence on the actual delivery of NICE guidance is more limited. NICE and the Healthcare Commission are working together to improve the availability of indicators related to the delivery of NICE guidance, and this also forms a key plank of NICE s Implementation Programme. The coverage of NICE guidance in the Healthcare Commission s assessment activities is shown in Table Other evidence on clinical outcomes or the delivery of effective care is limited to national targets assessment or service reviews. The Healthcare Commission is currently reviewing its overall approach to clinical effectiveness assessment, including the potential for a wide-ranging basket of clinical indicators to operate within its wider surveillance activities There is also a range of other bodies whose findings contribute to the monitoring of effective care, notably the national clinical audits (which are usually, but not exclusively, funded by the Healthcare Commission and operated by the Royal Colleges or Health and Social Care Information Centre (HSIC)). However, even taking all these activities together with those of the Healthcare Commission, NICE s Implementation Programme, and other initiatives such as the Better Metrics project, there is no comprehensive national programme that covers all key areas of clinical care There is a wide range of activities which potentially offer support to the delivery of clinically effective care, but also some gaps Knowledge provision (on what is effective practice) is (potentially) well covered by initiatives; whereas training on evidence-based practice has declined substantially since its heyday in the late 1990s (probably mirroring the decline of the Table 1. Coverage of NICE guidance in the Healthcare Commission s assessment activities Healthcare Commission assessment Technology appraisals Clinical guidelines NICE guidance Interventional procedures Public health guidance National clinical audits Yes Service reviews Yes Yes Assessment of core standards Yes Yes Yes Assessment of developmental standards Yes Yes New national targets Yes Yes National reviews Yes Independent sector inspections Yes Yes Yes 18

140 Section 1: Activities in the NHS related to supporting clinical effectiveness evidence-based medicine (EBM) movement as a popular method for promoting clinical effectiveness) Incentives (targets, financial incentives) are strong in those areas also covered by the early NSFs (particularly heart disease; cancer and mental health are well-supported by targets but not in the Quality and Outcomes Framework (QOF), though there is more mental health coverage in the latest revision) NICE s Implementation Programme includes a numbers of tools to support the delivery of its guidance. There is generic advice on implementation through their How to guide, which identifies steps separately for technology appraisals, clinical guidelines and interventional procedures guidance. Implementation advice tailored to specific topics, accompanied by slide sets, is available for more recent guidance and includes example action plans. NICE is planning to produce a How to influence practice guide, which may cover some of the more detailed aspects of implementation not yet covered by their tools (e.g. diagnostic analysis to identify barriers to and facilitators of the delivery of high quality care, and intervention development techniques to systematically develop the most suitable tailored interventions to improve care) The NHS Institute for Innovation and Improvement has a broader role to support the NHS in service transformation/improvement. In particular, its Delivering Quality and Value programme aims to improve efficiency as well as effectiveness of care. Outputs from this programme include the Better Value, Better Care Indicators and nine Focus On documents to date for high volume clinical topics. These documents provide illustrative care pathways and overarching organisational characteristics that appear to lead to best practice in these areas, as well as case studies and field tests. Within specific care groups or disease areas, there may also be practical support available through other national Institutes (e.g. the National Institute for Mental Health in England (NIMHE)) or national support teams (e.g. for heart disease or diabetes). The functions of the latter teams are currently being reviewed by the Office of the Strategic Health Authorities. Local initiatives 5.19 There are a number of expectations related to clinical effectiveness laid on Trusts, including in national guidance 9 and the Standards for Better Health. 7 There is also a legal requirement for organisations to fund NICE-recommended technology appraisals, as well as more recent clarification on how Trusts should proceed in situations where new treatments have not yet been, or are not planned to be, appraised by NICE The clinical and cost-effectiveness standard within Standards for Better Health states: Core standards C5. Healthcare organisations ensure that: a) they conform to NICE technology appraisals and, where it is available, take into account nationally agreed guidance when planning and delivering treatment and care; b) clinical care and treatment are carried out under supervision and leadership; c) clinicians continuously update skills and techniques relevant to their clinical work; and d) clinicians participate in regular clinical audit and reviews of clinical services. Developmental standard D2. Patients receive effective treatment and care that: a) conform to nationally agreed best practice, particularly as defined in National Service Frameworks, NICE guidance, national plans and agreed national guidance on service delivery; b) take into account their individual requirements and meet their physical, cultural, spiritual and psychological needs and preferences; c) are well coordinated to provide a seamless service across all organisations that need to be involved, especially social care organisations; and d) is delivered by healthcare professionals who make clinical decisions based on evidencebased practice Healthcare organisations had to declare their performance against the core standards, including C5 above, for the first time at the end of 2005/06. After selective inspection of 20% of Trusts, which in some cases will have resulted in amendments to their self-assessment, their performance was published in the Healthcare Commission s 2005/06 Annual Health Check (AHC) in October For this 9 HSC 1999/065: Clinical Governance in the New NHS 10 HSC 1999/176; Good Practice Guidance. Department of Health,

141 Section 1: Activities in the NHS related to supporting clinical effectiveness AHC, the percentages of Trusts declaring fully met for C5 were: (a) 85%; (b) 94% (c) 96% (d) 95%. The first Trust declarations on the developmental clinical and cost effectiveness standard will not be available until 2007, and only for acute and mental health Trusts Other surveys of the implementation of clinical governance have tended to suggest that less progress has been made on quality improvement than on quality assurance. For example, a national survey of healthcare managers perceptions suggested that they perceived higher achievement against items concerning structural change, corporate accountability and risk management than against those concerned with quality improvements and outcomes. Benchmarking and the use of clinical indicators were two of the items which respondents considered were particular areas of underachievement The local activities listed in Appendix 1 are drawn from example clinical effectiveness strategies and from the author s and reviewees experience. Clinical effectiveness initiatives are not separate from the core business of organisations e.g. the various system reform/redesign initiatives are all relevant. In terms of current reforms, one of the key initiatives is commissioning, and ensuring this is based on quality standards is essential hence the NICE initiative to produce commissioning guides While some relevant activities may have been missed, we are not aware of local organisations with the capability to carry out detailed diagnostic analysis, intervention development, or comprehensive monitoring of key clinical outcomes or proxy outcomes. 11 Freeman T, Walshe K. Achieving progress through clinical governance? A national study of health care managers perceptions in the NHS in England. Quality and Safety in Health Care, 2004; 13:

142 Section 2 How to approach the issues of clinical effectiveness and barriers to implementation of guidelines or effective practices Summary 6.1 When it comes to promoting clinically effective practice, everyone is sure that they know what needs to be done. Their solutions usually reflect discipline, areas of expertise or interest; they seldom agree, and are more likely to be belief- than evidence-based. Different players in healthcare use different approaches to changing clinical practice; most of these approaches are more based on belief than on scientific evidence. Implementing such changes seldom entails a single action; it usually demands good planning and a combination of different interventions. Before a strategy to implement change is selected, the obstacles to change should be identified. Evidence-based medicine should be complemented by evidence-based implementation. 6.2 This section presents a series of frameworks that offer a structure for thinking about improving clinical effectiveness in the context of the NHS in England. The frameworks reflect: levels of organisation within healthcare; priorities for choosing topics for attention; a model for diagnosing and treating problems of clinical effectiveness; available empirical treatments for clinical effectiveness problems. Introduction 6.3 When it comes to promoting clinically effective practice, everyone is sure that they know what needs to be done. Their solutions usually reflect discipline, areas of expertise or interest; they seldom agree, and are more likely to be belief- than evidence-based. The reason for suggesting the need for an explicit, systematic approach to clinical effectiveness is illustrated by Grol (1997). 12 That improvements are possible in many areas of clinical care has become increasingly clear. The different players within health care, however clinicians, epidemiologists, health services researchers, educationalists, social scientists, economists, health authorities often have different ideas on the best strategies to improve practice and the best way of making changes. An example Let us assume that aggregated data, collected by health authorities, disclose that the rate of caesarean section in a specific district is exceptionally high. A committee is formed with experts and representatives of various interests to develop plans for improving obstetric care. Hearing the problem, all are worried. The clinician either denies there is a problem or proposes setting up a well designed course to increase clinicians knowledge and skills. OK, says the clinical epidemiologist, but we first need to know what the evidence is on the indications for a caesarean section. We should perform a meta-analysis and come up with evidence-based guidelines to disseminate among the obstetricians. No, says the educational expert: that is a top down approach and such strategies will usually fail. Form small groups of doctors and let them discuss the problem, using cases and experiences from their own practices as the basis for local arrangements on new routines. 12 Grol R. Personal paper: Beliefs and evidence in changing clinical practice. BMJ, 1997; 315:

143 Section 2: How to approach the issues of clinical effectiveness and barriers to implementation of guidelines or effective practices We should take a look at the facts first, says the health services researcher. Let us set up a multicentre audit first and collect data on actual variation between hospitals and include data on casemix. Feeding this information back to the hospitals will probably stimulate improvement. You are all focusing too much on the individual doctor, says the management expert. The problem is not the doctor, but the system. We should analyse the process of decision making and performing the caesarean sections and see what structures determine the process. Next we need a quality improvement team. This is all too much talking, says the representative of the health authorities. Doctors are sensitive only to what happens to their budgets. We need to put a pressure on them to limit the number of caesarean sections per hospital, give hospitals a reasonable budget, and provide the obstetricians with an incentive when they reduce the rate. 6.4 Having expanded the arguments within the paper, Grol s summary points are: Different players in healthcare use different approaches to changing clinical practice; most of these approaches are more based on belief than on scientific evidence. Implementing such changes seldom entails a single action; it usually demands good planning and a combination of different interventions. Before a strategy to implement change is selected, the obstacles to change should be identified. Evidence-based medicine should be complemented by evidence-based implementation. Frameworks 6.5 This section presents a series of frameworks that offer a structure for thinking about improving clinical effectiveness in the context of the NHS in England. The frameworks reflect: levels of organisation within healthcare; priorities for choosing topics for attention; a model for diagnosing and treating problems of clinical effectiveness; available empirical treatments for clinical effectiveness problems. 6.6 Each of these frameworks, in its own way, acknowledges both the complexity of healthcare delivery and the fact that local circumstances will vary. Whilst there are interventions that have been shown in research studies to be effective across a range of contexts, the introduction of such interventions into a local healthcare setting will always be dependent on an understanding of local context. This understanding of context can be central to successful clinical effectiveness. The frameworks all identify the importance of, but do not seek to control, this local variation but clearly identify it as something that should be understood at a local level. Levels of engagement seeing the whole picture 6.7 Efforts to improve the quality of care, particularly for chronic diseases which are complex to manage, need to occur at and be coordinated across multiple levels such as the patient, clinician, team, organisation, policy. In their article in Milbank Quarterly, 13 Ferlie and Shortell said: 6.8 Fuelled by public incidents and growing evidence of deficiencies in care, concern over the quality and outcomes of care has increased in both the United Kingdom and the United States. Both countries have launched a number of initiatives to deal with these issues. These initiatives are unlikely to achieve their objectives without explicit consideration of the multilevel approach to change that includes the individual, group/team, organization, and larger environment/system level. Attention must be given to issues of leadership, culture, team development, and information technology at all levels. A number of contingent factors influence these efforts in both countries, which must each balance a number of tradeoffs between centralization and decentralization in efforts to sustain the impetus for quality improvement over time. The multilevel change framework and associated properties provide a framework for assessing progress along the journey. (The emphasis is mine.) 6.9 This framework highlights the importance of considering the multiple levels at which healthcare is delivered and the interplay between them. 13 Ferlie EB, Shortell SM. Improving the quality of health care in the United Kingdom and the United States: a framework for change. The Milbank Quarterly, 2001; 79(2):

144 Section 2: How to approach the issues of clinical effectiveness and barriers to implementation of guidelines or effective practices Choosing topics 6.10 Many of the taxonomies for choosing topics relate to guideline development. They vary slightly in emphasis but address many of the same general issues. Three examples are offered The first, from NICE, is summarised here and is in full in Appendix 2. NICE s criteria clearly refer to the specific context in which this taxonomy operates. The selection criteria take into account: burden of disease (population affected, morbidity, mortality); resource impact (i.e. the cost impact on the NHS or the public sector); policy importance (i.e. whether the topic falls within a government priority area); whether there is inappropriate variation in practice across the country; factors affecting the timeliness or urgency for guidance to be produced The criteria used by Shekelle and colleagues 14 focus on attributes of the clinical condition: 6.13 Guidelines can be developed for a wide range of subjects. Clinical areas can be concerned with conditions (abnormal uterine bleeding, coronary artery disease) or procedures (hysterectomy, coronary artery bypass surgery). Given the large number of potential areas, some priority setting is needed to select an area for guideline development. Potential areas can emerge from an assessment of: the major causes of (modifiable) morbidity and (premature) mortality for a given population; uncertainty about the appropriateness of healthcare processes (i.e. inappropriate variation); evidence that they (clinical interventions) are effective in improving patient outcomes; the need to conserve resources in providing care The criteria for topic selection given by Burgers and colleagues 15 are less explicit about these but go on to include dimensions of the perceived feasibility of addressing the process: the topic concerns a relevant problem that occurs frequently, and guideline development allows improvement in health or cost reduction; it is possible to define the topic and focus on the most crucial aspects; there is uncertainty or difference of opinion about the best care; there is a need to bring together scientific knowledge and expertise, or there are new insights; sufficient scientific evidence is available; there is a real opportunity to achieve consensus on the final recommendations; it is possible to formulate feasible recommendations One consideration for the use of any set of criteria is the degree to which current clinical effectiveness activities cover the major relevant clinical conditions. A model for diagnosing and treating problems of clinical effectiveness 6.16 It is important to have an integrated structure with which to represent various steps and sequences of a process to change behaviour and improve clinical effectiveness. There are several such available, one of which, from Grol 16 (2005), is reproduced overleaf The topic selection criteria above feature within the top boxes of this model. The Ferlie and Shortell levels feature throughout the model. Thus local context could be part of what generated the initial problems, would influence the structure of the planning process and would be an important consideration in the diagnostic analysis of performance group and setting. It would also shape the implementation plan A model such as this may look simple. However, when clinical governance leads from a (small) number of NHS organisations were interviewed about what evidence-based strategies they could routinely and systematically use to promote clinical effectiveness, they did not identify any systematic approach. 17 They reported that, in 14 Shekelle PG, Woolf SH, Eccles M, Grimshaw J. Clinical guidelines: Developing guidelines. BMJ, 1999; 318: Burgers J, Grol R, Eccles M. Clinical guidelines as a tool for implementing change in patient care. In: Grol R, Wensing M, Eccles M (Eds). Improving Patient Care: Implementing change in clinical practice. Oxford, Elsevier, Grol R, Wensing M. Effective implementation: A model. In Grol R, Wensing M, Eccles M (Eds). Implementation of Change in Clinical Practice, pp Oxford, Elsevier, Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay C, Vale L, Whitty P, Eccles M, Matowe L, Shirren L, Wensing M, Dijkstra R, Donaldson C. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technology Assessment, 2004; 8(6) iii-iv,

145 Section 2: How to approach the issues of clinical effectiveness and barriers to implementation of guidelines or effective practices Figure 1. Implementation of change: a model Research findings or guidelines Problems identified, good experiences or best practices Planning clear aims coordination/team involving target group budget time schedule Development of concrete proposal/ targets for improvement or change Adapting or improving proposal Analysis of performance, target group and setting Supplementary analyses Development/selection of strategies and measures to change practice New strategies Development, testing and execution of implementation plan Adapting plan (Continuous) evaluation and (where necessary) adapting plan Goals not achieved terms of available skills and resources, all they planned were postal distribution of educational materials, audit and feedback and educational meetings. Similarly, in the USA at an American College of Chest Physicians meeting in 2004, out of 87 participants (a 77% response rate) completed a survey about feasibility, acceptability, and cost for each of six implementation strategies. Respondents indicated that distribution of educational material and educational meetings were feasible, acceptable and of low cost; differences between the rating of these two strategies were not statistically significant. Respondents reported educational outreach visits, computer reminders and patient-mediated interventions as feasible and acceptable, but rated educational outreach visits as more costly. Audit and feedback scored lower than all other implementation strategies in terms of feasibility and acceptability; respondents also rated audit and feedback as unfavourably as educational outreach visits in terms of cost The skills, structures and resources to systematically integrate the use of a model such as Grol s at a local level are not known. Available empirical treatments for clinical effectiveness problems 6.20 Another time when it is helpful to have a framework is when thinking about the range of available effective treatments for a clinical effectiveness problem. This step represents Grol s Evidence-based implementation (the effectiveness of interventions is dealt with in Section 3) and fits into the Development/selection of strategies box of Grol s model On page 25, we present the (empirically-based) taxonomy of interventions used by the Cochrane Collaboration Effective Practice and Organisation of Care (EPOC) Group. The purpose of presenting the taxonomy is to demonstrate both the range and complexity of the available candidate treatments. There are two important points regarding the taxonomy. Firstly, it is empirically derived; this means that it will not be absolutely exhaustive. Secondly, there is a considerable volume of published evidence behind many of the techniques listed (see Section 3) EPOC reviews, and hence the taxonomy, include professional, financial, organisational or regulatory interventions. A summarised list is presented here; full details can be obtained from the Data Collection Checklist at 18 Schunemann HJ, Cook D, Grimshaw J, Liberati A, Heffner J, Tapson V, Guyatt G. Antithrombotic and Thrombolytic Therapy: From Evidence to Application. The Seventh ACCP Conference on Antithrombotic and Thrombolytic Therapy. Chest, 2004; 126: 688S 696S. 24

146 Section 2: How to approach the issues of clinical effectiveness and barriers to implementation of guidelines or effective practices Professional interventions a) Distribution of educational materials b) Educational meetings c) Local consensus processes d) Educational outreach visits e) Local opinion leaders f) Patient-mediated interventions g) Audit and feedback h) Reminders i) Marketing j) Mass media. Financial interventions Provider interventions a) Fee-for-service b) Prepaid c) Capitation d) Provider salaried service e) Prospective payment f) Provider incentives g) Institution incentives h) Provider grant/allowance i) Institution grant/allowance j) Provider penalty k) Institution penalty l) Formulary. Patient interventions a) Premium b) Co-payment c) User fee d) Patient incentives e) Patient grant/allowance f) Patient penalty. Organisational interventions Provider-orientated interventions i) Communication and case discussion between distant health professionals. Patient-orientated interventions a) Mail order pharmacies b) Presence and functioning of adequate mechanisms for dealing with patients suggestions and complaints c) Consumer participation in governance of healthcare organisation. Structural interventions a) Changes to the setting/site of service delivery b) Changes in physical structure, facilities and equipment c) Changes in medical records systems d) Changes in scope and nature of benefits and services e) Presence and organisation of quality monitoring mechanisms f) Ownership, accreditation, and affiliation status of hospitals and other facilities g) Staff organisation. Regulatory interventions A regulatory intervention is any intervention that aims to change health services delivery or costs by regulation or law. These interventions may overlap with organisational and financial interventions. a) Changes in medical liability b) Management of patient complaints c) Peer review d) Licensure. a) Revision of professional roles b) Clinical multidisciplinary teams c) Formal integration of services d) Skill mix changes e) Continuity of care f) Arrangements for follow-up g) Case management h) Satisfaction of providers with the conditions of work and the material and psychic rewards 25

147 Section 3 Current knowledge on the effectiveness of strategies to promote the uptake of clinically effective practice Summary Evidence of what to do (effectiveness of interventions) to promote clinical effectiveness should come from (systematic reviews of) randomised controlled trials (RCTs). Evidence of how to do it should come from formative evaluations, ideally conducted within or alongside RCTs. There is a large and increasing amount of evidence to inform the uptake of clinically effective practice. Whilst there may be methodological issues within this literature, it represents a valuable resource to underpin evidence-based clinical effectiveness. Introduction 7.1 When reading an evaluation of an intervention that they might wish to use, a reader should consider aspects of both the internal and external validity. Internal validity will relate to elements of study design and how unbiased, or true, the results are. External validity will relate to attributes of the population studied and how they relate to the population to which the reader wishes to apply the results. 7.2 In using evidence about the effectiveness of strategies to promote the uptake of clinically effective practice both these considerations apply. Evaluative designs 7.3 Evidence about the effectiveness of strategies to promote the uptake of clinically effective practice comes from a number of sources. Healthcare is complex, with a myriad of factors able to have an impact on the process and outcome of care. This very complexity means that, to understand the unbiased effect of an intervention, the gold standard design is the RCT, with its ability to control for known and unknown confounders. There are now many hundreds of these and they are routinely drawn together in systematic reviews. 7.4 There are situations where it is not feasible to use an RCT design and other, non-experimental, designs are used. Where it is not possible to identify a control site (e.g. mass media interventions), interrupted time series designs are the most robust design. For infrequent occurrences (e.g. hospital takeovers) case study designs are appropriate. However, any non-experimental design lays the onus of avoiding bias at all stages (design, conduct and interpretation) onto the investigator. 19 This problem is topically illustrated by the differing interpretations of the impact of the Institute for Healthcare Improvement (IHI) 100,000 Lives Campaign. The IHI report is of 122,300 lives saved; an independent assessment suggests that this estimate may be systematically biased and that the true figure is unknowable but considerably smaller. 20,21 The context for RCTs of strategies to promote clinically effective practice 7.5 RCTs, however, do not sit alone they represent a point in a scientific continuum 22 of evaluation that draws on prior work exploring issues such as the underlying theory, the range and role of moderating and mediating factors on the behaviours of interest and the development and pilot testing of 19 Shadish, WR, Cook, TD, Campbell, DT. Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Boston, Houghton-Mifflin, Wachter RM, Pronovost PJ. The 100,000 Lives Campaign: A scientific and policy review. Joint Commission Journal on Quality and Patient Safety, 2006; 32: Authors reply pp Berwick DM, Hackbarth AD, McCannon CJ. IHI replies to The 100,000 Lives Campaign: A Scientific and Policy review. Joint Commission Journal on Quality and Patient Safety, 2006; 32: Campbell M, Fitzpatrick R, Haines A, Kinmonth AL, Sandercock P, Spiegelhalter D, Tyrer P. Framework for design and evaluation of complex interventions to improve health. BMJ, 2000; 321(7262):

148 Section 3: Current knowledge on the effectiveness of strategies to promote the uptake of clinically effective practice Table 2. MRC Framework for the evaluation of complex interventions Evaluation of drugs Pre-clinical Phase I Phase II Phase III Phase IV Evaluation of implementation strategies Theory Modelling Exploratory trial Definitive RCT Long-term implementation and sustainability candidate interventions. This is commonly referred to as the Medical Research Council (MRC) Framework for the evaluation of complex interventions and is represented in Table 2 above. 7.6 Whilst RCTs represent the gold standard for evaluating what happened, they say little about how observed effects may have been achieved or why. To understand this it is necessary to conduct process evaluations (also known as formative evaluations 23,24 ) embedded within trials these are conducted much less commonly, though examples do exist. 25,26 Development work, such as that conducted within Phases 1 and 2 of the MRC Framework, also offers the means of identifying the factors to be measured within a process evaluation. Synthesising clinical effectiveness literature 7.7 The synthesis of literature in this area is challenging. As with all reviews, how the question (which the review is addressing) is framed will influence the literature that is synthesised. Identifying the literature is not straightforward as it is spread across a wide number of journals and there are no core journals that would be likely to contain a large proportion of the core literature (hence EPOC maintaining a specialist register of relevant studies). Having identified the studies, data extraction is complex with studies often reporting multiple study outcomes with no clearly identified primary endpoint. This can be compounded by design problems, principally failing to allow for the clustered nature of data which results in an overestimate of statistical significance. Data synthesis is then problematic. Because of the heterogeneity of endpoints, reviews often use narrative summary. When studies cannot be re-analysed, the use of the median effect size is the most informative way of quantifying the effect. However, as the quality of the literature improves it is possible to envisage this improving. The effect of computerising guidelines 25,26 The study evaluated the use of a computerised decision support system (CDSS) to implement evidence-based clinical guidelines for the management of adult patients with asthma or angina seen in 60 general practices in northern England. The RCT evaluation showed that there were no significant effects of CDSS on consultation rates, process of care measures (including prescribing) or any quality-of-life domain for either condition. A usage log across all practices showed that levels of use of the CDSS were low. Interviews conducted with physicians in five participating practices provided insights into why this was so. Interviewees were largely enthusiastic about the benefits of computing for general practice, and were optimistic about the potential for computers to present guidelines in a manageable format. However, the CDSS was felt by most practitioners to be difficult to use and unhelpful clinically. They believed that they were already familiar with the content of the guidelines, although they did not always follow recommendations for reasons that included limitations of the guidelines, patient preferences, lack of incentives and perceived structural barriers. The investigators concluded that even if it is possible to solve the technical hardware and software problems of producing a system that fully supports chronic disease management, there remains the challenge of integrating CDSS into clinical encounters in which busy practitioners manage patients with complex, multiple conditions. 23 Hulscher M, Laurant M, Grol R. Process evaluation of change interventions. In: Grol R, Wensing M, Eccles M (Eds). Implementation of Change in Clinical Practice. Oxford, Elsevier, Stetler CB, Legro MW, Wallace CM, Bowman C, Guihan M, Hagedorn H, Kimmel B, Sharp ND, Smith JL. The role of formative evaluation in implementation research and the QUERI experience. Journal of General Internal Medicine, 2006; 21: S Eccles M, McColl E, Steen N, Rousseau N, Grimshaw J, Parkin D, Purves I. Effect of computerised evidence-based guidelines on management of asthma and angina in adults in primary care: cluster randomised controlled trial. BMJ, 2002; 325: Rousseau N, McColl E, Newton J, Grimshaw J, Eccles M. Practice based, longitudinal, qualitative interview study of computerised evidence-based guidelines in primary care. BMJ, 2003; 326:

149 Section 3: Current knowledge on the effectiveness of strategies to promote the uptake of clinically effective practice Figure 2. New primary studies related to clinical effectiveness Volume of literature New Papers 7.8 As with many other areas of clinical practice, the volume of literature relevant to promoting clinical effectiveness is rapidly increasing. The Cochrane Collaboration EPOC Review Group maintain a register of relevant studies. The number of new primary studies by year of publication is shown in Figure This also applies to systematic reviews. When EPOC conducted an overview of systematic reviews of professional interventions for the time period they identified 41 reviews. An ongoing update has identified 78 new systematic reviews of professional interventions; organisational, financial and regulatory interventions are being included for the first time and there are 295, 20 and 9 of these respectively Systematic reviews in this general area may include study designs other that RCTs. The degree to which this occurs will vary from topic to topic and review to review. If a reviewer feels their question can be answered solely by RCTs (as is becoming increasingly common in the light of the increasing literature) then they will usually include only RCTs. However, if there are not any (or only a few) RCTs the reviews will often include study designs other than RCTs. For this reason EPOC includes interrupted time series designs and controlled before and after designs meeting pre-specified criteria. The focus of systematic reviews 7.11 Systematic reviews of the evidence of the effects of strategies to promote the uptake of clinically effective practices fall into two broad groups. Firstly there are intervention-based reviews of strategies to influence the behaviour of individuals or organisations. These tend to be reviews of the effectiveness of an empirically defined intervention (such as reminders or audit and feedback) across a range of healthcare professionals, settings and clinical conditions. Secondly there are reviews of strategies to improve care for a defined clinical condition (e.g. diabetes mellitus) or problem (e.g. poor prescribing) (condition- or problembased reviews). These are usually reviews of the effectiveness of various empirically defined strategies across a range of healthcare professionals and settings, but only for a single clinical condition The two approaches produce complementary information. Which you would choose to read depends upon what question you wish to address. If, at a health system level, a quality improvement budget is being invested across a range of conditions, then the intervention-based reviews are likely to be most informative. If the question is how best to improve care for patients with a specific condition, such as diabetes mellitus, then conditionbased reviews addressing studies of diabetes care are likely to be most informative. However, it will often be the case that cross referencing across the two bodies of reviews will be informative. Current knowledge: an overview 7.13 For the purposes of this scoping report, any attempt to draw together the messages from this literature will inevitably be an overview. The examples have been chosen either because they have been conducted to high methodological standards, or because they are the best that is available, or both. 27 Grimshaw JM, Shirran L, Thomas R, Mowatt G, Fraser C, Bero L, Grilli R, Harvey E, Oxman A, O Brien MA. Changing provider behavior: an overview of systematic reviews of interventions. Medical Care, 2001; 39(8 Suppl 2): II

150 Section 3: Current knowledge on the effectiveness of strategies to promote the uptake of clinically effective practice 7.14 The report focuses on the scale of the review and the summary messages. There are methodological issues with most reviews in this area; the interested reader is directed to the original publications The reviews are presented in the two groups: intervention-based and condition/problem-based. The contents of the EPOC Review Group module of reviews is summarised at the end as it contains examples of both. Intervention-based reviews of strategies to influence the behaviour of individuals or organisations Effectiveness and efficiency of guideline dissemination and implementation strategies (Grimshaw et al ) 7.16 Two hundred and thirty-five studies reporting 309 comparisons met the inclusion criteria. The overall quality of the studies was poor. Seventythree per cent of comparisons evaluated multifaceted interventions, although the maximum number of replications of a specific multi-faceted intervention was 11 comparisons. Overall, the majority of comparisons reporting dichotomous process data (86.6%) observed improvements in care; however, there was considerable variation in the size of the observed effects both within and across interventions. Commonly evaluated single interventions were reminders (38 comparisons), dissemination of educational materials (18 comparisons), and audit and feedback (12 comparisons). There were 23 comparisons of multifaceted interventions involving educational outreach. The median absolute improvement in performance across interventions ranged from 13.1% in 13 cluster randomised comparisons of reminders, 8.1% in four cluster randomised comparisons of dissemination of educational materials, 7.0% in five cluster randomised comparisons of audit and feedback, and 6.0% in 13 cluster randomised comparisons of multifaceted interventions involving educational outreach. The authors found no relationship between the number of component interventions and the effects of multi-faceted interventions. Organisational interventions: a structured review of reviews (Wensing et al, 2006) A total of 36 reviews, including 684 studies (of rigorous evaluations of organisational changes, and published between 1995 and 2003) were included; not all were high-quality reviews. Twenty-one of the 36 reviews were of high quality as defined by the review authors. The reviews were too heterogeneous for quantitative synthesis Overall: Revision of professional roles (nine reviews) can improve professional performance, although effects on patient outcomes remain uncertain. Revision of roles seemed especially effective in preventive care, but the effects of specialised nurses in primary care remain uncertain. Multidisciplinary teams (five reviews) can improve patient outcomes. They have primarily been tested in highly prevalent chronic diseases. Integrated care systems (eight reviews) can improve patient outcomes and save costs. They have been extensively tested in highly prevalent chronic conditions. Knowledge management (six reviews) can improve professional performance and patient outcomes (mainly by the implementation of computers in clinical practice settings). The effects of quality management (two reviews) on professional performance and patient outcomes remain uncertain. Professional performance was generally improved by revision of professional roles and computer systems for knowledge management. Patient outcomes were generally improved by multidisciplinary teams, integrated care services, and computer systems. Cost savings were reported from integrated care services The authors concluded that, whilst there is a growing evidence base of rigorous evaluations of organisational strategies, the evidence underlying some strategies is limited, and for no strategy can the effects be predicted with high certainty. Condition- or problem-based reviews Evidence-based Practice Centres evidence reports 7.20 In early 2003, the Institute of Medicine (IOM) in the United States released its report Priority Areas for National Action: Transforming Health Care Quality. The report listed 20 clinical topics for which best practices were strongly supported by clinical evidence. The rates at which these practices had been implemented in the United States had been disappointingly low, at a cost of many thousands of 28 Wensing M, Wollersheim H, Grol R. Organisational interventions to implement improvements in patient care: a structured review of reviews. Implementation Science, 2006, 1: 2. 29

151 Section 3: Current knowledge on the effectiveness of strategies to promote the uptake of clinically effective practice lives each year. In response, the Agency for Healthcare Research and Quality (AHRQ) engaged the Stanford-UCSF Evidence-based Practice Center (EPC) to perform a critical analysis of the existing literature on quality improvement (QI) strategies for a selection of the 20 disease and practice priorities noted in the IOM Report. The methods and three published reports summarised here represent the current products As well as representing high quality reviews, these are also noteworthy for being commissioned as a strategic response to an identified quality problem Further details of these reports can be found by selecting the Healthcare Services then Quality Improvement and Patient Safety links at Closing the Quality Gap: A Critical Analysis of Quality Improvement Strategies: Volume 1 Series Overview and Methodology. August In Volume 1 of Closing the Quality Gap, we provide an overview of our methods and the theoretical underpinnings of the field, which we will rely on to review and analyze the literature on the quality gap in a number of the IOM-identified priority areas that will appear in subsequent volumes. We then set forth our methodology: our reviews generally are restricted to studies that are likely to have strong validity (randomized controlled trials, well controlled before-after studies, and interrupted time series studies). To ensure consistency across our reviews, we introduce a taxonomy for nine QI strategies: 1. Provider reminder systems. 2. Facilitated relay of clinical data to providers. 3. Audit and feedback. 4. Provider education. 5. Patient education. 6. Promotion of self-management. 7. Patient reminders. 8. Organizational change. 9. Financial, regulatory, or legislative incentives. Closing the Quality Gap: A Critical Analysis of Quality Improvement Strategies: Volume 2 Diabetes Care 7.24 Main results: Fifty-eight articles reporting a total of 66 trials met the established inclusion criteria. The most common interventions employed were organizational change in 40 trials, patient education in 28 trials, and provider education in 24 trials. Fiftytwo trials involved interventions employing more than one QI strategy, with a median of two strategies per trial and a maximum of five The included trials reported a median absolute reduction in HbA1c of 0.48% (interquartile range: 0.20% to 1.38%), and a median improvement in clinician adherence of 4.9% (interquartile range: 3.8% to 15.0%). Trials in the lower two quartiles of sample size reported substantially larger effect sizes, as did non-randomized trials, strongly suggesting the presence of publication bias, with publication of smaller non-randomized trials occurring more often when reported improvements are large. Multi-faceted trials reported a median reduction in HbA1c of 0.60% (interquartile range: 0.30% to 1.40%), compared to a median reduction of 0.0% (interquartile range: -0.08% to 0.16%) for trials of a single intervention (p = 0.01). The benefit of employing more than one QI strategy appeared to persist among larger, randomized trials, but the small number of studies limits the reliability of this impression The investigators did not find any specific type of QI strategy to confer unambiguous benefit. Provider education and disease management were the only strategies to approach statistical significance, compared with interventions absent these strategies Conclusion: The authors analysis showed no particular type of QI to have an advantage over others, but suggested that employing at least two strategies provides a greater chance of success than single-faceted interventions, in terms of improving glycemic control or provider adherence. These conclusions are limited by probable publication bias favoring smaller trials and non-randomized trials, and the confounding presence of multiple QI strategies in a given intervention, as well as important patient and provider factors, and organizational characteristics. Closing the Quality Gap: A Critical Analysis of Quality Improvement Strategies: Volume 3 Hypertension Care 7.28 Main results: Sixty-three articles reporting a total of 82 comparisons met the inclusion criteria. Studies of hypertension identification were found to be too heterogeneous for quantitative analysis. The majority of screening studies were clinic-based (with a few offered at work sites), and the most common strategies involved patient and/or provider reminders. These generally showed positive results; several studies found that patients were more likely to know their blood pressure or attend clinic visits after receiving reminders. 30

152 Section 3: Current knowledge on the effectiveness of strategies to promote the uptake of clinically effective practice 7.29 Across all studies with a variety of strategies, the median reductions in systolic blood pressure (SBP) and diastolic blood pressure (DBP) were 4.5 mmhg (interquartile range: 1.5 to 11.0) and 2.1 mmhg (interquartile range: -0.2 to 5.0) respectively. The median increase in the proportion of patients in the target SBP range and target DBP range was 16.2% (interquartile range: 10.3% to 32.2%) and 6.0% (interquartile range: 1.5% to 17.5%) respectively. Studies that focused on improving provider adherence showed a range of median reduction of 1.3% to a median improvement of 3.3% across all QI strategies. Overall, patient adherence showed a median improvement of 2.8% (interquartile range: 1.9% to 3.0%) Conclusion: The findings of this review suggest that QI strategies appear, in general, to be associated with the improved identification and control of hypertension. It is not possible to discern with complete confidence which specific QI strategies have the greatest effects, since most of the studies included more than one QI strategy. All of the assessed strategies may be beneficial under some circumstances, and in varying combinations. There may be other useful strategies that have not been studied in trials meeting the inclusion criteria for evidence-based review; it is not possible to draw conclusions about these strategies. Closing the Quality Gap: A Critical Analysis of Quality Improvement Strategies: Volume 4 Antibiotic Prescribing Behaviour 7.31 Main results: Fifty-four studies reporting a total of 74 trials met the inclusion criteria; 34 studies (reporting 41 trials) addressed the treatment decision, and 26 studies (reporting 33 trials) addressed the selection decision. Six studies evaluated both decisions. Study methodologic quality was generally fair. Nearly all studies took place in outpatient primary care clinics Studies addressing the antibiotic treatment decision: Most studies addressed prescribing for acute respiratory infections (ARIs). Interventions were effective at reducing prescribing, with a median absolute effect of -8.9% (interquartile range: 12.4% to 6.7%). No individual QI strategy (or combination of strategies) was more effective at reducing prescribing. Within clinician education, active educational strategies appeared more effective than passive strategies. When extrapolated to a population level, strategies targeting general antibiotic prescribing appeared to reduce antibiotic prescribing more than strategies targeting prescribing for a single condition. Few studies addressed secondary endpoints; patient satisfaction was not worsened by QI interventions, but effects on antimicrobial resistance or costs could not be assessed Studies addressing the antibiotic selection decision: Interventions targeted prescribing for ARIs or urinary tract infections (UTIs). Interventions were effective, with a median absolute improvement in prescribing of recommended antibiotics of 10.6% (interquartile range: 3.4% to 18.2%). Clinician education alone appeared more effective than education in combination with audit and feedback, but this finding likely represents confounding. Very few studies addressed secondary outcomes Conclusion: Quality improvement efforts appear generally effective at reducing both inappropriate treatment with antibiotics and inappropriate selection of antibiotics. While no single QI strategy was more effective than others, active clinician education may be more effective than passive education, particularly for addressing the antibiotic treatment decision. Greater reductions in overall prescribing may be achieved through efforts targeting prescribing for all ARIs, rather than targeting single conditions. The available evidence is of only fair quality, and further research on the costeffectiveness and potential harms of these interventions is needed. Other AHRQ reviews 7.35 There are other AHRQ-commissioned reviews that are not part of the Closing the Quality Gap series. They cover end-of-life care and outcomes, costs and benefits of health information technology, and quality-based purchasing strategies. I have included a summary of the latter because of its potential specific relevance to the work of the HLG. Strategies To Support Quality-based Purchasing: A Review of the Evidence 7.36 Although evidence of quality problems has been available for years, purchaser interest in qualitybased purchasing (QBP) is a recent phenomenon. Furthermore, employers who support QBP have expressed uncertainty about how to measure quality, especially outcomes, and what incentives to offer to stimulate performance improvement We evaluated 5,045 publications. Nine were randomized controlled trials, and many of these did not report key characteristics of the incentive or the context in which incentives were applied. Incentives used included additional fee-for-service, quality 31

153 Section 3: Current knowledge on the effectiveness of strategies to promote the uptake of clinically effective practice bonuses, and public release of performance data. The results were mixed: among the 11 performance indicators evaluated, 7 showed a statistically significant response to QBP strategies while 4 did not. We also found 18 ongoing research projects, none randomized. These will yield data about the approaches to QBP currently in use, provider awareness of and concerns about QBP, and some preliminary estimates of the potential impact of QBP Regarding assessments of outcomes reports, we found that, under reasonable assumptions and applications, outcomes reports generate meaningful information about provider performance. Providers with good (expected) performance are unlikely to be labelled as poor quality in any given period, and very unlikely to be mislabelled more than once in a 3-year period, even if one allowed approximately 10% of hospitals to be labelled poor performers annually. In addition, hospitals with superior performance were quite likely to be identified as such at least once in 3 years Conclusions: Little is known about the impact of QBP on clinical performance. However, it does appear that basing incentives on measurements of outcomes is feasible without undue risk to the reputation or financial status of good hospitals. Ongoing research will only address some of the gaps in our knowledge about QBP, suggesting that much more additional research is needed. This should include comparisons of alternative QBP approaches and qualitative assessment of the barriers to and facilitators of quality improvement in response to QBP incentives. Cochrane Collaboration EPOC Review Group reviews 7.40 When systematic reviews are compared, in general Cochrane reviews are found to be of higher quality than non-cochrane reviews. They therefore represent a source of high quality evidence. The EPOC Review Group in the Cochrane Collaboration Database of Systematic Reviews hosts nine reviews of interventions aimed at professionals, 23 reviews of organisational interventions and six condition- or problem-based reviews. The titles are listed in Appendix 3 and the full reviews can be found by going to the Cochrane EPOC Group link at _clsysrev_crglist_fs.html or through the National Library for Health at The EPOC overview of reviews 7.41 In order to illustrate the broad messages about the effects of interventions, the following draft text illustrates the overview of systematic reviews of interventions aimed at changing the behaviour of individual healthcare professionals. As with previous examples, this is meant to be illustrative; the final referencing of this text is incomplete The Cochrane EPOC Review Group supports reviews of interventions to improve healthcare systems and healthcare delivery. It has identified over 4,500 randomised and quasi-experimental studies of professional, organisational, financial and regulatory interventions within its scope. EPOC has prepared two overviews of systematic reviews (Bero 1998, Grimshaw 2002) and is currently updating this. It has identified over 150 systematic reviews of professional behaviour change strategies. In this section, we summarise the results of key reviews selected because they were high quality and most up to date. We provide a definition of each intervention, the likely mechanism of action of the intervention, the results of the key review(s) and additional comments relating to the practical delivery of the intervention (including resources required). Interventions were considered: generally effective if two-thirds or more identified studies demonstrated improvements; having mixed effects if between one-third and two-thirds of identified studies demonstrated improvements; and generally ineffective if less than one-third of identified interventions demonstrated improvements. Wherever possible, we describe the range of effect sizes observed within reviews. Printed educational materials 7.43 EPOC defined printed educational materials as the distribution of published or printed recommendations for clinical care, including clinical practice guidelines, audio-visual materials and electronic publications. The materials may have been delivered personally or through mass mailings. In general, printed educational materials target knowledge and potentially skill gaps for individual healthcare professionals Farmer and colleagues identified 21 studies of the effects of printed educational materials, including six cluster randomised trials. Based upon the cluster randomised trials, printed educational materials appear generally effective, resulting in a median absolute improvement of care of 4.9% across studies. Printed educational materials are commonly used, are relatively low cost and generally feasible in most settings. 32

154 Section 3: Current knowledge on the effectiveness of strategies to promote the uptake of clinically effective practice Educational meetings 7.45 EPOC defines educational meetings as the participation of healthcare providers in conferences, lectures, workshops or traineeships. An important distinction is between didactic meetings (that largely target knowledge barriers at the level of individual healthcare professional/peer group) and interactive workshops (that can target knowledge, attitudes and skills at the level of the individual healthcare professional/peer group level) O Brien and colleagues identified 32 randomised trials of educational meetings. Interactive or mixed/interactive educational meetings were generally effective, resulting in moderate effects (between 11 20% absolute improvements in care). In contrast, didactic meetings were largely ineffective. Educational meetings are commonly used; the main cost relates to the release time for healthcare professionals and they are generally feasible in most settings. Educational outreach 7.47 EPOC defines educational outreach or academic detailing as use of a trained person who met with providers in their practice settings to give information with the intent of changing the provider s practice. The information given may have included feedback on the performance of the provider(s). Avorn and Soumerai suggest that educational outreach derives from social marketing approaches to target an individual s knowledge and attitudes. Typically the detailer aims to get a maximum of three messages across during a minute meeting with a healthcare professional. The detailer will tailor their approach to the characteristics of the individual healthcare provider and will typically use additional provider behaviour change strategies to reinforce their message. Most studies of educational outreach have focused on changing relatively simple behaviours in the control of an individual physician; for example, the choice of drug to prescribe Grimshaw (2004) identified 13 cluster randomised trials of multi-faceted interventions to implement clinical practice guidelines. Educational outreach was generally effective, resulting in a median absolute improvement of care of 4.9% across studies. The effects of educational outreach for changing more complex behaviours is less certain. Educational outreach programs have been used across a wide range of healthcare settings, especially to target prescribing behaviours. Educational outreach requires considerable resources, including the costs of detailers and preparation of materials. Nevertheless, Mason observed that educational outreach may still be an efficient way to change prescribing patterns. Local opinion leaders 7.49 EPOC defines local opinion leaders as use of providers nominated by their colleagues as educationally influential. The investigators must have explicitly stated that their colleagues identified the opinion leaders. Opinion leadership is the degree to which an individual is able to influence other individuals attitudes or overt behaviour informally in a desired way with relative frequency. This informal leadership is not a function of the individual s formal position or status in the system; it is earned and maintained by the individual s technical competence, social accessibility, and conformity to the system s norms. When compared with their peers, opinion leaders tend to be more exposed to all forms of external communication, have somewhat higher social status and be more innovative. However, the most striking feature of opinion leaders is their unique and influential position in their system s communication structure; they are at the centre of interpersonal communication networks (interconnected individuals who are linked by patterned flows of information). Opinion leaders target the knowledge and attitudes of their peer group Doumit (2007) identified 12 randomised trials of opinion leaders. Opinion leaders were generally effective for improving appropriate care, resulting in a median absolute improvement of care of 10.0% across studies. The majority of studies in healthcare have used an instrument derived by Hiss to identify opinion leaders; this seeks to identify individuals who are up to date, good communicators and humanistic. The potential success of opinion leaders is dependent upon the existence of intact social networks within professional communities. Grimshaw and colleagues (2006) observed that the existence of such networks varied across communities and settings within the UK. They also observed that opinion leaders were condition specific; in other words, colleagues identified different opinion leaders for different clinical problems. Doumit (2006) also observed that opinion leaders were not stable over time. The resources required for opinion leaders include costs of the identification method, training of opinion leaders and additional service costs. Audit and feedback 7.51 EPOC defines audit and feedback as any summary of clinical performance of health care over a specified period of time. The summary may also 33

155 Section 3: Current knowledge on the effectiveness of strategies to promote the uptake of clinically effective practice have included recommendations for clinical action. The information may have been obtained from medical records, computerised databases, or observations from patients. Adams and colleagues observed that healthcare professionals often overestimated their performance by around 20 30%. Audit and feedback target healthcare provider/peer groups perceptions of current performance levels, and are useful for creating cognitive dissonance within healthcare professionals as a stimulus for behaviour change Jamveldt (2005) identified 118 randomised trials of audit and feedback. Audit and feedback alone, audit and feedback with educational meetings, audit and feedback as part of multi-faceted intervention were generally effective, resulting in a median absolute improvement of care of 10.0%. In general, larger effects were seen if baseline compliance was low. The resources required to deliver audit and feedback include data abstraction and analysis costs and dissemination costs. The feasibility of audit and feedback may depend on the availability of meaningful routine administrative data for feedback. Reminders 7.53 EPOC defines reminders as patient- or encounter-specific information, provided verbally, on paper or on a computer screen, which is designed or intended to prompt a health professional to recall information. This would usually be encountered through their general education, in the medical records or through interactions with peers, and so remind them to perform or avoid some action to aid individual patient care. Computer-aided decision support and drugs dosage are included. Reminders prompt the professional to remember to do important items during professional patient interactions Grimshaw (2004) identified 14 cluster randomised trials of reminders to implement clinical practice guidelines. This review found that reminders were generally effective, resulting in a median absolute improvement of care of 14.1%. The majority of studies have been undertaken in highly computerised US academic health science centres (Shekelle 2006) and their generalisability to other settings is less certain. Most studies have examined the effects of relatively simple reminders; the results of more complex decision support systems for chronic disease have been less successful. The resources required vary across the delivery mechanism and there is insufficient knowledge about how to prioritise and optimise reminders. Multi-faceted interventions 7.55 EPOC defines multi-faceted interventions as any intervention including two or more components. Multi-faceted interventions potentially target different barriers in the system. Grimshaw et al (2004) explored whether there was a dose response curve for multi-faceted interventions and observed that effect sizes did not necessarily increase with increasing numbers of components. They observed that Few studies provided any explicit rationale or theoretical base for the choice of intervention. As a result, it was unclear whether researchers had an a priori rationale for the choice of components in multi-faceted interventions based upon possible causal mechanisms, or whether the choice was based on a kitchen sink approach. It is plausible that multi-faceted interventions built upon a careful assessment of barriers and coherent theoretical base may be more effective than single interventions. Multi-faceted interventions are likely to be more costly than single interventions. If planning multifaceted interventions, it is important to consider carefully how components are likely to interact to maximise benefits. 34

156 Appendix 1 Activities in the NHS related to supporting clinical effectiveness With acknowledgement to Anne Moger, whose initial draft for the High Level Group on Clinical Effectiveness formed the basis of this appendix. The structure of the table is: National level Organisations/activities relating to setting evidence-based standards Organisations/activities supporting the delivery of evidence-based standards Inputs (knowledge, skills, other resources) Processes that support delivery and/or focus on outcomes Organisations/activities relating to monitoring the delivery of effective care Regional/SHA-level initiatives Inputs (knowledge, skills, other resources) Processes that support delivery and/or focus on outcomes Local initiatives Inputs (knowledge, skills, other resources) Processes that support delivery and/or focus on outcomes Monitoring the delivery of effective care. Organisation Activity National level SETTING EVIDENCE-BASED STANDARDS Department of Health (DH) NHS Cancer Plan (2000) Paediatric Intensive Care Reports of CNO s Taskforce and the National Coordinating Group (1999) National Service Frameworks for: Coronary Heart Disease Mental Health Older People Diabetes Long-term Conditions Renal Services Children, Young People and Maternity Services Chronic Obstructive Pulmonary Disease (in development) Standards for Better Health (DH 2004) Domain 2 Clinical and cost-effectiveness Other heathcare strategy documents For example: National Stroke Strategy (in development) Winning ways: working together to reduce healthcare associated infection in England. Our NHS, our future, forthcoming NHS Next Stage Review by Lord Ara Darzi. 35

157 Appendix 1: Activities in the NHS related to supporting clinical effectiveness Organisation National Institute for Health and Clinical Excellence (NICE) Professional bodies (those identified from a web search as developing guidelines) (not exhaustive): British Association for Emergency Medicine British Association for Sexual Health British Dietetic Association British Society of Paediatric Dentistry British Thoracic Society Chartered Society of Physiotherapy Royal College of Nursing Royal College of Obstetricians and Gynaecologists Royal College of Physicians Royal College of Speech and Language Therapists Royal College of Surgeons Activity NICE guidance Technology appraisals Clinical guidelines Interventional procedures guidance Public health guidance Inappropriate Interventions Programme new workstream within Technology Appraisals and Clinical Guidelines programmes Contribute to Standards for Better Health. Most professional bodies develop clinical guidelines and all are responsible for setting quality standards for their profession. Several operate clinical effectiveness committees. They may also have R&D/evaluation programmes and/or run clinical audits. (Those that are involved with national clinical audits are identified separately below under monitoring.) Academy of Royal Medical Colleges SUPPORTING DELIVERY OF EVIDENCE-BASED STANDARDS INPUTS (KNOWLEDGE, SKILLS, OTHER RESOURCES) Research synthesis to underpin standards National Library for Health (NLH) Recommendation in Good Doctors, Safer Patients (subject to consultation), that the clinical professions, led by the Academy of Medical Royal Colleges and with input from representatives of patients, establish standards in areas of specialised medical practice. Cochrane Collaboration, Database of Abstracts of Reviews of Effects (DARE), Clinical Knowledge Summaries (e.g. British Medical Journal (BMJ) Clinical Evidence), NHS Health Technology Assessment (HTA) programme, Bandolier. (BMJ Clinical Evidence series is not available to NHS staff in England (though it is in Scotland and Wales). Available through personal or institutional subscription.) The NLH aims to deliver a world class information service, benefiting the NHS by improving the quality of care by enabling evidence-based decision making, and supporting education and research. It will save users time, extend services to previously unserved groups, and improve the cost effectiveness of library provision. Includes electronic access to: evidence-based reviews (Cochrane Collaboration, DARE, Clinical Knowledge Summaries, NHS HTA programme, NLH specialist libraries, Bandolier) 36

158 Appendix 1: Activities in the NHS related to supporting clinical effectiveness Organisation NHS Connecting for Health (which is responsible for delivering the National Programme for IT (NPFIT)) Director of Clinical Knowledge, Process and Safety University of Oxford Centre for Evidence Based Medicine (CEBM), Critical Appraisal Skills Programme (CASP) (the Public Health Resource Unit, University of Oxford) and a range of universities. Activity guidance (NICE, other clinical guidelines, care pathways and protocols) drugs guidance (British National Formulary etc) primary databases (Medline etc). Also working in partnership with NLH: Map of Medicine, which aims to provide quick access to best practice guidelines (presented in algorithm form and amenable to local modification). Based on NICE guidance where available. Relevant activities include: NHS Care Records Service (potential to link to decision support systems) Electronic transmission of prescriptions (e-systems to reduce errors etc). Large number of projects linked with NLH (e.g. Better Prescribing), Map of Medicine, to potentially provide online access to care pathways, guidelines etc. Knowledge-based commissioning plan: 1. The Annual Population Value Review 2. The Annual Review of Evidence, Process, Outcome and Configuration (EPOC) 3. The National Innovation Management Project 4. The National Clinical Networks Development Programme. Training in evidence-based practice: CEBM: research into and training in EBM, including annual week-long residential course in teaching EBM and MSc in evidence-based health care CASP: bespoke courses aimed at public health and management. PROCESSES THAT SUPPORT DELIVERY AND/OR FOCUS ON OUTCOMES Department of Health Targets (Public Service Agreement (PSA) 03/06 and 05/08, and Local Delivery Plan (LDP) 03/06; performance monitoring requirements for LDP 05/08) e.g. thrombolysis call to needle times Guidance to the NHS on requirements to implement NICE guidance HSC 1999/065: Clinical governance in the new NHS Quality and Outcomes Framework (QOF) Clinical domain of QOF uses evidence-based indicators; financial incentives in GP contract directly linked to these Guidance on clinical governance HSC 1999/176; DH Good Practice Guidance 2006 Health Reform in England and related strategies Financial Sustainability Review ongoing work around clinical variation/cost effectiveness Programme budgeting Links to the Knowledge-based Commissioning Plan; the current lead is Andrew Jackson from DH Finance, Leeds Commissioning frameworks/toolkits e.g. Stroke (Action on Stroke Services: an Evaluation Toolkit (ASSET)), diabetes. 37

159 Appendix 1: Activities in the NHS related to supporting clinical effectiveness Organisation National Institute for Health and Clinical Excellence (NICE) NHS Institute for Innovation and Improvement Activity NICE implementation programme NICE has set up a programme to help support implementation of NICE guidance. The aim of the programme is to embed the implementation of NICE guidance into core systems for clinical effectiveness within the local NHS. To achieve this, the programme has three key strands: a leverage approach working through national initiatives such as those for inspection and commissioning; providing practical support tools; and evaluating change. The implementation team does not get involved in developing the guideline recommendations but works alongside the guideline developers, the communications team and field-based teams to: ensure intelligent dissemination to the appropriate target audiences actively engage with the NHS, local government and the wider community work nationally to encourage a supportive environment provide tools to support putting NICE guidance into practice demonstrate significant cost impacts either costs or savings at local and national levels evaluate uptake of NICE guidance share learning develop educational material to raise awareness of NICE guidance and encourage people to input into its development. NICE produces several implementation tools, including a How to guide, implementation advice (specific to recently produced guidance) and slide sets. A How to influence practice guide is in development. The How to guide identifies implementation steps separately for technology appraisals, clinical guidelines and interventional procedures guidance. The steps are: assessing relevance; identifying a clinical lead; baseline assessment ( what needs to change as a result of this guidance? ); developing an action plan and assessing costs; providing funds/disseminating guidance and implementing plan; reviewing and monitoring. Implementation advice (the How to guide steps tailored to specific topics) accompanied by slide sets, is available for more recent guidance and includes example action plans. To collate information on uptake of NICE recommendations, a database is provided on the NICE website, Evaluation of Reviews of NICE Implementation Effectiveness (ERNIE). The NHS Institute for Innovation and Improvement aims to support the NHS in service transformation/improvement. Of particular relevance to clinical and cost-effectiveness: Delivering Quality and Value Priority Programme The aim is a paradigm shift in clinical efficiency and effectiveness, resulting in local health systems being able to provide efficient, clinically and cost-effective healthcare for the benefit of patients. Focus On: High Volume Care (This includes the programme s documents and outputs from the work on high volume care patient pathways (nine high volume clinical topics to date) Focus On: Productivity and Efficiency Better Value, Better Care Indicators (This provides information about the metrics for better value and better care published in October 2006). 38

160 Appendix 1: Activities in the NHS related to supporting clinical effectiveness Organisation Other national institutes Charitable foundations e.g. The Health Foundation National clinical support teams All national programmes are currently under review by the Office of Strategic Health Authorities Activity For example, National Institute for Mental Health in England (NIMHE), (part of the Care Services Improvement Partnership (CSIP)) Examples of relevant activities include: High impact changes for mental health services Acute Inpatient Care programme. From their website: The Health Foundation wants to make the quality of healthcare in the UK the best it can be. Working with others, we are helping to shape a future healthcare system that offers safe, effective and responsive care for all. We are a charitable foundation and operate independently from government, political parties or other interest groups... Our work is currently focused around five main aims: building and making public the knowledge base for quality and performance improvement developing leaders to improve health and healthcare services supporting organisational efforts to improve quality and performance in health and healthcare services engaging clinicians in quality improvement engaging patients for better health and healthcare outcomes. Cancer Services Collaborative Improvement Partnership (CSCIP) The CSCIP is a national NHS programme that supports local cancer service teams (Networks) to improve their cancer and diagnostic services and help reduce their waiting times. Working closely with the National Cancer Programme, the CSCIP works with Networks to achieve the objectives of the NHS Cancer Plan 2000 for all cancer services (excluding paediatrics) and associated services. NHS Heart Improvement Programme Role is to: Support cardiac network development Ensure spread of service improvement within cardiac networks Facilitate the introduction of new service improvement tools and recognised clinical innovations. National Diabetes Support Team (NDST) The NDST was set up to support healthcare professionals as they strive to implement the Diabetes National Service Framework (NSF) standards. NHS Clinical Governance Support Team (CGST) The CGST has set its stall out as a learning organisation that uses the knowledge of its staff and its relationships with other NHS organisations to support those who shape the healthcare experience for patients, carers, and the public. Clinical decision support systems See NHS Connecting for Health. In the meantime, most GP systems have some decision support functionality. Also, bespoke systems such as some diabetes registers. R&D Implementation Research (also referred to as Knowledge Translation Research) The study of methods to promote the uptake of research evidence into practice is currently largely absent from the UK. The closest NHS funded national programme is the Service Delivery and Organisation (SDO) R&D Programme. The Medical Research Council (MRC) Health Services and Public Health Research Board lists implementation as one of its research areas. There are also a number of universities that specialise in implementation research, including Newcastle and Manchester. 39

161 Appendix 1: Activities in the NHS related to supporting clinical effectiveness Organisation Activity MONITORING THE DELIVERY OF EFFECTIVE CARE Healthcare Commission (HC) SDO programme Aims to produce research evidence directed at improving the organisation and delivery of health services, and to promote the uptake and application of that evidence in policy and practice. Institute of Health and Society, University of Newcastle Led by Professor Martin Eccles, the programme involves studies of how to implement evidence in clinical practice. National Primary Care Research and Development Centre, Manchester Led by Professor Martin Roland, the centre s research is organised around four main areas: organisations, quality, self-management and workforce. The Healthcare Commission has responsibility for assessing performance against Standards for Better Health, and new and existing national targets, in individual healthcare organisations. The clinical and cost-effectiveness standard domain outcome states that Patients achieve health benefits that meet their individual needs through healthcare decisions and services based on what assessed research evidence has shown provides effective clinical outcomes. The detailed clinical and cost-effectiveness standard is: Core standards C5 Health care organisations ensure that: a) they conform to NICE technology appraisals and, where it is available, take into account nationally agreed guidance when planning and delivering treatment and care; b) clinical care and treatment are carried out under supervision and leadership; c) clinicians continuously update skills and techniques relevant to their clinical work; and d) clinicians participate in regular clinical audit and reviews of clinical services. C6 Health care organisations cooperate with each other and social care organisations to ensure that patients individual needs are properly managed and met. Developmental standard D2 Patients receive effective treatment and care that: a) conform to nationally agreed best practice, particularly as defined in National Service Frameworks, NICE guidance, national plans and agreed national guidance on service delivery; b) take into account their individual requirements and meet their physical, cultural, spiritual and psychological needs and preferences; c) are well coordinated to provide a seamless service across all organisations that need to be involved, especially social care organisations; and d) is delivered by healthcare professionals who make clinical decisions based on evidence-based practice. Healthcare organisations declared their performance against the core standards, including C5 above, for the first time at the end of 2005/06. After selective inspection of 20% of Trusts, which in some cases will have resulted in amendments to their self-assessment, their performance was published in the Healthcare Commission s 05/06 Annual Health Check (AHC) in October The first Trust 40

162 Appendix 1: Activities in the NHS related to supporting clinical effectiveness Organisation Department of Health Royal Colleges and clinical societies, including: Royal College of Physicians Royal College of Surgeons British Cardiac Society University of York National Clinical Audit Support Programme (Health and Social Care Information Centre (HSIC)) Activity declarations on the developmental clinical and cost-effectiveness standard will not be available until 2007, and then only for acute and mental health Trusts. This will be accompanied by publication of an information toolkit including clinical effectiveness indicators on cancer, heart disease, stroke and mental health for all acute and mental health Trusts. New and existing national targets assessment includes assessment on smoking, cancer, heart disease, sexual health, mental health and hospital-acquired infection. Other relevant performance assessment activities within the Healthcare Commission include: service reviews, the acute hospital portfolio programme; national clinical audits (see later); some special investigations; and outcomes reported by patients. National monitoring of PSA, LDP targets, including national patient experience surveys. National clinical audits: Most are funded by the Healthcare Commission (HC) but there are some other funders, e.g. British Heart Foundation (cardiac rehabilitation audit at University of York). Full list of HC-funded audits (including those in development) is: Cancer Bowel cancer Head and neck cancer Lung cancer Oesophago-gastric (stomach) cancer Mastectomy and breast reconstruction Child and maternity Neonatal intensive care Heart Adult cardiac surgery Paediatric cardiac surgery and interventions Cardiac interventions (e.g. opening up heart artery) Myocardial infarction (MINAP) (heart attack) Rhythm management (pacing/implantable cardiac defibrillators) Heart failure Cardiac ambulance services Cardiac patients pathway and outcomes Long-term conditions Diabetes Renal services (kidney disease) Mental health Violence in mental health settings Dementia services and prescribing Psychological therapies Older people Continence: RCP audit Stroke: hospital services Carotid endarterectomy Services for people who have fallen Stroke: patients pathway, focusing on primary care. 41

163 Appendix 1: Activities in the NHS related to supporting clinical effectiveness Organisation Various national clinical indicator initiatives QQUIP the Quest for Quality and Improved Performance Activity Secondary Uses Service (SUS) SUS is a system designed to provide timely, pseudonymised, patient-based data and information for management and clinical purposes other than direct patient care. The data is made available through the NHS Care Records Service. These secondary uses will include functions such as healthcare planning, commissioning, public health, clinical audit, benchmarking, performance improvement, research and clinical governance. National Centre for Health Outcomes Development (NCHOD): NCHOD is a unique national resource concerned with all aspects of health outcomes assessment. It was created in April 1998 following the outsourcing of the Department of Health s Central Health Outcomes Unit and is based jointly at the London School of Hygiene and Tropical Medicine, University of London and the Department of Public Health, University of Oxford. Its current work programme is funded by The Information Centre for Health and Social Care. NCHOD is involved in three main groups of activities: design and development of measures of health outcome; production of comparative health outcome indicators using available routine data; in the form of the Compendium of Clinical and Health Indicators; and electronic publication of extensive statistical and bibliographic information about health outcomes in the Clinical and Health Outcomes Knowledge Base. NCHOD is currently supporting the DH Information for Choice Taskforce among other initiatives. Better Metrics ( etrics.cfm) The better metrics project, which began in 2004, aims to improve the way the performance of health services is measured and monitored by: developing metrics that are more relevant to the work of doctors, nurses, and others who provide care to patients; identifying metrics that are already being used successfully by organisations to monitor and improve performance; sharing metrics with the Connecting for Health programme, as part of the process for developing electronic patient records, to improve routine monitoring of key clinical areas. Better metrics have already been suggested for 13 key health areas including cancer, heart disease and stroke, diabetes and mental health.... These metrics have been developed in collaboration with national clinical directors from the Department of Health and their colleagues in strategic health authorities, primary care trusts and other organisations in the NHS. Dr Foster Intelligence, including their Intelligent Board project ( Other examples include a New Clinician Outcomes and Benchmarking Tool for consultant clinicians, surgeons and their managers. A five-year, 2.5 million research initiative of The Health Foundation. It has been set up to help answer three fundamental questions about healthcare in England: What is the current state of quality and performance? What works to improve quality and performance? Are we getting value for money from what is spent on the NHS? The website brings together data to reveal hidden national and international trends on diseases, quality of care and the 42

164 Appendix 1: Activities in the NHS related to supporting clinical effectiveness Organisation Regional/SHA-level initiatives Activity INPUTS (KNOWLEDGE, SKILLS, OTHER RESOURCES) SHAs cost-effectiveness of treatments and government reforms. It is aimed at healthcare policy makers, researchers, clinicians, managers and patient groups so that they can make informed decisions based on the best possible data available. QQUIP collects and analyses data from a wide range of already published and publicly available sources including databases such as the Organisation for Economic Co-operation and Development (OECD) Health Data, the Department of Health, the Healthcare Commission, professional databases and peer-reviewed published studies. It houses easy to use, at a glance data on priority areas such as cancer, heart disease, diabetes and mental health. It also looks at how well the NHS is performing in terms of its effectiveness, safety, responsiveness, efficiency and equity of access to care. SHA workforce initiatives, e.g. funding of specific evidence-based training initiatives. PROCESSES THAT SUPPORT DELIVERY AND/OR FOCUS ON OUTCOMES SHAs Regional organisations Local initiatives Clinical effectiveness strategies INPUTS (KNOWLEDGE, SKILLS, OTHER RESOURCES) Library initiatives CPD, skills development Service redesign and/or system reform initiatives, e.g. funded implementation and evaluation of lean thinking or six sigma in a number of organisations across the patch. Local implementation of NPFIT; SHAs have recently taken on operational responsibility. Potentially provides the opportunity to tune the programme to local clinical priorities. Performance management of clinical priorities of commissioners and non-foundation Trusts. Clinical networks, e.g. cancer, heart disease. Collaborative projects, e.g. mental health. Public Health Observatories (PHOs) and cancer registries Monitoring of regional health data, which includes data on healthcare outcomes. Academic-service collaborations, e.g. Public Health Institutes focusing on getting evidence into practice. Example strategies tend to cover: assessment of health needs, establishing evidence-based standards, implementation of NICE guidance and NSFs, other guidelines and protocols, evaluation of those standards including clinical audit and patient partnership initiatives, implementation of recommendations and re-evaluation, processes in place to support the professional development of staff. e.g. Evidence-based summaries, e -alerts to new NICE guidance. e.g. Evidence-based practice and R&D training (including via funding for Masters programmes). PROCESSES THAT SUPPORT DELIVERY AND/OR FOCUS ON OUTCOMES NICE guidance implementation activities; other guideline implementation activities Integrated care pathways Local application of the NICE How to guide, e.g. participation in the NIMHE funded project to pilot implementation of stepped care for depression, as part of implementing NICE depression guidelines. Care pathways should be the quality-assessed and evidence-based way of consistently delivering high quality care for a particular circumstance (Bandolier 2003). The evidence base is usually provided by an evidence-based guideline. 43

165 Appendix 1: Activities in the NHS related to supporting clinical effectiveness Organisation Activity Clinical decision support systems Clinical pharmacy services They tend to: MONITORING THE DELIVERY OF EFFECTIVE CARE Clinical audits Other NICE guidance monitoring examine the external evidence for individual technologies combine this with local knowledge and experience and conditions involve a number of different disciplines of people, in a team decision and creating ownership of the product measure the results of the actions have information systems feeding back to the team on a timely basis amend the pathway in the light of results. e.g. GP computer systems, diabetes registers. Performance management reports At a minimum, to Trust Board. Quality and Outcomes Framework (QOF) Benchmarking programmes Commissioner provider dialogue R&D evaluations of specific initiatives e.g. Medicines Management Services; PCT Pharmaceutical Advisers. At a minimum, these would be expected to cover: participation of clinicians in relevant national clinical audits; audits on the implementation of NICE technology appraisal guidance. e.g. Reviewing prescribing information. e.g. Gateshead PCT: in addition to their routine reporting and ad-hoc reports on NICE guidance implementation, they have developed a set of local high level indicators. These cover issues relating to improving quality and minimising cost growth. These indicators (reported quarterly to the Board) monitor progress, take account of local prescribing patterns, the direction and magnitude of change required. The current set of local indicators include implementation of NICE guidance, for example: use of selective SSRIs in depression, and use of proton pump inhibitors in management of dyspepsia. e.g. Identification of differential GP practice performance on clinical QOF indicators. e.g. Subscribing to CHKS, Dr Foster Intelligence. e.g. Service specifications, primary/secondary care protocols. e.g. Gateshead: for the last two years Gateshead PCT and Gateshead NHS Foundation Trust (FT) have included clinical effectiveness indicators in their legally binding contract. This means that the FT routinely report to the PCT on a basket of quality indicators, and on a set of locally agreed quality and safety targets. This has included specific expectations of performance on preventive medication following myocardial infarction (2005/06 target) as well as diagnosing and establishing patients with heart failure on cost-effective medication (2006/07 target). e.g. Active engagement in programmes of applied research in partnership with local universities. 44

166 Appendix 2 Department of Health selection criteria for referral of topics to NICE (From 1) Is it appropriate for NICE to provide guidance on the topic? In particular, a) is the proposed topic within NICE s remit? b) has NICE already provided guidance or is NICE developing guidance on the proposed topic? c) is the proposed topic emerging as a future public health issue? d) is the proposed topic an ultra-orphan disease? 2) Would guidance promote the best possible improvement in public health and well-being and/or patient care, and the reduction of inequalities in health, given available resources? In particular, are one or more of the following satisfied? a) does the proposed guidance relate to one of the public health or NHS clinical priority areas, or to other health-related government priorities? b) does the proposed guidance address an area of action where better evidence of costeffectiveness would be expected to lead to substantive cost-efficiencies in the delivery of quality programmes or interventions? c) does the proposed guidance relate to one or more interventions or practices which might impact significantly on NHS or other societal resources (financial and other)? d) does the proposed guidance relate to one or more interventions from which the NHS could disinvest without detriment to cost-effective patient care, thus freeing up resources for use elsewhere in the NHS? And, for public health topics, one of the following: e) does the proposed guidance relate to an area of public health action that has high policy priority in the Department of Health and/or across government? f) does the proposed guidance address an area of public health action that promotes population health or well-being, and/or relates to a significant burden of avoidable disease, disability, injury or early death in the population as a whole or in specific population subgroups? And, for clinical topics, one of the following: g) does the proposed guidance address a condition which is associated with significant morbidity or mortality in the population as a whole or in particular subgroups? h) does the proposed guidance relate to one or more interventions or practices which could: i) significantly improve patients or carers quality of life; and/or ii) reduce avoidable morbidity; and/or iii) reduce avoidable premature mortality; and/or iv) reduce inequalities in health; relative to current standard practice if used more extensively or more appropriately? 3) Will NICE be able to add value by issuing guidance? In particular, taking into account whether: a) there is available in the topic area a substantive or developing body of research or related evidence, where plausible linkages between public health or clinical actions and outcomes can be demonstrated; and/or b) there is emerging evidence of widespread variation in the efficacy of public health actions in the topic area, such that publication of formal guidance would make a significant difference to improving the effectiveness of public health programmes or interventions; and/or 45

167 Appendix 2: Department of Health selection criteria for referral of topics to NICE c) there is evidence and/or reason to believe that, in the absence of guidance, there is or will be: i) inappropriate clinical practice, and/or ii) inappropriate variation in clinical practice, and/or iii) inappropriate variation in access to clinical interventions and/or treatment (between geographical areas or social groups). 4) Would it be timely for NICE to provide guidance on the proposed topic? In particular, a) would the guidance still be relevant and timely at the expected date of publication, and/or b) for new clinical interventions, does the balance of advantage for patient care lie with appraisal at time of launch or at some specified future date, taking account of: i) the possible impact on uptake or equity of access in the absence of guidance at time of launch; ii) for surgical and related interventions, whether safety and efficacy have already been assessed (or will be assessed in the near future) by the Interventional Procedures Advisory Committee; and/or c) is there a degree of urgency for guidance caused by factors other than those listed above, e.g. is there significant public concern, is this a new disease, or is this emerging as an important new area for public health action? July

168 Appendix 3 Intervention-based and clinical condition-based reviews and protocols in the Effective Practice and Organisation of Care Review Group of the Cochrane Library Further details of these reviews and protocols can be found by going to the Cochrane EPOC Group link at _clsysrev_crglist_fs.html Reviews Intervention-based reviews Professional interventions 1) Audit and feedback: effects on professional practice and healthcare outcomes 2) Computerised advice on drug dosage to improve prescribing practice 3) Continuing education meetings and workshops: effects on professional practice and healthcare outcomes 4) Educational outreach visits: effects on professional practice and healthcare outcomes 5) Guidelines in professions allied to medicine 6) Interprofessional education: effects on professional practice and healthcare outcomes 7) Local opinion leaders: effects on professional practice and healthcare outcomes 8) Tailored interventions to overcome identified barriers to change: effects on professional practice and healthcare outcomes 9) Teaching critical appraisal skills in healthcare settings. Organisational interventions 1) Capitation, salary, fee-for-service and mixed systems of payment: effects on the behaviour of primary care physicians 2) Care home versus hospital and own home environments for rehabilitation of older people 3) Discharge planning from hospital to home 4) Effectiveness of intermediate care in nursing-led inpatient units 5) Expanding the roles of outpatient pharmacists: effects on health services utilisation, costs, and patient outcomes 6) Hospital at home versus inpatient hospital care 7) Institutional versus at-home long-term care for functionally dependent older people 8) Interventions to improve outpatient referrals from primary care to secondary care 9) Interventions to promote collaboration between nurses and doctors 10) Lay health workers in primary and community healthcare 11) Mass media interventions: effects on health services utilisation 12) Medical day hospital care for the elderly versus alternative forms of care 13) Nursing record systems: effects on nursing practice and healthcare outcomes 14) On-site mental health workers in primary care: effects on professional practice 15) Organisational infrastructures to promote evidence-based nursing practice 16) Patient reminder and patient recall systems for improving immunisation rates 17) Pharmaceutical policies: effects of reference pricing, other pricing, and purchasing policies 18) Specialist outreach clinics in primary care and rural hospital settings 19) Strategies for integrating primary health services in middle- and low-income countries at the point of delivery 20) Substitution of doctors by nurses in primary care 21) Target payments in primary care: effects on professional practice and healthcare outcomes 22) Telemedicine versus face to face patient care: effects on professional practice and healthcare outcomes 23) Telephone consultation and triage: effects on healthcare use and patient satisfaction. 47

169 Appendix 3: Intervention-based and clinical condition-based reviews and protocols in the EPOC Review Group Clinical condition- or problem-based reviews 1) Dietary advice given by a dietician versus other health professional or self-help resources to reduce blood cholesterol 2) Effects of interventions aimed at changing the length of primary care physicians consultation 3) Improving health professionals management and the organisation of care for overweight and obese people 4) Interventions to improve antibiotic prescribing practices for hospital inpatients 5) Interventions to improve antibiotic prescribing practices in ambulatory care 6) Interventions to improve the management of diabetes mellitus in primary care, outpatient and community settings. Protocols Intervention-based review protocols 1) Alternatives to inpatient mental health care for children and young people 2) Case management: effects on professional practice and healthcare outcomes 3) Changes in out-of-pocket payments on utilisation of healthcare services 4) Computer-generated paper reminders: effects on professional practice and healthcare outcomes 5) Continuous quality improvement: effects on professional practice and patient outcomes 6) Educational Games for Health Professionals 7) Effectiveness of shared care across the primaryspecialty care interface in chronic disease management 8) Electronic access to health information and/or knowledge by health professionals to improve practice and patient care 9) Improving surgical practice a systematic review of effective education strategies to improve surgical technical skills in operative procedures 10) Local consensus processes: effects on professional practice and healthcare outcomes 11) Manual paper reminders: effects on professional practice and healthcare outcomes 12) On-screen computer reminders: effects on professional practice and healthcare outcomes 13) Printed educational materials: effects on professional practice and healthcare outcomes 14) Public release of performance data 15) Supervision outreach visits to improve the quality of primary healthcare in low- and middle-income countries 16) The impact of health financing strategies on access to health services in low and middle income countries. Clinical condition- or problem-based review protocols 1) Comprehensive geriatric assessment for older adults admitted to hospital: a systematic review 2) Interventions for improving the appropriate use of imaging in people with musculoskeletal conditions 3) Interventions for increasing the proportion of health professionals practising in under-served communities 4) Interventions for promoting information and communication technologies adoption in healthcare professionals 5) Interventions for reducing medication errors in children in hospital 6) Interventions for reducing preventable drug-related hospital admissions or preventable drug-related morbidity in primary care 7) Interventions to improve hand hygiene compliance in patient care 8) Interventions to improve the management of asthma in primary care settings 9) Interventions to increase clinical incident reporting in healthcare 10) Interventions to increase the use of screening and brief intervention programmes for hazardous alcohol consumption by patients in primary care settings 11) Interventions to reduce waiting lists for elective procedures 12) Non-clinical interventions for reducing unnecessary caesarean section 13) Outreach and Early Warning Systems (EWS) for the prevention of intensive care admission and death of critically ill adult patients on general hospital wards 14) Pharmaceutical policies: effects on rational drug use, an overview of 13 reviews 15) Pre-hospital emergency pathways for people with suspected stroke 48

170 Appendix 3: Intervention-based and clinical condition-based reviews and protocols in the EPOC Review Group 16) Preventive staff-support interventions for health workers 17) Primary and community healthcare professionals in hospital emergency departments: effects on process and outcome of care and resources 18) Psychologically mediated effects of the physical healthcare environment on work-related outcomes of healthcare personnel 19) Sensory environment on health-related outcomes of hospital patients 20) Smart home technologies for health and social care support. 49

171 Crown copyright p 500 Oct 07 (CWP) Produced by COI for the Department of Health If you require further copies of this title quote /Report of the High Level Group on Clinical Effectiveness and contact: DH Publications Orderline PO Box 777 London SE1 6XH Tel: Fax: Textphone: (8am to 6pm Monday to Friday) /Report of the High Level Group on Clinical Effectiveness may also be made available on request in Braille, on audio, on disk and in large print.

172 Appendix 17. Recording activity pilot spread sheet Psychology work survey Recording activity pilot

173 Recording Activity Pilot As part of New Ways of Working, etc. we are having to work out methods of costing our Services. I do not want everything we do loaded on to our cost per client contact. I have agreed with the powers that be to try this method essentially to estimate the % of time we spend over a typical month, doing the different things in our job descriptions. Method: 1. Estimate for each of the four weeks in June 2006 the % of time spent in each activity. 2. Look back over your diary to help with reliability and validity. 3. Each week should add up to 100%. 4. Use the best guess you can to reflect how time is spent 5. Assume a week is 37.5hrs. Work out the hours/mins per activity, then calculate % as hrs/37.5 x 100. Easy. It took me approximately 1 hour to do the 4 weeks. I know this is a pain, but if we don t come up with something that reflects our Agenda for Change Job Descriptions, someone will come up with a crude and unhelpful alternative. Can you return to Kath/Katrina by 18 August. I will collate it all and feedback to see what we think! Thanks Paul

174 ACTIVITY DEFINITIONS Direct Clinical Direct Contacts = EPEX recorded time Supervision = Both giving and receiving Consultation = Talking about clients (not entered on epex) to anybody Related Admin = All travel, admin, etc that is client related Teaching Delivered Preparation = Your preparation to teach Delivery = Teaching contact time Related admin/travel = To teaching NOTE: teaching can be formal/informal basically you telling someone/group things about Psychology not related to a specific client Teaching Received Contact time = Length of session you attended Related admin travel = To being taught Research Planning = Thinking about projects Doing = Directly engaged in Related Admin/Travel = To research NOTE: Research is audit, evaluation, projects, etc. It s a catch all. It usually relates to service/group issues rather than individual clients. Planning/Policy Solihull meetings = All non client meetings concerning only Solihull PCT service Solihull Admin/travel = As above Non Solihull meetings = All non client meetings concerning services outside Solihull PCT Non Solihull Admin/Travel = As above Other Annual Leave/Sick If you are unable to code or can t make an informed guess about how time was spent try and allocate time somewhere, if you can As it says on the tin.

175 NAME: RECORDING ACTIVITY PILOT Week 1 - % Week 2 - % Week 3 - % Week 4 - % Overall average % Example of Overall % ACTIVITY June 5-9 June June June % % % % % % Direct Clinical Work Direct Contacts 40 Supervision 5 Consultation 2 Related Admin/Travel/Management 15 Teaching Delivered Preparation 1 Delivery 2 Related Admin/Travel 2 Teaching Received Contact Time 3 Related Admin/Travel 2 Research Planning 5 Doing 7 Related Admin/Travel 0 Planning/Policy Solihull Meetings 3 Solihull Admin/Travel 3 Non-Solihull Meetings 2 Non-Solihull Admin/Travel 2 Other Anything not covered above 6 Annual Leave/Sick 0 TOTAL

176 Appendix 18. Step by step guide to job planning

177 Step by step guide to job planning 7 7 Step by step guide to job planning Introduction This section of the workbook outlines the suggested steps that could be taken in order to achieve an agreed job plan. The job planning process is described in Schedule 3 of the consultant contract. A more detailed description and guidance on job planning can be found in the document Consultant Job Planning Standards of Best Practice provided in Appendix 1 of this workbook. These documents should be read alongside this one. If a consultant works for more than one employer, one should be identified as the lead employer who will assume responsibility for agreeing all of the job plan. The job plan should cover: The consultant s main duties and responsibilities The scheduling of commitments Accountability arrangements professionally and managerially Agreed personal objectives and their relationship with wider service objectives The support needed to fulfil the job plan. In developing the job plan the following should be borne in mind: It is a prospective agreement setting out duties, responsibilities and objectives for the coming year It should cover all aspects of a consultant s professional practice It may be modelled on the previous year s plan The plan may be wholly or partly team based It should include local and national service objectives It should include personal objectives Resources and support required are agreed and stated The process is separate from, but linked to, appraisal. Who is involved? Many people can be involved in the job planning process. The three most important are: Consultant or team of consultants for part of the process (as each consultant will have their own personal objectives and personal schedule) Medical Manager Lead Clinician, Clinical Director or Medical Director Chief Executive Although the job plan is an agreement between the employer and the consultant, in practice the detailed discussion will take place between the consultant and their medical manager, whose responsibility it is to draw up the draft job plan. The Chief Executive s responsibility is to ensure that all consultants have agreed job plans as well as to sign them off. 7.1

178 7 Step by step guide to job planning Job plan review There should be a job plan review every year. Much of the information required for the job plan review is the same or similar to that required for the appraisal process. In addition, one outcome of the appraisal process is a personal development plan. It would seem sensible, therefore, that the two processes should occur as close together in time as possible, with appraisal preceding job planning. If there is a significant change in any aspect of the job plan during the year it may be necessary to have an interim job plan review. The purpose of the review is to: Consider what has affected the job plan Consider progress against the agreed objectives Agree any changes to duties and responsibilities Agree a plan for achieving personal objectives Agree support needed from the organisation Review relationship with private practice. Information needs The information needed to inform job planning will differ between specialties and even between consultants in the same specialty. It is not possible, therefore, to give a definitive list of what might be required. The different parties to the job planning process also require different sets of information. The following checklist provides suggestions about the sort of information required. The list is by no means exhaustive, nor is every item mentioned necessary. Information checklists Consultant: Last year s job plan. If this is the first job plan or if either party feels that there might be need for significant change in the current job plan, a sensible diary kept over an appropriate number of weeks. Included in this section starting on page 23 is the diary developed by the BMA and the Department of Health. You do not have to use this diary, but you may find it helpful List of main clinical responsibilities for employer Workload figures broken down in a meaningful way Timetable of private practice commitments List and scheduling of any fee paying services Teaching commitments CPD/CME requirements Personal development plan List and time commitment of other duties and responsibilities for main employer List and time commitment of duties and activities for other organisations such as work for trades unions, GMC, CHAI, Royal Colleges. Clinical audit and clinical governance issues to be addressed Support provided by the organisation and support required Ideas for improvements to service quality, range or performance Thoughts on blockages to efficient service delivery. 7.2

179 Step by step guide to job planning 7 Medical Manager: Quantity and quality targets for the directorate and performance against them by the team and individually in the previous year Knowledge of the relevant priorities within the local delivery plan Changes in services being required of, or offered by, the directorate Clinical audit and clinical governance issues affecting the directorate Knowledge of the resource base of the directorate including numbers of staff, changes in skill mix and those services, space and equipment available Understanding of current and new initiatives within the directorate or Trust. In addition, the consultant and/or the medical manager may require information from other sources in order to complete the job plan accurately. Such information might include: Activities for other employers in the case of joint appointments Changes in practices and/or services of other directorates or other providers National clinical audit or clinical governance issues Changes in the health provision requirements of the local health community Information from tertiary centres regarding referrals The Royal Colleges, particularly regarding workload and changes in clinical practice The requirements of medical schools The needs of doctors in training Feedback from trainees. Before the discussion If the job planning process is to have meaning and to be helpful to both the consultant and the organisation, some preparatory work is necessary by both parties. It is very likely that a diary will need to be kept for a period of time to assess accurately the amount of time consultants spend on different activities. Consultants and medical managers will need to agree how long a diary needs to be kept. This will vary according to the predictability of the workload of individual consultants, but should be kept for the duration of at least one full on-call rota. Other information will also need to be collected. In order to give some structure to the process it is suggested that the pro forma on the pages following the diary be used by the consultant whilst preparing for the job planning discussion. If the job planning discussion takes place immediately after the annual appraisal then much of the information can be shared. As the personal development plan will have been agreed at the appraisal session, it can immediately inform the job planning discussion. Medical managers and consultants need to agree at the outset who should prepare the first draft job plan. Whilst this is the responsibility of the medical manager, by agreement the consultant may undertake this task." 7.3

180 7 Step by step guide to job planning Diary The following diary has been adapted from that prepared by the British Medical Association s Central Consultants and Specialists Committee for the survey on hours of work and pattern of working that took place during It has been tailored to reflect the definitions of work and out of hours as set out in the Heads of Agreement and consultant contract framework agreement. The purpose of the diary is to help inform the job planning process. Consultants may find it helpful to note down when they are undertaking private practice or other additionally remunerated work, particularly work of this nature that is irregular. If, however, should they prefer not to do this, there is no requirement to do so. Predictable emergency work (e.g. post take ward rounds) should be programmed where possible into the working week and count towards programmed activities. Less predictable emergency work should be handled, as now, through on-call arrangements. For this reason, consultants may wish to distinguish between predictable and unpredictable on-call duties in their diary. Completing the diary On each page of the diary there are two tables. On the left hand side of the page is the diary and on the right hand side is a list of activity codes. For each day consultants should: a Put the date underneath the day. b At the end of every day, consultants should assign the relevant code to each half-hour block in the first column and write it in the relevant box. Half-hour blocks may be bracketed together for ease of completion. Please remember this is a survey of a consultant s WORK activities, not their private life. All activities not related to NHS or other additionally remunerated work are covered by the single code N2. c The second column should be used to record on-call status. Half-hour blocks may be bracketed together for ease of completion. 7.4

181 Step by step guide to job planning 7 Monday Tuesday CODES Direct clinical care 7AM to 7:30 7:30 to 8:00 8:00 to 8:30 8:30 to 9:00 9:00 to 9:30 9:30 to 10:00 10:00 to 10:30 10:30 to 11:00 11:00 to 11:30 11:30 to Noon Noon to 12:30PM Work code On-call? Work code On-call? Emergency attendance Out-patient or other clinic Operating session (including anaesthetists) Ward round Other patient treatment or relative consultation Telephone advice to hospital Multi-disciplinary meetings about direct patient care Investigative, diagnostic or laboratory work Public health duties Travelling time between sites, not to usual place of work Patient administration C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11 12:30 to 1:00 1:00 to 1:30 1:30 to 2:00 2:00 to 2:30 2:30 to 3:00 3:00 to 3:30 3:30 to 4:00 4:00 to 4:30 4:30 to 5:00 Supporting professional activities Training Continuous Professional Development Teaching Audit / Clinical Governance Job Planning / Appraisal Research Clinical management S1 S2 S3 S4 S5 S6 S7 5:00 to 5:30 5:30 to 6:00 6:00 to 6:30 6:30 to 7:00 7:00 to 7:30 7:30 to 8:00 8:00 to 8:30 Additional responsibilities Caldicott guardian Audit lead or Clinical governance lead Clinical tutor Medical / clinical directors and lead clinicians PAs by substitution or additional remuneration Other additional responsibilities A1 A2 A3 A4 A5 8:30 to 9:00 9:00 to 9:30 9:30 to 10:00 10:00 to 10:30 10:30 to 11:00 11:00 to 11:30 11:30 to Midnight Other duties Trade union duties AAC external member NCAA, GMC, CHAI Work for Royal Colleges Other D1 D2 D3 D4 D5 Midnight to 12:30 12:30AM to 1:00 1:00 to 1:30 Additionally remunerated work (see notes on p23) (e.g. private practice P1, Category 2 work P2 and other additionally remunerated work P3) P 1:30 to 2:00 2:00 to 2:30 2:30 to 3:00 3:00 to 3:30 Non-work activity Absent from work (annual or sickness leave) Other (i.e. time spent not working) N1 N2 3:30 to 4:00 4:00 to 4:30 4:30 to 5:00 5:00 to 5:30 On-call status (column 2 see notes on p23) Predictable on-call 1 Unpredictable on-call 2 5:30 to 6:00 6:00 to 6:30 6:30 to 7AM 7.5

182 7 Step by step guide to job planning Wednesday Thursday CODES Direct clinical care 7AM to 7:30 7:30 to 8:00 8:00 to 8:30 8:30 to 9:00 9:00 to 9:30 9:30 to 10:00 10:00 to 10:30 10:30 to 11:00 11:00 to 11:30 11:30 to Noon Noon to 12:30PM Work code On-call? Work code On-call? Emergency attendance Out-patient or other clinic Operating session (including anaesthetists) Ward round Other patient treatment or relative consultation Telephone advice to hospital Multi-disciplinary meetings about direct patient care Investigative, diagnostic or laboratory work Public health duties Travelling time between sites, not to usual place of work Patient administration C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11 12:30 to 1:00 1:00 to 1:30 1:30 to 2:00 2:00 to 2:30 2:30 to 3:00 3:00 to 3:30 3:30 to 4:00 4:00 to 4:30 4:30 to 5:00 Supporting professional activities Training Continuous Professional Development Teaching Audit / Clinical Governance Job Planning / Appraisal Research Clinical management S1 S2 S3 S4 S5 S6 S7 5:00 to 5:30 5:30 to 6:00 6:00 to 6:30 6:30 to 7:00 7:00 to 7:30 7:30 to 8:00 8:00 to 8:30 Additional responsibilities Caldicott guardian Audit lead or Clinical governance lead Clinical tutor Medical / clinical directors and lead clinicians PAs by substitution or additional remuneration Other additional responsibilities A1 A2 A3 A4 A5 8:30 to 9:00 9:00 to 9:30 9:30 to 10:00 10:00 to 10:30 10:30 to 11:00 11:00 to 11:30 11:30 to Midnight Other duties Trade union duties AAC external member NCAA, GMC, CHAI Work for Royal Colleges Other D1 D2 D3 D4 D5 Midnight to 12:30 12:30AM to 1:00 1:00 to 1:30 Additionally remunerated work (see notes on p23) (e.g. private practice P1, Category 2 work P2 and other additionally remunerated work P3) P 1:30 to 2:00 2:00 to 2:30 2:30 to 3:00 3:00 to 3:30 Non-work activity Absent from work (annual or sickness leave) Other (i.e. time spent not working) N1 N2 3:30 to 4:00 4:00 to 4:30 4:30 to 5:00 5:00 to 5:30 On-call status (column 2 see notes on p23) Predictable on-call 1 Unpredictable on-call 2 5:30 to 6:00 6:00 to 6: :30 to 7AM

183 Step by step guide to job planning 7 Friday Saturday CODES Direct clinical care 7AM to 7:30 7:30 to 8:00 8:00 to 8:30 8:30 to 9:00 9:00 to 9:30 9:30 to 10:00 10:00 to 10:30 10:30 to 11:00 11:00 to 11:30 11:30 to Noon Noon to 12:30PM Work code On-call? Work code On-call? Emergency attendance Out-patient or other clinic Operating session (including anaesthetists) Ward round Other patient treatment or relative consultation Telephone advice to hospital Multi-disciplinary meetings about direct patient care Investigative, diagnostic or laboratory work Public health duties Travelling time between sites, not to usual place of work Patient administration C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11 12:30 to 1:00 1:00 to 1:30 1:30 to 2:00 2:00 to 2:30 2:30 to 3:00 3:00 to 3:30 3:30 to 4:00 4:00 to 4:30 4:30 to 5:00 Supporting professional activities Training Continuous Professional Development Teaching Audit / Clinical Governance Job Planning / Appraisal Research Clinical management S1 S2 S3 S4 S5 S6 S7 5:00 to 5:30 5:30 to 6:00 6:00 to 6:30 6:30 to 7:00 7:00 to 7:30 7:30 to 8:00 8:00 to 8:30 Additional responsibilities Caldicott guardian Audit lead or Clinical governance lead Clinical tutor Medical / clinical directors and lead clinicians PAs by substitution or additional remuneration Other additional responsibilities A1 A2 A3 A4 A5 8:30 to 9:00 9:00 to 9:30 9:30 to 10:00 10:00 to 10:30 10:30 to 11:00 11:00 to 11:30 11:30 to Midnight Other duties Trade union duties AAC external member NCAA, GMC, CHAI Work for Royal Colleges Other D1 D2 D3 D4 D5 Midnight to 12:30 12:30AM to 1:00 1:00 to 1:30 Additionally remunerated work (see notes on p23) (e.g. private practice P1, Category 2 work P2 and other additionally remunerated work P3) P 1:30 to 2:00 2:00 to 2:30 2:30 to 3:00 3:00 to 3:30 Non-work activity Absent from work (annual or sickness leave) Other (i.e. time spent not working) N1 N2 3:30 to 4:00 4:00 to 4:30 4:30 to 5:00 5:00 to 5:30 On-call status (column 2 see notes on p23) Predictable on-call 1 Unpredictable on-call 2 5:30 to 6:00 6:00 to 6:30 6:30 to 7AM 7.7

184 7 Step by step guide to job planning Sunday CODES Direct clinical care 7AM to 7:30 7:30 to 8:00 8:00 to 8:30 8:30 to 9:00 9:00 to 9:30 9:30 to 10:00 10:00 to 10:30 10:30 to 11:00 11:00 to 11:30 11:30 to Noon Noon to 12:30PM Work code On-call? Emergency attendance Out-patient or other clinic Operating session (including anaesthetists) Ward round Other patient treatment or relative consultation Telephone advice to hospital Multi-disciplinary meetings about direct patient care Investigative, diagnostic or laboratory work Public health duties Travelling time between sites, not to usual place of work Patient administration C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11 12:30 to 1:00 1:00 to 1:30 1:30 to 2:00 2:00 to 2:30 2:30 to 3:00 3:00 to 3:30 3:30 to 4:00 4:00 to 4:30 4:30 to 5:00 Supporting professional activities Training Continuous Professional Development Teaching Audit / Clinical Governance Job Planning / Appraisal Research Clinical management S1 S2 S3 S4 S5 S6 S7 5:00 to 5:30 5:30 to 6:00 6:00 to 6:30 6:30 to 7:00 7:00 to 7:30 7:30 to 8:00 8:00 to 8:30 Additional responsibilities Caldicott guardian Audit lead or Clinical governance lead Clinical tutor Medical / clinical directors and lead clinicians PAs by substitution or additional remuneration Other additional responsibilities A1 A2 A3 A4 A5 8:30 to 9:00 9:00 to 9:30 9:30 to 10:00 10:00 to 10:30 10:30 to 11:00 11:00 to 11:30 11:30 to Midnight Other duties Trade union duties AAC external member NCAA, GMC, CHAI Work for Royal Colleges Other D1 D2 D3 D4 D5 Midnight to 12:30 12:30AM to 1:00 1:00 to 1:30 Additionally remunerated work (see notes on p23) (e.g. private practice P1, Category 2 work P2 and other additionally remunerated work P3) P 1:30 to 2:00 2:00 to 2:30 2:30 to 3:00 3:00 to 3:30 Non-work activity Absent from work (annual or sickness leave) Other (i.e. time spent not working) N1 N2 3:30 to 4:00 4:00 to 4:30 4:30 to 5:00 5:00 to 5:30 On-call status (column 2 see notes on p23) Predictable on-call 1 Unpredictable on-call 2 5:30 to 6:00 6:00 to 6: :30 to 7AM

185 Step by step guide to job planning 7 Other information to be collected by the Consultant Background details Name GMC/GDC registration no Main employer Other employer(s) Date of completion of relevant appraisal Start date of new job plan Timetable & workload information List information on current main job that have you collected (delete those listed not collected) 1 Current job plan 2 List of main clinical responsibilities 3 Workload figures 4 Diary showing work other than on call Information required about your on-call 1 Actual rota frequency (state frequency) 1 in 2 Timetable of predictable PAs 3 Diary showing unpredictable on-call work 7.9

186 7 Step by step guide to job planning Information on activities that are supporting professional activities List of additional duties and responsibilities for main employer and time commitment Duty Time commitment List of external duties for other organisations, both NHS and non NHS, and time commitment Duty Time commitment

187 Step by step guide to job planning 7 List of private practice commitments Type of clinical work Scheduled time List of regular fee-paying services Type of clinical work Scheduled time List of non-regular fee-paying services Type of clinical work Frequency

188 7 Step by step guide to job planning Objectives List of the clinical and service objectives for you and your team as you understand them List of your personal objectives, as agreed in the appraisal process List of CPD/CME requirements 7.12

189 Step by step guide to job planning 7 Clinical audit and clinical governance issues Ideas for service improvements What do you feel prevents you from working efficiently? Detail below 7.13

190 7 Step by step guide to job planning What resources do you need to complete your duties and objectives effectively? Detail below Currently provided: Not currently provided, but required: 7.14

191 Step by step guide to job planning 7 Preparing the job plan There should now be sufficient information for a draft job plan to be prepared. This needs to include the following elements: sessions, timing content and where they will happen on-call commitments and the work that is generated as a consequence of being on-call commitments to the organisation that do not occur on a regular, weekly basis additional duties for the organisation commitments to other organisations personal objectives the support required to undertake the job that is already provided by the organisation additional support needed. The job planning meeting It is essential that both parties allow sufficient, uninterrupted time (usually one hour, although more time may be required for first job plans or where significant changes are being proposed). It is also essential that sufficient time has been allowed to prepare for the meeting, that all relevant information has been collated and that a provisional job plan has been drawn up for discussion. Both parties should make every possible effort to agree job plans. In the rare circumstances where a consultant and employer fail to reach agreement on the content of a job plan, either initially or at a job plan review, they should follow the mediation and appeal procedures set out in Schedule 4 of the Terms and Conditions. Once the job plan has been agreed, a copy should be sent to the Chief Executive for formal ratification and a copy retained by both parties. 7.15

192 Appendix 19. Psychological therapy staff possible time allocations by banding

193 Psychological Therapy Staff - Possible Time Allocations by Banding Percentage of Time Band 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 9 DCC = 20% 8d DCC = 30% 8c DCC = 50% 8b DCC = 70% 8a KEY Direct Clinical Care Supporting Professional Activity Other Face-to-face contact Receiving training/cpd Trade union duties Group work Teaching (preparation and delivery) Work for professional organisation / representing the profession / contributing to professional practice Clinical telephone contact Clinical consultancy/liaison MDT Meeting about direct patient care (Not CPA Meeting) CPA Meeting (Standard and enhanced) Travel (between sites, not to usual place of work) Patient administration Receiving clinical supervision Delivering clinical supervision Peer supervision (Clinical) Audit/Clinical Governance/Service Improvement & Development activities Research Management and leadership (delivering) Management and leadership (receiving) Other

194 Appendix 20. Rough guide to influencing commissioners

195 The Rough Guide to Influencing Commissioners Claire Maguire Consultant Clinical Psychologist PEC Member Bury PCT BACKROUND Tim Cate asked me to write a short briefing paper on how to influence Commissioners following a presentation I made at the North West DCP s AGM. I am a Consultant Clinical Psychologist working within a Specialist Mental Health Trust as a Clinical Lead for Primary Care Mental Health. About four and a half years ago the Chief Executive of our local PCT invited staff from Mental Health Services to apply for a position on the Professional Executive Committee. I was appointed as the Other Healthcare Professional Member of the Committee in Autumn I was successfully reappointed last year to sit on the Committee for a further four years (if there still is a PEC then!). The Role of the PEC PEC s were established as part of a PCT s governance arrangements in April It s role was to lead the PCT Board through detailed thinking on priorities, service policies and investment plans with decisions about how to take these forward largely delegated to the Executive. It was described as the engine room of the PCT Model. The new commissioning framework, published by the Department of Health in July 2006, announced a review of the role and composition of the PEC to ensure that this part of the PCT structure can be fit for purpose in the new NHS market. Every PCT will be currently reviewing the role of their PEC in this new climate. The Department of Health affirm the need to retain PEC s in the future, however their form and function may alter. How can you influence the PEC and influence commissioning? Understanding the role of the PEC is critical to understanding how to influence Commissioners. Psychologists can be involved on a number of levels. 1) It is really important to first of all find out who are the key players in your area. Each PEC is made up of a group of senior clinicians and senior management. The core membership will be GP s and other Professionals such as Dentists, Nurses, and Pharmacists. Then there will be members from the Allied Health professionals, such as Speech and Language Therapists, OT s, Physiotherapists and even Psychologists depending upon your local area. It is worth finding out how to become a PEC member in your local area. The appointments to PEC s are competency based and are often filled on a rolling basis so members come up for election at different times. 2) Find out who is the Mental Health Lead for your PCT. It is vital to know who champions Mental Health issues at PEC meetings. If no one does offer to do it! 3) There are clear Mental Health Targets that PCT s have to meet. These are described in the Mental Health National Service Framework. Each PCT is

196 asked to assess their performance against these targets each autumn jointly with local Mental Health providers and Social Services. Find out who is responsible for the presentation of these targets to the PCT. They may already contact you each Autumn when they are preparing their data for the SHA. Offer to help them and see how well the targets are met in your locality. You can access the information about the targets on the CSIP websites. 4) Usually these targets are discussed at Mental Heath Local Implementation Teams (LIT s) so if you do not already attend your local LIT start going. You will find that you will be listened to if you can provide good quality services that meet these targets and each year there is also a themed review of services. Be proactive. 5) Psychologists have particular skills to offer around the Governance arrangements for Mental health Services. It is useful to find out who your Clinical Governance Lead is at the PCT and indeed make links with the Governance Leads within Mental Health Trusts. In this area we are setting up a Psychotherapeutics Committee along the lines of a Drugs and Therapeutics Committee (which is something both GP s and PCT s are familiar with) This committee will be looking at the governance arrangements around Psychological Therapies. The PCT s locally are very happy to be part of this structure as it helps them identify and commission safe services. 6) Step out of your comfort zone. PCT s need to commission services for the whole population that they serve not just those people who are in contact with specialist Mental Health Services. We need to look at what Psychologists can offer to the Wellbeing agenda. This requires being up to date with National Policy and Guidance and being creative about how we can contribute to areas such as the care of people with Long Term Conditions, Psycho-oncology and other physical health conditions (especially those mentioned in NICE guidelines) Many PCT staff working to deliver Active Case Management would value Clinical supervision if it was made available. We also need to be more willing to engage with the Voluntary sector who may already provide services in you locality. 7) Work closely with your Director for Public Health. It will increasingly be their role to identify priorities. They are actively working to reduce health inequalities, promote good health and encourage public involvement. 8) Embrace new ways of working and ensure that new roles are couched in high quality governance frameworks. For example, many Graduate workers will go on to Clinical Training and it is important to influence these new members of our profession positively. Support Time and Recovery workers and Associate Psychology roles also are opportunities, not threats if handled positively. It is important to influence Training Courses to provide a Workforce fit to deliver new ways of working. 9) Get involved with your patients (and their carers) experience and shape services to meet needs and provide an evidence base to support change. 10) Use Research skills in a broader way to support service decisions and collate good quality data to support bids for further finances. 11) Finally, last but no means least, find out all you can about Practice Based Commissioning and how that is being implemented in your area. Good communication with PbC commissioners and consortia will be essential. This will be a challenge but it will also be an opportunity to encourage development of provision and patient choice.

197 I hope this is helpful. It is by no means an exhaustive list and it may well be very different in your area. A lot does depend upon the personalities of the key people and the strength of Partnership arrangements. However a great deal can be achieved by being an enthusiastic champion for the individuals, families, and groups that we work with and for. The time frame for PCT s to initiate change can be very swift so it is important not to sit on the sidelines and observe but to get involved (where possible) and shape the change. Claire Maguire.

198 Appendix 21. Fairer commissioning power point presentation

199 Fairer Commissioning OHOCOS: Third Sector Summit 12 March 2007 Bob Ricketts Head of Demand-Side Reform Department of Health

200 Fairer Commissioning Commissioning environment: Health Reform Commissioning defined Failure of commissioning Unfair playing field Commissioning challenge: Re-building commissioning: vision & progress since OHOCOS Commissioning Framework for Health & Well-being Next steps

201 Commissioning - part of a comprehensive health reform programme Money following the patients, rewarding the best and most efficient providers, giving others the incentive to improve (transactional reforms) Choice & Commissioning (demand-side reforms) Better care Better patient experience Better value for money More diverse providers, with more freedom to innovate and improve services (supply-side reforms) A framework of system management, regulation and decision making which guarantees safety and quality, fairness, equity and value for money (system management reforms)

202 Reform is about creating an NHS which Knows the quality of services, and rewards excellence Listens to users, and designs services to suit their needs choices Improves quality, responsiveness, efficiency, equality Develops & empowers organisations and staff Light touch monitoring with robust safeguards

203 Commissioning Definition: The means by which we secure the best value for patients & taxpayers. By best value we mean: the best possible health outcomes, including reduced inequalities the best possible healthcare within the resources made available by the taxpayer At the heart of commissioning are the millions of individual decisions of patients and clinicians that lead to the provision of care and the commitment of resources. Behind these clinical decisions lies a range of separate but related processes that collectively make up commissioning a commissioning cycle. Health Reform in England: update and commissioning framework July 2006

204 The Commissioning Cycle National targets Reviewing service provision Deciding priorities Assessing needs Published prospectus Designing services Petitions Seeking public and patient views Patient / Public Shaping the structure of supply Managing performance (quality, performance, outcomes) Referrals, individual needs assessment; advice on choices; treatment / activity Managing demand

205 Commissioning Roles: Commissioners play at least three roles: Advocate for the individual Advocate for communities Guardian of taxpayers money Taken together they create dilemmas and trade offs the essence of the role of commissioning Commissioning for quality improvement The Health Foundation, October 2006

206 Failure of Commissioning NHS has commissioned for over a decade, but Command & control delivery model has consistently reinforced the provider line Commissioners have lacked robust levers Simplistic cost & volume approach Waiting as the main control on demand Not all available levers have been used Inadequate regulatory regime bail out Low investment in developing commissioners Seen as low status stories & role models? Highly variable & fragmented practice Lack of legitimacy (linked to voice & patient /public engagement) Very limited range of providers

207 Failure of Commissioning Social care has commissioned for well over a decade: Very successful in driving down costs Greater understanding of the principles of good commissioning* Much more professional procurement approach Improving engagement of users* Improved commissioning to meet the needs of BME communities* Increased consultation with children & young people* The State of Social Care in England,

208 Failure of Commissioning Social Care: progress remains gradual *: Focus on outputs, not outcomes Extent and form of engagement varies considerably* Focus on specific groups rather than whole population* Limited Market Analysis and Development* Little Commissioning for Quality * New initiatives represent small investment of total resources* Few Councils focusing on people who fund their own care or those excluded from services by high eligibility criteria* *The State of Social Care in England,

209 An unfair playing field The indictment has been set out often: Lack of awareness amongst commissioners of the Third Sector Inconsistency of procurement and regulatory practices Great variation in when/whether to use grants or contracts Burden of disproportionate procurement practices pre-qualification requirements, guarantees/bonds Poor funding practice: lack of full cost recovery > subsidy Poor funding practice: short-termism = barrier to investment Most TSOs have not seen any general improvement in funding practices since 2002, and in some cases funding practices are perceived to have worsened Burden of disproportionate monitoring Working with the Third Sector National Audit Office June 2005

210 An unfair playing field

211 An unfair playing field

212 Re-launching Commissioning Vision + direction set out in: Commissioning Framework (July) Practice-Based Commissioning Guidance (November) NHS Contract hospitals (December) Care Resource Utilisation demand management (December Commissioning Framework for Health, Care & Well-being (March - consultation) To follow: Guidance on Free Choice (Spring) Choice Framework (late-spring, for consultation) NHS Contract hospitals & out-of-hospital services, including Third Sector (Autumn 2007 for 2008/09)