Interim evaluation of the Victorian Life Sciences Statement: Healthy Futures

Transcription

1 COMMERCIAL IN CONFIDENCE E X E C U T I V E S U M M A R Y Interim evaluation of the Victorian Life Sciences Statement: Healthy Futures Delivering better health, research and jobs for Victorians Prepared for Department of Business and Innovation 14 February 2013 THE CENTRE FOR INTERNATIONAL ECONOMICS

2 The Centre for International Economics is a private economic research agency that provides professional, independent and timely analysis of international and domestic events and policies. TheCIE s professional staff arrange, undertake and publish commissioned economic research and analysis for industry, corporations, governments, international agencies and individuals. C A N B E R R A Centre for International Economics Ground Floor, 11 Lancaster Place Majura Park Canberra ACT 2609 GPO Box 2203 Canberra ACT Australia 2601 Telephone Facsimile cie@thecie.com.au Website S Y D N E Y Centre for International Economics Suite 1, Level 16, 1 York Street Sydney NSW 2000 GPO Box 397 Sydney NSW Australia 2001 Telephone Facsimile ciesyd@thecie.com.au Website DISCLAIMER While TheCIE endeavours to provide reliable analysis and believes the material it presents is accurate, it will not be liable for any party acting on such information.

3 Interim evaluation of the Victorian Life Sciences Statement: Healthy Futures 3 Contents Fkey finding 1 Summary 4 Overview of Healthy Futures 4 Evaluation approach 6 Assessment of performance 7 Interim impacts for Victoria 12 Key evaluation messages from this review 12 Summary of key findings 17 BOXES, CHARTS AND TABLES 1 Stated objectives of the Healthy Futures program 5 2 Interim assessment of Healthy Futures (Horizon 1) 8 3 Enhanced Performance Monitoring Framework tool 9

4 4 Interim evaluation of the Victorian Life Sciences Statement: Healthy Futures Summary Healthy Futures is a $ million Victorian Government capital investment to enhance the health and economic wellbeing of Victorians and maintain the state s competitive strength in medical research. Healthy Futures has enabled the building of a wide range of strategic capital works that has provided physical space, workforce development and enabling technologies for enhanced innovation. It has created opportunities to improve the quality of medical research and contribute to the future health of Victorians. As an interim evaluation, this assessment focuses on evaluating the outcomes and impacts of Healthy Futures to date or Horizon 1 ( ). The interim evaluation also attempts to identify early signs and indicators of impacts as well as performance measures over the medium-term (Horizon 2) and thereafter (Horizon 3). There is early evidence that the scale and quality of medical research has increased as a result of Healthy Futures, and that more collaborative and translational research behaviours are being embedded among researchers. Overall, the projects have delivered on the outcomes anticipated to date, including attraction of world- class researchers, expansion of training opportunities and access to capital infrastructure and equipment, improved access to competitive funding, increased use of platform technologies and collaboration through colocation. Tangible signs of new outputs of research (health interventions, treatments and medicines and associated downstream economic impacts) are expected in future Horizons. Each of the Healthy Futures initiatives is well placed to deliver on their objectives, understanding their contributory and facilitative role in achieving improved health outcomes. This interim evaluation draws together several findings in terms of what is already working well and highlights important considerations for future assessment of the suite of Healthy Futures initiatives. Overview of Healthy Futures The Victorian Life Sciences Statement: Healthy Futures (referred to hereon as Healthy Futures) is a $ million capital investment to enhance the health and economic wellbeing of Victorians and maintain the state s competitive strength in medical research. In essence, Healthy Futures: created new or expanded world class medical research infrastructure to expand capacity and help attract the best and brightest people to generate commercial and clinical opportunities from a world-class research base; and

5 Interim evaluation of the Victorian Life Sciences Statement: Healthy Futures 5 enhanced access to health services and workforce, health information, and health priorities. This access will ultimately contribute to access to new treatments for Victorians the best medicines, treatments and healthcare and hence to ensure they continue to enjoy high quality health services. The stated objectives of the Healthy Futures initiative at its time of development are outlined in box 1. 1 Stated objectives of the Healthy Futures program The Victorian Life Sciences Statement emphasises the unique window of opportunity to build upon existing strengths and take the next steps in medical research to solve critical health problems. The following objectives of Healthy Futures are specified. Capture new opportunities in areas of research where Victoria has critical mass and a competitive advantage. Attract investment and generate high quality jobs by maintaining and growing the international reputation of Victoria s research institutes. Maximise opportunities for continued growth in national and international collaborations and partnerships. Deliver major benefits to business and industry by encouraging the commercialisation of medical research. Create a healthier future by enabling research to translate speedily into practical health benefits for the entire community. Healthy Futures captured an important moment in time, successfully capitalising on a limited window of opportunity in terms of the availability of financial capital and the congruence of political will at the State and Commonwealth level. The Healthy Futures investment was motivated by the need to address space constraints and overcrowding of medical research infrastructure and to realise potential scale economies and increased productivity of labour from increased collaborative research efforts. It was also initiated during a period of substantial innovation investment by the Victorian Government, reflecting a commitment to stimulate infrastructure and capability for innovation to underpin long-term economic and social outcomes for the state. Healthy Futures also sought to leverage additional philanthropic and Commonwealth funding available given the favourable economic and financial landscape at the time. While this creates challenges in attributing changes to Healthy Futures, it does mean that the ultimate impacts associated with Healthy Futures are much larger than those that would have been achieved in isolation. In summary, Healthy Futures made several important contributions to the value proposition for Victoria as being the place to do medical research in Australia, and indeed the world. It also contributes to the broader long-term objective of boosting the level of high quality medical research and associated clinical translations.

6 6 Interim evaluation of the Victorian Life Sciences Statement: Healthy Futures Evaluation approach The evaluation methodology for Healthy Futures addresses the appropriateness, effectiveness, and efficiency of the suite of projects within a program logic framework. It spreads the program logic across three distinct time horizons given the long time lag between research and workforce development, discovery, development, and translation into clinical applications. In the case of Healthy Futures, the time lag is extended further due to additional time required to secure the necessary resources associated with capitalising effectively on infrastructure spending which required additional leveraged capital funding, leveraged operating funding, recruitment and purchasing before additional research activities could get underway. An innovative feature of this interim evaluation is that it tackles key challenges in attribution for the Healthy Futures set of projects, particularly by incorporating qualitative approaches to develop a systems view of changes associated with the program. For outcomes to date, inevitably many are associated and/or in some way linked to earlier or subsequent measures. In addition, there is an extended payback period for these types of investments and early signs of potential future impacts need to be assessed, despite intensified challenges of attribution over the longer term. The key methodological steps involved in this interim evaluation included the following. Literature review the information gathering phase reviewed relevant available information regarding the what, why, and how of the Healthy Futures initiative. This included a review of Healthy Futures program documents and status updates, Commonwealth and State Government policy statements, institutional annual reports, independent evaluation reports and wider literature. Stakeholder consultation face to face meetings were held with each Healthy Futures program beneficiary (for instance, various medical research institutes (MRIs), providers of tertiary health services, CSIRO and other enabling medical research infrastructure providers) and relevant Victorian government agencies. Data collection existing Healthy Futures evaluation surveys from 2008 to 2012 were supplemented by open-ended stakeholder questioning. Bibliometric publications and citations data, National Health and Medical Research Council (NHMRC) funding data, Cooperative Research Centres (CRC) funding data, National Survey of Research Commercialisation data (including regarding the scale of MRI patents, licences and invention disclosures and the number of spin-off companies) and data from the Victorian Government s program of support for independent Medical Research Institutes were also collected and analysed. Quantitative analysis Computable General Equilibrium modelling was undertaken to estimate the indirect impacts of increased capital expenditure associated with Healthy Futures to date. Qualitative analysis focus was placed on case studies of behavioural changes and impacts, attracted expertise and associated new research outcomes, an assessment of governance arrangements and horizons assessment of future impacts. The methodological steps discussed above were used to undertake a horizons assessment of Healthy Futures meaning that each component of the program logic framework was

7 Interim evaluation of the Victorian Life Sciences Statement: Healthy Futures 7 considered in the context of timing of incidence. As an interim evaluation, this assessment focuses on evaluating Healthy Futures in the program logic framework over Horizon 1 (that is, from 2006 to 2012 only). The interim evaluation also attempts to identify early signs and indicators of impacts as well as performance measures over the medium-term (Horizon 2) and thereafter (Horizon 3). However, later evaluation of Healthy Futures will determine the full scope of impacts and benefits over these future periods. A summary of the appropriateness, efficiency and effectiveness of Healthy Futures as at 2012 is set out in chart 2. Assessment of performance The assessment above builds upon DBI s existing Performance Monitoring Framework to identify key indicators of program outcomes/impacts over various periods in order to assess the appropriateness, effectiveness, and efficiency of the program elements. Chart 3 below summarizes the Performance Monitoring Framework developed by DBI for Healthy Futures, and identifies additional assessment measures that were developed as part of this interim evaluation. The chart also shows which indicators are believed to be directly attributable to Healthy Futures (pink) or partly attributable to Healthy Futures (black). Assessment of Horizon 1 impacts demonstrates that Healthy Futures has made a genuine difference to the scale and scope of medical research, workforce development and enabling technologies in Victoria. All elements of the program are assessed to be largely appropriate as a single event package of measures,1 although the extent to which they fill gaps in existing capacity and enhance the productivity of research inevitably varies. Healthy Futures has already delivered a range of economic and social returns to Victoria. In addition, returns to date are expected to grow in future as the induced scale of research, and changes in collaborative behaviours lead to increased research output, clinical translations, and improved health outcomes. Indeed the outputs of Healthy Futures are found to be critical to the probability of future success. Its value also lies in improving the efficiency of other research and innovation spending in the State. Healthy Futures projects have been positive enablers to achieving the overarching objectives of the program, particularly in terms of positive workforce impacts, research outputs and the branding value for Victoria as a destination for medical research. 1 Single event refers to Healthy Futures as being a one-off investment in medical research infrastructure. Hence, the positive assessment of Healthy Futures does not necessarily infer that additional medical infrastructure investment would be warranted and/or would generate the same benefit profile.

8 2 Interim assessment of Healthy Futures (Horizon 1) Objectives Inputs Outputs Capture new opportunities in areas of research where Victoria has critical mass and a competitive advantage Healthy Futures capital funding Attracted funding to complete capital builds New purpose-built facilities to support biomedical research and capital improvements in medical/research training New platform technology resources developed Enabling facilities underpinning multiple institutions and access to equipment Increased output and employment in the building and construction phase Attract investment and generate high quality jobs by maintaining and growing the international reputation of Victoria s research institutes Horizon Financial Impact to Date New equipment Leveraged funding for operating costs People involved in training courses on the use of Healthy Futures funded infrastructure Increased number of new medical training places capacity and uptake of PhD/Masters candidates under supervision in HF supported activity Access to attracted world class scientists Maximise opportunities for continued growth in national and international collaborations and partnerships Total leveraged funding of $508.1 million ($2.20 per public $ invested) Assessment of appropriateness: Achieved Increase Victoria GSP by about $170 million per annum Increase household consumption by around $77 million per annum 1480 employment years created since 2006 Deliver major benefits to business and industry by encouraging the commercialisation of medical research Comments Access to new equipment (e.g. bioprocessing facility) and enabling technologies (e.g. VeRSI) Access to new sources of operating expenditure from competitive funding and philanthropic/other sources (HF seeded new funding sources). For instance, the Australian Cancer Data Grid has been awarded independent grants for ongoing operation (e.g. Roche, Sanofi Aventis, Novartis grants) Career paths for next generation researchers Create a healthier future by enabling research to translate speedily into practical health benefits for the entire community Attraction of world-class international researchers. For instance, eight of the twelve group leaders working at the ARMI were high profile international recruits Boost to scale, quality and functionality of research facilities (with benefits in terms of greater collaboration and multi-disciplinary teams). For instance, space constraints for WEHI researchers were solved and teams were bought closer together into a common facility Workforce enhancement via new training opportunities. For instance, new research positions available at Healthy Futures institutes and more medical university places available in rural and regional Victoria 8 Interim evaluation of the Victorian Life Sciences Statement: Healthy Futures Assessment of efficiency: Achieved Outcomes Increased medical and clinician researchers, engineers and support staff Steady/ access to peer reviewed (NHMRC/ARC) research funding Use of new platform technologies by research groups Expatriate researchers and clinician researchers attracted back to Victoria collaboration through colocation Increased clinical trials providing increased income for hospitals Annual fee for service income of over $1 million from industry accruing to the Bioprocessing facility Maintenance of access to competitive Commonwealth research grants (e.g. NHMRC, CRC, Commonwealth Education Investment Fund, ARC, CSIRO) More strategic prioritisation of research. For instance, the Victorian Cancer Agency focuses on priority areas for improving health outcomes. Providing for a faster, more efficient process to conduct clinical trials at multiple sites Upskilling the medical workforce Impacts GSP impacts associated with enhanced expenditure from attracted labour from interstate/ overseas and additional capital investment Small but positive increase in productivity of labour employed in medical research in Victoria Assessment of effectiveness: Achieved Key Achieved in full Achieved as expected to date

9 3 Enhanced Performance Monitoring Framework tool Indicators from DBIs Performance Monitoring Framework Additional indicators developed as part of this interim review APPROPRIATENESS Evidence that medical research and training capabilities were inadequate Evidence of potential loss of competitive advantage Consistency with DBI and broader Government objectives Alignment with Commonwealth Government objectives, policy and investments Alignment with best practice research trends Focus on additionality (non-duplicative and game changing) Primarily support for basic, pre-commercialisation and publicly funded research Focus on system-wide performance Alignment with the goals of non Government stakeholders Investment unlikely to crowd out commercial activity given market failures EFFECTIVENESS EFFICIENCY LESSONS LEARNED/ FUTURE DIRCERTION Impact on attraction and retention of researchers, clinicians and future leaders Increase in quality jobs in innovative industries Leveraging non-state Government funding for facilities and infrastructure Encouraging national/international linkages, partnerships and collaboration Efficiency of infrastructure delivery Lessons from delivery of Healthy Futures to date Capacity to rapidly translate medical research into improved health outcomes Increase in high quality education and training Attracting investment in and maximising new research opportunities Increasing research focus on clinical / commercial outcomes Enhancing capacity, efficiency and integration of existing facilities Efficiency in the use of DBI resources Opportunities for future improvement in design and delivery of Healthy Futures Maintaining/growing Victoria s biomedical research capabilities Recognition of Victoria as a world leader in biomedical research and technology Enhancing the Victorian medical and research training environment Efficiency of administrative costs Adequacy of resources to enable Healthy Futures to achieve its objectives Creation of career paths for the next generation of researchers in Victoria Strengthen Victoria s reputation and international presence Opportunity for prioritisation of research activity Impact on access to new equipment Attract competitive funding, industry/philanthropic support for operating expenditure Increase industry partnerships, spin-off companies and commercialisation activity Direct economic benefits Victorian GSP, employment, household consumption Achieving scale and integration Increase clinical trial activity in Victoria Key Evidence of the impact directly attributable to Healthy Futures Evidence of the impact partly attributable to Healthy Futures Interim evaluation of the Victorian Life Sciences Statement: Healthy Futures 9 Data source: The CIE

10 10 Interim evaluation of the Victorian Life Sciences Statement: Healthy Futures In terms of outputs: Healthy Futures has already realised positive impacts on access to new equipment, both directly and indirectly. For instance, Healthy Futures has directly provided Victorian researchers access to a bioprocessing facility (creating a substantial private revenue stream of $1.2 million annually by making access open to industry); equipment required for research using bioresources (at a bioresources facility established for the Austin Biomedical Alliance); and New Grid infrastructure. Indirectly, the impending availability of new infrastructure and larger research capacity provided the impetus and business case for advanced biomedical imaging equipment (Victorian Biomedical Imaging Capability) in facilities across Melbourne, including the new Neuroscience facilities; in the majority of cases, Healthy Futures has been an effective recruitment tool and has assisted to meet workforce challenges across the spectrum of need. It has already been successful in assisting Victoria to attract world-class researchers. For instance, eight of the twelve group leaders now working at the Australian Regenerative Medicine Institute (ARMI) were high profile international recruits. In addition, the Walter and Eliza Hall Institute (WEHI) has attracted back key research staff after various stints in overseas research institutes following Healthy Futures investments. Healthy Futures has also better enabled the creation of career paths for the next generation of researchers in Victoria by enhancing the quality and quantity of research opportunities; Healthy Futures has improved workforce development by facilitating improvements in the quality of training and development, including enhancements to facilities in rural and regional areas. For instance, the Medical University Places Capital Infrastructure program built teaching, training and research facilities, tutorial rooms, libraries, student accommodation and other education amenities to a) improve opportunities for medical training in regional Victoria, and b) encourage medical graduates to work in rural areas. In addition, increased physical capacity at Healthy Futures medical research institutes has increased the number of research training positions available. For instance, the WEHI currently has 80 PhD students, of which some are directly attributable to the additional physical space that has resulted from Healthy Futures; as well as bringing direct economic benefits through net capital inflow, Healthy Futures also strengthens Victoria s international presence and reputation as a world-class destination for medical research. Increased reputation and leadership of Victorian research is evidenced by, for instance, by the ARMI being selected to host the European Molecular Biology Laboratory (EMBL) Australia. Associate Membership with this global centre of excellence2 provides Australian researchers with access to state-of-the-art infrastructure and technologies, expertise and networks in Europe; new opportunities to forge collaborations with European researchers and institutions; and access to European funding opportunities. ARMI has also recently attracted a partnership with Japan s Systems Biology Institute highlighting the ARMI s success in forging strong links with internationally recognised research organisations. Healthy Futures has improved opportunities for clinical translation. Clinical practice aligned and integrated with discovery research is a strong feature of research culture 2 EMBL is the most cited scientific institution outside of the USA in molecular biology and genetics, with an extremely high impact of an average of 51.9 citations per paper.

11 Interim evaluation of the Victorian Life Sciences Statement: Healthy Futures 11 and practice, which underpins Victoria s reputation and performance in health sciences. This was given a significant boost through the Healthy Futures investment. At the WEHI, new facilities have enabled much closer interaction with clinicians at the Royal Melbourne Hospital. In addition to direct benefits resulting from more rapid and efficient translation of research findings to clinical practice, this new association has encouraged a more active research interest amongst clinicians, which in turn is likely to result in improved clinical care and outcomes for patients. Further, a new test developed by researchers at the Burnet Institute offers a simpler and cheaper way of monitoring HIV in sufferers and a wristwatch that continuously monitors the health of patients with Parkinson s disease developed by Australian Centre for Neuroscience and Mental Health Research (ACNMHR) scientists was announced at the international BIO2009 Conference. At this relatively early stage, Healthy Futures has achieved several successful outcomes: it has achieved scale and integration, which has helped grant recipients continue to attract major national and international competitive funding, as well as industry and philanthropic support; it has enhanced collaborations on new platforms underpinning future medical science outcomes; it has provided the opportunity for prioritisation of research activity following integration of research institutions and teams. For instance, investment by Healthy Futures in the Victorian Cancer Agency (VCA) enabled a coordinated approach on strategy for cancer research in Victoria. Its strategy is increasingly to focus on priority areas for improving health outcomes. The strategy includes initiating new research areas where there is clear evidence that current research effort is misaligned relative to priority health issues; it has led to some early signals of success in indicators of clinical translation of research efforts of MRIs including new and pre-existing international patents and income from clinical research projects; and there are also positive developments in clinical trial activity that are partly attributable to Healthy Futures. The streamlining of ethics approval for multi-site trials has directly addressed time and cost factors that influence trial location decisions for multinational pharmaceutical companies and multi-site clinical trial activity in Victoria over the last few years. This will have been at least partly facilitated by Healthy Futures. Data from the Australian and New Zealand Clinical Trials Registry shows an increasing trend in the number of registered trials occurring at the broader state level. To date there are few early examples of industry partnerships induced by Healthy Futures. An exception is Victorian technology company Global Kinetics Corporation, which worked closely with researchers from the Florey Neuroscience Institutes to develop a prototype micro-electro-mechanical wristband device. It is expected that spin off impacts for biotechnology start ups are expected to increase and evolve in a way that is at least partly attributable to various Healthy Futures projects. This is because there is a strong supply side element to attracting downstream activity in medical research in Victoria. Biotechnology start ups and existing biotech and pharmaceutical companies are attracted by a strong supply base of people, skills, and intellectual property all of which have been enhanced by Healthy Futures.

12 12 Interim evaluation of the Victorian Life Sciences Statement: Healthy Futures It is also expected that the increased reputation brand and leadership of Victoria s medical research sector will be evidenced in the future by additional extramural staff attraction and in rising research income sourced internationally. Further, the quality and quantity of Victorian research outputs are expected to continue to rise evidenced by increasing publication rates and citations received by Victorian researchers. Interim impacts for Victoria Healthy Futures seed funded a total program spend of $932 million in Victoria, leveraging over $701 million from other Government and non-government sources that is unlikely to have been invested without Healthy Futures. On average, Healthy Futures funding leveraged $4 for every dollar invested by the Victorian Government, which varied across projects from $1.20 to $18.60 per dollar invested. Including financing from the Victorian Government in addition to Healthy Futures, each dollar of state government funding leveraged $2.20 from Commonwealth, philanthropic, institutional and other sources. Based on economy-wide modelling on Victoria, by , Healthy Futures is estimated to have increased Victorian GSP and household consumption by approximately $170 million and $77 million respectively. Of this, $81 million represents the return to labour, which based on average wage rates in Victoria, is equivalent to close to employment years created. Large future economic impacts are expected over subsequent Horizons, which are expected to show more direct benefits in terms of research collaboration, industry partnerships, and ultimately improved clinical treatments and therapies that will improve health outcomes. Healthy Futures has made a strategic and lasting impact on the scale and quality of research activity and behaviours to make it more likely that successful clinical translations will result over time, producing improved health outcomes over the medium to longer term. Given the nature of drug development and the need for MRI initiated research to be developed by start up companies and/or downstream users, it is highly likely that Healthy Futures has and will generate positive impacts in terms of spin off companies over time. In terms of future research outputs, Horizon 2 is expected to witness an increase in the number of international publications in peer reviewed journals, while in Horizon 3, the number of citations attracted by Victorian-authored publications is expected indicating the rising quality of research outputs. In essence, Healthy Futures is expected to have made a lasting impact to the foundation that matters in terms of probability of successful, outcomes based research. Key evaluation messages from this review The interim evaluation of Healthy Futures is a positive one and there are findings that reinforce positive elements of the program and serve to highlight areas where future effort

13 Interim evaluation of the Victorian Life Sciences Statement: Healthy Futures 13 and resources may need to be brought to bear in order to best ensure that the potential benefits from the investment are realised. Governance and administration Projects of the size of Healthy Futures take considerable resources to govern and administer well. The following levels of governance are recognised as being important in assessing Healthy Futures: project implementation and subsequent management for each Healthy Futures initiative, particularly the delivery of intended outputs this is the responsibility of the leaders of the institutions granted Healthy Futures funding (e.g. WEHI, VCA); program oversight and review of the performance, outcomes and impacts for the overall Healthy Futures program this is the responsibility of the DBI; and policy oversight and review of the appropriateness and effectiveness of Healthy Futures program in relation to higher-level, longer-term, and wider-ranging government goals for health and medical sciences in Victoria this is the responsibility of the Victorian Government. Project Healthy Futures has a project governance component, given that various discrete projects each have their own strategic intent and high level governance structure in place. There are examples where this project level of strategic governance was very sound. The WEHI was an exemplar in this regard, complementing overall governance through its Board/Audit and Risk Committee with a dedicated project oversight committee. The high-level New Building Sub-Committee met monthly over the course of the project. The Sub-Committee received full executive support from the Institute and was attended by key project consultants (project management, architects, and quantity surveyors) and additional consultants as required. The Sub-Committee also included opportunities for DBI and Department of Health participation from the outset. In additional to steering the capital development, the Sub-Committee monitored the Institute's obligations to funders of the project and compliance with all reporting requirements. This approach ensured strong governance for the specific project and strengthened communication with DBI and its higher-level (program) governance role. Project governance was arguably less strong with respect to the Australian Cancer Data Grid project, where there could have been stronger emphasis on business planning, strategic intent in the activities undertaken and focus on bringing the new platform to end-users. Greater emphasis on these elements should assist with building broader stakeholder support and more widespread use of project outputs. In terms of enhanced project level governance, in addition to regular reporting to DBI, leaders of the respective projects within the Healthy Futures program could be involved in more regular and collaborative self-review, with reflections shared among program beneficiaries. This would encourage active learning across the Healthy Futures community, through which governance of individual projects should improve over time.

14 14 Interim evaluation of the Victorian Life Sciences Statement: Healthy Futures Program Healthy Futures was a grants-based program, with much of the project management the responsibility of medical research institutes and agencies. DBI s program governance role (oversight and review of performance, outcomes and impacts of the overall Healthy Futures program) is considered appropriate. DBI had generally sound strategic governance structures in place for interacting with each Healthy Futures project, with interdepartmental consultation established wherever relevant, arranged as required on an as-needed basis. These arrangements generally enabled DBI to be close enough to projects to genuinely ensure their most appropriate, effective and efficient implementation, with some qualification relating specifically to the administrative resources allocated to project oversight. With no specific allowance made in the program budget for government administration, in-kind government resources were likely to have been spread too thin. The total administrative costs of the Healthy Futures program were estimated by DBI to be $1.66 million since program inception to June This represents 0.7 per cent of the Healthy Futures investment of $ million and 0.4 per cent of total program costs when subsequent Victorian Government funding is included. We note however, the substantial in-kind support that was mobilised by the funding recipients to deliver Healthy Futures projects. Australian National Audit Office (ANAO) suggests that there is no apparent benchmark for the ratio of administrative costs to program costs. Costs observed in audits of grant programs range from as low as 1 per cent to as high as 35 per cent.3 The appropriate share of administrative costs to program funds will vary according to: the size of the program smaller programs will have a higher proportion of fixed costs, which will raise the proportion of administrative costs to program funds; and the risk associated with the program higher risk individual projects or programs will require more costly risk treatments including closer monitoring. Given these factors, it is likely that the administrative resources allocated to Healthy Futures were too limited. The ANAO suggests that insufficient administrative resources to manage grant programs increase the risk that the program s objectives may not be achieved in an efficient, effective and timely manner. That said, administrative resources do appear to have been managed effectively for contract management and project oversight activities. DBI as lead agency also undertook extensive reporting to Cabinet, in partnership with the Department of Health. This was particularly the case in the period from inception to late 2010, which was a critical period in the development of each initiative. No criticisms were raised by any stakeholders on the role of DBI in managing and reviewing any of the Healthy Futures projects. 3 Australian National Audit Office 2002, Administration of Grants: Better Practice Guide, p.14.

15 Interim evaluation of the Victorian Life Sciences Statement: Healthy Futures 15 Policy Healthy Futures is a good example of governance working well at the policy level. Healthy Futures is a highly strategic set of initiatives that are well complemented by, and consistent with, the broader innovation framework of the Victorian and Commonwealth Governments. Consistent with the Commonwealth s innovation strategy Powering Ideas and the Victorian Government s recent Technology Plans for the Future, Healthy Futures focuses on enhancing Victoria s competitive strength and reputation in science and innovation and continued priority investment in medical research. Healthy Futures had longer-term expectations in mind, which were captured in the specific objectives defined at the time of program inception, and have continued to be referred to by program beneficiaries when reflecting on Healthy Futures today. The importance of enabling system evaluation capability in future Longer-term impacts of a program such as Healthy Futures will be dispersed through the medical sciences system in Victoria. Future evaluation of Healthy Futures should capture if, and how, the system performance targets are being met over time. This is particularly important given that impacts genuinely attributable to Healthy Futures are likely to be small and understate their true value. Improved data collection could help to achieve this, possibly facilitated by more administrative resources being dedicated to project oversight. For instance, not all recipients provided evaluation surveys to DBI and in most cases, responses were incomplete. While we note that the data collection tool was less relevant to some projects, while others were subject to separate and specific program evaluations, incomplete survey data has created challenges for this evaluation and will continue to do so. It is therefore important that greater compliance with reporting responsibilities be achieved. Further, more detailed and timely information on medical science, public health and business innovation metrics for Victoria, to help assess changes in priority policy areas over time. It is also important to focus on drivers of medical sciences performance and outcomes. For example, bibliometric assessments of research outputs and their impact by subject area and institution can be valuable for understanding the international competitiveness of research capability in Victoria and therefore the potential to compete beyond Victoria for research funding. Similarly, business innovation metrics for firms involved in developing and deploying medical technologies derived from research in Victoria s medical research institutes can help understand constraints and opportunities affecting business growth. The current Healthy Futures survey tool is well designed and appropriate. However, incomplete responses to date limit their usefulness for program evaluation and data analysis purposes. In order to decrease the compliance burden and increase response rates, consideration may be given to sourcing data through existing structures. For instance, MRIs could be asked to provide responses to the National Survey of Research Commercialisation and elaborate where deemed appropriate. Further, bibliometric data

16 16 Interim evaluation of the Victorian Life Sciences Statement: Healthy Futures may be sourced from independent sources, which would reduce the compliance burden on MRIs, and also allow for comparative data analysis. Options for future investment should be explored Collective effort to increase access to operational funding is likely to be needed to realise greater value from the infrastructure investments made through Healthy Futures.4 This is likely to depend on leadership to facilitate a collaborative effort for identifying and capitalising on funding options (including Commonwealth and international funding agencies, commercial and philanthropic investors), thereby strengthening Victoria s overall competitiveness in attracting such investment. Securing ongoing operational funding is an important issue, although it is not a direct issue for this evaluation. All stakeholders were clear during the development of the initiatives that operating funding was not provided for under Healthy Futures and all stakeholders agreed that operating funding would be obtained from existing funding sources. Operating funding is an issue being considered in the McKeon Strategic Review of Health and Medical Research in Australia, which advocates that MRIs receive at least 60 per cent indirect cost loading for national competitive grants. In its response to this consultation paper, the Victorian Government emphasised the importance of predictability and certainty of funding for health and medical researchers over the longer-term. Consistent with this view, it is important to articulate what the investment options are for building on Healthy Futures achievements to date. Consideration should also be given to encouraging and rewarding successful implementation. As most Healthy Futures projects are still at relatively early stages (relative to their longer-term value), there is plenty of scope to influence the overall performance of the investments made. Further State Government investment in medical sciences should therefore encourage and reward project recipients to maximise the impact of the (sunk) investment already made. This could include on-going expectations that project recipients define what is being done, and what more is needed to maximise long-term value, as part of future funding requests related to the infrastructure established through Healthy Futures. Tell the Healthy Futures Story It is important to promote the success and value of the strategic investment in Healthy Futures. As investment priorities and mechanisms vary over time, there is merit in ensuring a record of how different approaches work, at different stages of sector development and maturity. 4 Operational funding may refer to a) indirect cost funding, b) direct research funding from non- Victorian Government sources and c) equipment and infrastructure funding. Future references to operational funding include all of the above.

17 Interim evaluation of the Victorian Life Sciences Statement: Healthy Futures 17 Healthy Futures was implemented during a period of substantial infrastructure investment in medical sciences and other innovation areas in Victoria. The aggregate impact of these investments is inevitably complex and hard to track over time. This means it may be difficult to ensure future policy makers appreciate critical features of this investment approach. The Healthy Futures story should therefore be recorded in a format that is easily absorbed by future policy audiences. This interim evaluation of Healthy Futures will be important in this regard. Summary of key findings Key finding #1: Healthy Futures seed funded a total program spend of $932.3 million, leveraging over $508 million from non-victorian Government sources that is unlikely to have been invested without Healthy Futures. Key finding #2: On average, Healthy Futures and subsequent Victorian Government funding leveraged $2.20 from Commonwealth, philanthropic, institutional and other sources for every dollar invested by the State Government. This varied across projects from $1.00 to $4.40 per dollar invested. Key finding #3: Healthy Futures has had a positive impact on access to new equipment, both directly and indirectly. In the case of the bioprocessing facility, a private revenue stream has already been created. While this is a positive outcome and demonstrates private value, it remains the case that Healthy Futures investments were not intended to establish profit-making centres, but rather to support and advance Victoria s medical research base. Key finding #4: By , Healthy Futures is estimated to have increased Victorian GSP and household consumption by approximately $170 million and $77 million respectively. Of this, $81 million represents the return to labour, which based on average wage rates in Victoria, is equivalent to close to employment years created. Key finding #5: Healthy Futures has enhanced the quality and quantity of research training opportunities and enabled career paths for early career researchers in Victoria, and has significantly increased the capacity for regional medical training. Key finding #6: Healthy Futures has already been successful in assisting Victoria to attract world-class researchers. As well as bringing direct economic benefits through net capital inflow, it also strengthens Victoria s international presence and reputation. Key finding #7: Healthy Futures has enhanced collaborative approaches to research, including through e-research and ICT-enabled platforms and has better enabled clinicians to be engaged with research. Key finding #8: In the few years since Healthy Futures investments became operational, Victoria has been able to maintain its first-place status in terms of access to competitive Commonwealth research grants.

18 18 Interim evaluation of the Victorian Life Sciences Statement: Healthy Futures Key finding #9: Healthy Futures was an important first step in helping to improve the viability of a clinical research environment in Victoria. The streamlining of ethics approval for multi-site trials has directly addressed critical time and cost factors that influence trial location decisions for multinational pharmaceutical companies and multi-site clinical trial activity in Victoria over the last few years will have been at least partly facilitated by Healthy Futures. Key finding #10: Healthy Futures has had an important impact on supporting the conditions for future positive translations from research to clinical practice and treatments. Key finding #11: Healthy Futures has made a strategic and lasting impact on the scale and quality of research activity, and behaviours it has reinforced make it more likely that successful clinical translations will result. It is the finding of this evaluation that improved health outcomes will be a result of the program over the medium to longer term (Horizons 2 and 3). Key finding #12: Given the nature of drug development and the need for MRI initiated research to be developed by start up companies and/or downstream users, it is highly likely that Healthy Futures will generate positive impacts in terms of spin off companies. Key finding #13: The quantity and quality of Victorian research outputs have been rising steadily since 2002 and indicate that, in terms of research output, Victoria is currently punching above its weight. Healthy Futures is expected to reinforce this trend. Horizon 2 is expected to witness an increase in the number of international publications in peer reviewed journals, while in Horizon 3, the number of citations attracted by Victorian-authored publications is expected, which would indicate the rising quality of research outputs. Key finding #14: Healthy Futures is expected to have an important impact on the international profile, reputation and leadership of Victorian research. Healthy Futures initiatives have already attracted world class researchers and peaked the interest of global centres of excellence such as European Molecular Biology Laboratory and Japan s Systems Biology Institute. Key finding #15: The Department of Business and Innovation should continue to promote and facilitate the linkages between medical research institutes and private industry in order to help institutes diversify their funding base.

19 Interim evaluation of the Victorian Life Sciences Statement: Healthy Futures 19 THE CENTRE FOR INTERNATIONAL ECONOMICS