Department of Canadian Heritage ( DCH or The Department ) Summary of the Evaluation of the Canada Media Fund 2010-2011 to 2013-2014 ( Report ) TOPICS: A) Overview B) Relevance 1. Responsiveness to the Needs of Canadians 2. Alignment with Government of Canada s Priorities 3. Alignment to Federal Roles and Responsibilities C) Performance - Effectiveness 1. Outputs 2. Immediate Outcomes 3. Intermediate Outcomes 4. Ultimate Outcomes 5. Unintended Impacts D) Performance Efficiency & Economy 1. Demonstration of Efficiency and Economy 2. The Support of Canadian Convergent Television and Digital Content 3. Duplication 4. Governance E) Other Evaluation Questions 1. The Performance Measurement Framework 2. The Requirement for the Second Platform in the Convergent Stream 3. Two Streams of Funding 4. Underrepresented Genres 5. Leveling the Playing Field 6. Additional Funding Triggers 7. Minority French- and English-language Production F) Recommendations G) Looking Forward
ANALYSIS: A) Overview Overall, the Department s evaluation of the CMF is very positive. Broad recurrent themes of alignment (with both the government and industry), effective and responsive frame the majority of the Department s analysis and conclusions throughout the Report. The Evaluation Approach and Methodology section clarifies that the evaluation was led by the Department s Evaluation Services Directorate, with some components conducted by the Department s Policy Research Group and an external consulting firm. The Report highlights the variety of methods used in the evaluation (including, amongst others, a literature review, key informant reviews, case studies and an expert panel), but also notes that certain areas of activity (e.g., performance of CMF Board members and CMF staff) were excluded from the scope of evaluation. The Report concludes with seven recommendations and an accompanying timeline for deliverables and next steps. B) Relevance 1. Responsiveness to the Needs of Canadians The Report concludes that the CMF is the single most important source of financing for Canadian digital content in the drama, documentary, children and youth, and variety and performing arts genres. Further, the CMF directly addresses the distinct issues associated with the small size of the English and French-language Canadian markets, including, but not limited to, the financial disincentives associated with the creation of original programming as compared to acquiring less expensive foreign content and requiring support tailored to the specificities of each language market. Additionally, DCH references numerous studies that confirm that Canadians value and seek access to Canadian content and the CMF responds to such demand as it ensures that Canadian digital content is created and made available on multiple distribution platforms. This CMF-funded television and digital content remains popular with Canadians and there is increasing evidence which shows that Canadians are engaging with CMF supported content and applications created exclusively for the Internet, wireless and other platforms and devices. The Report notes that while television is expected to be the primary platform by which Canadians are accessing Canadian content in the near future, as digital technology continues to expand the delivery methods by which content is made available, the CMF is vitally relevant to ensure that Canadians continue to have access to a wide range of content on a variety of platforms. Finally, the Department observes that the Government of Canada s objectives in connection with official language minority communities in Canada are reinforced by the CMF through its support for productions in Aboriginal and other diverse languages and the French and English-language minority communities. 2
2. Alignment with Government of Canada s Priorities The Report concludes that the CMF is very well aligned to federal Government priorities and departmental strategic outcomes as both the mandate and objectives of the CMF support the Government s priority for the creation of a vibrant Canadian culture and heritage. Further, DCH notes that the CMF is aligned to the Department s Program Alignment Architecture and supports the strategic outcome that "Canadian artistic expressions are created and accessible at home and abroad". The CMF is also consistent with the departmental priority of taking full advantage of digital technology. Finally, the CMF reinforces the Government of Canada s digital agenda by strengthening Canada s capacity to lead the digital economy with its support to Canadian digital content and applications. 3. Alignment to Federal Roles and Responsibilities The Department holds that the CMF is very well aligned to federal government roles and responsibilities as it is an instrument by which the federal government can support Canadian broadcasting policy. Additionally, the CMF supports the Department and its responsibility to ensure that the broadcasting and digital communications sectors contribute to the realization of the policy objectives set out in the Broadcasting Act. Further, as noted above, the Report acknowledges that the CMF s support of the development of leading-edge digital content and applications is aligned to the federal government s development of a Canadian digital economy for Canadians. The Department specifically references that the CMF directly supports the government s Digital Canada 150 strategy, which identifies the production of Canadian digital content as one of its five pillars. One of the outcomes of this strategy is that Canadians are (a) able to see themselves reflected in digital content and have a broad range of choices in the way they access that content and (b) for creative industries to have greater capabilities to seize digital opportunities, promote Canadian content and play a more prominent role in the global marketplace. C) Performance Effectiveness 1. Outputs DCH concludes that, during the evaluation period, the CMF catalyzed the creation of high-quality convergent television content distributed on multiple platforms, interactive digital media content and software applications. The Report notes that of all CMF- supported digital content projects, about half (48%) had valueadded original digital media content created for a second platform, while others made the television property available via video-on-demand (VOD) or through digital distribution on unlicensed video streaming platforms. In the Experimental Stream, the CMF enabled projects to develop, produce or market leading-edge interactive digital products and software applications. The most common content type of projects funded through the Experimental Stream was Games, which comprised nearly half (46%) of all projects supported in production. 3
2. Immediate Outcomes DCH concludes that, during the evaluation period, the CMF stimulated increased availability of highquality content on more than one platform. In the English-language market, there was a 30% increase in the number of CMF-funded program hours which were licensed and scheduled in peak viewing hours. In the French-language market, there was a 12% increase in the number of CMF-funded program hours broadcast. In the Convergent Stream, as stated above, almost half of projects were made available with valueadded interactive digital media content developed for or distributed on a second platform. The Report specifically highlights, however, that while value-added digital media content is well regarded in the industry for its promotional use, the CMF s additional rich and substantial threshold (which is more stringent than merely value added ), should be examined and potentially revised. In the Experimental Stream, projects were supported either for commercial exploitation or to be made available for free to the public. The Report specifically highlights, however, the tension between the commercial and innovation objectives of the Experimental Stream. 3. Intermediate Outcomes The Department holds that the CMF contributed to the quality of Canadian digital content made available as Canadians (a) are watching more hours of CMF-funded Canadian Television programming and (b) have increased their consumption of Canadian digital content distributed on at least two platforms and are engaging with Canadian interactive digital media content and applications. The Report notes that while foreign programming continues to be very popular among viewers in the English-language market in prime time, the demand for CMF-funded programming garners the same proportion as supply in connection with Canadian content. As this is not the case for non-cmf funded programming, DCH states that this suggests the CMF funds quality programming Canadians want to watch. In the French-language market, DCH observes that Canadian programming dominates and CMF-funded programming continues to be very popular. The Report notes that while one quarter of the broadcast schedule is made up of CMF-funded programming, two-fifths of the viewing is dedicated to CMFsupported programs. In addition to viewership, the Report notes that projects funded under both the Convergent and Experimental Streams captured a number of domestic and international awards. While DCH concedes the difficultly in accurately measuring digital media content consumption and engagement, the Report states there is evidence that Canadians are significantly engaging with and consuming convergent digital media content funded through the Convergent Stream as well as interactive digital media content and software applications funded through the Experimental Stream. Further, the Report holds that the CMF has made progress towards its target of interactive digital media projects and software applications funded through the Experimental Stream being released to the market and there is evidence that such projects have been adopted by mainstream media stakeholders. On the other hand, however, the Department states that some projects (for which case studies were conducted) revealed a difficulty in measuring or meeting their usage targets. While in some examples 4
this was attributed to the difficulty with measuring audiences, in others, projects were said to have been particularly high-risk due to their highly experimental nature, including use of unproven technologies that did not deliver on their initial promise. 4. Ultimate Outcomes The Department concludes that the CMF has made significant strides towards achieving its ultimate expected outcome: high-quality Canadian convergent digital content is being created and consumed on multiple platforms in Canada and abroad, and Canadians are engaging with leading-edge interactive digital media products and software applications created with support from the CMF. The Report notes that Canadians are significantly consuming Canadian digital cultural content created with the support of the CMF: There has been an increase in the number of hours that Canadians are tuning in and watching CMF-supported programs, far exceeding the CMF s targets in both the English- and Frenchlanguage markets. Canadian digital content and applications are also being consumed abroad and integrated to the mainstream, as evidenced by international awards bestowed on funded productions and by the number of international markets in which sales of funded projects were made. 5. Unintended Impacts The Department acknowledges both the positive and negative unintended impacts as a result of the CMF. In terms of unintended positive impacts, the Report notes that the fusion of the Canadian Television Fund (CTF) and the Canada New Media Fund (CNMF) and the creation of the CMF has had a positive impact on the fostering of collaboration between the television and digital media sectors and encouraging the development of an independent video games sector. In terms of unintended negative impacts, the Department notes the following: due to the increasing consolidation in the broadcasting industry, decision-making has become concentrated in a small number of large corporate groups of broadcasters; while the use of the audience criterion of total hours tuned appears to reward repeats over original broadcasts, this issue has been mitigated in part by the CMF s Original First Run Factor; the Convergent Stream s requirement for a second platform may have resulted in projects of minimal added value being made; and there is tension between the objectives of commercialization and innovation in the Experimental Stream. Specifically, the requirement for innovation in projects funded through the Experimental Stream appears at times to be incompatible with the objective of having projects commercialized or used by Canadians. 5
D) Performance Efficiency and Economy 1. Demonstration of Efficiency and Economy The Department concludes that, overall, the evidence indicates that the CMF is administered in an economical fashion, with its administrative expense ratio ranging from 4.3 percent to 4.8 percent between 2010-2011 and 2013-2014 (and well within the maximum ratio of 6 percent required by the Contribution Agreement). Further, the popularity of CMF-funded programming shows that when given the choice, Canadians are interested in watching high quality Canadian programming. They value distinctive and high quality Canadian content that is culturally relevant, based on Canadian stories and issues. 2. The Support of Canadian Convergent Television and Digital Content DCH concludes that the CMF continues to be the best way for government to support Canadian television and digital content as it provides an effective framework that ensures both public policy and industry objectives are met. Further, no alternative program mechanisms or designs were discerned that would increase its effectiveness. 3. Duplication The Report concludes that CMF funding is complementary to other programs being offered in Canada. Further, the approach being taken in Canada which relies on access to television and digital media funding via multiple sources is consistent with approaches being taken internationally. 4. Governance The Report concludes that the right governance structure is in place to deliver the CMF program effectively, the CMF is well managed by DCH and the changes to the Board of Directors in the transition from the CTF to the CMF have been beneficial. Further, while some stakeholders suggested improvements that could be made to program delivery (e.g., file processing times, capacity of staff to resolve problems), the current administration model and delivery processes are effective. Finally, regular consultations conducted by the CMF as required by the Contribution Agreement are identified as a success with stakeholders. E) Other Evaluation Questions 1. The Performance Measurement Framework The Report notes that while the performance measurement implemented by the CMF captures considerable information on key performance indicators identified in the Performance Measurement, Evaluation and Risk Strategy particularly in regards to television audiences, limitations include a lack of access to measurement of audiences for original broadcasts (as opposed to repeat airings) and a lack of reliable data on international sales. 6
The Report also notes a lack of reliable data with respect to audiences for digital media content projects funded through the Convergent and Experimental Streams, as well as other commercial and innovation outcomes of projects funded through the Experimental Stream. Finally, DCH relays that the Digital Media Performance Measurement Framework developed by the CMF is said to be the first step in a critical need expressed by the industry, with the aim of adopting a standardized approach to measuring the performance of interactive digital media. 2. The Requirement for the Second Platform in the Convergent Stream The Report concludes that the requirements for second platform content have been successful in encouraging a greater volume of convergent content that is being consumed by Canadians. Further, given the evolution of the industry and changing consumption habits of Canadians, the Department holds that the exceptions made for convergent projects distributed through VOD or linear streaming in the Convergent Stream are appropriate. Finally, the Department notes that the CMF s heightened requirement that many projects include rich and substantial digital media content, may need to be revised. 3. Two Streams of Funding DCH concludes that the CMF s two streams of financing continue to be the optimal way of delivering the program. Further, the Department notes that there is evidence that supports a review on whether CMF funds should be available to web series through both the Convergent and Experimental Streams to ensure that this form of content is not being disadvantaged. 4. Underrepresented Genres DCH observes that there is evidence that supports the CMF s current role in funding the four underrepresented genres of programming as the unique financing challenges facing these genres (in both language-markets) are still present. 5. Leveling the Playing Field The Report concludes that the measures introduced by the CMF to level the playing field between broadcasters to ensure fair competition are effective. Further, the consolidation that has taken place in the broadcasting industry since the CMF s launch has required additional measures to achieve this goal. 6. Additional Funding Triggers The Department concludes that the rationale for extending funding triggers to unlicensed Canadian entities (that exclude a majority role for licensed Canadian broadcasters) is not sufficiently supported by the evidence at the present time. 7
DCH, however, does note that the CMF currently has the latitude to introduce pilot initiatives (e.g., English POV Program) which allow licence fees by foreign broadcasters to be considered eligible licence fees, on the condition that Canadian broadcasters provide the larger share of the eligible licence fee. 7. Minority French- and English-language Production The Report concludes that the CMF has met its requirements for the production of minority language programming (i.e., providing a minimum of 10 percent of French-language production support to minority-language projects each year) and the CMF plays an important role in strengthening the capacity of the minority language production sector. F) Recommendations The Report recommends that the Department undertakes to do the following: 1. Encourage the CMF to consider whether additional steps can be taken to reward audience success in light of consolidation in the industry since the CMF s launch. 2. Encourage the CMF to ensure that the industry has the necessary flexibility to determine which forms of value-added content are best suited to each individual project. 3. Work with the CMF to review its support for web series. 4. Encourage the CMF to clarify the objectives of the Experimental Stream as concerns commercialization and innovation. 5. Encourage the CMF to consider ways to improve Canada s television and digital content industry s competitive position on the global stage. 6. Undertake a review of the CMF s performance measurement, evaluation and risk strategy. 7. Encourage the CMF to continue its efforts to improve the measurement of its performance. G) Looking Forward The Report notes that the Canadian media ecosystem is experiencing rapid change that is difficult to predict but which could impact the CMF, including consumer adoption of online television and regulatory changes implemented by the CRTC. Possible consequences of these moves include (a) reduced viewing of CMF-funded programs, (b) a loss of resources as the result of lower subscribership and less money in the traditional system, (c) a greater strategic emphasis on foreign participation in programming made by Canadians, and (d) consolidation in the production sector. The Department concludes that in light of these possible changes, it may be prudent for the CMF and DCH to continue to monitor the environment closely if it is deemed necessary to adjust policies to ensure the continuance of the CMF s vital role. 8