conférences dans le 8e Semestre Ecole Politecnique Benchlearning Final Conference Measuring egovernment Impact Pr. Jean Pierre Noel Professeur Jean Pierre Noel [Projectmanagement]8. Term Facility Management
INDEX THE PROJECT BENCHLEARNING THE PILOTS BENCHLEARNING COMMUNITY Professeur Jean Pierre Noel [Projectmanagement]8. Term Facility Management 2
Professeur Jean Pierre Noel [Projectmanagement]8. Term Facility Management 3
The project Rooted in egep s findings egovernment Economics Project, Benchlearning is a mean to test the comparability of impact indicators, to build measurement awareness and capacities, to share good practices. On a voluntary and flexible basis, 12 public agencies covering 10 European countries have freely committed themselves to join 3 Pilot egovernment Benchlearning exercises on a 2-yearly time span. Through a systematic data gathering, the Agencies will prove whether egovernment services and applications are finally delivering the expected outcomes. Professeur Jean Pierre Noel [Projectmanagement]8. Term Facility Management 4
Back to 2005: the egovernment Economics Project (egep) Expenditure Study egov costs monitoring methodology Expenditure estimate for EU25 Total ICT: 36.5 billion (2004) egov only: 11.9 billion (2004) Measurement Framework Measurement Framework About 90 indicators Implementation methodology egovernment Productivity GDP Growth Economic Model Scenarios show that future egovernment research and programmes (2005-2010) could boost EU25 GDP by up to 1.54 percent Professeur Jean Pierre Noel [Projectmanagement]8. Term Facility Management 5
The egep measurement framework Efficiency Cashable financial gains Better empowered employees Financial & organisational Value Democracy Better organisational and IT architectures Inter-institutional cooperation Openness and participation Transparency and accountability Political Value Net Costs Set-up Provision Maintenance Reduced admin. burden Effectiveness Increased user value & satisfaction Constituency Value More inclusive public services Professeur Jean Pierre Noel [Projectmanagement]8. Term Facility Management 6
How could it be used? EU25 Benchmarking Very simple fully comparable indicators EU must agree with Member States indicators Methodology for new benchmarking egep National level monitoring of egov Less simple indicators, some comparability problems A national level unit can Impose indicators top down Build consensus on most comparable ones Micro-level business case & measurement Sophisticated indicators, no comparability problems A public agency can: Select any of egep indicators First use them for ex ante business cases Then for steady and continuous measurement Use egep implementation tools Professeur Jean Pierre Noel [Projectmanagement]8. Term Facility Management 7
Lessons from egep: impact measurement difficulties Level International (EU25) Type Policy system benchmark Difficulties score Cooperation Comparability Feasibility 4 4 4 Overall score HIGH Member State (holistic) Public policy benchmark Cooperation Comparability Feasibility 3 3 3 MEDIUM-HIGH Member State (within vertical and/ or region) Individual public agency (voluntary) Organisational benchmark Measurement/ Internal benchmark Cooperation Comparability Feasibility Cooperation Comparability Feasibility 2 1 3 1 0 2 MEDIUM LOW 4=High; 3=Medium-high; 2=Medium; 1=Low; 0= null Professeur Jean Pierre Noel [Projectmanagement]8. Term Facility Management 8
2008-2009: the Benchlearning challenge A bottom-up collaborative benchmarking based on a peerto-peer experimental exchange among fairly comparable public agencies from at least two different EU Member States, designed as a symmetric learning process, that ( ) will implement and calculate more sophisticated indicators in a chosen area of impact the ICT enabled services the selected agencies provide and in the process will build transformative capacities. Professeur Jean Pierre Noel [Projectmanagement]8. Term Facility Management 9
Why to benchlearn? To benchmark only some egep indicators: The simplest and more comparable. To boost the public sector s impact evaluation capabilities: Focus on most sophisticated impact indicators; Measurement capacities are built bottom-up. To provide the involved agencies with tangible benefits: Opportunity to look at processes complexity; Identify enabling and hindering factors. Professeur Jean Pierre Noel [Projectmanagement]8. Term Facility Management 10
Benchlearning = bottom-up benchmarking Benchlearning is: Voluntary, bottom up and learning oriented; Flexible, with no need of uniform rigid indicators. Gradually scalable from micro to meso and macro: Groups of similar organisations; Groups of similar verticals / regions; Groups of similar countries. Provides insights and learning on the egov value chain: Key drivers and success factors; Main barriers; Organisational processes and input. Professeur Jean Pierre Noel [Projectmanagement]8. Term Facility Management 11
What benchlearning is aiming to achieve ACTIVITY Analysis of eservice set-up and delivery processes AIM To understand the success factors and barriers behind processes Elaboration of new impact indicators To collaboratively test its feasibility and comparability Peer-to-peer exchange of experiences To build awareness on good practices Transfer knowledge on measurement tools To enable PAs measure their own performance To extrapolate the promising areas where EU can become a global leader Professeur Jean Pierre Noel [Projectmanagement]8. Term Facility Management 12
Benchlearning vs. benchmarkig TEST AND LEARNING BEST IN CLASS Ag.1 Ag.8 vs Ag.1 Ag.2 Ag.3 Ag.4 Ag.5 Ag.6 Ag.7 Ag.8 Ag.9 Ag.13 Ag.11 Ag.12 Ag.13 Professeur Jean Pierre Noel [Projectmanagement]8. Term Facility Management 13
Benchlearning vs. benchmarking: outcomes CAPACITIES AND LESSONS RANKING Ag.1 Ag.8 vs Ag.13 Ag.8 Ag.1 2 1 3 Ag.13 Professeur Jean Pierre Noel [Projectmanagement]8. Term Facility Management 14
How benchlearning works (1/2) First year measurement Set-up: Letter of intent from the participating agencies; Running of a kick-off meeting with all participating agency. As is and mainstreaming: Review of existing measurement systems and data; Analysis of organisational strategy and context; Draft report on indicators and preliminary measurement. First full measurement (or zero measurement): Data gathering instruction to agencies; Remote support to agencies to gather the data; Validation of data and calculation of indicators. Professeur Jean Pierre Noel [Projectmanagement]8. Term Facility Management 15
How benchlearning works (2/2) Set-up (same as Y1); Second year measurement As is and mainstreaming (same as Y1); Second full measurement (same as Y1). Continuous exchange of information Exchange activities (www.epractice.eu/community/benchlearning); Inter-agency workshops. Sustainability actions Provision to the agencies of a measurement organisational model (processes and roles); Final recommendations and final report. By end of Dec 2009 Professeur Jean Pierre Noel [Projectmanagement]8. Term Facility Management 16
Expected project outcomes: egep 2.0 Pilot 1 Pilot 2 Pilot 3 EFFICIENCY ADMINISTRATIVE BURDEN REDUCTION CITIZEN CENTRICITY Public sector information indicator Efficiency indicator Standard cost model based indicator Work in progress on number of data field indicator Plurality of subjective and objective metrics Work in progress on a combined index Simplified version of the egep Measurement Framework: egep 2.0 Professeur Jean Pierre Noel [Projectmanagement]8. Term Facility Management 17
Pilot 1 Efficiency gains About the Pilot Pilot measurement of efficiency gains of cadastral eservices: Development of a set of indicators to measure the efficiency gains and savings due to the delivery of cadastral eservices; Definition of the indicators in line with the agencies requests to assess the added value of online cadastral information supply in terms of internal and social impact; Data gathering and analysis (volumes, eservice costs, organisation and customer satisfaction programs); Shared experiences among agencies. Agencies involved: Agenzia del Territorio (Italian National Cadastre) Oficina Virtual del Cadastro (Cadastre Virtual Office of Spain) Lantmäteriet (National Land Survey of Sweden) Observers: Regional agencies (Emilia-Romagna, Catalonia) Professeur Jean Pierre Noel [Projectmanagement]8. Term Facility Management 18
Pilot 2 Administrative burden About the Pilot Piloting of indicators in the Administrative Burden Reduction Field Proof-of-concept service: Business registration Data sources: Standard Cost Model Focus groups and interviews Methodology: Specific focus on users (both business and civil servants) Qualitative and quantitative approach to burden measurement Agencies involved: Pilot agencies G2B: Belgium: FPS Economy Slovenia: Ministry of Public Administration Greece: Ministry of Public Administration Project Observers: Fedict, Dutch Ministry of Interior, Greek Information Society Observatory Professeur Jean Pierre Noel [Projectmanagement]8. Term Facility Management 19
Pilot 3 User centric impact About the Pilot Pilot measurement of user centric impact of national government portals: Understand what the EC has already done on measuring e-government and specific indicators developed for user-centric impact and benchmarking national portals Development of a measurement framework that can be used to benchmark the user-centricity of national government portals. The framework measures 5 key aspects: content richness, service sophistication, user choice and control, quality control and design for usability Shared experiences among agencies. Agencies involved: DirectGov (British egovernment portal) Service-public.fr (French egovernment portal) Mojauprava (Croatian egovernment portal) Additional egovernment portal data collected from Italy and Hungary Professeur Jean Pierre Noel [Projectmanagement]8. Term Facility Management 20
Benchlearning groups: how to manage them Start simple Generate ownership Micro level only, single public organisations: 3-4 learning organisations managing homogeneous services Voluntary participation: Participants self-interested in capacity building/learning. Clear mandate and leadership buy-in: Groups assembled from similar countries: Leverage existing collaboration networks. Groups to be assembled not by facilitator. Multi-stakeholders but firm governance: Third party facilitators (EU contractors, governments ): Intense and in-depth work. Exchange and consensus But with clear lines of accountability. Professeur Jean Pierre Noel [Projectmanagement]8. Term Facility Management 21
Benchlearning community within epractice.eu Sharing experience and cases Having a say in the blog session Suggesting and attending new events Exchanging ideas on eservice impact measurement and evaluation Recommending documents for the Community library Professeur Jean Pierre Noel [Projectmanagement]8. Term Facility Management 22
What results will be presented tomorrow What kind of indicators we selected for the measurement of Efficiency Gains, Administrative Burden Reduction and User Centricity; How we managed the comparability issues due to the structural differences of public services in the different countries; How we took into account the organisational process and the eservice costs for the service provision in each country; What have been the results of terms of learning; How we quantified the benefits from egovernment; ICT-enabling of processes leads to significant savings both for businesses and internally within governments. The results obtained in the pilots illustrate the business cases for better and more convenient egovernment solutions. Professeur Jean Pierre Noel [Projectmanagement]8. Term Facility Management 23