Balanced Scorecard Approach for Benchmarking Rural ICT Projects K.M. Nazri, Tariq Zaman, M.O.Rosita. The contribution of Information Communication Technologies (ICTs) is significant in rural development especially to bridge the digital divide. However the ICT projects in rural communities are confronted with challenges of performance management, access to infrastructure, limited formal education, insufficient training and capacity building, financial and political constraints, and social and cultural challenges. To overcome these challenges the organizations focus on strategic planning and constant performance measurements. The Balanced Scorecard is a performance measurement system that supplements traditional financial measures from three additional perspectives: customer perspective, internal business perspective and innovation and learning perspective. In the paper we explore the modified Balanced Scorecard for rural ICT projects and discuss the monetary and non- monetary issues that affect the performance of ICT projects in rural communities. We also examine the strategic process of designing and implementing the project on the base of Balanced Scorecard approach. This paper will highlight some of the lessons learnt during our replication and deployment process of the project using the ebario Project as an initial guide. Keywords: Rural ICT Project, Balanced Scorecard, Performance Measurement Universiti Malaysia Sarawak (UNIMAS), Sarawak, Malaysia kmnazri@fit.unimas.my tariqzaman@lawyer.com morosita@fit.unimas.my -14-
Introduction ICT projects in rural communities are confronted with challenges of performance management, access to infrastructure, limited formal education, insufficient training and capacity building, financial and political constraints as well as social and cultural challenges. One way to handle these challenges is by focusing on strategic planning and performance management. The Balanced Scorecard is a useful tool for strategic planning and management where it has been used worldwide extensively in business and industry by government, profit and nonprofit organizations [1]. It is also has been used as part of performance evaluation and benchmarking tool for supply chain management [4,5]. Performance measurement The management of technical and technology development projects are becoming challenging to professionals in an increasingly competitive marketplace. Successful project implementers and stakeholders demand that their projects should be sustainable and achieve the goals and objectives of the stakeholders. The ICT projects in rural communities faces many problems pre and post project implementation i.e. lack of infrastructure, lack of skilled human resources, less opportunities for coordination with community, and lack of awareness in community about new technologies. Some of the motivations like community eagerness to encounter with new technology and active participation help the organizations to implement the projects successfully although measuring the success of the ICT projects for rural development is complicated and varies in terms of financial benefits, behavior and perception changes, building and strengthening the relationships, and service delivery. On one hand the financial benefits from ICT projects is a catalyst for community active participation and on the other hand community self-reliance and the eagerness to learn and seek new knowledge are the factors that contribute in making the project sustainable in future. So during and after the transfer of technology to rural communities, regular measurement of project s performance is extremely important. The absence of or inadequate performance measurement would result in failure of the project not only in terms of wasted time and development cost involved but also the wider negative effects on relationships with community, increasing frustrations, and potential loss of competitiveness. Performance measures are benchmarks for evaluating how ICT investments can be more efficient, effective, and obtain better results to support stakeholders mission, goals, and objectives. Measurement is vital in determining what a project is accomplishing and whether the results are being achieved. The objectives of the common performance measurement activities are; I. To Identifying and tracking progress against project goals II. Identifying opportunities for improvement III. Comparing performance against both internal and external standards -15-
Reviewing the performance of a project is also an important step when formulating the direction of the future strategic activities. The Balance Scorecard is a performance measurement tool that is unique in contrast to other methods. It links strategy with performance and goes beyond the traditional financial metrics in determining whether or not an organization has been successful in implementing and achieving the objectives. It can also measure the current performance of the project in comparison to the organization s vision and objectives. Balanced Scorecard Approach The Balanced Scorecard Approach provides a clear solution on what organizations should measure in order to balance the financial perspective. This approach also helps organizations to manage besides measure. It enables the clarification of vision and strategy and translates it into action. It provides feedback around both internal business processes and external outcomes in order to continuously improve strategic performance and results [2]. Robert Kaplan of Harvard University and David Norton, an American management consultant, has proposed the balanced scorecard as a means to evaluate corporate performance from four different perspectives. The approach suggest to view the organization by taking into account the four perspectives which is financial, customer, internal business processes and learning and growth (see Table 1). Next step is to develop metrics, collect data and analyze it relative to each of the perspectives. Perspective(s) Customer perspective (value-adding view) Financial perspective (shareholders view) Internal perspective (process-based view) Learning and growth perspective (future view) Mission To achieve our vision, by delivering value to our customers To succeed financially, by delivering value to our shareholders To satisfy our shareholders and customers by promoting efficiency and effectiveness in our business processes To achieve our vision, by delivering value to our customers Table 1: The four perspectives in a balanced scorecard Before building the balance scorecard for rural ICT projects, several steps need to be applied. There is work done by [3] on building balanced scorecard for information systems (IS) that measures and evaluates IS activities based on several perspectives such as business value, user orientation, internal process, and future readiness. The author suggested that a balanced IS scorecard can be the foundation for a strategic IS management system provided that certain development guidelines are followed, appropriate metrics are identified, and key implementation obstacles are overcome. A few steps in building IS balanced scorecard was recommended by [2,3] which can be followed: -16-
1. Create awareness for the concept of the balanced scorecard among top management and project team members 2. Collect and analyze data on the following items: Rural ICT project s strategy specific objectives and goals related to the strategy traditional metrics already in use for performance measurement; and potential metrics related to the four balanced scorecard perspectives; 3. Clearly define the specific objectives and goals of the project or functional area from each of the four perspectives; 4. Develop a preliminary balanced scorecard based on the defined objectives and goals of the enterprise and the approach outlined in this paper; 5. Receive comments and feedback on the balanced IS scorecard from management, and revise it accordingly; 6. Achieve a consensus on the balanced IS scorecard that will be used by the organization; and 7. Communicate both the scorecard and its underlying rationale to all stakeholders. Apart from that, there is also tool that addresses the need of information exchange within communities. The community information toolkit [6] developed by the Knight Commission and Monitor Institution in United Kingdom offers a tool that consists of four main parts which is exploring the role of information in addressing local needs and opportunities, ways of implementing information into the information ecosystem through the use of information community checklists, creating community information score card and finally planning for action. Information is regarded as an important asset in our projects. Therefore the third part is to be of our interest which is the creation of community information score card. In this part, information is analyzed and later translates into score card. The Scorecard addresses the three elements of a community information ecosystem: the supply of information, the infrastructure for delivering information and the skills needed to access that information and engage with the community. All these elements can be taken into account when designing our own balance score card especially when involving community information. Proposed Rural ICT Project Balanced Scorecard Perspective(s) Mission Objectives KPIs (qualitative & quantitative) Customer perspective (value-adding view) To provide reliable ICT access to rural areas Establish and maintain a good relationship with rural community and project s representative Exploit ICT opportunities Provide ICT training whenever needed Survey from participants of the projects (community representative) -17-
Perspective(s) Mission Objectives KPIs (qualitative & quantitative) Financial perspective (shareholders view) TTo succeed financially, by delivering value to our shareholders Ensure that ICT projects provide business value Control ICT costs Financial statements Year end reports Budget Internal perspective (process-based view) To deliver support and services in an efficient and effective manner Efficient in planning and setting up ICT telecentre Efficient in maintaining ICT telecentre Provide platform for rural community to give feedbacks Efficient in handling any problems that arises Number and list of trainings provided Maintenance plan Monthly report Meetings minutes Learning and growth perspective (future view) To look for ways to achievesustainable project through continuous improvement and preparation for future challenges Anticipate any future problem that may arises Continuously enhance skills through training, research and development Regularly upgrade and update software, hardware and related applications. Research emerging technologies that may be use for ICT development in the rural areas. List of trainings List of research conducted Table 2: Proposed Rural ICT Project balanced scorecard The main idea of Balanced Scorecard is to create an equal or balance between financial and non-financial, achieving and non-achieving and internal and external systems indicators. The core components for rural ICT project are the planning, implementation and evaluation. During the planning, the project needs to identify the key stakeholder. For rural ICT project, the stakeholders could be categorized into two main groups, the direct and the indirect stakeholders. The direct stakeholders are the project funder and the community or the project recipient. The indirect stakeholders are the external advisors, non-government organization, and the telecommunication providers. The stakeholder will identify the direction of the project and the need for the ICT project. Apart from setting up the plan for the project, the stakeholder also required to analyze and to find out what might be the possible need from the project. In another word, the key stakeholders will outline the key metrics or the key indicators. These metrics will cover the financial and non-financial perspectives. Nevertheless, the nature of the ICT project itself need to be clear to the community and the project funder. In this paper, the project funder will be referred to the Centre of Excellence (CoE) for Rural Informatics. The term CoE then will be used herewith in this paper. Figure 1 below indicates that the stakeholders need to come up with attainable metrics to achieve the common goal which is to establish a successful rural ICT project. -18-
Figure 1: Rural ICT Project s Metrics The first perspective of balance scorecard is the customer perspective. As for rural ICT, the mission is to ensure that the project will provide the reliable ICT access to rural areas. Before the ICT to be introduced further to the rural community, there is a need to establish and maintain good relationship with the community. This form of relation is to bridge the gap between the CoE and the project representative with the community. From there, the project at the start will indentify the resources available from the community. Human resources and raw material is the key to build the ICT centre later. Surveys are conducted to look at the demographic of the community. From the financial perspective, the main goal of the project is to have the community to lead or handle the project later. In the beginning, most of the cost will be provided by the CoE. CoE plays the major role in searching for grants for the rural ICT project. Grants obtained might be from government agencies or non-government agencies and International organisations. It also ranged from technological infrastructure to the socio-economic studies. The financial expenses mostly encompass the ICT equipment (VSAT, computers and the internet), the labor charges, the logistics, telecentre building materials, travelling and the training. Of course there are also other cost incur during the implementation process. Moreover, the requirement of financial report will do much help to know how much been spent or how much been earn by the ICT centre. The financial statement could be formed into two parts. The first part is when the project started to roll. In this stage, the authors have identified three main groups whom support the project namely, the CoE, community and other organization. Figure 2: Contributors of Rural ICT Project -19-
At the internal perspective (process-based view), the requirement is to deliver support and services in an efficient manner. Once the ICT centre been set up, there are need to have efficient in planning and maintaining the ICT telecentre. The three main groups as in Figure 2 will emphasis the smooth running of the centre. The centre will cater and provide various platforms of knowledge sharing for the community. Nevertheless, the community or the user of the centre need to provide their feedback on the ICT infrastructure provided. Some would agree that the technology part of the project might not have much of the problem. As the technology need to be installed based on the user manual correctly. The issues arise from the community or people that engage with the technology. CoE together with the community should emphasis on the information architecture. It is the process of keeping track on who is involved in the flow of the project and what that process looks like from beginning to end (an overall picture), and any information that needs to be acquired throughout the project (a process of documentation). This is a crucial step in ensuring that the technology solution is delivering business value and provides a roadmap for implementing the technology part of the project. Ensuring learning and growth perspective, continuous training and education is very important. CoE, Community and other related organization to the project have to achieve the metrics to make sure the return of investment and business value of the project. Persistency in gaining and interacting with the community whom are then the final user during the development process will ensure the project have correctly addressed what it need from the feedback mentioned earlier and the involvement from the beginning of the project. Survey from participants of the projects (community representative) To assess the community perception of telecentre project influences on their economic, social and cultural system a survey has been conducted where 15 community members participated and provided feedback. The basic aim of the survey was to explore the community perception that how the telecentre project and activities have affected the community life. Three main sources of influence have been derived from telecentre project i.e Telecentre Project Activities, Infrastructure, and Technology while one aspect of community system has been derived for each Balanced Scorecard corresponding perspective i.e. for Finance, Economy, for Learning and Growth, Knowledge and Education, for Stakeholders Participation, culture and for Internal system Community Collective Activities (Figure 3). Figure 3: Telecentre Project Influences on Community Systems -20-
Methodology The survey has been conducted on the base of structured questionnaire with Kg Buayan community (see Appendix 1 for the questionnaire). Questions were designed to assess the relative strengths of the project influences and corresponding community systems as indicated in Figure 3. The questionnaire format was adopted in order to simplify the responses for the community. The methodology to develop the questionnaire, and analysis adopted from the previous work of authors [8] and [7]. Fifteen respondents from different social and cultural background have been selected from Kg Buayan community. The respondents include farmers, religious leaders, tourist guides, and members of community council (JKK), students and women entrepreneurs. Our results (Table 2) are based on the responses of our subjects/respondents. An interpreter from local community, a seventeen year old girl, assisted the process. She speaks local language (Kadazan Dusun), Bahasa Malaysia and English. Results The respondent were asked to rate each source of influence on each aspect of community system on a scale from; 1 for little influence, 2 for some influence and 3 for a lot of influence. Each set of values was then compiled against each set of questions, producing the following scores in table. Project s Source of Influences Telecentre Project (Activities) Telecentre Infrastructure (Building) Telecentre Technology (Computers and Internet) Community Economic, Social and Cultural Systems Community Knowledge Activities Economy and Culture Total Ranking (Gotongroyong) Education 37 38 15 22 112 2 23 22 29 30 104 3 31 45 37 23 136 1 Total 91 105 81 75 352 Ranking 2 1 4 3 Table 2: Source of influence on each aspect of community system -21-
The study examined the influence of ebuayan Telecentre project on the community economic, social and cultural system. Generally, the picture that emerges indicates that technology influences are the strongest on the community system, while the most effected community system is the knowledge and education. The study examined the influence of ebuayan Telecentre project on the community economic, social and cultural system. Generally, the picture that emerges indicates that technology influences are the strongest on the community system, while the most effected community system is the knowledge and education. Conclusion The benchmarking and performance management of rural ICT projects are quite different from research and business organizations, where the measurement has always been seen as numbers and precisions. The Balanced Scorecard can give a qualitative analysis of rural ICT projects, which can help the researchers to realize the community based management of ICT project. In this paper we have presented the results of customer perspective based on our proposed Balanced Scorecard while future directions of this research include coming up with the tools for analysis of financial, internal perspective and learning and growth perspective. Acknowledgements (if any) The authors would like to thank ISITI-CoERI and the Faculty of Computer Science & Information Technology, Universiti Malaysia Sarawak in supporting this project. -22-
References [1]Balanced Scorecard Basics: Available at URL: http://www.balancedscorecard.org/bscresources/aboutthebalancedscorecard/ta bid/55/default.aspx. Accessed on 9th August. [2]Robert S. Kaplan and David P. Norton, Using the Balanced Scorecard as a Strategic Management System, Harvard Business Review (January-February 1996): 76. [3]M. Martinsons, et al., The balanced scorecard: a foundation for the strategic management of information systems. Decision Support Systems 25 (1) (1999) 71 88. [4]Xu Xiaoping and Li Chen, The Supply Chain Performance Evaluations Indicator System Based on Benchmark Balanced Scorecard, Proceedings in the 4th International Conference Wireless Communications, Networking and Mobile Computing, WiCOM '08 published in IEEE. [5]Bhagwat, R., & Sharma, M. K., Performance measurement of supply chain management: A balanced scorecard approach, Journal of Computers and Industrial Engineering, Volume 53 Issue 1, August, 2007. [6]Community Information Toolkit, Knight Commission on the Information Needs of Communities in a Democracy, Informing Communities: Sustaining Democracy in the Digital Age, Washington, D.C.: The Aspen Institute, October 2009. Copyright by John S. and James L. Knight Foundation and the Monitor Institute. All rights reserved. Website: http://www.infotoolkit.org, Accessed on 6th October. [7]Harris, C. A. & R. W. Harris () Information and Communication Technologies for Cultural Transmission among Indigenous Peoples. The Electronic Journal of Information Systems in Developing Countries, 45. [8]Zaman, T., N. Kulathuramaiyer & A. Y. Wee. 2010. Formulating Strategic Directions for Indigenous Knowledge Management System. In 2nd Semantic Technology And Knowledge Engineering Conference (STAKE 2010). Kuching, Malaysia: MIMOS BERHAD and University Malaysia Sarawak (UNIMAS). [9]Hao Xiaoling, Sun Qiang. The Information Performance Evaluation Framework, Implementation and Case Sttudies [M]. Tsinghua University Press. 2005,7. -23-
APPENDIX 1 Survey Questionnaire on Influences of Telecentre Project on Buayan community. 1. Nama: 2. Jantina: ο Male/ Lelaki ο Female/ Perempuan 3. Umur: ο 16-20 ο 21-30 ο 31-40 ο 41-50 ο 50-60 ο 60+ 4. Status Perkahwinan: ο Bujang ο Kahwin ο Bercerai ο Janda In the following table there is a list of 4 aspects of community economic, social and cultural systems corresponding to Balanced Scorecard 4 Perspectives (Finance, Learning and Growth, Stakeholders Participation and Internal system). There are also 3 sources of telecentre project influence. Please consider each source of influence on your community systems and score each one according to how influential they are; 1 for little influence, 2 for some influence and 3 for a lot of influence. For example, if you think Telecentre Project (Activities) has some influence on your knowledge and education; circle the 2 under Knowledge and Education against a. Telecentre Project (Activities) And if you think on Economy, Community Activities and Culture is not influential at all, then circle 1, as follows: Dalam jadual berikut, terdapat senarai 4 aspek iaitu komuniti ekonomi, sosial dan budaya masyarakat yang selaras dengan Balanced Scorecard 4 Perspektif (Kewangan, Pembelajaran dan Pertumbuhan, Penyertaan Pihak Berkepentingan dan Sistem Dalaman). Terdapat juga 3 sumber projek telecentre yang berpangaruh. Sila mempertimbangkan setiap sumber pengaruh ke atas sistem komuniti anda dan nyatakan markah mengikut tahap pengaruh mereka; '1 'bagi pengaruh yang sedikit, '2' untuk beberapa pengaruh dan '3 'bagi banyak pengaruh. Sebagai contoh, jika anda berpandapat Projek Telecentre (Aktiviti) mempunyai pengaruh ke atas pengetahuan dan pendidikan anda; bulatkan '2 'di bawah Ilmu dan Pendidikan terhadap Projek Telecentre (Aktiviti) dan jika anda berpendapat ' Ekonomi ',' Aktiviti Komuniti 'dan' Budaya 'tidak berpengaruh sama sekali, maka sila bulatkan '1', seperti berikut: a. Telecentre Project (Activities) Economy Community Economic, Social and Cultural Systems Knowledge and Education Culture Community Activities (Gotongroyong) 1 2 3 1 2 3 1 2 3 1 2 3 Now go ahead and enter your own scores in the table below: a. Telecentre Project (Activities) b. Telecentre Infrastructure (Building) c. Telecentre Technology (Computers and Internet) Community Economic, Social and Cultural Systems Economy Knowledge and Education -24- Culture Community Activities (Gotongroyong) 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3