SAINT KITTS AND NEVIS

Size: px

Start display at page:

Download "SAINT KITTS AND NEVIS"

Edwina Hampton
6 years ago
Views:

The World Bank Human Development Network -

Results (SABER) Public Disclosure Authorized

Authorized Public Disclosure Authorized 67663

Information System (EMIS) COUNTRY REPORT Emilio

1 The World Bank Human Development Network - Education System Assessment and ing for Education Results (SABER) Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized SAINT KITTS AND NEVIS Education Management Information System (EMIS) COUNTRY REPORT Emilio Porta, Jennifer Klein, Gustavo Arcia and Harriet Nannyonjo February 2012

2 Acknowledgements This report was prepared by a team led by Emilio Porta, Senior Education Specialist at the Human Development Network/Education at the World Bank; and consisting of Gustavo Arcia, Consultant to the Human Development Network/Education of the World Bank and Senior Economist at Analítica LLC in Miami, Florida; Jennifer Klein, Consultant to the Human Development Network/Education at the World Bank, and Harriet Nannyonjo, Senior Education Specialist, LCSHE, World Bank. The report was prepared under the guidance of Elizabeth King, Robin Horn and Chingboon Lee. The views expressed here are those of the authors and should not be attributed to the World Bank Group. All data contained in this report is the result of collaboration between the authors, the Organization of Eastern Caribbean States, and participants in the benchmarking exercise. All errors are our own. This benchmarking study arose from an active partnership between the Education Reform Unit of the Organization of Eastern Caribbean States () and the World Bank. The benchmarking exercise was done during an workshop conducted in Castries, St. Lucia, from January 23 to January 28, 2011, with the participation of government officials from Antigua & Barbuda, the Commonwealth of Dominica, Grenada, St. Kitts &, St. Lucia, and St. Vincent & the Grenadines. A delegate from Montserrat also attended as an observer. The workshop and benchmarking exercise were done under the invaluable leadership of Marcellus Albertin, Head of the Education Reform Unit (OERU) at the. His unflagging support, enthusiasm, and institutional supervision were fundamental for the cooperation of all participants and for the success of the workshop. To him we owe a great deal of gratitude. We would like to thank the OERU staff that helped us with workshop logistics, especially Emma Mc Farlane- Jouavel and Beverly Pierre. We would also like to thank the workshop participants: Doristeen Etinoff, Priscilla Nicholas, and Patricia George from Antigua & Barbuda; Ted Serrant, Robert Guiste, and Weeferly Jules from Dominica; Pauleen Finlay, Michelle Peters, and Imi Chitterman from Grenada; Gregory Julius from Monserrat; Quinton Morton, Ian Gregory, and Laurence Richards from ; Kendall Khodra, Nathalie Elliott, Sisera Simon, Evariste John, and Valerie Leon from St. Lucia; Dixton Findlay, Keith Thomas, and Junior Jack from St. Vicent & Grenadines; Darrel Montrope, Jacqueline Massiah, Sean Mathurin, and Loverly Anthony- Charles from the. Abbreviations EMIS MOE OECD SABER SEAT UIS UNESCO Education Management Information System Ministry of Education Organization for Economic Cooperation and Development Organization of Eastern Caribbean States System Assessment and ing for Education Results SABER EMIS Assessment Tool UNESCO Institute for Statistics United Nations Educational, Scientific, Cultural Organization

3 SAINT KITTS AND NEVIS ESTABLISHED Aspect of Data Quality Prerequisites of Quality Assurances of Integrity Methodological Soundness Accuracy and Reliability Serviceability Accessibility 1

4 BACKGROUND Education Data in St. Kitts and The Education Management Information System (EMIS) in St. Kitts and is part of the Education Planning Division of the Ministry of Education. It functions in collaboration with the Chief Education Officer and other education officers. Four key objectives of St. Kitts EMIS have been established: Ü Ü Ü Ü Collect and analyze educational data. Inform policymakers. Provide feedback to stakeholders. Monitor and support schools. FACILITIES AND EQUIPMENT. All schools are equipped with administrative computers and broadband Internet access, which allow for the internet-based exchange of information and -based communication. EMIS data are stored the EMIS office and are backed up both electronically and in hard copy. Occasional issues with the server have been noted as a challenge by EMIS staff. EMIS STAFF. St. Kitts EMIS staff includes an EMIS Director, secretary, and Education Officers. Also, each school selects an Information and Communication Technology (ICT) Champion, who is the link between the EMIS staff and the school. EMIS DATA. Data on students and teachers are collected from schools and other educational institutions, and additional teacher data is collected from the Human Resources Unit. The Ministry of Education s Project Management Unit and schools supply infrastructure data, and budget data is collected from schools and the Statistics Unit. DATA COLLECTION. Data collection begins in September with the submission of questionnaires to the 24 public primary schools, nine private primary schools, eight public secondary schools, and three private secondary schools. The annual questionnaires are collected by October 31, but additional student and teacher attendance forms are submitted each term. DATA PROCESSING. The Ministry of Education EMIS department aggregates the source data received from various sources. The Statistical Officer verifies accuracy and consistency of the data, and errors are communicated to the school/institution. The Statistical Officer also designs the output tables. PUBLICATIONS. At the end of the school year in August, St. Kitts strives to publish a statistical bulletin that presents EMIS data and the financial expenditure of the Ministry of Education. Also, the EMIS provides annual statistics upon request to regional and international affiliates including the Educational Research Unit (OERU) and UNESCO. 2

The EMIS in ESTABLISHED: In January 2011, St. Kitts and EMIS was assessed using the SABER-EMIS Assessment Tool (SEAT) and overall the EMIS was categorized as ESTABLISHED (0.65).

5 The EMIS in ESTABLISHED: In January 2011, St. Kitts and EMIS was assessed using the SABER-EMIS Assessment Tool (SEAT) and overall the EMIS was categorized as ESTABLISHED (0.65). Among the six Organisation of Eastern Caribbean States () countries, St. Kitts tied with Dominica for the highest overall score (Table 1). St. Kitts outperformed the average on most of the SEAT s Aspects of Quality except two Methodological Soundness and Assurances of Integrity. St. Kitts Methodological Soundness score was only slightly below average due to one EMERGING score on the scope of statistics, but St. Kitts had many EMERGING sub-component scores on the Assurances of Integrity that resulted in the lowest average score of all the countries. St. Kitts was the highest of the countries on two aspects Accuracy & Reliability and Serviceability but neither of the two Aspects had MATURE average scores. Figure 1. SABER EMIS Scores in the Countries The next sections of this country report will analyze St. Kitts performance on the sub-components of each Aspect of Quality in order to present a detailed portrait of the strengths and weaknesses of St. Kitts EMIS and many concrete actions that the country can take to improve education data quality. Table 1. SABER EMIS Scores in the Countries (2011) Dominica Antigua Grenada St. Kitts St. Vincent St. Lucia Pre-Requisites of Quality Assurances of Integrity Methodological Soundness Accuracy and Reliability Serviceability Accessibility Overall Latent

6 PREREQUISITES OF QUALITY ESTABLISHED: St. Kitts has ESTABLISHED (0.66) the Prerequisites of Quality (Figure 2) necessary to support an EMIS and was above the average score, but still has a few areas in need of improvement. Processes are in place to ensure that human, physical, and data resources are efficiently managed (Table 2, 0.6) but current EMIS staffing and training levels are inadequate (0.5). Quality procedures are in place and enforced by management (0.8), but there are no reviews by external bodies or user surveys to monitor the quality of data (0.9). Figure 2. Prerequisites of Quality ESTABLISHED The legal framework specifying responsibilities for collecting and disseminating data are not specific to the education sector (0.1). An informal agreement exists for data sharing and coordination among most educational levels and institutions, but it is not fully implemented; there is no agreement for data sharing or coordination with private universities (0.2). Statistical reporting is encouraged through an informal agreement, but there are no formal legal mandates or penalties for institutions that do not report data (0.4). St. Kitts could improve the Prerequisites of Quality by formalizing informal agreements, establishing the formal legal framework for the EMIS, and hiring and training additional EMIS staff. Table 2. Prerequisites of Quality: Subcomponents 0.1 Responsibility for collecting and disseminating education data is clearly specified 0.2 Data sharing and coordination among different agencies are adequate Individual/personal data are kept confidential and used for statistical purposes only Statistical reporting is ensured through legal mandate and/or measures to encourage response Staff, facilities, computing resources, and financing are commensurate with the activities Processes and procedures are in place to ensure that resources are used efficiently Education statistics meet user needs and those needs are monitored continuously Processes are in place to focus on quality 0.9 Processes are in place to monitor the quality of data processes Processes are in place to deal with quality considerations in planning the stat program Mechanisms exist for addressing new and emerging data requirements

ASSURANCES OF INTEGRITY EMERGING: Assurances of Integrity was St. Kitts only EMERGING average score (0.44) and the only instance where St. Kitts scored drastically below the average (0.55).

7 ASSURANCES OF INTEGRITY EMERGING: Assurances of Integrity was St. Kitts only EMERGING average score (0.44) and the only instance where St. Kitts scored drastically below the average (0.55). A closer look at the sub-components of this Aspect reveals a wide range of scores from LATENT to MATURE. Only informal mechanisms protect the professional independence of the data producing institution, which can ensure that statistics are produced on an impartial basis (Table 3, 1.1). More formal arrangements could be developed to improve the score on this sub-component. Figure 3. Assurances of Integrity in the EMERGING Choices of data sources are technically justified and staff are encouraged to enforce technical criteria (1.3). The terms and conditions under which statistics are collected, processed, and disseminated are difficult to find but made available on request (1.5). No notice is given on changes in methodology, source data, or statistical techniques (1.8). St. Kitts earned its only MATURE score by having wellknown professional guidelines for staff behavior (1.1). Even though professional credentials are already considered for recruitment and promotion (1.2), professionalism could be furthered by encouraging staff to publish and having a peer review process in place. Table 3. Assurances of Integrity: Subcomponents 1.1 Statistics are produced on an impartial basis Professionalism of staff is actively promoted Choices of data sources and statistical techniques are made solely by statistical considerations Agency is entitled to comment on erroneous interpretation and misuse of statistics 1.5 Terms and conditions are available to the public Public is aware of internal governmental access to statistics prior to their release 1.7 Products of education statics agency are clearly identified Advanced notice is given of major changes in methodology, source data, and statistical techniques Guidelines for staff behavior are in place and are well known to the staff 0.00 Latent

METHODOLOGICAL SOUNDNESS ESTABLISHED: In terms of Methodological Soundness, St. Kitts s EMIS is ESTABLISHED (0.67). St. Kitts scored just below the average (0.

8 METHODOLOGICAL SOUNDNESS ESTABLISHED: In terms of Methodological Soundness, St. Kitts s EMIS is ESTABLISHED (0.67). St. Kitts scored just below the average (0.69) and had the same score as both Grenada and St. Lucia (Figure 4). St. Kitts earned one MATURE score on this Aspect of Quality due to its use of the internationally accepted standards and guidelines for structure, concepts and definitions established by the UNESCO Institute for Statistics (UIS) and the Education Reform Unit (OERU) (Table 4, 2.1). St. Kitts also follows the International Standard Classification of Education (ISCED) in all education sector data except expenditure data (2.3). Expanding the use of ISCED to expenditure data would ensure complete consistency with ISCED and improve St. Kitts s score on this subcomponent. Figure 4. Methodological Soundness in the countries ESTABLISHED Currently, St. Kitts s EMIS produces around 69 percent of UIS indicators annually, which results in a EMERGING benchmark on the scope of statistics sub-component (2.2). Expanding the scope of statistics produced to 100 percent of UIS and OECD indicators is ideal and can enable additional domestic, regional, and international education policy analysis. Table 4. Methodological Soundness: Subcomponents 2.1 Overall structure, concepts and definitions follow regionally and internationally accepted standards, guidelines, and good practices Scope is in accordance with international standards, guidelines, or good practices Classification systems are consistent with international standards, guidelines, or good practices

ACCURACY AND RELIABILITY ESTABLISHED: The Accuracy and Reliability of St. Kitts s EMIS statistics is ESTABLISHED () (Figure 5). St. Kitts score was the highest of the countries, and on four of 10 sub-components, St.

9 ACCURACY AND RELIABILITY ESTABLISHED: The Accuracy and Reliability of St. Kitts s EMIS statistics is ESTABLISHED () (Figure 5). St. Kitts score was the highest of the countries, and on four of 10 sub-components, St. Kitts s EMIS was MATURE (Table 5). Data compilation in St. Kitts employs sounds statistical techniques including appropriate sample sizes for survey data and updated school registries for census data (3.5). Statistical discrepancies in intermediate data are always assessed and investigated (3.8). There are systematic processes in place for monitoring errors and omissions and the results are made public (3.9). Also studies and analyses of revisions are carried out routinely. A feedback loop is implemented based on the studies to inform the process and the findings are made public (3.10). Figure 5. Accuracy and Reliability ESTABLISHED In order to improve, St. Kitts could 1) collect data on school characteristics and education demand (3.1), 2) develop procedures to update, standardize, and properly reference source data (3.2), 3) document information on sampling errors and imputed data (3.4); 4) always validate intermediate results against other information (3.7) and finally 5) ensure that education data are provided within six months after the end of the school year to other source providers (3.3). Table 5. Accuracy and Reliability: Subcomponents Source data are obtained from comprehensive data collection that takes into account country-specific conditions Data are reasonably confined to the definitions, scope, classifications, and time of recording required 3.3 Source data are timely (6 months after event) Other data sources, such as censuses, surveys, and administrative records, are routinely assessed Data compilation employs sound statistical techniques to deal with data sources Other statistical procedures (data editing, transformations, and analysis) employ sound statistical techniques Intermediate results are validated against other information where applicable Statistical discrepancies in intermediate data are assessed and investigated Statistical discrepancies and other potential indicators or problems in statistical outputs are investigated Studies and analyses of revisions are carried out routinely and used internally to inform the processes

10 SERVICEABILITY ESTABLISHED: The Serviceability of St. Kitts s EMIS data is ESTABLISHED (0.79) and is far above the average (0.55). St. Kitts s lowest score on any subcomponent was, which indicates that a strong foundation for Serviceability is currently in place. St. Kitts met the MATURE benchmark for Periodicity by producing an annual census of enrolments, teachers, schools, and financial data (Table 6, 4.1), but the timeliness of releasing the data could be improved: Currently administrative census data are available six to 12 months after the initiation of the school year when ideally this data should be released within two months (4.2). Time series data are available for five to 10 years, and there are procedures in place for the revision of time series data (4.4). Crosschecking is done regularly but consistency checking is done only for administrative census data (4.3). When schoolreported data was checked for consistency against household survey data, the difference between the two sets of data was less than five percentage points, thus earning St. Kitts a MATURE score on subcomponent 4.5. Figure 6. Serviceability in the ESTABLISHED St. Kitts could improve its Serviceability by publishing administrative data within two months after the initiation of the school year, increasing the availability of time series data to more than 10 years, and strengthening systems for revisions. Table 6. Serviceability: Subcomponents 4.1 Periodicity follows dissemination standards 4.2 Timeliness follows international dissemination standards 4.3 Statistics are consistent within the dataset Statistics are consistent or reconcilable over a reasonable period of time Statistics are consistent or reconcilable with those obtained through other data sources and/or statistical frameworks 4.6 Revisions follow a regular and transparent schedule 4.7 Preliminary and/or revised data are clearly identified

11 ACCESSIBILITY ESTABLISHED: St. Kitts earned the second highest Accessibility score of the countries (0.61/ESTABLISHED), but St. Kitts sub-component scores ranged from LATENT to MATURE. St. Kitts had three MATURE scores: 1) EMIS statistics are clearly presented with disaggregation and underlying data for charts (Table 7, 5.1); 2) data are released to all users at the same time (5.4) and 3) a data catalog is available so users can request data according to their needs (5.7). Figure 7. Accessibility in the ESTABLISHED Data are available electronically (5.1), but data are not released on a pre-announced schedule (5.3) and yearbooks are only printed upon request (5.2). Metadata are documented and updated, but only available upon request (5.6). The release of non-published data and non-confidential data is discretionary (5.5) and no catalogs of publications and other services are available (5.9). Most statistical releases identify a contact person in case of required assistance and assistance is monitored (5.8). Accessibility is one of the key missions of an EMIS because it creates and maintains the public image of the EMIS and enables greater accountability. St. Kitts could develop a more accessible EMIS by establishing and pre-announcing the schedule for data releases, offering improved assistance for users, and creating a catalog of publications and other services. Table 7. Accessibility: Subcomponents 5.1 Statistics are presented to facilitate proper interpretation and comparisons (layout, clarity of texts, tables, and charts) 5.2 Dissemination media and format are adequate 5.3 Statistics are released on a pre-announced schedule Statistics are made available to all users at the same time 5.5 Statistics not routinely disseminated are made available upon request 5.6 Documentation on concepts, scope, classifications, basis of recording, data sources, and statistical techniques is available, and differences from internationally accepted standards, guidelines, or good practices are annotated 5.7 Levels of detail are adapted to the needs of the intended users 5.8 Contact points for each subject field are publicized Catalogs of publications and other services, including information on any charges, are widely available 0.00 Latent

The World Bank Human Development Network - Education System Assessment and Benchmarking for Education Results (SABER)

The World Bank Human Development Network - Education System Assessment and Benchmarking for Education Results (SABER) Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized