Decentralized Basic Education 1: Management and Governance
The DBE1 Service Provider Program; an Evaluation
March 2011 This report is one of a series of special reports produced by Research Triangle Institute (RTI), Implementing Partner for the USAID-funded Improved Quality of Decentralized Basic Education (IQDBE) program in Indonesia
The DBE1 Service Provider Program; an Evaluation Contract 497-M-00-05-00029-00
Prepared for USAID/Indonesia Prepared by RTI International 3040 Cornwallis Road Post Office Box 12194 Research Triangle Park, NC 27709-2194
The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government.
Table of Contents
Contents Table of Contents ........................................................................................... ii Introduction .................................................................................................... 1 Program overview .......................................................................................... 2 Program evaluation; the methodology ............................................................ 3 The UPI school level program, West Java ..................................................... 4 The Sampoerna Foundation school-level program ........................................ 8 The UPI district program, West Java ............................................................ 11 The UMS district program, Central Java ...................................................... 14 The UNM district program, South Sulawesi.................................................. 18 Lessons Learned .......................................................................................... 21 Recommendations ....................................................................................... 24 Appendix 1: Survey Instrument – Service Provider Evaluation, March 2011 .............................................................................................. 27 Appendix 2: Report on evaluation of Service Provider Program, UPI, December 2009 ............................................................................. 30 Appendix 3: Abbreviations, Acronyms and Glossary ................................... 35
Introduction The key strategy for both dissemination and sustainability in DBE1 is to develop facilitators and service providers who can take over the program, using the manuals and training modules developed under the project, and implement it in districts - without further project support. The main agent for dissemination and sustainability of school level programs is the large group of trained ‘district facilitators’. Meanwhile for district level interventions, and to some extent also for school level programs, DBE1 has been working with a group of universities and one NGO. For the purposes of DBE1, there are three types of ‘service provider’: 1. institutions which are independent of the schools and education systems to which they provide a service, consisting of training and consultancy; these are universities and potentially NGOs and independent consulting agencies; 2. institutions from within the education system which provide a similar service; these potentially include government agencies such as LPMP and, for example, the West Java provincial education office which may provide a service facilitating implementation of the asset management system in districts; and 3. ‘District facilitators’; who are individuals trained by DBE1 to facilitate school level programs; the majority of these are school supervisors (pengawas) working under district education offices. This report focuses on the first type. It does not consider the role of ‘district facilitators’ in disseminating DBE1’s school level programs. This program has been evaluated in depth in a previous report.1 Nor does it consider the role of government agencies as service providers as it is too soon to do so. One other type of agency which has become active in disseminating DBE1 methodologies is the implementation teams for other donor-funded projects, including the World Bank’s BEC-TF and AusAID’s SEDIA, which have begun disseminating district level interventions. These actors, although very significant, are outside the scope of the current evaluation. The purposes of this evaluation report are: 1. to assess the effectiveness of the service provider program, 2. to assess the interest and commitment of each service provider organization in further developing the program in 2011, and 3. to identify lessons learnt in order to improve future programs.
1
DBE1, July 2010, Implementing School-Based Management in Indonesia, The DBE1 Experience: 2005 – 2010: Impact Study
The DBE1 Service Provider Program; an Evaluation
1
Program overview In 2009, DBE1 ran a pilot program to develop the Indonesian University of Education (UPI) in Bandung as service provider. Following this pilot, in late 2009 two more universities were selected to be developed as service providers for district-level activity, making a total of three for the 2010 program: in West Java the Indonesian University of Education, or Universitas Pendidikan Indonesia (UPI), in Central Java Universitas Muhammadiyah Surakarta (UMS), and in South Sulawesi Universitas Negeri Makassar (UNM). The NGO, PATTIRO, 2 was also selected but was later dropped from the program as described below. Each of these institutions provided personnel to work with DBE1 specialists in implementing core methodologies in one newly identified district in each of the three provinces. Table 1, below, shows the number of personnel trained in each program. It should be noted that although these persons have received several days of classroom and on-the-job training, they have yet to be certified. Criteria for certification will be developed in the first half of 2011. Table 1: District Level Service Provider Personnel Trained 2010 Program
Number trained
BOSP
7
AKPK
7
SIPPK
7
Renstra
7
At the same time in 2010 DBE1 worked with the Sampoerna Foundation’s School of Education (SSE) to develop their capacity as a service provider for DBE1’s school level programs. The original stated objectives of the service provider program are as follows: 1. To reach agreements with institutions of higher education, NGOs or consulting organizations to authorize institutional associates to participate in the Service Providers training program 2. To train and certify a cadre of professionals to provide services to assist and improve capacities of local governments in education finance analysis and education development planning and policy development 3. To link DBE1 trained Service Providers to potential clients such as districts or private school networks.
2
Pusat Telaah dan Informasi Regional
2
The DBE1 Service Provider Program; an Evaluation
The Sampoerna Foundation program was evaluated in workshops held at the SSE campus in Jakarta in August 2010 and March 2011. Also in March 2011 the university partnership program was evaluated in a series of joint workshops held with each of the three partner universities. This report is a result of that evaluation. In addition, the report takes into account information from a number of secondary sources, including: 1. DBE1 progress reports, including annual and quarterly reports; 2. internal reports on (1) monitoring and evaluation of the school level program conducted by DBE1 and UPI in West Java in December 2009, (2) a meeting between DBE1 and UPI in February 2011, and (3) a report from the coordinator of the program in UPI.
Program evaluation; the methodology This evaluation considers both the perspectives of DBE1 and the perspectives of the service providers. In order to gain the DBE1 perspective, reports from specialists and advisors were considered along with a report of monitoring and evaluation conducting in West Java in 2009. In order to obtain the service provider perspective, a series of one-day workshops was conducted in March 2011 with each of the three partner universities. A report from the director of UPI’s research body (LPPM) was also considered. The Sampoerna Foundation program was evaluated in workshops conducted in August 2010 and March 2011. The methodology employed: (1) a survey instrument (see Appendix 1) to elicit the responses of individuals (participants and program supervisors) from each institution, and (2) a focus-group discussion approach to further explore responses to the questions. (In the case of the Sampoerna Foundation evaluation only the second method was used.) The following questions formed the basis of the individual survey instrument and focus group discussion questions. Individual Perspective 1. Why did you decide to join the DBE1 Service Provider program? 2. Did the program meet your expectations? Did you get what you were hoping for? 3. What did you learn? 4. What do you need to learn more of in order to be an effective 'service provider'? Organizational Perspective 5. Did this program meet the needs of your organization / university? 6. Is the idea of being a 'service provider' in line with the vision and mission of your organization? 7. Do you think there is a market for this service? Who? Where?
The DBE1 Service Provider Program; an Evaluation
3
8. How your organization can better meet the demands / needs of the market? 9. How could DBE1 help you to achieve these objectives? 10. Has the DBE1 program provided you with content that assists you in your core work (teaching, research etc.)? General 11. Are you interested to join the program in 2011 Service Provider (if any)? 12. How could the program be improved? Analysis of the individual and group survey responses, together with the other sources mentioned is summarized below. The two programs which aimed to prepare service providers for school-level interventions are discussed first, followed by the three programs with service providers for district level interventions. The report then concludes with a summary of lessons learned and recommendations.
The UPI school level program, West Java The program In 2009 DBE1 formed a partnership with service provider UPI to disseminate school level programs. A series of training events was delivered to prepare university lecturers to train and mentor final-year education students during their obligatory community service program (KKN) who, in turn, trained and mentored schools in the basic components of school based management: school committee strengthening, SDS, school development planning. A team of ten senior advisors and specialists from DBE1 provided the intensive training to 24 UPI personnel over nine days, in three sessions, in May-June 2009. The training commenced with school visits and concluded in the third session with a focus on district level interventions. Also participating were representatives of the Provincial Education Office, Provincial MORA office and provincial LPMP. Keynote addresses were given by senior personnel from MONE’s Secretariat for School-Based Management under the Directorate for Kindergarten and Elementary Schooling, the Head of the Provincial Education Office (Kepala Dinas Pendidikan Provinsi Jawa Barat), Assistant Rector and senior academic staff of UPI, the Head of the District Education Office from Sukabumi and Karawang districts and USAID. Subsequently, in collaboration with the West Java Provincial Education Office and UPI, leadership training was provided for 100 elementary school heads from target schools in Kabupaten Bandung Barat and Kota Cimahi to prepare them for the program ahead of the KKN student placements. The workshop took place at UPI campus in Bandung in July and was provided by the staff who joined the TOT program. The training for 1,000 KKN students was also
4
The DBE1 Service Provider Program; an Evaluation
provided in July. The KKN was then conducted over one month from July 24 to August 29, 2009. Working in teams of 10 or so, the students assisted the schools over a one month period; 37 in Cimahi and 63 in West Bandung District, to produce school plans (RKS) and in some cases introduced the DBE1 School Database System (SDS). The DBE1 perspective The following is based mainly on the findings of monitoring and evaluation conducted in December 2009 to assess the impact of the program.3 In December 2009 a DBE1 team visited 26 schools, 26% of the 100 schools that received assistance under the program. The purpose of the monitoring was to determine the extent to which DBE1’s school-based management methodologies had been successfully implemented at school level through this program. The monitoring consisted of school visits and interviews with principals and teachers as well as reviews of documents including SDS and RKS. It was found that, on average, each school was allocated approximately 11-13 students, coordinated by a field supervisor from UPI who visited the school once a week. Most schools regarded the KKN students as active enough in helping the school; furthermore quite a number of the KKN students wanted to teach in the classroom, although their primary task was school based management. As a result of the intervention, all schools had formed KK-RKS (working groups for school planning). Of the 26, eight had completed an RKS and were able to show this in hardcopy. Six already had a district-approved plan (RPS) prior to the intervention, and three had softcopies of partcompleted RKS. The remaining nine schools could not show the team an RKS, either in softcopy or hardcopy form. Various reasons were given: the plan was not provided by the KKN students, the school principal left it at home, or it was at the school but stored in a locked cabinet. Three of the schools had a complete SDS, including the BOS format K1-K6 and LMS. A further six schools had almost completed their SDS. The remaining 17 schools had no SDS and gave a variety of reasons for this, including: it was taken by the students and the school was not given a softcopy, the school does not have a computer, an error / virus, and it was taken by the staff member who manages SDS. A small number of schools established a new school committee, facilitated by the KKN students. Most significantly, the schools generally indicated that they were not empowered by the students. In preparing the RKS and SDS, the school typically served only as a provider of data. The process of preparation of the RKS and SDS was completed solely by the students. 3
DBE1, January 2010, Hasil MonEv Sekolah Dampingan UPI di Kota Cimahi dan Kabupaten Bandung Barat (Internal Report)
The DBE1 Service Provider Program; an Evaluation
5
The Service Provider perspective A meeting was held between DBE1 and UPI on February 16, 2011, to discuss the service provider program. The Chairman of UPI’s research body, LPPM, gave a report on the program. Also in attendance were the Secretary of LPPM and five service provider personnel. The objective was for UPI to report to DBE1 on the activities carried out since 2009, and to discuss program constraints and expectations for the future. The meeting confirmed the commitment of UPI and interest in continuing the program in 2011. UPI expressed the hope that further training will be given to strengthen their capacity as a service provider. Constraints to the success of this program from the perspective of UPI were as follows: 1. There were too few UPI trainers to train students and lecturers before they plunged directly into the school KKN program. 2. Due to budget limitations, UPI was unable to provide KKN students with hardcopies of the modules. Only softcopies were provided. 3. The school-based management themed KKN program requires more time for the supervisor to monitor the school. While for most programs three monitoring visits are sufficient, for the school-based management program, six visits are required. This has an impact on financing the implementation of the KKN program. 4. Given that the aim is to build the capacity of school stakeholders, the time allowed for KKN was too short, and so the results were not optimal; the capacity building process can take a rather long time and requires high intensity support. These preliminary findings are supported by the findings of the structured evaluation workshop held subsequently in UPI on March 1st, 2010. Six participants of the 2009 program attended along with the Head of UPI’s research body and six participants from the 2010 student community service program, described above. All respondents indicated that they would like to join another program if the opportunity were given. They felt that the program was well aligned to their professional and institutional objectives: ‘The program was conceptually aligned with the theory developed by UPI, the objective is good, and the process, in terms of material preparation, was good.’ In particular the program helps academic staff to achieve the three mandated objectives of universities: teaching, research and community support. However, in terms of implementation, it was felt that the program was not yet optimal due to: (1) limited time, (2) limited funds, (3) limited human resources, (4) the varied response of the market (expectations were sometimes
6
The DBE1 Service Provider Program; an Evaluation
too high), and (5) an inadequate system for organization and coordination of district personnel, education offices, schools (teachers and principals) the district, the superintendent, and school committee. The participants were appreciative of the new knowledge they had acquired from DBE1. However, in order for them to become a more effective 'service provider’, further training is required and the process of school mentoring needs to be more intensive. According to the director of its research body, the university is also considering conducting a stakeholder impact evaluation in the schools which received assistance from KKN students. The participants felt that there is a clearly defined market for UPI as a service provider in this field. Furthermore it was felt that this market could be better tapped through establishing MOUs with district governments. Opportunities to socialize and present to stakeholders could be more numerous and intensive. It was suggested that a ‘road show’ to district heads and senior officials would be a good strategy to promote UPI as a service provider. It was also suggested that a good strategy would be to repackage and publish the existing DBE1 school-based management modules in book form in partnership with UPI and the Department of Education (national and provincial). Conclusions The results of this program were mixed. On the negative side of the equation, the students failed to empower or provide enough knowledge and information about the DBE1 program to schools. They saw their task as the preparation of RKS and SDS, rather than to help and teach the schools how to develop an RKS and use the SDS program. This is perhaps not surprising. The students were relatively junior and generally lacked both the status and experience required to support the implementation of school-based management methodologies in schools. Moreover the program was extensive and adopted a ‘cascade’ approach resulting in limited supervision of the students in the field and a limited understanding of the methodologies among the students. Academic staff members were selected for inclusion in the program by senior UPI staff and were not all well suited to the role, generally lacking any theoretical or practical experience in educational management. In part as a result of this experience, DBE1 participated in a joint selection process with clearly defined selection criteria for participants in the district level program which followed and is described below. On the positive side, the program succeeded in engaging UPI as a service provider for DBE1 programs and, significantly, in engaging senior officials from both district and provincial education offices. Further, the program provided training in DBE1 school-based management programs to a cohort of 24 UPI academic staff, who in turn provided training to some 1,000 students. Finally, 100 principals received training in school leadership and the placement of the 1,000 students in 100 schools as part of the annual KKN program resulted in the introduction of key school-based methodologies,
The DBE1 Service Provider Program; an Evaluation
7
particularly RKS and in some cases SDS and strengthened school committees to these schools. It should also be noted that the ‘Thematic KKN program’ implemented by UPI covering RKS, SDS, Leadership, and School Committee training has continued each year since the 2009 pilot. In the first year there were 1,000 students in the program, 24 lecturers, and 100 primary schools in the city of Cimahi and West Bandung. In 2010, 1,000 students, and 28 faculty coaches took part in 100 primary schools in the same two districts. In the 2011 plan, KKN will be held in July with a target of 1,500 students, 50 lecturers, and 150 primary schools spread over Cimahi, West Bandung, Subang, and the City of Bandung. However, the RKS and SDS components of the school-based management package are currently under review as part of MONE’s national program. UPI does not wish to create confusion. It was therefore decided that if by July there is no clarification or further information regarding this matter, UPI only will focus on leadership and school committees in this year’s program. DBE1 has now also trained a number of elementary school supervisors in Cimahi in leadership and school committee strengthening. Several school committees have also been trained. It was suggested that these can become partners for UPI in implementing the KKN program and DBE1 has advised UPI of the 21 schools that already receive leadership and school committee training, the supervisors’ names, and the names of participants who were trained.
The Sampoerna Foundation school-level program The program Responding to a request made to USAID, in 2010 DBE1 implemented a program to develop the capacity of the Sampoerna Foundation’s School of Education (SSE) and Outreach and School Development Program (then called the Teacher Institute). The training aimed to enable Sampoerna Foundation trainers to make use of DBE1 materials in their in-service school development program and also possibly in pre-service teacher training. DBE1 provided training on the overall DBE1 approach, leadership training, BOS reporting, school committee strengthening, SDS and school development planning (RKS). In all, 18 days of training were provided between February and May 2010 by DBE1 specialists and district facilitators. Most of this training was given at the SSE campus in Jakarta. DBE1 also made an agreement with the Sampoerna School of Education (SSE) to jointly adapt DBE1 school based management materials to use in preservice teacher training. SSE had requested technical assistance in developing a credit earning program within the school based on the existing DBE1 modules and materials. DBE1 agreed to support SSE by assigning a specialist
8
The DBE1 Service Provider Program; an Evaluation
to work with the SSE Team to convert the DBE1 program into a credit earning program to be integrated into the formal degree program of S1 and/or S2 level at SSE. Work on mapping of DBE1 methodologies and SSE curriculum was completed by September. It was anticipated that by the end of December, an integrated curriculum framework would have been prepared. However this has not eventuated due to changing priorities within SSE in 2010. The DBE1 perspective The Sampoerna Foundation has an impressive profile with a new teacher training institute and a history of implementing corporate-social responsibility programs for private companies in the basic education sector. While the managers and field trainers from the Sampoerna Foundation have built strong experience in the delivery of programs to improve teaching and learning and, to some extent, to help implement school-based management, it is clear that they lack experience in linking these programs to government policy and systems, especially given the current dynamic regulatory context. It is this perspective which DBE1 has been able to bring to the Sampoerna Foundation, potentially resulting in improvements and enhancing their approach. The managers and trainers from the Sampoerna Foundation participated actively and enthusiastically throughout the program. As a result, the trainers have reportedly adopted and adapted many of the tools from DBE1 schoolbased management package for use in the field. In particular, they have incorporated aspects of RKS and Leadership materials into their training and have recently used DBE1 school committee training modules in a program funded by the Beyond Petroleum (BP) Corporation in Bintuni, Papua Barat. DBE1 materials have reportedly been used in one way or another in at least 20 field activities since the training in 2010. The Service Provider perspective In August 2010 an evaluation workshop was conducted with DBE1 and Sampoerna managers and staff to review the program and discuss next steps. The response of participants and of the institution was enthusiastic. The Foundation especially found that the training increased their understanding of government regulations, which in turn enables them to explain and justify the legal basis for school improvement programs, particularly school-based management. All of the training material was found to be relevant. As most of the training program had been class-based it was felt that more hands-on field experience is needed to enable the Sampoerna trainers to be fully confident with the DBE1 material. In August 2010 it was agreed that the cooperation could be continued under an amended TOC, however this did not eventuate due to changing priorities within the organization. Sampoerna inservice trainers appeared to be enthusiastic about the possibility of being certified as facilitators under the DBE1-MONE certification scheme and it was agreed that they could join DBE1 dissemination activities as service providers
The DBE1 Service Provider Program; an Evaluation
9
at the expense of their organization. Due to conflicting priorities, this did not occur in 2010. The offer was repeated in 2011. In order to determine the extent to which the Sampoerna Foundation had adopted or adapted DBE1 materials and the level commitment to follow up in 2011, a second meeting was held in March 2011. The meeting was attended by Dr Paulina Panen, Dean of SSE), along with five members of the academic faculty and training team. This meeting provided an opportunity for DBE1 to update Sampoerna Foundation on the project and on changes in the regulatory and government policy context. Sampoerna requested copies of updated materials for SDS and RKS. The Sampoerna Foundation appears very keen to continue to work with DBE1 and further develop their capacity and curriculum. As in 2010 there are two programs in which they hope to make use of DBE1 methods and materials: (1) the Outreach and School Development program, and (2) the SSE graduate studies program. The Foundation undertook to provide a written report on field activities in which DBE1 materials were adopted or adapted. In 2011 they hope to make more use of DBE1 materials and will reportedly appreciate the opportunity to join DBE1 in the field as either trainers or participants. Meanwhile, the SSE is relocating to a new campus in Pancoran. In May SSE is holding a Celebration Week. They requested a presentation from DBE1 on School-Based Management, a Teachers Perspective. The SSE’s Masters Degree program is known as Education Leadership and Management (ELM) and is based on material from the Principal’s Academy at Columbia University. SSE would like to incorporate more aspects of the DBE1 material into this curriculum to increase the local relevance. They are currently finalizing two packets for the post-graduate program: Finance and Marketing. The program is delivered in two modes: on-campus and e-learning. They welcome further support from DBE1. The SSE is also currently applying to extend its license from MONE’s Directorate for Higher Education (Dikti) for the graduate (S-1) program, which expires October 2011 after the initial two years operation. For this they need to complete a feasibility study and prepare an academic paper (naskah akademik). They have also adapted material from DBE2 for use in developing good practice for teaching in higher education. Conclusions The aim of the program was to develop the capacity of Sampoerna as a service provider in DBE1’s school-based management methodologies. The program succeeded in introducing DBE1’s methodologies and in developing the understanding of SSE trainers and managers of the policy and regulatory basis of school-based management in Indonesia. As a result, aspects of the DBE1
10
The DBE1 Service Provider Program; an Evaluation
approach to school-based management have been adapted and adopted into Sampoerna’s in-service training program. However, further field work would strengthen the capacity and confidence of Sampoerna trainers to deliver training using DBE1 materials in the field. The level of commitment and interest of Sampoerna Foundation in working with DBE1 in 2011 is confirmed. For a very small investment of time, DBE1 could gain a good return in terms of dissemination by (1) taking a more proactive role in involving the Sampoerna trainers in school-level dissemination activities, and (2) providing some limited assistance to SSE to help them develop their ELM curriculum.
The UPI district program, West Java The program In 2010 DBE1 worked with the Universitas Pendidikan Indonesia (UPI) in West Java to develop the capacity of the institution to provide a service in consulting and training to improve the management and governance of education at district level. DBE1 worked with six professional university staff, three specializing in education finance related DBE1 programs (AKPK and BOSP) and three specializing in education planning related programs (SIPPK and Renstra). These participants were jointly selected in a merit-based process conducted on campus in late 2009. The program was coordinated internally by the university’s research body (LPPM). An introductory training on DBE1 programs was conducted with the staff from UPI and the other two institutions over five days in Bogor in February 2010. Following this the focus of the program moved from the national to the provincial level. Some additional training was provided by DBE1 provincial specialists before the team began giving technical assistance to district planning and education finance teams in the district of Cimahi. The service providers worked between March and September with DBE1 specialists and counterparts from local government to conduct financial analysis (BOSP and AKPK) and strategic planning (SIPPK and renstra) in Cimahi. In July, with the support of UPI and DBE1 the Cimahi Education Office Renstra Development Team started to develop the plan, specifically that part relating to objectives, activities, and costs involved. As a result of this process, team members gained a better understanding of effective budget allocations required to support planned activities of the Education Office. In August the Renstra Development Team estimated the Office’s yearly expenditures and made necessary adjustments between the estimation and available budget. Subsequently, the team reviewed results of work with DBE1 and the service providers.
The DBE1 Service Provider Program; an Evaluation
11
As the next step, the document was presented to other stakeholders in a public consultation session in September. The event was chaired by the Head or the Education Office and attended by more than 60 people including the representatives from DPRD, Bappeda, the Assistant for the Development Unit, Education Council, NGOs, media, school principals, and school committee members. There are two key issues out of many worth mentioning. First, the Chair of the district parliament’s education commission made a commitment to increase the 2011 budget for teaching and learning from 0.8% (as indicated in the AKPK for 2009) to 5%. Second, the Education Office agreed to provide funds in APBD 2011 for disseminating DBE1’s school-based management package. The latter point was discussed between DBE1’s COP and the Mayor of Cimahi two days prior to the public consultation. The DBE1 perspective In West Java, as in other provinces, the service providers played a rather limited role in the earlier stages of the renstra process which gradually increased as the program progressed. Service providers played a much more important role during the in-service support. As to the AKPK and BOSP components, service providers were the key actors in the facilitation process from the very beginning and consequently DBE1 provincial specialists only provided some technical back-up support. Progress in implementing the different DBE1 programs at the district level was very satisfactory. The Service Provider perspective Discussions between DBE1 and UPI on February 16th confirmed that UPI hopes to continue and further develop the collaboration with DBE1. This was reconfirmed in March. The coordinator of the UPI program, Professor Sumarto, Head of the university’s research body, LPPM, also indicated that he hopes the understanding of renstra methodology gained by the service provider personnel can be shared with UPI’s own renstra team so that they can use the same approach internally. Much of the DBE1 material can reportedly be used as course material within UPI. The university is also considering conducting research on the implementation of strategic planning in the city of Cimahi. The following were identified by Professor Sumarto as constraints in the district level service provider program: 1. The involvement of the service providers was hampered in Cimahi by conflicting schedules which often made it impossible for them to attend activities. It was difficult to align schedules of the district education office and DBE1 with the availability of service providers. 2. The DBE1 specialists who worked with the service providers were changed several times, causing problems with program continuity. Each specialist approached the task slightly differently and with a 12
The DBE1 Service Provider Program; an Evaluation
different style making it difficult for the service providers and even for the Education Office team itself, to follow the process well. 3. The role of service providers in mentoring activities with the Cimahi Education Office was insufficient. The activities were still dominated by DBE1 specialists. This is because the service providers were given only two days training at the province, after which they immediately dived into the program. The evaluation workshop conducted on March 1st confirmed these findings and added more. Members of the team indicated that they were motivated to join the program by an interest in increasing their knowledge of education finance, management and governance, and in gaining a practical experience of working in teams and in implementation in the field. In general, participants felt that the program met their expectations, although there were some unexpected obstacles. Among others, it turned out that the education bureaucracy in the district was more complex than expected, the need for more understanding of teamwork was evident, and time management was a problem. In order to become a more effective service provider, participants felt that they needed more time to share experience and opinions with DBE1 specialists, more field experience (‘jam terbang’) more focused workshops and the opportunity to learn by using the products of previous DBE1 activity such as completed BOSP, AKPK and renstra from other districts. It was agreed that the DBE1 Service Provider program is in line with the mission of the organization. The market for UPI services is clearly defined, consisting of schools, government and community organizations, higher education institutions, and especially district governments. In order to increase the potential of UPI to access this market, it was felt that UPI should develop strategic partnerships with education stakeholders, particularly NGOs and provincial and district governments along with more effective networking in the field and marketing to stakeholders. Other recommendations include: publication of promotional material, an intensive road-show to meet with district heads and senior officials, and conducting follow-up research in target districts to better determine the needs of the ‘market’. Conclusions The program succeeded in achieving its initial objectives of developing the capacity of UPI as service provider for DBE1’s district level programs. However further training and field experience is required before UPI can be fully confident to act as an independent service provider. The commitment of the institution to working with DBE1 to develop that capacity is clear. The individuals within the team are generally strong, especially in the finance area. However the program was hampered somewhat by frequent changes in DBE1 personnel, problems with scheduling, and a need for more on-the-job training of UPI personnel by DBE1, prior to, during, and after field activities.
The DBE1 Service Provider Program; an Evaluation
13
In order to achieve good results in a follow-up program in 2011, it will be helpful to jointly select additional personnel to join the team and to prepare DBE1 specialists to take on a more proactive mentoring role with the service provider personnel. DBE1 could also support UPI in publishing materials under UPI’s banner (with due acknowledgement to USAID, DBE1, MONE and MORA) and possibly conducting a joint ‘road show’ to promote UPI as a service provider within districts.
The UMS district program, Central Java The program
In Central Java, DBE1 worked with the Universitas Muhammadiyah Surakarta (UMS) in the City of Solo (Surakarta). Four senior academic staff were jointly selected for participation in the program; two to focus on finance (BOSP and AKPK) and two on data and strategic planning (SIPPK and Renstra). The program was coordinated within the university by the postgraduate studies department. Following the national training in Bogor described above, preparatory training was conducted one week before the work commenced at the district level. The two-day training covered the district training strategy, steps in the training process and training content. Following this training, the personnel worked with DBE1 specialists to facilitate the financial analysis and planning activities. In Central Java, it was intended that the national NGO, PATTIRO, also be included in the program. However, this proved to be impossible as the district government refused to work with PATTIRO as a result of the organization’s previous work as a critic of government and the exposure of a corruption case. After some deliberation and consultations with PATTIRO it was decided that they would withdraw from the program at this point. Working with the service provider, DBE1 completed BOSP, SIPPK and renstra development for the Education Office in Surakarta. The AKPK component was not completed, in part due to limited availability of qualified DBE specialists to assist the process in this period. Results of these activities were presented to stakeholders. As an outcome of internal consultation held in July, BOSP team members revised unit cost calculations for each level. A public consultation event was conducted in July in the District Secretary’s Office and results of the public consultation session were presented to Mayor of Surakarta and Head of the Regional Development and Planning Body (Bappeda) two weeks later. It is hoped that the Mayor of Surakarta will issue a decree to support the use of BOSP results and that implementation will start in 2011 and be used by schools as basis to develop their budgets.
14
The DBE1 Service Provider Program; an Evaluation
A public consultation session was also held to discuss the Surakarta Education Office Renstra in July. The Head of the Surakarta Development and Planning Body (Bappeda) advised that the District Education Office’s document should be finalized in conjunction with Municipality’s development plan. As the Mayor of Surakarta was newly installed in early August, it will be some months before the city’s new vision and mission as well as Mid-Term Development Plan (RPJMD) is finalized. In order to ensure that the Education Office’s Renstra is based on the district’s new plan, it should be finalized then. The renstra process proved to be so successful in Surakarta that the methodology was subsequently picked up by the District Health Office and the District’s Regional Planning and Development Body (Bappeda). DBE1 provided some assistance to these agencies to prepare a renstra. The DBE1 perspective
In Central Java, DBE1 provincial specialists played a dominant role in conducting data analysis (SIPPK) as the service providers lacked the necessary computer skills. Once the data analysis stage was completed, service providers and DBE1 provincial specialists shared the task of providing technical support to the district planning team on an equal basis. In summary, the service providers’ capacity is adequate to provide BOSP and education planning support, but major work remains to be done to develop their computer skills. The UMS service provider personnel are all quite senior academic staff. While this seniority is an asset when it comes to consulting with officials in districts, it can be a problem in that senior staff tend to have greater time commitments making it difficult to fully participate in the field work. Furthermore, the computer skills required for the field work are more likely to be found amongst more junior academic staff. The service provider perspective In his remarks at the workshop on March 8th, the program coordinator, Head of the Post-Graduate program for UMS, expressed strong support for the service provider program. In addition to developing the capacity of UMS as a service provider, the program has resulted in DBE1 material being integrated into postgraduate (S2) programs within the university. Professor Harsono indicated that the DBE1 program could be further disseminated through the Muhammadiyah network of schools and teacher training institutions. Professor Sutama, secretary of UMS post-graduate program, reiterated this support and added that further work on LAKIP and management systems would be appreciated. The service provider personnel reported that they were motivated to join the program by an interest in expanding their knowledge of the ‘education world’, both practical and theoretical. All are senior academics, teaching post-graduate programs. As one participant commented:
The DBE1 Service Provider Program; an Evaluation
15
‘Since I am responsible for the education finance program, it is important to have practical experience from field to complement the theory given to students.’ SIPPK can strengthen the policy analysis component of the strategic management post-graduate program, while in the context of teaching statistics SIPPK provides good material for statistical analysis. BOSP is helpful for preparation of RKAS taught in education finance courses. In general, the participants indicated that the program met their expectations. However, not entirely, because what was given at the training in Bogor was not fully followed up. Further training is required after the TOT to strengthen understandings so that individuals are better prepared when plunging into the field (particularly for BOSP). Meanwhile the renstra team reported that they received extra training as preparation the day before ‘jumping in’ so there was some reinforcement for individuals first. The hope is that much more information can be provided. As well as further training and field experience to deepen their understanding of education finance and planning, the participants indicated that they would like to learn about other programs including Asset Management and Human Resource Management. In order to further increase their capacity as service providers, the team felt that they need advanced training in strategic planning including the implementation of renstra in the field. Is it used and to what extent are the objectives achieved, for example, in Renja and LAKIP and district policy? Also, what are the outcomes of BOSP in the field; is there an impact on the policy of the Bupati / Mayor? All indicated that they feel the need for more ‘flying hours’ as well as exposure to other material. The team felt that the DBE1 program meets institutional needs for UMS by providing opportunities for faculty to take a wider role in education. There is relevance to the development of post-graduate subjects. The DBE1 program has provided additional material and insights. This is relevant for courses in educational policy development. For example, what should the quality of teachers be, in order that graduates are aligned with user expectations? Moreover, the program aligns with the vision and mission of UMS. It supports the participation of both the institution and academic staff in the development of education. In particular, becoming a service provider supports the mission of UMS to provide community service (pengabdian masyarakat). This is included in the official three duties (Tri Dharma) of higher education in Indonesia: teaching, research and community service. In this context, DBE1 material can also be followed up with research as well as enrich teaching programs. It was reported that BOSP and renstra materials have already been used as course material. Data generated through the program has also been provided as material for research by students and lecturers, although it has not yet been analyzed. The service provider program also supports the more general mission of
16
The DBE1 Service Provider Program; an Evaluation
Muhammadiyah to create a community of excellence through solid planning (‘…menciptakan masyarakat utama melalui perencanaan yang mantap’). It was felt that there is a strong market for UMS as a ‘service provider’. Particularly within the Muhammadiyah education community: elementary, middle, and high schools, and Muhammadiyah tertiary institutions. There are 164 such institutions in Indonesia; in Java five large universities. Beyond this, district governments are a prime market along with other government and private education networks. In order to better meet the needs of this market, it was felt that UMS needs to improve the quality of individual personnel, by providing education, training, and increased practical field experience. Does the service provider function need to be institutionalized in a special section within UMS? It was felt that it could be proposed as a center of study of Management Education (School Management) in the post-graduate department. To support this, it was hoped that DBE1 can provide guidelines / manuals / software to UMS, including those which were given during the closing program and those that can be accessed through www.dbe-usaid.org, providing additional training in other materials and to other faculty, and increasing practical time in the field. The following recommendations were made to improve the program in the future: (1) add more time for field experience, (2) Provide increased opportunities for preparation / briefing / reflection before plunging into the field, and (3) try to improve coordination of schedules between DBE1, academic staff, and district government. If there is a clash with scheduled teaching hours, classes can be arranged for another time. However, it is difficult to manage clashes if staff have other assignments outside the city. Conclusions The program achieved its aim of developing the capacity of UMS as a service provider for DBE1’s district level programs. However, as with the other service provider institutions, further training and field experience is required before UMS can be fully confident to act as an independent service provider. The commitment of the institution to working with DBE1 to develop that capacity is clear. UMS is well placed to act as a service provider not only for districts in Central Java (and beyond) but also within the Muhammadiyah network of schools and tertiary institutions. Further work could also consolidate the use of DBE1 materials within post-graduate courses in UMS and use of DBE1 experience and materials as a basis for research to support broader policy development. Compared with those in the other institutions, the personnel participating in the program from UMS are relatively senior. Partly as a result of this, scheduling problems were frequent and problems were evident in the use of computer software. It would be helpful if additional team members were selected to fill this gap.
The DBE1 Service Provider Program; an Evaluation
17
The UNM district program, South Sulawesi The program In South Sulawesi, DBE1 worked with the Universitas Negeri Makassar (UNM). In late 2009, four professional university staff were jointly selected, two to specialize in education finance related DBE1 programs (AKPK and BOSP) and two to specialize in education planning related programs (SIPPK and Renstra). The program was coordinated internally by the university’s research body (LemLit). However, Professor Nurdin, the Director of the division responsible for external partnerships (Pembantu Rektor IV, Kepala Bidang Kerjasama), who oversees the service provider program, recommends that further work be situated under the newly formed School Effectiveness Unit. In February 2010, the four staff joined the introductory training in Bogor. Following this, additional training was provided by DBE1 provincial specialists before the team began giving technical assistance to planning and education finance teams in the district of Barru. This preparatory training was organized at the DBE1 provincial office. Between March and August the team worked with DBE1 specialists and counterparts from local government to conduct financial analysis (BOSP and AKPK) and strategic planning (SIPPK and renstra) in Barru. The analysis and reports for both AKPK and BOSP in Barru District were finalized in July. In August results of these calculations were presented with the new Barru Education Office Renstra to members of district Parliament and Education Council in two separate sessions. During the morning session with the District Parliament, the Head of the Parliament raised three main education-related issues currently faced by District stakeholders: (1) school infrastructure not aligned to schools’ needs and conditions (i.e. some schools have more class rooms than needed while others do not have enough rooms), (2) the high cost of education, and (3) low education quality. The meeting was also attended by members of the Parliamentary Education Committee. A similar session was held for Head and other members of Barru Education Council (Dewan Pendidikan) in the afternoon of the same day. Representatives of schools, media, and NGOs also attended. During the final quarter of 2010 and the first quarter of 2011, DBE1 continued to support capacity development of service provider personnel by involving them in different programs such as the SIPPK and Renstra development process in Soppeng. DBE1 also hopes to provide further opportunities in 2011 for the service providers to take part in SIPPK and renstra development in Pangkep district and BOSP in Makassar City. UNM is also reportedly about to commence a program to disseminate BOSP in East Kalimantan, independently of DBE1.
18
The DBE1 Service Provider Program; an Evaluation
The DBE1 perspective
In South Sulawesi, the service providers took the lead in facilitating BOSP and AKPK components with DBE1 provincial specialists only providing technical back-stopping at the district level. The service providers facilitated the last stages of the BOSP preparation process by themselves as well as in-service support for AKPK preparation. They also prepared the presentation materials for the internal consultation completely independently. As in other provinces, during the initial stages of the renstra preparation process, service providers played only a minor role in district facilitation. However, during the process their role gradually increased and they played a central role in the last two workshops in the planning process. The program for Barru was completed in the third quarter of 2010. The commitment of individuals within the team is clearly evident in the participation in additional work with Soppeng District. However, because they were already busy with their own job responsibilities, the UNM personnel were not always able to take part fully in the programs. A possible solution to this problem that has been suggested is to choose those who are still junior lecturers in their university and have ample time to join programs such as DBE1’s. The service provider perspective At the evaluation workshop held in Makassar on March 4th, strong support was indicated for the service provider program and further cooperation with DBE1. Professor Nurdin, Pembantu Rektor IV, attended for the full day along with Professor Asfah (former DBE1 Coordinator and Head of the university’s research body, Lemlit). As in the other service provider institutions, the program was found to strongly align to the vision and mission of UNM, supporting the institution to achieve its objectives in enriching programs of study with practical methodologies and field experience, providing a basis for relevant research and supporting community service programs including, potentially, the annual student KKN program. Reportedly DBE1 approaches have already been informally incorporated into teaching programs and some research has already been conducted using DBE1-based data. In addition, UNM academic staff who are former DBE1 personnel are currently implementing a case study research program to implement school-based management approaches in ten disadvantaged urban schools in Makassar. Service provider personnel indicated that they were motivated to participate in the program as it aligns with the vision and mission of the institution (UNM), especially community service. In their view, the experience strengthened their knowledge and capacity as education professionals and strengthens the subjects taught through practical experience from the field. The program reportedly met their expectations in a very positive way. Individuals’ interest in gaining practical knowledge was answered in the field, they gained new
The DBE1 Service Provider Program; an Evaluation
19
understandings on planning and budgeting, and the need for synergy from the center to the regions. Also mentioned was the need to accommodate local knowledge through consultation, and accurate data on education funding needs and education budget policy in planning. Some important new understandings gained by participants in the program were identified as follows: (1) planning should be supported by data so that it is valid; (2) the concept of making as much effort as possible to ensure that the plan is implemented, (3) the idea that the Education Office should accommodate the needs of schools in planning, (4) expanding horizons in the field of education through an understanding of education policy, and (5) understanding the amount required in relation to education budget allocation. The following four points were identified as necessary for participants to become more effective as service providers: 1. Printed technical instruction manuals; books, rather than softcopy or photocopies. (It was understood that development of the manuals was ongoing during the implementation period, making this difficult in 2010.) 2. More intensive coordination: information could be more open, the program objectives needs to be better understood from the beginning. There was limited time to meet with service providers. It would be helpful to have discussions prior to the implementation of activities in the field. 3. More time for participation in the field. Limited financial resources may have reduced the involvement of service providers and as a result, the DBE1 specialists were still dominant in the field activity. 4. Involvement of all personnel in all programs, although specialization remains. It was felt by participants that the market for UNM as a service provider in this field is diverse, including not only Dinas Pendidikan Kabupaten/Kota but private sector businesses, public and private enterprises as sponsors through CSR programs and private teaching institutions. In order to meet the needs of this market, it was felt that more personnel should be trained as service providers and current personnel should be given more training. Study programs for both graduate and post-graduate courses need to be strengthened, especially in the area of programs for educational administration/management and preparatory programs for school principals. It would also be helpful to ‘repackage’ DBE1 materials and training in line with the needs of the market. Promotional material, such as leaflets, brochures and a website will assist. In order to achieve the above, assistance from DBE1 was requested. In order to improve the program in the future, it was felt that UNM should establish a body to formalize the service provider program, and qualified
20
The DBE1 Service Provider Program; an Evaluation
personnel need a certificate from DBE1 and a letter of authority (SK) from UNM. More coordination is required prior to implementing activities along with reflection on the results of each activity. Expanded MOUs are needed between UNM and districts. Conclusions The program achieved the objective of developing the capacity of UNM as service provider for DBE1’s district level programs. As with the other service provider institutions, further training and field experience is required in order for UNM to become fully confident to act as an independent service provider. In particular it will be helpful to provide more training during the course of the program. UNM’s commitment to working with DBE1 to develop that capacity is confirmed. Among other things, the evaluation found that the service provider personnel were often busy with their own job responsibilities and as a result were not always able to take part in DBE1 programs. A possible solution to this problem is to choose those who are still junior lecturers in their university and have ample time to join the programs. The idea of ‘repackaging’ DBE1 materials and publishing under an UNM banner echoed similar suggestions made in Bandung and Solo.
Lessons Learned The purposes of this evaluation report are: 1. to assess the effectiveness of the service provider program, 2. to assess the interest and commitment of each service provider organization in further developing the program in 2011, and 3. to identify lessons learnt in order to improve future programs. On the whole, the service provider has effectively achieved its objectives which were, in summary, to reach agreements with institutions of higher education, to train and certify a cadre of professionals to provide services, and to link these service providers to potential clients such as districts or private school networks. Further work is now required to strengthen the capacity of the institutions and to link the service providers to potential clients. The commitment of each of the three universities and of the Sampoerna Foundation to expand and further develop the program in 2011 is confirmed. The service provider program meets the needs of individuals and of the four institutions and is well aligned to their missions, supporting the achievement of institutional goals in relation to community service, as well as enriching teaching and research programs.
The DBE1 Service Provider Program; an Evaluation
21
Some important lessons have been learned from this program. These are described below. School level programs The outcomes of the school level program in UPI were somewhat less satisfactory than for the district level program and for the Sampoerna Foundation program. 1. The UPI KKN program was successful in introducing school-based management programs to a number of schools and districts and has been continued by UPI, independent of DBE1. However, the program failed to effectively empower the schools. This failure is a result of relatively junior students being entrusted to deliver the program in schools with inadequate training or understanding of their role. While there is value in the KKN approach, it is questionable whether this is an appropriate mechanism for disseminating DBE1 methodologies in schools. 2. In contrast, the Sampoerna Foundation facilitators trained in DBE1 school-based management methodologies are all experienced trainers, well accustomed to empowering schools through their programs. The training was intensive, enabling the participants to gain a relatively comprehensive understanding of the material. DBE1 materials and approaches have now been adapted and adopted in the field to strengthen the Foundations in-service program. While the initial training program did not provide sufficient opportunity for participants to gain practical experience in the field, it is clear that they have already incorporated aspects of DBE1 material into their approach and have delivered programs as an independent service provider. The commitment of both UPI and the Sampoerna Foundation to support the continuation of the program in 2011 is clear. However, while it seems that while Sampoerna Foundation trainers are exactly the right people to deliver DBE1 programs in schools, in the case of the UPI KKN students the opposite is true. District level programs Outcomes of the district program were much more positive. The progress in implementing DBE1’s district level finance and planning methodologies in these three districts was faster and more efficient than in the original DBE partner districts because the DBE1 staff were more experienced and DBE1 purposely districts that were already highly committed to undertake these activities as the venue for on-the-job training for the service providers. Moreover, results were very encouraging in terms of potential policy impact as described. These facts indicate: (1) a high level of commitment from the three districts, (2) a maturity in the methodology as a result of refinement over the last three years, and (3) a high-level of competency amongst the team of specialists working with the service
22
The DBE1 Service Provider Program; an Evaluation
providers. In all three service provider institutions the commitment to continue the program is very evident. However, there are a number of areas of the program that need strengthening and improvement during the remainder of the project. The following are important lessons have been learned that will improve the program. 1. As described, it proved difficult in all provinces to harmonize the agendas of the three parties involved in service provider capacity development because of the work load at each of the three home organizations: the university, District Education Office, and DBE1, which had at the time competing commitments to other districts. This, at times, resulted in a situation whereby the work at the district level had to be implemented without active involvement of some or all assigned service provider personnel. 2. The role of PATTIRO, an NGO included in the original service provider training, proved problematic in Surakarta City, where specialists from PATTIRO were to provide training to government officials. It appears that as a result of PATTIRO’s previous activity as a ‘watchdog’ on government, the district was unwilling to work with the group as a service provider. In order to continue the program it was necessary to withdraw the PATTIRO personnel. This may be regarded as a lesson learnt for the future. 3. In all three locations, the service provider personnel require further training and field experience in order to become fully confident as independent service providers in the delivery of these programs. 4. Although the experience varies between provinces, it is clear that the training delivered at province and district level should be more structured and better planned on the part of the DBE1 specialist team. While in some cases participants received good briefings before going into the field, this was not always the case and it seems that, with some exceptions, the service provider personnel felt that they were ‘left behind’ in the delivery of the program, with DBE1 personnel taking the lead and not always effectively mentoring the service provider personnel as they proceeded with the program. Some TOT training for DBE1 personnel together with allocation of time and resources for preand post- activity briefings would address this problem. 5. In all three institutions there is an interest in and a need for expanding the number of personnel involved. In each case the makeup of the team is slightly different. Each has strengths and weaknesses described above. Joint selection of new team members should take account of these conditions. For example, in Surakarta it will be helpful to recruit a couple of younger academic staff with computer skills to complement the team.
The DBE1 Service Provider Program; an Evaluation
23
6. In all three institutions there is an interest in (1) integrating DBE1 materials into course material and (2) using the DBE1 experience and data as a basis for research which could subsequently inform policy. This interest has resulted in some informal use of materials and some initial research or research planning in all cases. 7. Despite the above constraints the service providers have become skilled in delivering DBE1 education programs, although additional capacity development work remains to be done. The enthusiasm of the institutions and individuals participating in the program is high, linked to an awareness of the demand for service provision. This was confirmed in each case by senior academic staff from the university. For example, in South Sulawesi where the service provider representatives have discussed the potential to use their new skills and understandings in assisting districts to develop renstra after the election of new district heads later in the year. UNM personnel in this province have already assisted in the SIPPK updating and renstra process in Soppeng since the completion of the program in Barru. UNM also has plans to disseminate BOSP in East Kalimantan. Similar experiences are echoed in the other locations.
Recommendations On the basis of this evaluation, it is recommended that further training for school level programs with UPI not be provided in 2011. Sampoerna Foundation trainers should be encouraged to participate (at the expense of SF) in dissemination programs in order to gain the field experience they need to consolidate their class-based learning. For this to eventuate, DBE1’s Jakarta-based coordinator will be assigned to coordinate and ensure information is passed to the Sampoerna Foundation in a timely manner. In addition, consideration can be given to assigning a DBE1 specialist to provide limited assistance to the Sampoerna School of Education (SSE) to help them develop an integrated curriculum for the Masters level Education Leadership and Management (ELM) program. The district level programs should be further developed at the three current universities: Universitas Pendidikan Indonesia (UPI) in West Java, Universitas Muhammadiyah Surakarta (UMS) in Central Java and Universitas Negeri Makassar in South Sulawesi. The service provider program can be expanded in these institutions by jointly selecting additional personnel from within each partner university to work with DBE1 personnel on programs to disseminate district level interventions in new districts. Given the success of the current program it is likely that the field of candidates from within each institution may be somewhat larger enabling the inclusion of specialists from each university who can fill the skill gaps evident in the first round. In particular it will be helpful to recruit some specialists with computer 24
The DBE1 Service Provider Program; an Evaluation
expertise to complement technical backgrounds and planning or finance experience. Each service provider team should ideally include some senior academic staff who have the status to effectively consult at a high level along with some more junior personnel who have greater flexibility of time and more familiarity with the computer software required. In addition the service provider program can be expanded to include at least two additional institutions, including LPMP centers to disseminate the personnel management program. (This is in line with the Scope of Services for a DBE1 extension until December 2011.) Additional recommendations, which apply to both current and new service provider programs are as follows: 1. A strategy to certify individuals as qualified to provide training and consulting services in specified DBE1 programs should be jointly developed and implemented. Crtiera for certification should be developed in collaboration of users of the service provider program including district government staff who have been involved in the program. 2. Consideration should be given to enabling universities to repackage and publish DBE1 manuals and materials under their own banners (giving due acknowledgement to USAID, DBE1, MONE and MORA). 3. Consideration should also be given to supporting the service provider organizations to conduct a road show to districts within their provinces to promote their service. DBE1 could assist be providing promotional material. 4. Key specialists within the DBE1 team in each province should be identified as responsible for each service provider program (Finance, Planning etc). Effort should be made to avoid changing staff during implementation periods. These specialists should be given some direction (or TOT) in how to maximize their role as a mentor for service providers, including (1) improving communication and coordination of schedules, (2) providing structured training and providing feedback in the field (prior to, during, and following field work. 5. More field time should be provided to service provider personnel from the first round to consolidate their learning. For experienced personnel this may include inclusion in programs outside their province. As much field time as possible should be provided to new personnel. 6. Partner universities should be encouraged to incorporate DBE1 materials into course materials and to use the DBE1 experience as a basis for research. DBE1 has received informal approaches from a number of institutions with a possible interest in adapting and incorporating the materials and methodologies into pre-service teacher training programs within courses such as school leadership and educational administration. Discussions should be held with universities including the current three
The DBE1 Service Provider Program; an Evaluation
25
partners along with, possibly, UNM in Malang and Unila in Lampung with a view to co-developing and trialing curriculum modules.
26
The DBE1 Service Provider Program; an Evaluation
Appendix 1: Survey Instrument – Service Provider Evaluation, March 2011
Service Provider Evaluation Survey Individual Tujuan dari survey ini adalah: 1. Mengidentifikasi sejauh mana program Service Prvider yang dilakukan oleh DBE1 memenuhi kebutuhan Anda dan organisasi Anda 2. Mengidentifikasi bagaimana program dapat ditingkatkan
Jawablah pertanyaan-pertanyan di bawah ini dengan jujur dan terbuka Informasi Individu Peran/Spesialisasi: Supervisor BOSP AKPK SIPPK (DPISS) Renstra RKS SDS Leadership Komite Sekolah Jumlah hari yang dihabiskan di lapangan bersama dengan DBE1 di tahun 2010 (untuk SP tingkat kab/kota) Jumlah hari yang dihabiskan di lapangan melalui KKN (untuk SP tingkat sekolah)
Pengalaman Individu (for Field Specialists) 1. Mengapa Anda memutuskan untuk bergabung dengan Program Service Provider DBE1? (Why did you decide to join the DBE1 Service Provider program in 2010?)
The DBE1 Service Provider Program; an Evaluation
27
2. Apakah program ini memenuhi harapan Anda? Apakah Anda mendapatkan apa yang Anda harapkan? (Did the program meet your expectations? Did you get what you were hoping?)
3.
Apa yang Anda pelajari? (What did you learn?)
4. Apa yang Anda perlukan untuk belajar lebih banyak untuk menjadi service provider yang efektif? (What do you need to learn more of in order to be an effective ‘service provider’?)
Pengalaman Organisasi (for all) 1. Apakah program ini memenuhi kebutuhan organisasi Anda? (Did this program meet the needs of your organization/university?)
2. Apakah gagasan untuk menjadi service provider sejalan dengan visi dan misi organisasi Anda? (Is the idea of being a ‘service provider’ in line with the vision and mission of your organization?)
3. Apakah Anda pikir ada pasar untuk layanan ini? Siapa? DImana? (Do you think there is a market for this service? Who? Where?)
28
The DBE1 Service Provider Program; an Evaluation
4. Bagaimana organisasi Anda dapat lebih baik memenuhi permintaan/kebutuhan pasar? (How could your organization better meet the demand/needs of that market?)
5. Bagaimana DBE1 dapat membantu Anda untuk mencapai tujuan tersebut? (How could DBE1 help you to achieve these objectives?)
6.
Apakah program DBE1 memberikan Anda konten yang dapat membantu Anda dalam pekerjaan inti? (Has the DBE1 program provided you with content that can assist you in your core work?)
a. b. c. d.
Pelatihan guru (Training Student Teachers) Program Pasca Sarjana (Post Graduate Programs) Melakukan penelitian (conducting research) Lainnya…? (other ..?)
Improving the program (for all) 1. Apakah Anda tertarik untuk bergabung dengan program Service Provider di tahun 2011 (jika ada)? (Are you interested in joining another Service Provider program in 2011?)
2. Bagaimana program tersebut ditingkatkan? (How could the program be improved?)
Terima Kasih atas Partisipasi Anda dalam Survey ini
The DBE1 Service Provider Program; an Evaluation
29
Appendix 2: Report on evaluation of Service Provider Program, UPI, December 2009 HASIL MONEV SEKOLAH DAMPINGAN UPI DI KOTA CIMAHI DAN KABUPATEN BANDUNG BARAT A. PENDAHULUAN Pada tanggal 7-11 Desember 2009 Tim DBE1 mengunjungi 26 sekolah (26%) dari 100 sekolah yang telah mendapatkan dampingan UPI untuk mendiseminasikan program DBE1. Tujuan dari monitoring ini adalah untuk melihat sampai sejauhmanakah keberhasilan program DBE1 di tingkat sekolah yang dilakukan oleh UPI. Monitoring dilakukan dengan melakukan wawancara dengan kepala sekolah dan guru serta dokumen review seperti dokumen RKS ataupun SDS. Nama-nama sekolah yang dimonitor dapat dilihat pada matrix di bawah. B. TEMUAN Beberapa temuan yang dapat kami laporkan adalah sebagai berikut: 1. Tidak semua kepala sekolah mengikuti workshop/sosialisasi Program DBE1 yang dilakukan di kampus UPI. Dari 26 sekolah yang kami intervie, terdapat 10 kepala sekolah tidak mengikuti dengan berbagai alasan misalnya waktunya bersamaan dengan kegiatan yang telah diprogramkan sebelumnya, tidak ada undangan, dan ada 1 kepala sekolah yang hadir tetapi masuk ke ruangan lain. 2. Dari 26 sekolah tersebut, tingkat penyelesaian RKS maupun SDS berbeda-beda; A. Rencana kerja Sekolah (RKS a) Sebanyak 8 sekolah telah selesai menyusun RKS dan telah dicetak, termasuk di dalamnya adalah RKAS 2009/10; b) Sebanyak 6 sekolah telah mempunyai RPS (mereka masih menyebut dengan istilah RPS, bukan RKS) dan sudah disahkan oleh kepala UPTD pada bulan Juli 2009 (sebelum KKN mahasiswa/I dimulai). Semua sekolah ini berada di wilayah Kabupaten Bandung Barat dan RPS mereka berlaku mulai Tahun 2009-2013. c) Sebanyak 9 sekolah tidak dapat menunjukan keberadaan RKS, baik dalan bentuk sofcopy maupun hardcopy. Berbagai alasan yang dikemukakan mereka antara lain: tidak diberikan oleh mahasiswa yang KKN, ditinggal di rumah kepala sekolah, ada di lemari sekolah tetapi dikunci, d) Sebanyak 3 sekolah masih berbentuk soft copy dan sebagian dari mereka belum selesai 100%.
30
The DBE1 Service Provider Program; an Evaluation
B. Sistem Database Sekolah (SDS) 1. Sebanyak 3 sekolah mempunyai SDS yang lengkap, termasuk format BOS K1-K6 dan LMS 2. Sebanyak 6 sekolah mempunyai SDS dan hamper lengkap pengisiannya 3. Sisanya (17 sekolah) tidak mempunyai SDS dengan berbagai alasan seperti: dibawa mahasiswa dan sekolah tidak diberi softcopy, sekolah tidak punya computer, error/kena virus, dan dibawa staff yang mengelola SDS. C. Temuan lainnya: 1. Rata-rata setiap sekolah didampingi sekitar 11-13 mahasiswa yag dibimbing satu orang DPL (Dosen Pembimbing Lapangan) yang dating ke sekolah satu minggu sekali. 2. Sebagian besar sekolah menilai mahasiswa KKN cukup aktif dalam membantu pihak sekolah; malalahan tidak sedikit di antara mereka yang mau mengajar di kelas. 3. Semua sekolah sudah membentuk KK-RKS 4. Ada sebagian kecil sekolah yang membentuk Komite Sekolah yang baru dengan difasilitasi mahasiswa KKN 5. Sebagian besar sekolah merasa tidak ada pemberdayaan dari para mahasiswa. Dalam penyusunan RKS maupun SDS, pihak sekolah hanya berfungsi sebagai penyedia data saja. Proses penyusunan RKS maupun SDS selanjutnya sepenuhnya dilakukan oleh para Mahasiswa. 6. Sebagian besar Dosen Pembimbing Lapangan hanya bertemua 2X saja dengan pihak sekolah; yakni pada saat penyerahan dan penarikan mahasiswa. C. KESIMPULAN DAN SARAN 1. Kesimpulan a. Pihak UPI dalam hal ini mahasiswa KKN kurang maksimal dalam melakukan pendampingan sekolah, terbukti hanya 8 sekolah saja yang berhasil menyusun RKS, dan hanya sedikit saja yang mengisi SDS maupun mengerti bagaimana menggunakan SDS. b. Mahasiswa kurang memberdayakan maupun memberikan pengetahuan yang cukup tentang program DBE1 kepada sekolah. Penyusunan RKS maupun SDS dianggap sebagai tugas yang harus mereka laksanakan di
The DBE1 Service Provider Program; an Evaluation
31
sekolah, bukan membantu dan mengajarkan sekolah bagaimana cara menyusun RKS dan menggunakan program SDS. 2. Rekomendasi a. Perlu dilibatkan para pengawas sejak awal dalam kegiatan diseminasi program DBE1 sehingga mereka dapat membantu sekolah yang belum selesai menyusun RKS dan masih mengalami kesulitan dalam menyusun SDS. b. Saat pelatihan kepada mahasiswa harus diberi pengertian bahwa tugas mereka di sekolah adalah membantu sekolah menyusun RKS, bukan membuatkan RKS. c. Pihak UPI sebaiknya diberitahu bahwa DBE1 akan melakukan monitoring dan evaluasi kepada sekolah untuk mengetahui keberhasilan program DBE1 yang dilakukan oleh UPI ataupun service provider lainnya. MATRIK HASIL DISEMINASI PROGRAM DBE1 OLEH UPI Nama sekolah KOTA CIMAHI 1. SDN Sosial 2
RKS
SDS
Lengkap (termasuk RKT, RKAS)
Lengkap (termasuk LMS dan format BOS K1-K6)
SDS untuk profil lengkap, sehingga bisa digunakan untuk output profil, LMS, Lembar akreditasi, RKAS. Untuk BOS hanya baru terisi sebagian. Tidak ada
2.
SDN Baros Mandiri 4
Dalam bentuk soft copy dan belum selesai)
3.
SDN Cipageran Mandiri 1
4.
SDN Cibabat 5
Tidak ada (Manual RKS dimasukan dalam laporan pelaksanaan KKN Tematik Tidak ditemukan
5.
SD Hikmah Teladan
Lengkap
Lengkap
6.
SDN Melong Asih 5
Soft copy dan data rusak
7.
SDN Melong Asih 7
8.
SDN Utama 7
Hardcopy tidak ada. Soft copy dan data rusak Hardcopy tidak ada. Soft Copy ada (tidak ada breakdown pendanaan 4 tahunan) Lengkap (termasuk RKAS dan RKT)
32
Tidak dapat ditemukan (dibawa guru)
Soft Copy , Profil 3 tahun terakhir lengkap, BOS (K1-K6) lengkap
Keterangan
Kepsek tidak ikut pelatihan dan tidak paham apa itu RKS, SDS Kepsek ikut pelatihan dan sangat paham RKS maupun SDS, hanya saja soft copy dan hardcopy tidak ada di sekolah
Kepsek tidak ikut workshop Kepsek tidak ikut workshop
Profil sekolah lengkap, (3 tahun terakhir lengkap ), BOS diisi sebagin.
The DBE1 Service Provider Program; an Evaluation
Nama sekolah 9.
SDN Utama 3
RKS Lengkap
10. SDN Cibeber 3
Tidak ditemukan (sekolah sedang direhab) Kabupaten Bandung Barat 11. SDN Cipatik
Lengkap (RKT, RKAS)
12. SDN Cihampelas 3
RKS sudah dibuat sebelum ada mhs UPI. Berlaku 5 tahun (20092013) RKS versi UPTD sudah dibuat
13. SDN Cihampelas 1
14. SDN 3 Batujajar
Tidak ada
15. SDN Batujajar 7
Hardcopy tidak ada. Soft copy ada di rumah kepsek (belum selesai) RKS sudah dibuat sebelum ada mahasiswa KKN RKS sudah dibuat sebelum ada KKN mhs
16. SDN Cimamere 4
17. SDN1 Cimamere
SDS
Keterangan
SDS untuk pengisi data profil sekolah (3 tahun terakhir lengkap ), BOS tidak di isi. Tidak ada
Print out Profile sekolah ada Profile, Bos K1-K6
Profile lengkap, BOS K1-K6 belum diisi lengkap Tidak ada
Kepsek tidak ikut workshop di UPI Tidak ikut workshop di UPI
Sekolah mengeluh Karena mahasiswa UPI tidak meninggalkan hardcopy maupun soft copy sehingga sekolah tidak tahu apa-apa
Soft copy ada di rumah kepsek SDS Tidak ada
Kepsek tidak ikut workshop di UPI
SDS tidak ada
•
Kepala sekolah tidak ikut pelatihan
•
Kepsek merasa kurang ada manfaat adanya KKN mahasiswa
18. SDN Jayagiri 1 (gab. SDN Lembang 1 & 3)
Ada dan lengkap
Tidak ada (file dibawa mahasiswa; saat KKN sekolah belum punya computer)
19. SDN Cibogo 1
Belum selesai (hanya profile dan program 4 tahun)
Tidak ada (Komputer belum tersedia). Hardcopy juga belum tersedia
20. SDN 2 Kayuambon
Tidak ditemukan (sudah dikirim ke UPTD). Soft copy ada
21. SDN Cihanjuang II
Di lemari dan dikunci
SDS ada (profile sebagian) Operator sudah paham; BOS K16 belum digunakan DI lemari dan dikunci
The DBE1 Service Provider Program; an Evaluation
Sekolah kebingungan apakah harus menyusun RPS (versi Dinas) atau RKS Kepsek sangat paham betul tentang program DBE1. Kepsek tidak ikut pelatihan (diwakili Ibu Mimin). Saat wawancara kepsek tidak ada
33
Nama sekolah
RKS
SDS
22. SDN Cihanjuang I
RPS versi diknas
Tidak diajarkan; SDS diinstall oleh DBE1
23. SDIT Daarul Fikri
Ada dan lengkap (dijilid dengan laporan lain)
Tidak ada (dan tidak dikenalkan ke guru). Diinsntall lagi oleh DBE1
24. SDN Sariwangi (gab. SDN Jeungjing I dan II)
Versi Diknas Kab. Bandung Barat (2009-13)
Error (kena virus) dinstall ulang oleh DBE1
25. SDN 3 Jeungjing
Tidak ada di kantor (di rumah kepsek)
26. SDN Jeunjing Rigil Tengah
34
Tidak ada
Soft copy tidak ada; computer belum ada tidak ada pembelajaran ke sekolah
Tidak ada
Keterangan Para guru merasa mahasiswa KKN kurang aktif Kepala sekolah sangat tertarik untuk mendalami MBS dengan DBE1 • Ikut pelatihan •
Kepsek maupun guru puas dengan kinerja mhs/i. Mereka paham SDS dan RKS
•
Datang ke pelatihan tetapi salah masuk ruangan
•
Komputer belum ada
•
Tidak ikut pelatihan (tidak ada undangan);
•
Sekolah merasa kecewa dengan mhs KKN
•
Sekolah merasa hanya sebagai penyuplai data
The DBE1 Service Provider Program; an Evaluation
Appendix 3: Abbreviations, Acronyms and Glossary
Abbreviations & Acronyms ADD
Alokasi Dana Desa [Village Budget Allocation]
APBD
Anggaran Pendapatan dan Belanja Daerah [District Government Annual Budget]
APBN
Anggaran Pendapatan dan Belanja Negara [National Government Annual Budget]
AusAID
Australian Agency for International Development
Balitbang
Badan Penelitian dan Pengembangan [Research and Development Body]
Bappeda
Perencanaan Pembangunan Daerah [Regional Development Planning Agency]
Bappenas
Perencanaan Pembangunan Nasional [National Development Planning Agency]
BIA
BOS (Bantuan Operational Sekolah) Impact Analysis
BOP
Bantuan Operasional Pendidikan [Education Operational Grants]
BOS
Bantuan Operational Sekolah [school grants]
BOSP
Biaya Operasional Satuan Pendidikan [School Unit Cost]
BP
British Petroleum
BRR
Bureau for Reconstruction and Rehabilitation (Aceh and Nias)
BSNP
Badan Standar Nasional Pendidikan [National Education Standard Board]
CA
Capacity Assessment
CLCC
Creating Learning Communities for Children
COP
Chief of Party
CSO
Civil Society Organization
DAU
Dana Alokasi Umum [general budget allocation from central government to local governments]
DBE
USAID Decentralized Basic Education Project
The DBE1 Service Provider Program; an Evaluation
35
DBE1
Decentralized Basic Education Project Management and Governance
DBE2
Decentralized Basic Education Project Teaching and Learning
DBE3
Decentralized Basic Education Project Improving Work and Life Skills
DEFA
District Education Finance Analysis
DPISS
District Planning Information Support System
DPRD
Dewan Perwakilan Rakyat Daerah [district parliament]
DSC
District Steering Committee
DTT
District Technical Team
EMIS
Education Management Information Systems
ESP
Environmental Services Program [USAID project]
GDA
Global Development Alliance
GDP
Gross Domestic Product
GGSP
Good Governance Sektor Pendidikan (Good Governance in The Education Sector)
GOI
Government of Indonesia
IAPBE
Indonesia-Australia Partnership in Basic Education [AusAID project]
ICT
Information and Communication Technology
ILO
International Labor Organization
Jardiknas
Jaringan pendidikan nasional – national education network
KADIN
Indonesian Chamber of Commerce
Kandepag
Kantor Departemen Agama [District Religious Affairs Office]
KKG
Kelompok Kerja Guru [teachers’ working group]
KKRKS
Kelompok Kerja RKS [school RKS team]
KTSP
Kurikulum Tingkat Satuan Pendidikan [School Unit Curriculum]
LG
Local government
LGSP
Local Governance Support Program [USAID project]
LOE
Level of Effort
LPMP
Lembaga Penjamin Mutu Pendidikan [Education Quality Assurance Body]
M&E
Monitoring and Evaluation
MAPENDA
Madrasah dan Pendidikan Agama [Religious and Madrasah Education]
MBE
Managing Basic Education [USAID project]
MBS
Manajemen Berbasis Sekolah (SBM=School Based Management)
36
The DBE1 Service Provider Program; an Evaluation
MCA
Millennium Challenge Account
MGMP
Musyawarah Guru Mata Pelajaran [Subject-based Teachers Association]
MI
Madrasah Ibtidaiyah [Islamic primary school]
MIS
Madrasah Ibtidaiyah Swasta [private madrasah; MIN State Madrasah]
MOU
Memoranda of Understanding
MSS
Minimum Service Standards
MTs
Madrasah Tsanawiyah [Islamic junior secondary school]
Musrenbangdes
Musyawarah Perencanaan Pembangunan Desa [Village Development Planning Forum]
NGO
Non Governmental Organization
P4TK
Pusat Pengembangan dan Pemberdayaan Pendidik dan Tenaga Kependidikan [Center for Educators and Education-Related Personnel Capacity Building]
PAG
Provincial Advisory Group
PAKEM
Pembelajaran Aktif, Kreatif, Efektif, dan Menyenangkan [AJEL: Active, Creative, Joyful, and Effective Learning]
PADATIWEB
Pangkalan Data dan Informasi berbasis WEB. MONE database system
PCR
Politeknik Caltex Riau, Pekanbaru
PDIP
Pusat Data dan Informasi Pendidikan [Education Data and Information Center]
PDMS
Project Data Management System
Permendiknas
Peraturan Menteri Pendidikan Nasional [Minister of National Education Regulation]
PKBM
Pusat Kegiatan Belajar Mengajar [Teaching and Learning Center]
PMP
Performance Monitoring Plan
PMTK
Peningkatan Mutu dan Tenaga Kependidikan [Quality Improvement of Education and Education Staff]
PPA
Public-private alliances
Ranperda
Rancangan Peraturan Daerah [Draft of District Regulations]
RAPBS
Rencana Anggaran, Pendapatan, dan Belanja Sekolah [School Budget Plan]
Rembuk Nasional National meeting
The DBE1 Service Provider Program; an Evaluation
37
RKAS
Rencana Kegiatan dan Anggaran Sekolah [School Activities and Budget Plan]
RKS
Rencana Kerja Sekolah [School Work Plan]
RKT
Rencana Kerja Tahunan [Annual Work Plan]
RKTL
Rencana Kerja Tindak Lanjut [Future Action Plan]
RPJMD
Rencana Pengembangan Jangka Menengah Daerah [District Mid-Term Development Plan]
RPK
Rencana Pengembangan Kapasitas [Capacity Development Plan]
RPPK
Rencana Pengembangan Pendidikan Kabupaten/Kota [District Education Development Plan]
RPS
Rencana Pengembangan Sekolah [School Development Plan]
RTI
RTI International
SBM
School-based management (see MBS)
SD
Sekolah Dasar [primary school]
SIMNUPTK
Sistem Informasi Manajemen - Nomor Unik Pendidik dan Tenaga Kependidikan (Management Information System of Unique Number of Educator and Education Staff)
SIPPK
Sistem Informasi Perencanaan Pendidikan Kabupaten/Kota [District Planning Information Support System]
SMP
Sekolah Menengah Pertama [junior secondary school]
SNP
Standar Nasional Pendidikan [National Standards for Education]
SOAG
Strategic Objective Agreement [USAID and Menko Kesra]
SOTK
Struktur Organisasi dan Tata Kerja [Organizational and Work Structure]
SPM
Standard Pelayanan Minimum [Minimum Service Standard]
STTA
Short-Term Technical Assistance
SUCA
School Unit Cost Analysis
TraiNet
TraiNet Administrator & Training [USAID reporting system]
UPTD
Unit Pelaksana Teknis Dinas [Technical Implementation Unit]
USAID
United States Agency for International Development
WIB
Waktu Indonesia Barat [Western Indonesian Standard Time]
38
The DBE1 Service Provider Program; an Evaluation
Glossary Badan Kepegawaian Daerah
District Personnel Board
Bupati
Head of a district
Departemen Agama
Ministry of Religious Affairs
Departemen Keuangan
Department of Finance
Departemen Pendidikan Nasional
Ministry of National Education
Dewan Perwakilan Rakyat Daerah
District Parliament (DPRD)
Dinas
Provincial, district, or city office with sectoral responsibility
Dinas Pendidikan dan Kebudayaan (Dinas P&K)
Provincial or district educational office
Gugus
School cluster
Kabupaten
District (administrative unit), also referred to as a regency
Kanwil Agama
Provincial Religious Affairs Office
Kecamatan
Sub-district
Kepala Dinas Pendidikan
Head of provincial or district education office
Kepala Sekolah
School principal
Komisi
Committee in national or local legislatures
Komite sekolah
School committee
Kota
City (administrative unit)
Madrasah Ibtidaiyah
Islamic primary school (MI; MIS Swasta; MIN Negeri)
Madrasah Tsanawiyah
Islamic junior secondary school (MT)
Madrasah Pendidikan dan Agama
Department of Religious Affairs directorate for Islamic religious schools (Mapenda)
Menko Kesra
Coordinating Ministry for People’s Welfare
Pengawas
School inspector
Renstra Satuan Kerja Perangkat Daerah (Renstra SKPD)
Strategic Plan for local government work unit (e.g. District Education Development Plan)
Sekolah Dasar
primary school (SD)
Sekolah Menengah Pertama
junior secondary school (SMP)
Surat Keputusan
Decree/defining conditions, outcomes of a decision
Wali Kota
Mayor
Widyaiswara
Trainer
The DBE1 Service Provider Program; an Evaluation
39
