Evaluation of an Alternative Learning System for Youths at Risk of Involvement in Urban Violence in Bagong Silang, Manila, Philippines-Outcome Analysis, Cost-Analysis, and Economic Evaluation


 Background: Globally, violence disproportionately affects young people, leading to injury, hospitalisation, death, social dysfunction, and poor mental wellbeing. Moreover, it has far-reaching economic consequences for whole nations, due to loss of productivity. Research suggests that attaining a higher level of education promotes factors which insulate youths from poverty and violence. Purpose: In this study we investigated the outcomes, the cost, and the efficiency of a non-formal education program with an additional psychosocial component. The short-term outcome measure was an increase in educational attainment, a crucial step for youth empowerment. The program analysed was the Alternative Learning System (ALS) offered by the Balay Rehabiliation Centre in Bagong Silang, an urban slum in Manila, which targeted out of school youth. Methods: The cost-effectiveness analysis of ALS compared to a ‘do nothing approach’ was performed from the perspective of the service provider. The study sample comprised 239 learners who were enrolled in the ALS during 2015-2018. For the comparator ‘do nothing approach’, a counterfactual scenario was hypothesised. The average cost of the intervention per enrolled learner, and the incremental cost effectiveness ratio (ICER) for passing the Accreditation and Evaluation (A&E) exam at elementary or secondary level, were calculated. Results: The ALS intervention studied resulted in 41%(n=97) of the learners passing the examination over a period of four years (from 2015 – 2018). The estimated total cost of the intervention was $371,110, corresponding to $1,550 per enrolled learner. The incremental cost-effectiveness ratio for a pass in the exam was found to be $3,830. Compared to other alternative learning interventions, the ALS intervention as used in Bagong Silang was found to be more cost-effective.Conclusion: From the service provider perspective, the ALS for out-of-school young people was found to be a valuable investment to benefit poor young people living in slums in Manila.


Introduction
Globally, violence puts an immense strain on health care expenditure, and has far reaching consequences for national economies, due to loss of productivity and an increase in the expense of law enforcement (1). Household poverty and neighborhood deprivation are both strongly related to youth violence (2,3).
Among young people, violence is often used as a tool to gain self-esteem when they are exposed to adversities. This is often seen among people who are forced to live under impoverished circumstances (2). Violence is a characteristic of the area where our study was performed, Bagong Silang, a Barangay (a large municipal entity) in the city of Manila with a population of approximately 240,000 (4). It was established in the 1970s as a relocation site for slum-dwellers who had previously lived in different parts of Metro Manila. Since its establishment, there have been immense socioeconomic and political problems in Bagong Silang, associated with experiences of all types of violence and torture (4,5).
Common problems are lack of employment opportunities, socioeconomic inequalities, teenage pregnancies, child abuse and drug addiction (4). The educational level is generally low, which is a risk factor for poverty and violent behavior, especially among young people. (6). In 2015, around 14% of young people in the age group of 15-29 years living in Manila dropped out of school (7), mainly due to lack of personal interest and in addition because of the high cost of education (8). Based on the overview of the empowerment theory applied to youth violence prevention programs, we hypothesized that youths can be empowered to move out of poverty and reduce the risk of exposure to violence by raising their educational level and equipping them with the knowledge and skills necessary for the labor market (9).
An Alternative Learning System (ALS) program was initiated in 2004 by the Department of Education, Government of the Philippines, throughout the country with the aim of providing school drop outs with access to free basic education (10). In 2015, the national average of the target population enrolled in ALS was less than 10% and in Manila enrollment was around 6% (7). This program was one part of a larger youth empowerment program targeting children, adolescents and young adults in the urban slums in Manila who are especially exposed to violence.
The ALS program is a non-formal education program which is free of charge for the participants (referred to as "learners" from this point). Enrollment in ALS allows the learners to take the ALS Accreditation and Evaluation (A&E) Exam, offered by the government, where a pass is considered comparable to graduation from the formal education system (11). Through implementation of different programs utilizing various delivery methods, ALS strives to reach and help learners in ways that t their distinct needs (10).

Description Of The Intervention
The study described here investigated the ALS program as planned and run by the Balay Rehabilitation Centre (BRC) in Bagong Silang, Manila. The ALS program was advertised in Bagong Silang at the beginning of each academic year and new learners were recruited on a voluntary basis. The learners were rst screened to ascertain their educational status, and classi ed into three levels: Basic Literacy; ALS Elementary; ALS Secondary, This terminology is de ned in the International Standard Classi cation of Education (ISCED) for the formal mode of education (12). At the time of recruitment, all learners were also interviewed about their experiences of violence in the community. Their psychosocial needs were assessed by individual interviews, focussed group discussions and home visitation to check living conditions and family dynamics. Based on their needs, targeted psychosocial interventions were offered to 22% of learners. These included case management through individual or group counselling sessions, psychoeducational sessions on topics such as anger management, self-esteem and reproductive health, and welfare assistance, including nancial support for education and mental healthcare.
Educational sessions providing instruction in six interrelated learning streams (communication skill, scienti c literacy and critical thinking, mathematical and problem solving skills, life and career skills, understanding self and society and digital citizenship) are offered as required for the A&E examination (13). These sessions were offered three times a week at the ALS Centre in Bagong Silang, a small compound rented for conducting ALS teaching sessions and psychosocial activities. In the Basic Literacy program, the sessions lasted two hours, while for ALS Elementary and ALS Secondary they lasted 4-5 hours. The sessions ran for 10 months per year, from January to October, and at the end learners took the ALS Accreditation and Evaluation (A&E) Exam conducted by the Department of Education (10). Two ALS instruction managers (teachers) conducted the education sessions. One psychosocial worker managed the psychosocial activities. A project supervisor, a project coordinator, one researcher and an administrative staff supported the smooth implementation and running of the intervention.
The aim of this study was to evaluate the ALS intervention in Bagong Silang in terms of costs, short-term effects (education and employment), and cost-effectiveness from the service provider's perspective.

Methods
The costs and short-term effects were evaluated during the full period of the ALS program, January 2015 to October 2018. In addition to the implementation and running of the intervention, the period included the recruitment processes and examinations (7).
Data on outcomes was extracted directly from reports obtained from BRC. The primary short-term outcome measure used was educational attainment in terms of achieving an equivalency to formal school education, measured as passing the ALS Accreditation and Evaluation (A&E) examination within the period of the intervention, as this is a well-de ned and reproducible outcome (14,15). The examination is conducted at two educational levels: Elementary and Secondary, and attaining a pass at either level was considered a successful outcome of the intervention in this study. If a learner managed to pass both elementary and secondary level exams during the period, this was considered as two successful outcomes.
Data on long-term outcomes such as getting a job or a reduction in experience of violence were also collected by BRC through surveys. However, since the intervention only ended in 2018, the data on longterm outcomes was limited for the purpose of the current study. We therefore did not include these outcomes in the cost-effectiveness analysis, but rather described them statistically. Jobs were categorised as formal and informal pre-and post-intervention. A formal job was de ned as being involved in an occupation where the employee received a monthly salary. An informal job was working for a daily wage. Assessment of experience of violence was obtained from participants giving the answer 'yes' when asked whether they had experienced an incident of violence in interviews conducted pre-and post-intervention. The recall period for the experience of a violent incident was not included in the data for the pre-intervention stage. For the post-intervention stage, the recall period was same as the intervention period for the individual learner. Learners who did not respond to the surveys (pre and/or post) were categorized as 'not reported'. The association between educational attainment and gender, age, job outcomes and experiences of violence before and after the ALS intervention was determined by performing a Pearson's chi square test.
A cost analysis for the intervention was conducted from the service provider perspective. We included the costs borne by BRC during the recruitment process and the implementation and maintenance of the intervention. These costs were taken from the audited nancial reports of the organization for the years 2015-18. The costs that were identi ed as being explicitly attributable to the ALS intervention were directly allocated and labelled as "Direct cost". (Table 1) This was followed by identi cation of the overhead costs from the budget. Those were the costs that were partially attributable to the ALS program, and were allocated based on their monthly time contribution to ALS related activities. These were labelled as "Shared costs" and included the salaries of supporting staff in proportion to the time spent in the ALS intervention compared to their total work time. This cost allocation was ascertained by listing the work responsibilities of the staff members and then shortlisting those activities that were related to ALS.
Finally, an estimate of the monthly time spent on ALS-related activities was made by each individual staff member. The administrative cost and the planning and development cost which are also included in "Shared Costs" were identi ed on the basis of the involvement of staff members in these elds and its relation to the implementation, running and upkeep of ALS. Table 1 Description and categorization of cost items -Project support costs (BRC o ce rent, supplies, utilities, and maintenance) Table 1 All costs were evaluated using the consumer price index for 2018 (16). Finally, the costs were converted to USD using Purchasing Power Parity (PPP) exchange rates (17). The costs presented in the results were rounded to the nearest $10.
In the cost-effectiveness analysis (CEA) the ALS program was compared to a 'do nothing approach' where no educational intervention was implemented. As the design of the intervention is a one-group pre-post evaluation there was no equivalent group for comparison, so a counterfactual scenario was hypothesized based on contextual research. If youths drop out of school in this setting, they will be unemployed or involved in paid or unpaid low-skilled work, or be involved in criminal activities (8). We could assume that a young person who was not in school and had not enrolled in an ALS program would not increase his or her educational attainment in terms of passing the A&E exam. Apart from the need for educational input, the ALS A&E exam is only available to students enrolled in an ALS Centre, and there are none in Bagong Silang apart from the one run by BRC. We do acknowledge that a similar outcome could be produced if a school drop-out returned to formal schooling, but this comes at a cost and is further discussed below.
Costs and outcomes of the intervention are compared to the do-nothing scenario using an incremental cost-effectiveness ratio (ICER) that shows the extra cost for one extra positive outcome (pass in A&E exam). We also report the cost of the intervention per learner. To investigate the uncertainties in our results, two ambiguous parameters were identi ed and one-and multi-way sensitivity analyses were performed. Shared cost was estimated based on the self-reported time spent by the staff working for ALS and in the sensitivity analyses we varied this estimate by +/-50%. Further, the outcomes of attaining a pass in the ALS A&E examination at Elementary and Secondary level were considered equal in this study. In the sensitivity analyses we only considered a pass at the secondary level as a successful outcome, which reduced the rate of successful outcome from 41-34%. IBM SPSS Statistics version 26 were used for all statistical analyses (18).

Results
Of the total 239 learners enrolled in the intervention over the study period, 157 were children (12-17 years); 65 females and 92 males. Of the 82 adults (18 years old or more), 30 were females and 52 males. While everyone received the educational intervention only 22% of learners were found to need additional psychosocial interventions. Information on employment after participating in the intervention was available for 115 learners up to 31st January 2020. A signi cant association (p-value = 0.006) in likelihood of having a formal job was noted for those who had passed the A&E exam.

Experience of violence
The learners who reported on their experience of violence before enrolment in the intervention were also followed up until 31st of January 2020. Pre-intervention experience of violence was found to be an unfavourable factor in attaining a pass in the A&E exam. This association was signi cant (p-value = 0.02). No signi cant association was found between the results in the examination and post-intervention violence experience (p-value = 0.34).

Cost analysis
The total cost for the intervention during the period of January 2015 to September 2018 was 371,110 US dollars. The educational sessions of ALS accounted for 25% of the total cost, while the psychosocial intervention accounted for 14% of total cost. The total overhead costs (Shared cost) were almost the same as the Direct cost. The average cost of the intervention per enrolled learner (n = 239) was $ 1,550 (Table 3).

Discussion
This study investigated the outcomes, the cost, and the cost-effectiveness of the ALS intervention in Bagong Silang. The outcome of the intervention could be measured in terms of the success rate of learners passing the A&E examination at both elementary as well as secondary level. The national average of ALS learners passing the A&E exam was 30% and in this respect, this particular ALS intervention at Bagong Silang was found to be 11 percentage points more effective (7).
Besides the short-term outcome of the success rate in the A&E examination, another outcome measure of ALS was the job status of 115 learners who were followed up after their completion of ALS.
Approximately 60% reported that they were employed in the informal sector and 40% in the formal sector. A signi cant unadjusted association was found between passing the exam and getting formal jobs which suggest that a learner who passes the A&E exam is more likely to get a formal job. A 2018 report by the World Bank states that, in the Philippines, an individual who has enrolled at ALS and passed an A&E is twice as likely to get a formal job as someone who has not passed the exam (7). Since job status is a long-term outcome, and information is not yet available for 50% of the learners, who nished the course only recently, we have not yet drawn conclusions about the effectiveness of ALS for job outcome in this report. The staff of BRC are at present trying to collect more follow-up data. We will reevaluate the intervention when this follow-up data is available.
The bene ts of education for work prospects have been shown in other settings. For example, Kienzl et al found that nishing high school in the USA is associated with a reduced risk of unemployment compared to non-completers (19). However, this relation is time dependent; a person who completes high school with a shorter drop out period (less than four years) is more likely to have a positive economic outcome than a person who takes longer (19). Since the age of most learners in the ALS programme was 16-19 years of age, suggesting a short drop out period, we expect that the intervention will have a positive effect on future employment rates and further improve the cost-effectiveness of the ALS intervention and thus "the social return of investment". An aspect of the success of the ALS intervention that is harder to measure is whether it can reduce the exposure of the participants to experiences of violence. The available data concerning violence before and after the intervention did not show any striking change. However, the data on experience of violence are di cult to evaluate. Experience of violence is likely to have been under-reported among the enrollees (25% of those who had pre-intervention violence experience passed the A&E while 42% those who did not reported such experiences passed A&E (Table 2)). UNICEF has reported that 80% of youths aged 13-24 years in the Philippines had experienced some form of violence in their lifetime, whether in the home, school, workplace, community or during dating (20). To address the challenge for young people in Manila in terms of exposure to violent, BRC's ALS intervention has a psychosocial component designed to enhance the physical and mental welling of young people at risk, which we see as a strength of this intervention. The impact of violence goes beyond physical consequences. Toxic stress from experiencing violence can negatively impact the wellbeing and intellectual development of young people (21).
It is known that high prevalence of violence has a negative impact on the people living in slums like Bagong Silang in terms of perceived and actual safety, school attendance, and businesses opportunities. Therefore, an ALS intervention with an effective psychosocial component is likely to result in more selfesteem, self-mastery, and self-control for the participants, and also diminished behavioral/emotional problems and violence (7). In addition to addressing violence prevention the psychosocial interventions in this program were designed with an intention to address risk factors that were the reasons behind low attendance of learners and prevent them from dropping out.
ALS interventions are an important way of helping young people to become better quali ed for coping with adult life. However, such interventions must be nanced, and in discussions about nance it is important to have an evaluation of what they might cost. Until now, cost analysis has not been routinely undertaken for evaluation of such interventions. The cost of the ALS intervention discussed here was found to be $1,550 per enrolled learner, with an ICER of $3,830. The overhead costs accounted for a huge portion of the total cost, which means that a high proportion of money is not spent directly in project but rather in administration. For many donors, the overhead ratio is an important indicator for the e ciency of a non-governmental organization. However, a high overhead does not mean that an intervention is not cost-effective. We believe that the high overhead cost re ects on the need of higher investment for initial organizational development in resource limited settings. The proportion for overhead can be expected to decrease in long run, especially if a higher number of learners can be enrolled.
The ICER for ALS was found to be $3830/passed A&E Exam. As there is no internationally accepted value of improved education, or of social interventions targeting out-of-school youths, it is di cult to establish whether the intervention studied was cost-effective. Comparing our PPP results with those of similar interventions from similar contexts could indicate whether this was an e cient intervention.
Unfortunately, all reports of comparable studies found in the literature were performed in high income countries, which poses a challenge for comparison. Kendall  the ALS intervention in Bagong Silang was found to be more cost-effective than the other informal education programs.
Since the published studies are of doubtful relevance to the situation in Manila, a more useful way of determining the worth of the investment is to compare it with the cost of school education in the Philippines. There is no tuition fee for primary and secondary education in government run public schools in Philippines. Budgets for maintenance and operating expenses are prepared at school level which pose a challenge to calculate cost per student enrolled. Therefore, yearly tuition fee for private schools were referred. At elementary level (1-6 years) and secondary level (7-12 years) combined, the cost ranges from $ 54,160 -$ 111,500 in 2019, which is around $ 4,510-9,220 per student per year (24). The average cost of ALS per student per year in Bagong Silang was estimated to be $1,550, which is 1/3 to 1/6 of the cost of attending a private school.
This study includes some parameters with uncertainties that may affect the estimated ICER. We performed two "one-way sensitivity analysis" and one "multi-way sensitivity analysis" and although this tended to increase the ICER, it remained in the lower range of prior studies and the cost of attending private schools in the Philippines. We therefore consider that the ALS intervention is cost-effective from a service provider's perspective in achieving a higher level of education, which is a crucial step for youth empowerment, compared to a 'do nothing approach'. However, this should not be an argument for students for dropping out of formal education and participating in non-formal education instead. As education in a formal setting offer more opportunity of engagement in the learning process, it plays an important role in developing non-cognitive skills in young people. Such skills are crucial for realising desirable life outcomes thus offering a sustainable solution to poverty alleviation (9,25). Dropping out from schools is very common for at-risk youths living in Bagong Silang. These young people often need to work to earn money, and even though public schools are free, the learners are unlikely to follow the regular curriculum successfully as they may miss many classes or fall asleep in class due to fatigue from jobs. In addition, the public schools in the slums in the Philippines are not always a safe haven for the students and are often burdened by youth gangs. It is hard to keep the good teachers in violence-prone neighborhoods. This touches on the issues of inequality, as poverty itself forces children and young adults to work and thereby forgo formal education, resulting in them being less able to compete in the future labor market. Therefore, we consider ALS a reasonable investment on behalf of at-risk young people living in slums in Manila if they are dismissed/drop out from the public schools, as other educational options are scarce.

Strengths and limitations
A strength of this study was that this youth empowerment program followed the guidelines and curriculum set by the government, which facilitated a comparison with the national data. A limitation is that we did not have a control group with which we could make a direct comparison. However, we considered that the results could be reliably compared with a 'do nothing alternative' for young people who did not attended BRC's ALS intervention, as none of them would be able to get a pass in the examination, and there would be no cost associated with that outcome. We acknowledge that a similar outcome could be obtained if an out-of-school youth bene ts from the intervention in another nonformal program or returns to a formal school. However, based on our contextual research we consider this to be unlikely. In any case, the results would still indicate the value of ALS interventions although not necessarily indicating the cost-effectiveness of the ALS intervention under study.
In this study our emphasis for the analysis was on the immediate/short-term outcome of attaining higher level of education. Occupation is another determinant of socioeconomic status, and knowledge and skills are important factors determining job opportunities. Skills for a job can be gained through informal means as well. However, we assume that in the long run, completing secondary education would have better outcomes.
The intervention is also expected to have wider societal bene ts beyond the learner's better employment/income status and violence prevention. Taking a societal perspective in the economic evaluation could potentially have unmasked some interesting results and could have guided decision makers within the government for future investments. An important step involved when considering a societal perspective is to include opportunity costs of the participants, i.e., the individual cost for attending ALS (e.g., reduced work income). It is suggested that taking an ingredient approach for cost analysis would have facilitated a study from a broader perspective (23).

Conclusion
From a service provider's perspective, the ALS for out of school youths with an ICER of $ 3,830 per passed A&E Exam was found to be cost-effective compared to a 'do nothing approach'. We hope that there will be long-term bene ts that will improve the cost-effectiveness of the intervention, such as better job prospects, poverty reduction and reduction in experience of incidences of violence, as well as a change of safety perception in the community. Further research will be needed to evaluate these broader and longterm social and health bene ts of ALS. We conclude that the intervention was a good investment on behalf of underprivileged youths living in the urban slum of Bagong Silang.