Effectiveness of the Public Work Program
in Slovenia
Laura Juznik Rotar
This paper evaluates the direct effects of the Slovenian public work program on the youth. The effectiveness of the program is measured by re-employment probability. A Heckman correction of selection is applied to estimate the average program effects. The Heckman selection estimator is an extensively used method to control for selection on unobserved variables. It relies on the assumption that a specific distribution of the unobservable characteristics jointly influences participation and outcome. The results of the empirical study indicate that the public work program has a positive effect on employment in the short run. In the long run, however, the results are not so obvious.
Key Words: active employment policies, evaluation, Heckman procedure, selection, treatment effects jel Classification: C50, J38, j68
Introduction
Active employment policies are essentially public interventions in the labor market. For the measures of active employment policies in Slovenia in the year 2008 there was a budget of approximately 98.6m eur (current prices). Measures of active employment policies are financed from the state budget, and some of these resources are European resources. The most heavily financed are: training programs and programs dealing with social inclusion. Based on scientific methodology, there are few studies on the evaluation of the effectiveness of employment programs in Slovenia. This leads to overestimation of the results and inadequate distribution of resources.
Since it is a well-known fact that unemployment among young people is more sensitive to fluctuations in the business cycle than adult unemployment, we focus our empirical analysis on young unemployed persons. An additional reason is also that the public work program has not yet been evaluated for the youth.
The aim of the empirical analysis is to find out how effective the public work program is on future employment probability. We are interested
Dr Laura Juznik Rotar is a Senior Lecturer at the Faculty
of Commercial and Business Sciences, Slovenia
Managing Global Transitions 9 (3): 275-287
in the future employment probability of young unemployed who participated in the public work program, compared with young unemployed who did not participate in the employment program. Because it is not possible to identify the individual causal effect for inclusion in the employment program, it is necessary to introduce certain assumptions. In the empirical analysis, which is based on a rich database, we use the Heckman procedure. The rest of this paper is organized as follows: Short Overview of Public Work Program, Data and Variables, The Empirical Application, and Conclusion.
Short Overview of Public Work Program
Public work programs are local or state employment programs, which are intended for stimulation of development of new working places or preservation of current ones, or development of working abilities of unemployed people. Public work programs are organised in order to carry out social, educational, cultural, environmental, municipal, agricultural and other programs. Employers or other organizations may not organise public work program for those activities, of which goal is production of profit, or when the presence of the public work program on the market would lead to imperfect competition.
The public work program and number of people participating in the public work program are determined by the Government of the Republic of Slovenia within the frame of the program of measures of active employment policy for the current budgetary period. The public work program can also be accepted by the municipality, if it ensures funds for implementation of the public work program in whole. National employment service, orderers of the public work program, performers of public work program as well as unemployed persons can take part in implementing individual employment programs, whereas their mutual obligations are determined by a contract. Based on a special employment contract, unemployed persons can participate in the public work program. Such a contract is conducted with performers of the public work program. Working relations regulations are being used for the above mentioned contracts. Such regulations cover working hours, pauses and rests, night work, minimal holiday, safety and health at work and special protection of workers. For approximately one quarter of the duration of the public work program, the unemployed persons are to participate in training programs.
The aim of the public work program is to ensure that social and work-
table 1 Data on participants in public work programs in Slovenia in the period 2005-2008
Structure	2005	2006	2007	2008
By gender				
Women	3,087	2,727	2,089	2,460
Men	2,281	1,999	1,386	1,476
By age				
Up to 18	7	21	2	2
18 to 26	886	999	242	340
26 to 30	897	684	767	732
30 to 40	990	799	769	891
40 to 50	1,039	884	625	723
50 and above	1,549	1,339	1,070	1,248
Continued on the next page
ing engagement of target groups of uneployed persons with inclusion in the working environment, rising the level of their knowledge, working abilities and motivating participants for their own professional development, stimulation of employers for regular employment of participants as well as activating local communities to solve the unemployment problem on their own area.
Activities of the public work program are intended for those who are long-term unemployed and who have not managed to change their status so far. In the public work program preferentially are included the long-term unemployed, recipients of social assistance, older unemployed persons (above 50 years), young unemployed persons (up to 24 years) and seekers of a first job. In the year 2006 there were 4,726 participants in the public work program, in the year 2007 there were 3,475 and in the year 2008 there were 3,936 participants in the public work program (Ministry of Labour, Family and Social Affairs 2006; 2007; 2008).
In table 1 we show the data on participants in public work programs for the period 2005-2008 in Slovenia. The number of women participating in public work programs is for all years constantly higher than the number of men participating. The number of young people, up to 30 years, decreased by 716 from 2005 to 2008, while the number of people above 40 years decreased by 617. Among participants in public work programs the most are being unemployed for one year and more. In the period 2005-2008 the most significant fall in the number of unemployed
table 1 Continued from the previous page
Structure	2005	2006	2007	2008
By unemployment duration (waitingperiod)				
Up to 6 months	709	494	416	530
From 6 months to 1 year	923	589	220	282
1 year and more	2,737	2,055	2,075	2,273
3 years and more	999	1.588	764	851
By qualification/education				
I.	1,004	934	652	722
11.	221	199	114	128
III.-IV.	837	778	604	623
v.	2,163	1,793	1,268	1,532
VI.-VII.	1,143	1,022	837	931
Total	5,368	4,726	3,475	3,936
Exit to employment	772	814	675	596
persons by duration of unemployment was for those unemployed from 6 months to one year.
The majority of participants in public work programs had fifth level of education, whereas for those with first as well as with sixth and seventh level of education the number decreased. In 2008, 35.1% of participants in public work programs were included in social programs; 31.4 % of participants were included in the fields of public administration, education, culture, tourism, catering industry and sport; 19.1% of participants were included in the field of environment and ecology, construction, traffic and communications; 12.6% of participants were included in other services, whereas 1.9% of participants were included in the field of agriculture, forestry, fishery and hunting (Ministry of Labour, Family and Social Affairs 2008).
Data and variables
DATA
The data used in this empirical study cover a random sample of approximately 3000 unemployed persons collected from the unemployment register kept by the Employment Service of Slovenia. The unemployment register includes records of all individuals who have been registered with the Employment Service as unemployed persons and who are actively
searching for a job. The advantages of this source of data are the availability and accuracy of data, and the fact that the data can be shown at the lowest possible level (having regard to the protection of personal data). Whereas the disadvantage of such a database is that the data do not allow for international comparisons.
The target group in our empirical analysis represents young unemployed persons aged from 20 to 29. For each person used in this study we have data on registration dates, data on labor market status: unemployed person not included in the employment program and unemployed person included in the institutional training program, and individual characteristics.
Table 2 presents the descriptive statistics of some selected variables for a group of non-participants and a group of participants in the public work program. The data are for the years 2002 and 2003. Among the group of non-participants and the group of participants in the public work program there are differences in program characteristics as well as in individual characteristics. Duration of unemployment before the program start is shorter for the group of non-participants compared with the group of participants in the public work program. Secondly, the group of participants in the public work program consists of persons who registered quite early with the Employment Service, and thus also started earlier in the program. The differences are also in gender, age, education and region.
variables that potentially influence the process of
selection into the employment program
Variables that might influence the decision to participate in the institutional training program as well as future potential outcomes should be included in the conditioning set X, and therefore in the propensity score in order to avoid biased estimates of the causal effects. The key to addressing the sample selection (endogeneity) problem is to obtain an understanding of how different individuals end up in different programs. We discuss the main determinants of selection and then explain which observable variables are used to capture them. We focus on the determinants that can be divided into two groups: those required by legislation, and those that may be underlying the decisions of the employment consultant and the unemployed.
Inclusion into the institutional training program (or any other employment program) is based on the fact that the unemployed person has
table 2 Descriptive statistics
(1)	(2)	(3)
Gender (%)		
Male	53.4	37.2
Female	46.6	62.8
Age (average)	24.4	25.4
Region (%)		
Pomurska	5.3	14.9
Podravska	16.9	28.4
Koroška	5.1	0.7
Savinjska	12.7	35.6
Zasavska	2.2	3.2
Spodnjeposavska	2.9	4.6
jv Slovenija	4.8	3.7
Osrednjeslovenska	27.6	0.6
Gorenjska	9.8	1.5
Notranjsko-kraška	2.6	0.9
Goriška	5.8	3.8
Obalno-kraška	4-3	2.1
Education (%)		
Unfinished/finished elementary school	15.8	4.8
Lower vocational school (2 years)	3.1	2.8
Lower vocational school (3 years)	0.7	0.3
Vocational school (4 years)	18.7	15.9
High school	41.4	48.4
University (2 year degree)	1.7	6.6
University (4 year degree)	18.5	21.2
Master's degree	01	0
PhD	0	0
Duration of unemployment before the program start in months (average)	4-3	19-5
Number of observations	1346	725
notes Column headings are as follows: (1) variable, (2) non-participants, (3) participants in the public work program. 'Program start' for the group of non-participants is a hypothetical date randomly assigned by a procedure suggested by Lechner (1999b). 'Duration of unemployment before the 'program start' for the group of non-participants is based on this hypothetical date.
to be entered in the unemployment register kept by the Employment Service of Slovenia. Being entered in the unemployment register is, in accordance with legislation, a formal precondition for enforcement of rights from insurance in the case of unemployment, inclusion into occupational consultancy and into active employment programs. Given that the eligibility rules are met and that the unemployed person has been
registered with the Employment Service, the unemployed person is given the right to draw unemployment benefit.
At least two months after the unemployed person has been registered with the Employment Service the unemployed person, together with the employment consultant, prepares an employment plan. The unemployed person accepts all agreed obligations by signing the employment plan. The employment consultant and unemployed person meet at least every three months in order to discuss the job search efforts of the unemployed person since their last meeting, new job offers available, potential benefits of participating in employment programs, as well as potential adaptations of their strategy for getting the unemployed person back to work.
Usually it is the employment consultant - but it may also be the unemployed person - who proposes participation in the employment program to improve the chances of finding a job. In the case of participation in the employment program, the unemployed person signs a contract in which all formal rights and obligations are determined. In the case of participation in the employment program the unemployment benefit right remains. There is no legal entitlement to the employment program, and the employment consultant has a considerable amount of discretion when making a decision about program participation. However, the employment consultant has to use this discretion in accordance with the aims and objectives of the employment program. The situation and development of the labor market also has to be considered, and the employment consultant has to act on the basis of the principle of economic efficiency. In addition, the employment consultant has to take into account the aptitude of the applicant for specific jobs and his/her chances of completing a specific program successfully. In particular, the employment consultant's decision has to be guided by the consideration as to which of the measures available have the highest chances of success and are the least costly, that is, most efficient for a specific individual.
According to our knowledge of the 'average' selection process, the employment consultant's decision about referral of applicants to specific programs may be guided by two objectives: efficiency or equity. Employment consultants pursuing efficiency goals assign those individuals to the programs that are expected to benefit most from them. In contrast, equity goals require employment consultants to select the neediest individuals into the programs, where neediness is defined by some criterion, e. g. a high risk of becoming long-term unemployed. Variables that are
used in the empirical analysis to capture such selection are age, sex, education and region. We also included a variable describing unemployment history, that is, duration of unemployment before the program start. The importance of unemployment history can be found in Heckman and Smith (1995). We could also capture the unemployed features by other indicators such as last job position (short-term/long-term employment, qualified/unqualified worker), branch of activity.
From the point of view of the unemployed, their decision whether or not to participate in a program is guided by considerations very similar to those of the employment consultant. There are, however, additional reasons for joining or not joining a program: if the unemployed person sees no chance of finding a job anyway, with or without a program, he/she may prefer not to join a program which reduces his/her free time. We capture this fact by using their unemployment history as well as regional variables as a proxy. Finally, legislation also provides a rather strong incentive to participate in the employment program as the unemployed person is still entered in the unemployment register, even if he/she does not find a job after the program ends. In this case, he/she can still enforce the right to unemployment benefit. We could control for this fact with the variable enforcement of unemployment benefit at the beginning and at the end of a spell.
Having discussed potentially important factors and variables available for the empirical analysis, the question is whether or not some important groups of variables are missing. One such group could be described as motivation, ability, social contacts (see, for example, Lechner 1999a). We could capture such attributes by using professional achievements, achievements at their working place, independence at work, confronting with changes. Unfortunately our database does not currently allow for such information. As usual for these variables, we have to rely on their indirect effects.
Empirical Application
The Heckman selection estimator is an extensively used method to control for selection based on unobserved variables. It relies on the assumption that a specific distribution of the unobservable characteristics jointly influences participation and outcome. By explicitly modeling the participation decision (estimating the first step equation pw?; = Xiy1 +Ziy2+u, generally using a Probit specification, where pwp stands for public work program and is a choice variable), it is possible to derive a variable that
can be used to control for the potential correlation between the residual of the outcome equation and that of the selection equation. By including this new variable alongside the observable variables (X¡) and the public work program dummy in the second step (outcome) equation, Heckman can generate unbiased estimates of att (att stands for average treatment effect on the treated). However, as for example with the iv approach, credible implementation requires the selection equation to contain an instrument, and the identification of a suitable instrument is often an obstacle to proper implementation (Heckman 1998; Vanden-berghe and Robin 2004).
We focus on the impact of the public work program upon the youths' chances of finding a job (measured as the probability of being employed one/two years after the program start and expressed as the differential being equal to participant minus non-participant mean). The principal obstacle in obtaining unbiased estimates of the impact of the public work program on the chances of finding a job is the problem of selection. Individuals opting to participate in the public work program may differ from those opting not to in many aspects, some of which may be unobserv-able. If these unobservable characteristics also affect the job prospects of the individual, the equation to be estimated can be mis-specified and the estimated coefficient biased.
The direction of the bias cannot be determined in advance. It is possible that individuals who enroll in the public work program are mostly those whose self-esteem and motivation are low. The chances of those individuals, ceteris paribus, moving from unemployment into work are lower than the chances of other unemployed persons. This produces a downward bias in the coefficient of participation in the estimation which does not correct for selectivity (the effect of the public work program is therefore underestimated). On the other hand, it is possible that the public work program attracts more motivated and agile individuals, which produces an upward bias in the coefficient of participation in the estimation which does not correct for selectivity (see, for example, Vodopivec
1999).
To rectify this selection bias we run a Heckman model, where in the first stage the equation of participation in the public work program is estimated. The outcome of that stage is a new variable (the inverse Mills ratio), to be used as one of the covariates in the second stage - in the estimation of the equation of exit from unemployment.
As already mentioned, the model depends on the presence of a proper
(1)	(2)	(3)	(4)	(5)
Gender	0.136	0.0216	6.33	0.000
Age	0.179	0.0724	2.47	0.013
Age2	-0.004	0.0015	-2.53	0.012
Education	0.055	0.0062	8.87	0.000
Unemployment before the program start	0.036	0.0015	20.14	0.000
notes Column headings are as follows: (1) variable, (2) marginal effect, (3) standard	
error, (4) z, (5) P.	
table 4 Probability of being employed	one year/two years after the program start
One year after the program start	Two years after the program start
att Std. err. p	att Std. err. p
0.352 0.073 0.000	0.151 0.075 0.046
instrument in the first equation (choice equation). We have opted for a dummy variable 'region,' equal to 1 (Osrednjeslovenska, Gorenjska, Notranjsko-Kraska, Goriska and the Obalno-Kraska Region) and to 0 otherwise. This variable fulfills the first condition to be an instrumental variable candidate (Wooldridge 2002): to be correlated with the endogenous variable or choice variable pwp, ceteris paribus. As can be seen in table 3, the (marginal) effect of region on the participants' chances of finding a job is strongly significant and evident.
Whereas the second condition for a variable to be an instrumental candidate (non-correlation with the residuals of the outcome equation) cannot be tested; which makes the choice of an instrument largely dependent on sensible arguments. We believe that there are plausible circumstances that would make 'region' a valid instrument. If participation in a public work program is more frequent in regions that are less-developed and have more rural areas, the risk of overestimating the effectiveness of the public work program is serious. Since the values of the coefficients in table 3 are not so big and the sign of coefficients is not the same across variables, we could consider that this somehow reduces the risk of overestimating the effectiveness of the public work program. But it could still be the case that the relative prevalence of the public work program according to region somehow reflects demand-side factors, in which case the endogeneity problem would remain.
Results, as presented in table 4, however, show the impact of the public work program on participants' chances of finding a job, especially in the short run. With Heckman's two-stage method the probability of being employed one year after the program start is approximately 35% and is high. This is due to the correction of selection, suggesting that estimates exaggerate the effectiveness of the public work program. When we measure the probability of being employed two years after the program start, Heckman's two-stage method generates results that are lower and could be related to stigmatization of the participants in the public work program. However, in the long run, the probability of being employed falls (but is not statistically significant).
Conclusion
The main purpose of this paper was to evaluate the Slovenian public work program for young unemployed persons using re-employment probabilities as a key measure of effectiveness. The age group examined was 20-29.
The Heckman selection estimator used in the empirical analysis is an extensively used method to control for selection on unobserved variables. It relies on the assumption that a specific distribution of the un-observable characteristics jointly influences participation and outcome. By explicitly modeling the participation decision it is possible to derive a variable that can be used to control for the potential correlation between the residual of the outcome equation and that of the selection equation. By including this new variable alongside the observable variables and the public work program dummy in the outcome equation, Heckman can generate unbiased estimates of att . The results suggest that the public work program has a positive effect on employment in the short run, whereas, in the long run, the probability of being employed significantly falls, but is still positive. Only in the short run are the results statistically significant at the 5% level. According to international studies the results for public work programs are not so obvious. Martin and Grubb (2001) show that numerous studies have not been successful in decreasing the number of unemployed. On the other hand, the public work program has had some positive effects in countries which were faced with serious problems in the labor market especially in the short run. In Poland the public work program is effective only for women, whereas men had serious problems in finding regular employment (Kluve, Lehmann, and Schmidt 1999). Reasons for such a situation could be stigmatization of
participants in the public work program as well as the institutional background of active employment policy where renewal of the right to unemployment benefit is possible. An alternative explanation is given in Puhani and Steiner (1997). They suggest that target groups are poorly defined. In Finland the results are not so straightforward (Hâmâlâinen and Ollikainen 2004), whereas in France the effects on re-employment probabilities are even negative (Brodaty, Crépon, and Fougére 2000).
The study presented in this paper contributes to the relatively thin literature on the topic of evaluation of the active employment policies in Slovenia (especially for the youth) and presents the use of recent data of relatively good quality.
References
Brodaty, T., B. Crépon, and D. Fougére. 2000. 'Using Matching Estimators to Evaluate Alternative Youth Employment Programs: Evidence from France.' cepr Discussion Papers 2604, Centre for Economic Policy Research, London.
Hâmâlâinen, K., and V. Ollikainen. 2004. Differential Effects of Active Labour Market Programmes in the Early Stages of Young People's Unemployment. Helsinki: Government Institute for Economic Research.
Heckman, J. 1998. 'The Economic Evaluation of Social Programs.' In Handbook of Econometrics, edited by J. Heckman and E. Leamer, 5:1673-94. Amsterdam: Elsevier.
Heckman, J., and J. Smith. 1995. 'Assessing the Case for Social Experiments.' Journal of Economic Perspectives 9 (2): 85-110.
Kluve, J., H. Lehmann, and C. Schmidt 1999. 'Active Labor Market Policies in Poland: Human Capital Enhancement.' Journal of Comparative Economics 27 (1): 61-89.
Lechner, M. 1999a. 'Earnings and Employment Effects of Continuous off-the-Job Training in East Germany After Unification.' Journal of Business and Economic Statistics 17:74-90.
-. 1999b. 'Identification and Estimation of Causal Effects of Multiple Treatments Under the Conditional Independence Assumption.' izA Discussion Papers 91, iza, Bonn.
Martin, J., and D. Grubb. 2001. 'What Works and for Whom: A Review of oecd Countries' Experiences With Active Labour Market Policies.' Swedish Economic Review 8 (2): 9-56.
Ministry of Labour, Family and Social Affairs. 2005. 'Annual Report of National Employment Service in Slovenia for the Year 2005.' Ministry of Labour, Family and Social Affairs, Ljubljana.
-. 2006. 'Annual Report of National Employment Service in Slovenia for the Year 2006.' Ministry of Labour, Family and Social Affairs, Ljubljana.
-. 2007. 'Annual Report of National Employment Service in Slovenia for the Year 2007.' Ministry of Labour, Family and Social Affairs, Ljubljana.
-. 2008. 'Annual Report of National Employment Service in Slovenia for the Year 2008.' Ministry of Labour, Family and Social Affairs, Ljubljana.
Puhani, P., and V. Steiner. 1997. 'The Effectiveness and Efficiency of Active Labour Market Policies in Poland' Empirica 24 (3): 209-31.
Vandenberghe, V., and S. Robin. 2004. 'Evaluating the Effectiveness of Private Education Across Countries: A Comparison of Methods.' Labour Economics 11 (4): 487-506.
Vodopivec, M. 1999. 'Does the Slovenian Public Work Program Increase Participants' Chances to find a Job?' Journal of Comparative Economics
27 (1): 113-30.
Wooldridge, J. M. 2002. Econometric Analysis of Cross-Section and Panel Data. Cambridge, ma: mit Press.