Informatica 32 (2008) 289-291 289
The Modelling of Manpower by Markov Chains - A Case Study of the Slovenian Armed Forces
Damjan Škulj, Vasja Vehovar and Darko Štamfelj University of Ljubljana, Faculty of Social Sciences
E-mail: damjan.skulj@fdv.uni-lj.si, vasja.vehovar@fdv.uni-lj.si darko.stamfelj@fdv.uni-lj.si Keywords: manpower planning, Markov chains, controlled Markov chains Received: August 9, 2007
The paper presents a case study of manpower planning for the Slovenian armed forces. First, we identified 120 types of military segments (including civil servants). Next, administrative data were used to estimate the transitions between these segments for the 2001-2005 period. Markov chain models were then applied and 5-, 10- and 20-year projections were calculated. Various discrepancies were discovered between the projected structures of the military segments and the official targets. Finally, we addressed two optimisation questions: 'Which transitions would lead to the desired structure in five years' time?' and 'Which transitions would sustain the desired structure in the long run?'. A heuristic modelling approach was applied here, i.e. we used a simulation model with a specific loss function. To perform feasible simulations of the probabilities in a 120 x 120 matrix, only those transitions were simulated where experts had previously estimated that real measures existed to provide potential change (recruitment policy, regulation of promotion, retirement strategy etc).
Povzetek: V članku je predstavljen primer načrtovanja kadrov v Slovenski vojski.
1 Introduction
Efficient manpower planning is a crucial task of managing large organisations such as transportation or industrial corporations, the state administration or military systems. All of these systems comprise many segments of employees with specific roles and job descriptions. Skills needed to perform assigned tasks are usually acquired through special training or long work experience. Both a shortfall and a surplus of skilled staff can be costly and very inefficient. To prevent such difficulties, the future needs of personnel have to be predicted well in advance, while corresponding strategies to achieve the desired structure must be adopted.
Knowledge about these processes is important for predicting the future development of the manpower structure in complex organisations. In large systems, such predictions are usually based on previous experience. However, knowledge gained from such experience is often difficult to apply without appropriate mathematical or statistical models and corresponding computational tools.
Pending on the goals, various mathematical models can be applied in manpower planning. Obviously, the problems cannot be fully addressed by only using tools of the spread-sheet type. The choice depends on many factors, such as the size of the system being analysed, available knowledge of the processes that govern the system structure's dynamics, the methods available to control the processes and the ability to predict the consequences of actions concerning regulations. Moreover, the choice of the appropriate model often depends on its complexity. While complex models can
supply very accurate results, they often require data that are not easy to collect, or parameters that may only be vaguely known, especially if a very large number of them have to be specified. Consequently, the reliability of the resulting outputs is then put in question. For very large systems, simpler and more robust models are therefore often a better choice. A good overview of the existing models used in workforce planning can be found in Wang (2005). For other references, also see Grinold and Marshall (1977), Price et al. (1980), Purkiss (1981), and Georgiu and Tsantas (2002).
The most basic information that can be used to model manpower dynamics is the rate of transitions between different segments of the system, i.e. the transition probabilities. Transitions are usually consequences of either promotions, transfers between assignments or wastage and input into the system. Often transitions are controlled by certain rules that govern the system and cannot be arbitrarily changed. If this is the case, planning has to be especially careful since slight changes in policies can have considerable consequences on the future development of the manpower structure.
In several cases, the models used to predict the future structure of a dynamic system are based on Markov chains and their derivatives, such as semi-Markov chains. Both are based on the assumption that the rules governing the system's manpower dynamics do not change very often and that future dynamics will follow patterns observed in the past. While classical Markov chains view segments as homogeneous, semi-Markov chains additionally involve the time a person has spent in a segment, of course at the cost of the model's simplicity and therefore the possibility to reliably estimate its parameters. A thorough description of many
290 Informatica 32 (2008) 289-297
D. Skulj et al.
variations of Markov and some other manpower planning models can be found in Bartholomew et al. (1991), Vassiliou (1998) or Vajda (1978). Besides Markov models, other approaches to the problem are also possible, such as models based on simulations or system dynamics models (Wang, 2005). Applications of manpower models used in the specific case of military manpower planning can be found in Jaquette et al. (1977), Murty et al. (1995), Smith and Bartholomew (1988).
In our particular case of modelling the structure of the Slovenian armed forces, the number of segments alone was relatively large. Together with other potential problems related to the data collection, these were the main reason against using the more complex semi-Markov chains. Moreover, transitions between segments are surprisingly complex where, besides recruitment, promotions and wastage from the system, many more transitions to other segments also occur such as transitions from military to administrative positions and vice-versa.
Models based on Markov processes can be divided into the following groups, depending on the level of the structural control i.e. the ability to attain and maintain the desired structure:
1	Descriptive-predictive models. This group is mainly concerned with the development and analysis of a time homogeneous Markov model whose parameters are often based on historical observations. It can be used to predict the behaviour of a system in time. The models in this group have no intention to search for any kind of optimal control, but only give descriptions and forecasts. Several models of this group can be found in Bartholomew et al. (1991) or Price et al. (1980). In this paper, we use such a model in our first part to make predictions on the future development of the system.
2	Control theory models-normative models (Markov decision processes). This group tries to find optimal set of policies in order to minimise certain loss functions such as the cost of recruiting new workers or maintaining the existing structure. The basis of these models is the work of Howard (1960) and they can be found in Grinold and Stanford (1974), Zanakis and Maret (1981), Lee et al. (2001) or a very general treatment can be found in Li et al. (2007). Our approach to searching for the transitions leading to a desirable structure could be regarded as belonging to this class, although its key concern is to find any satisfactory policy rather than to select a particular policy satisfying some additional optimality conditions.
While in the first part we use the classical descriptive-predictive model, the structural control part of the problem partly belongs to the class of control theory problems, although it has a very specific form and thus does not fit well in the context of controlled Markov chains or related models. The main specific point is that the choice of feasible policies is severely restricted by the Ministry of Defence and the problem, at least at this stage, is thus not in finding a policy that would satisfy
some additional optimality criterion, but rather in finding any policy that leads to the desired structure of the system. A similar problem was theoretically studied by Antoncic (1990), but subject to less severe restrictions. We thus claim that our particular problem in fact belongs somewhere between the first and second groups of the above models, and that it requires some kind of a specific treatment.
The problem we are dealing with thus mainly consists of two parts. In the first part we identify relevant segments and the transitions between them. The goal is to make predictions of the future sizes of the segments if the current transitions ruled the system's dynamics. This would likely be the case if no further regulations were implemented. In the second part we study the attainability and maintainability problem. We first identify transitions that could be regulated by the Ministry of Defence by setting appropriate measures and policies. Then we tried to modify them within the limits given by the Ministry in order to achieve the required structure. The problem is complex mainly because transitions that are not controllable represent a substantial part of the system's dynamics and are therefore the main cause of the large discrepancies between the projected and the required structures. Being ruled by mechanisms unknown to us, in the best case we can assume that they will remain roughly the same in the following years. To achieve the required structure, we must then appropriately set the controllable transitions.
The estimated probabilities of the uncontrollable transitions are used as expectations of future transition probabilities and thus we effectively assume a deterministic model with two classes of transitions -fixed and controllable. The only objective at this stage is to find a sufficient set of transitions that would lead to the required structure. Among the models found in the existing literature none of them seemed quite appropriate for resolving this particular problem, although a modification of some existing analytical model to fit our framework is a promising direction for our further work. However, at this stage the approach using computer-based simulations proved to be sufficiently effective in producing satisfactory results. The idea is therefore to simulate a large number of randomly generated scenarios and pick those yielding a satisfactory structure after a given period. However, the implementation depends to a large degree on the particular specifics.
The practical implementation of the approach to solving our problem also contains a web-based user application that was developed to allow non-mathematicians to change the parameters and analyse the consequences. The application's user interface is designed to be both user-friendly and flexible so that it allows practically unlimited possibilities in testing different scenarios. The results of testing are displayed simultaneously in real time. The application is available on-line and no programmes other than web browsers are needed to run it.
The prediction part of the model is thus made in a fully interactive manner; however, the structural control part is still too complex to be implemented by ordinary
THE MODELLING OF MANPOWER BY...
Informatica 32 (2008) 289-297 291
users, especially because of the computational complexity which requires several manual adjustments during optimisation. In addition, the process of preparing the data to estimate the parameters is technically very demanding because it requires the combining of several software tools and is therefore not available in an automated form.
The paper has the following structure: Section 2 contains a description of the method used with some mathematical background, Section 3 describes implementation of the method for calculating projections of the manpower structure of the Slovenian armed forces and the administration of the Ministry of Defence, while Section 4 presents the results. Conclusions are presented at the end.
2 Description of the method
2.1 Basic model
The model used for manpower planning in the Slovenian armed forces and the administration of the Ministry of Defence is based on Markov chains. The description of the mathematical background of this and related models can be found in Bartholomew et al. (1991), Vassiliou (1998), and Grinold and Marshall (1977). The usual assumption of those models is that the system modelled consists of clearly defined segments and that the transitions between them are time-homogeneous and independent of history. To satisfy these requirements, the system must be sufficiently large to diminish the effect of random variations in time allowing transitions to be analysed on the aggregate level.
Markov chains are used to model random processes evolving over time. A crucial assumption used is the Markov property which is best described by saying that the process is "memoryless", which means that its future states only depend on the present state rather than its history. Another assumption which is usually posed is that transition probabilities are time-homogeneous, which means that they are independent of time. Of course, these requirements are often not entirely fulfilled; however, omitting them would result in more general non-homogeneous models (see Vassliou (1981, 1998), Gerontidis(1994)) that require additional data to estimate the parameters.
A sequence of random variables X1, X2, ..., Xn , ... is a Markov chain if every variable can assume any value from a set S = (i1, s2, ..., sm}, whose elements are called states. A particular realisation of the process is then a sequence of values from S. Mathematically we can describe the Markov property by requiring that P(Xn+1 = Sj I Xn = Si,„„ X1 = sk) = P(Xn+1 = Sj I Xn = s) = p<n>ij, where n denotes the time points. This means that the probability that the process will be in state Sj at time n+1, given that it is in state si at time n, is p<n)ij. if in addition p<n)ij= pij, for every n, the chain is homogeneous because the transition probabilities do not depend on time. The matrix P=(pij),
whose entries are transition probabilities, is called a transition matrix.
In the particular case of manpower modelling, states are the segments of the system, and transition probabilities are understood as relative frequencies. Thus transition probability pj is interpreted as the ratio of persons in segment s¡ at the time n that will make a transition into state Sj by the time n+1. Let yn be a statistical distribution over the set S, thus, y"¡ is the number of persons in state s¡. On the assumption of transition matrix P, the distribution at the next time point is obtained by multiplying vector yn by P, yn+1 = ynP. Thus, given the initial distribution and transition matrix we can predict the distribution at the next time point. Consequently, by repeating the same argument predictions for all future time points are enabled. The time interval used in our model is one year and so if a person makes more than one transition within a year this is recorded as a single transition.
The actual model used is based on the model described in Bartholomew (1991), where in addition a vector of recruits r is added. The model also allows wastage w, which accounts for those people who leave the system. According to this model, the transition matrix P need not be stochastic but only substochastic, i.e. the sum of rows may be less than 1, where the difference is wastage w, thus,
m
wi =1 - Z Pj.
i=1
We assume a model with a constant recruitment vector since the only recruitment to the system is births in the general population. The actual recruitment to military segments is then modelled as transitions between general and military segments. Thus the model used takes the following general form:
Yn = Pyn-1 + r.
Hence, the military segments used in the model are regarded as a subset of segments of the general population. The reason for this is that new employees are recruited from general segments whose size then determines the number of possible new recruits every year. So, for instance, diminishing generations of school children due to the lower birth rates seen in recent years affect the number of potential candidates to enter military service. For this, we assume that the proportion of them interested in such a career is roughly constant.
The whole population of the inhabitants of Slovenia was divided into segments relevant to the model. General population is, for instance, divided into the following six segments: non-active population, which mainly consists of pre-school and primary school children; secondary school pupils, students, working people, unemployed, and retired persons. Military segments are treated separately from the general population because their relatively small size does not affect the dynamics of the general population.
The total number of military segments is 120, which makes the total number of segments equal to 126. Of course, this is not the only possibility to model our
292 Informatica 32 (2008) 289-297
D. Skulj et al.
segments. If the general segments were omitted, for instance, the same effect would be achieved by adequately modifying the recruitment vector. The changes in the size of the military segments are consequences of the following factors:
1	transitions between segments;
2	wastage from the system (retirement...);
3	input from the general population (recruitment); and
4	input to the general population, which is modelled as births.
Because of the relatively small size of the military segments, wastage from the army may be neglected and treated as wastage from the system even though actually there may also be transitions to other segments of the general population. The model assumes a constant level of annual input to the system, which we assume is the current birth rate in Slovenia. Slight deviations from this assumption in the following years would not affect the model substantially. The size of the whole population follows general trends in society.
Recruitment into military segments is modelled by transitions from the general to the military segments. The model assumes that those transitions are regulated by a set of rules that do not change often. We adopt the assumption that the transitions will remain similar to those observed in the past if regulations do not change. If, for instance, 10% of officers in a segment were promoted to a higher rank, we assume that this will continue in the following years. This assumption is, of course, sometimes questionable because it is rarely true that all members of a segment have the same probability of being promoted. However, the available data did not allow the modelling of any heterogeneity within the segments.
The model involves a mixture between the deterministic and stochastic approaches. We initially start with stochastic transitions estimated on the base of historical transitions. These are then mixed with those transitions that are possible to regulate and are thus deterministic. Another model with a mixture of stochastic and deterministic transitions can be found in Guerry (1993), where only wastage transition probabilities are modelled as stochastic while promotion and demotion probabilities are considered constant.
2.2 Attainability and maintainability
The next step after making predictions based on the model is to exercise structural control i.e. to find transitions leading to the desired structure after a certain time period, or at least close to it. Thus we are solving the attainability problem. Methods of solving attainability problems have been discussed in the literature (see e.g. Davies (1973) or Nilakantan and Raghavendra (2005)), but none of the approaches seems to completely resolve our specific problem, where the task is to be accomplished by resetting a certain subset of transitions within given constraints. This is because not all transitions can be controlled, while those that can be controlled are not allowed to be set entirely arbitrarily.
Therefore, it is crucial to determine which transitions can be changed and to what degree. Once this is determined, the transitions yielding the best possible result are to be found. We find a sufficient solution using computer simulations. The idea behind the simulations is to generate many possible scenarios by varying the allowed transitions within the allowed intervals and selecting the most suitable ones.
A sufficient solution, that is the set of transitions leading close enough to the aimed structure, is thus sought within the limits that can be implemented. We thus first had to find out which transitions are possible to regulate and to what extent. We acquired this information in co-operation with the Ministry of Defence, which provided us with a set of all transitions that can be controlled and typically the upper bound for each transition in terms of the maximum ratio of employees from each segment that can make a transition to another segment. Typically, the transitions that can be controlled are promotions. The transitions capable of sufficiently improving the resulting structure were then sought within these bounds.
About 500 such transitions were identified and lower and upper bounds for the transition probabilities were given. To find optimal transitions within these bounds, several approaches seem possible. One option is to decrease transitions to segments with an excess of units and increase those to segments with deficiencies. However, changing a single transition also affects other segments and the effects are difficult to predict beforehand. Thus it is difficult to determine which transition probability has to be changed to obtain the desired result. Also, such an iterative algorithm would to a large extent depend on various assumptions and features of a particular example. It would be reasonable to expect good results with such an approach in a simpler model where the usual transitions are promotions and recruitment only; however, this was not the case with our model. Because of the large number of other possible transitions this was unsuitable for our model. The other natural approach is to try to find an acceptable solution by systematically examining all possible combinations of transitions within a given range. However, a simple calculation showed that such an approach would be impossible to implement in such a large model.
A remaining possibility is Monte Carlo simulations where, instead of changing transitions systematically, this is done by randomly changing them within a particular amount. More precisely, the change of a particular transition was obtained as
pj" = pf {l+d1r1 )+d 2 r,
where the symbols denote
d1 and d2 are the factors denoting the maximum amount of the change and
r1 and r2 are random numbers uniformly distributed over the interval [ D0.5, 0.5].
Further, we required that all the changes sum to 0 so that wastage remains as estimated, since it is an uncontrollable parameter. Thus, in every step some
THE MODELLING OF MANPOWER BY...
Informatica 32 (2008) 289-297 293
transitions are picked and varied randomly, where d1 and d2 are parameters of the algorithm, as well as the number of chosen transitions. The process is iterative. If the matrix obtained in this way leads to a better result, this new matrix is used again in the next iteration. Some tuning is, of course, necessary such as determining the number of transitions to be changed at each time and the optimal values of the parameters, which depend on the particular instance. So far we have not done any systematic analysis of the effects of the parameters on the efficiency of the algorithm.
It soon turned out that it is impossible to obtain an exact solution leading to the satisfactory structure. This is due to many factors such as the set of transitions we are allowed to change and the amount of changes we are allowed to make for a particular transition and especially the short time available to reach the desired structure. Therefore, we have to satisfy ourselves with a solution sufficiently near to the required one. The concept of the best possible solution is thus not unique but depends on more or less subjective criteria. To illustrate this, suppose we have two segments with 50 and 500 units targeted, respectively. Is it better to end up with 60 units in the first and 490 in the second one, or is it better to have 50 units in the first one and 550 in the second? Clearly, this depends on what we are trying to achieve. If the main goal is that absolute deviations from the required sizes of the segments are as small as possible, the first scenario is better; however, those in the segment of size 50 are likely to have more specialised working tasks and therefore it is probably more important to have a better result in this segment. In the sense of relative deviations from the target, the second possibility is clearly better. Thus, another possibility would be to minimise relative deviations from the optimal structure. But this would clearly be again suboptimal. It is difficult to say in general where in the middle is "the right" criterion.
We have some general requirements of optimality, such as:
•	Big deviations in one segment are less satisfactory than a large number of smaller ones, even though they sum equally.
•	In larger segments bigger absolute deviations are allowed than in smaller ones.
A criterion satisfying the above requirements is expressed through a mathematical function. A function satisfying those criteria is, for instance, the following one:
z


Xo )2
Xo + C
In the above function the symbols denote the following:
•	x: the size of the ith segment as a result of a given scenario;
•	xi0: target size of the ith segment; and
•	C: a constant - varying this constant changes the relative importance of either segments with a smaller number of units or those with a larger
number of units. A value that turns out as balanced in our particular case is C=50.
The above loss function may be understood as a distance function measuring the distance between the solution achieved in a given scenario and the optimal solution. The algorithm for attaining the desired manpower structure thus tries to minimise this distance.
Once the required structure is attained, the task is to find a strategy to maintain the obtained structure. Our goal is to solve it in a similar manner as the problem of attainability; that is, by setting the set of controllable transition to values that would, at least approximately, maintain the obtained required structure. Here too, simulations do the task sufficiently. The only modification from solving the previous task is that we set the initial and the target structure to be the same and then, using simulations, try to find transitions that preserve the structure. Effectively, we are thus seeking a transition matrix within an allowed set of matrices whose stationary distribution is the required distribution. We therefore call the resulting transitions stationary transitions.
3 Preparation of the data
3.1	Input data and population segments
The first phase of the analysis included the identification of segments from the available data. The military segments were identified on the basis of the administrative titles of employees, by regarding employees holding the same administrative title as members of the same military segment. The Ministry provided us with anonymised data (such as identification number, period of employment, administrative title, education etc.) for all employees in the Slovenian armed forces and the administration of the Ministry of Defence in the period between 1 January 1997 and 31 August 2006.
The most demanding task was estimating the transitions between the segments. They were calculated on the basis of past transitions identified from the data on employees in the Slovenian armed forces. This was possible thanks to the accurate manpower data of the Ministry of Defence.
In the data, every employee's transition from one segment to another and every extension of the employment contract in the same segment was registered as a new entry with the same identification number, and a new initial and final date of their employment contract. From that, transitions between various segments were calculated for every year in the period. Also, the number of people in each segment as at 30 June 2007 was obtained as a basis for calculating projections (as the initial manpower structure).
3.2	Standardisation of the data
The model requires identifying the employees' administrative titles (i.e. affiliation to military segments) for each year in the given period for which data are
294 Informatica 32 (2008) 289-297
D. Skulj et al.
available. An employee's administrative title usually does not change more than once a year. Therefore, the administrative titles were identified on 31 December for every year in the period between 1 January 1997 and 31 August 2006. The data enable the identification of administrative titles on this date for the period between 1997 and 2005.
However, the data did not enable us to identify from which of the six segments of the general population employees of the Ministry of Defence came from. It was also impossible to identify into which of the segments of the general population they went after they left the Ministry. For this reason, another segment called "outside" was introduced, which stands for the entire general population. Thus, transitions from the "outside" to military segments therefore denote new employments, while transitions to the segment "outside" from other segments denote deaths, retirements or terminations of an employment contract. Consequently, transitions between military segments and specific segments of the general population were estimated so that their cumulative effect at the aggregate level was equal to transitions between military segments and the segment "outside".
In the next phase, transitions between segments of every individual employee in the Slovenian armed forces and the administration of the Ministry of Defence were identified on the basis of comparisons between his or her segment in two consecutive years on 31 December.
Yet it is possible that a person made a transition more than once a year. In that case, all those transitions were only registered as a transition between the starting and ending segment since the person's segments were only checked on 31 December.
3.3 Estimating the transition matrix
For each year, the number of people in every segment and the number of transitions from one segment to another were counted. The number of transitions from a particular segment to another one was divided by the number of people in the starting segment. This gave us the transition probabilities between the segments for each year in the period. The average transition probability between two segments in the period was calculated as a weighted mean:
I'

I
i=1

where x denotes the transition probability in a certain year, and wt the number of people at the start of the same year.
However, the transitions have changed substantially in recent years due to changes in legislation and the process of professionalising the Slovenian armed forces. Consequently, some of the transitions that were possible in the past are no longer possible today, and the transition probabilities have also changed. For those reasons only the data for the four most recent years, that is for the
period from 31 December 2001 to 31 December 2005, were used to calculate the average transitions. Also in order to avoid such problems, the Ministry of Defence reviewed the transition matrix and marked those transitions that are no longer possible. The latter were then assigned the value 0 in the transition matrix, while the transitions on the diagonal (standing for the share of people who stay in the same segment each year) were correspondingly corrected so as to keep the same total proportion of people remaining in the system. The final result was transition matrix P with entries pj which summarise the proportion of employees of segment i that yearly make the transition to segment j.
The next step is to identify those transitions that can be regulated by the Ministry of Defence's personnel departments. By simulating changes in those transitions it is possible to identify a transition matrix that leads to the desired manpower structure in a certain period. The transitions that can be regulated are, for example, promotions and new employments. What we want to know, for instance, is the optimal share of people that can be promoted from one segment to another, and the optimal number of people that can be employed within the segment each year.
4 Results
4.1 Basic model
The results of our simulations show how the manpower structure will change over the course of time after 2006 if the future transitions are equal to the average transitions in the period from 31 December 2001 to 31 December 2005. A comparison of the projected manpower structure in 2010 and the desired structure also shows which military segments will face manpower shortages or surpluses.
The projected number of people in seven selected military segments (out of 120) in the period from 200720011 and in 2027 according to a continuation of the transitions observed in the period from 31 December 2001 to 31 December 2005, the desired manpower structure in 2010 and differences between the desired and projected structure in 20101 are presented in Table 1.
As can be seen from Table 1, in this scenario the manpower projection for 2010 differs considerably from the desired manpower structure: some military segments (like S1, S3, S4 and S5) will face a manpower shortage, while others (like S2, S6 and S7) will face manpower surpluses. The deviations are large in both absolute and relative numbers (i.e. an absolute deviation in relation to the desired number of employees in the selected military segment).
The table only contains numbers for the selected seven military segments since the data for all of the segments cannot be published due to data protection reasons. The names of the segments were also encoded for that reason.
THE MODELLING OF MANPOWER BY...	Informatica 32 (2008) 289-297 295
Table 1: Manpower structure for seven selected segments by the continuation of average transitions 2001-05
Year	Sg1	Sg2	Sg3	Sg4	Sg5	Sg6	Sg7
2007	51	944	700	125	33	488	635
2008	48	966	594	108	39	520	657
2009	46	973	505	97	46	551	692
2010	45	968	430	89	54	584	735
2011	43	953	368	84	61	619	781
2027	25	495	135	133	257	1267	1214
Desired 2010	67	853	760	145	81	573	642
Difference (proj. 2010 - desired 2010)	-22	115	-330	-56	-27	11	93
Difference in %	-33%	13%	-43%	-39%	-33%	2%	14
4.2 Attainability
Structural control was exercised on the basis of a transition matrix containing average transitions between 31 December 2001 and 31 December 2005, and the manpower structure of the Slovenian armed forces and
attaining the desired manpower structure was implemented in the following way. For each iteration five transitions were randomly chosen among the controllable transition to be modified using equation (1), where optimal values of r1 and r2 turn out to be 0.05 and 0.01, respectively. If the resulting matrix leads to a more desirable structure, the following iteration uses the new
Table 2: Attainability of manpower structure for seven selected segments (optimised transitions)
Year	Sg1	Sg2	Sg3	Sg4	Sg5	Sg6	Sg7
2007	51	944	700	125	33	488	635
2008	56	909	707	123	44	511	595
2009	59	876	713	127	54	528	588
2010	60	845	720	133	66	543	601
2011	61	816	728	140	77	558	624
2027	42	519	1015	228	294	890	900
Desired 2010	67	853	760	145	81	573	642
Difference (proj. 2011 - desired 2010)	-6	-37	-32	-5	-4	-15	-18
Difference in %	-9%	-4%	-4%	-3%	-5%	-3%	-3%
the administration of the Ministry of Defence on 30 June 2007. The target year for achieving the desired structure of 2010 specified by the Ministry of Defence was reset to 2011. By doing so, an additional year was added. Otherwise, the period for attaining the desired structure would have been too short. The improvements due to the structural control are measured in terms of the change in the loss function described in Section 2.2. It turned out that about 1,000 transitions could occur and 500 of those are controllable.
If the transitions in the period from 2007 to 2011 were on average the same as those in the period from 31 December 2001 to 31 December of 2005, the value of the loss function would be equal to 3,1332. The method of
Smaller values are better.
matrix as the initial one. After 200,000 iterations the value of the loss function stabilises at 283, which clearly indicates a significant improvement. Increasing the number of iterations does not yield a much greater improvement.
The projected number of people in selected military segments in the period from 2007 to 2011 and in 2027 by the continuation of transitions for attaining the desired structure, the desired structure in 2010, and differences between the desired structure in 2010 and the projected structure in 2011 are presented in Table 2.
Small differences between the desired structure and that achieved with the optimised transitions are also clearly seen in Table 2. The deviations are small in both absolute and relative numbers: none of the selected military segments will have a manpower shortage of
2
296 Informatica 32 (2008) 289-297
D. Skulj et al.
more than 37 people (S2) or 9% (S1), respectively. In Table 1, on the other hand, the projected number of employees in the segment (S3) with the largest deviation from the desired manpower structure faces a manpower shortage of 330 people or 43% of the desired number of employees.
4.3 Maintainability
The solution to the attainability problem described above provides a set of transitions that lead to a structure sufficiently close to the desired structure in a few years. It must be stressed that the current structure is quite far from optimal, which mainly reflects the fact that it transformed substantially in recent years due to the change from a conscript to a professional service. Therefore, to achieve the desired structure in only a few years the needs for recruitment and promotions tend to be higher than needed to merely sustain the structure once the desired structure is reached.
The next important step in exercising structural control is to find the transitions needed to maintain the desired structure after it has been reached. The method of finding such transitions is substantially the same as in the previous case. We take as an initial distribution the desired distribution and identify transitions, using simulations, so that they would preserve the structure. Even though the problem seems easier than the previous
one, it turns out that it is impossible to sustain the structure in all of its parts. This indicates that some transitions prevent the system's stability. Identifying those transitions and examining ways to improve the stability is one of the tasks that still has to be accomplished.
The projected number of people in the selected military segments in the period from 2011 to 2015 and in 2031 according to the continuation of stationary transitions, the desired manpower structure in 2010 and differences between the desired structure in 2010 and the projected structure in 2031 are presented in Table 3.
Unlike in Tables 1 and 2, the projected manpower structure presented in Table 3 was calculated using the projected manpower structure for 2011 in Table 2 as the
initial structure instead of the manpower structure on 30 June 2007. As can be seen, the projected manpower structure of the selected segments only slightly deviates from the desired one, even in 2031: the greatest manpower shortage does not exceed 26 people or 5%, respectively (S6).
5 Conclusion
In this paper we presented a case study of applying a Markov chain model in the Slovenian armed forces. First, extensive administrative effort was needed to obtain data on the individual status of all employees for the 2001-2005 period. We needed an exact assignment (of each person and for each year) to one of 120 key military segments (soldier, lieutenant etc.). This then formed the basis for calculating transition probabilities in a 120 x 120 matrix. The Markov chain model was then applied to these data. Considerable gaps were found in the projected sizes of the segments compared to the official targets. Of course, experts and decision-makers were roughly aware of these discrepancies but the results of the modelling provided much more explicit and elaborated evidence of the problems related to future trends.
However, the Markov chain model itself could not provide an answer as to how to achieve the desired manpower structure. This problem was successfully
addressed here by simulations. The simulation algorithm selected a solution closest to the target structure from a large number of computer-generated scenarios. A specific loss function was developed for this problem.
The approach described in this paper can be upgraded in several ways. For planning a smaller number of selected segments a semi-Markov model could be developed in which the "age" of the units in the segments (i.e. the time a person is employed in the segment) is considered in calculations of more accurate transition probabilities. Another direction is the implementation of a time non-homogeneous model instead of a time homogeneous model; this means that the transition probabilities could vary over time.
Table 3: Maintainability of manpower structure in seven selected segments in a long run
Year	Sg1	Sg2	Sg3	Sg4	Sg5	Sg6	Sg7
2011	61	816	728	140	77	558	624
2012	61	816	728	140	77	556	621
2013	61	816	728	140	77	555	617
2014	62	816	728	139	77	553	615
2015	62	816	728	139	77	551	612
2031	62	813	716	135	77	532	599
Desired 2010	67	853	760	145	81	573	642
Difference (proj. 2031 - proj. 2011)	1	-3	-12	-5	0	-26	-25
Difference in %	2%	0%	-2%	-4%	0%	-5%	-4%
THE MODELLING OF MANPOWER BY...
Informatica 32 (2008) 289-297 297
The accompanying web application that automated the above described calculation of manpower projections also demonstrated to be a very useful tool. The application used the existing manpower structure and some type of (assumed) transition matrix to calculate statistics and trends for past years and projections for the future.
Acknowledgement
We would like to thank the Slovenian Army for supporting this project and providing all the required help. We would also like to thank the four anonymous referees for their many useful suggestions and comments.
References
[1]	Antoncic, V.: Enodobni planski racun za velikost in sestavo, Metodoloski zvezki, Vol. 7, 1990, p. 118.
[2]	Bartholomew, D.J., Forbes, A.F., McClean, S.I., Statistical techniques for manpower planning, John Wiley & Sons, 1991.
[3]	Davies, G. S.: Structural Control in a Graded Manpower System, Management Science, 1973, Vol. 20(1), Theory Series, p. 76.
[4]	Georgiou, A.C., Tsantas, N.: Modelling recruitment training in mathematical human resource planning, Applied Stochastic Models in Business and Industry, John Wiley & Sons, Ltd. 2002, Vol. 18 (1), p. 53.
[5]	Grinold, R.C., Stanford, R.E.: Optimal Control of Graded Manpower System, Management Science, 1974, Vol. 20 (8), Application Series, p. 1201.
[6]	Grinold, R.C., Marshall, K.T.: Manpower Planning Models, North-Holland Pub Co, 1977.
[7]	Gerontidis, I.I.: Stochastic Equilibria in Nonhomogeneous Markov Population Replacement Processes, Mathematics of Operations Research, 1994, Vol. 19(1), p. 192.
[8]	Guerry, M.A.: The probability of attaining a structure in a partially stochastic model, Advances in Applied Probability, 1993. Vol. 25(4), p. 818.
[9]	Howard, R.A.: Dynamic Programming and Markov Processes, The Technology Press of MIT and John Wiley & Sons, Inc., New York, London, 1960.
[10]	Jaquette, D.L., Nelson G.R., Smith R.J., An Analytic Review of Personnel Models in the Department of Defense. 1977, RAND.
[11]	Lee, H.W.J., Cai, X.Q., Teo, K.L.: An Optimal Control Approach to Manpower Planning Problem,
Mathematical Problems in engineering, 2001, 7, p. 155.
[12]	Li, Y., Chen, J., Cai X.: An integrated staff-sizing approach considering feasibility of scheduling decision, Annals of Operations Research, 2007, Vol. 155 (1), p. 361.
[13]	Murty, K.G., Djang P., Butler W., Laferriere R., The Army Training Mix Model. Journal of the Operational Research Society, 1995. Vol. 46(3): p. 294.
[14]	Nilakantan, K., Raghavendra, B. G.: Control aspects in proportionality Markov manpower systems, Applied Mathematical Modelling, 2005, Vol. 29(1): p. 85.
[15]	Nilakantan, K., Raghavendra, B. G.: Length of service and age characteristics in proportionality Markov manpower systems, IMA Journal Management Mathematics, 2008, Vol. 19, p. 245.
[16]	Price, W.L., Martel A., Lewis K.A.: A Review of Mathematical Models in Human	Resource Planning. OMEGA, 1980. 8(6): p. 639.
[17]	Purkiss, C.: Corporate Manpower Planning: a review of models. European Journal of Operational Research, 1981. 8(4): p. 315.
[18]	Smith, A.R., Bartholomew, D.J.: Manpower Planning in the United Kingdom: An Historical Review, Journal of the Operational Research Society, 1988, Vol. 39(3), p. 235.
[19]	Vajda S.: Mathematics of Manpower Planning, John Wiley & Sons, Chichester, 1978.
[20]	Vassiliou, P.-C. G.: On the Limiting Behaviour of a Non-homogeneous Markov Chain Model in Manpower Systems. Biometrika, 1981, Vol. 68, p. 557.
[21]	Vassiliou, P.-C.G.: The Evolution of the Theory of Non-Homogeneous Markov Systems. Applied Stochastic Models and Data Analysis, 1998. 13: p. 159.
[22]	Wang, J.: A Review of Operations Research Applications in Workforce Planning and Potential Modelling of Military Training, DSTO Systems Sciences Laboratory, Edinburgh Australia, 2005.
[23]	Zanakis, S.H., Maret M.W., A Markovian Goal Programming Approach to Aggregate Manpower Planning. Journal of the Operational Research Society, 1981. 32(1): p. 55.