THE USE OF DIFFERENT TOBACCO AND RELATED PRODUCTS, 
WITH AND WITHOUT FLAVOURS, AMONG 15-YEAR-OLDS IN SLOVENIA
Helena KOPRIVNIKAR1*, Tina ZUPANIC1
1National Institute of Public Health, Trubarjeva 2, 1000 Ljubljana, Slovenia
Received: Jan 12, 2016
Accepted: Aug 29, 2016
Original scientific article
Background. Different tobacco and related products, like waterpipe, e-cigarettes, are gaining popularity 
among adolescents in different countries; the data for Slovenia is currently limited. The purpose of this paper 
is to present the latest data on the use of different tobacco and related products, with or without flavours, 
among 15-year old students in Slovenia.
Methods. Data for 15-year-old students were drawn from 2014 Slovene database of the cross-national survey 
Health Behaviour in School-Aged Children. The survey sample was selected with stratified two-stage sampling 
method. The survey was performed in schools with a self-administered web questionnaire. The survey 
questionnaire included international compulsory, selected optional and national questions, all on a variety of 
topics related to youth health behaviour.
Results. 25.2% of 15-year-old students reported current (past 30-day) use of any tobacco related product, 
mainly cigarettes (93.1% of users), followed by waterpipe (11.7%) and cigars, cigarillos and pipes (9.4%). 
Exclusive use of unconventional products is low (5.1% of users). 85.5% of users of any product used one product, 
48.5% of users used products with flavours. The use of different products, one or more products, and flavoured 
products was related to gender.
Conclusion. A comprehensive tobacco control and prevention must address all tobacco and related products.
Uvod. Med mladostniki v tujini postajajo vedno bolj popularni različni tobačni in s tobakom povezani izdelki, 
kot so vodne pipe ali elektronske cigarete, za Slovenijo pa imamo o tem na voljo omejene podatke. Namen 
prispevka je prikaz najnovejših podatkov o uporabi tobačnih in s tobakom povezanih izdelkov z okusom ali brez 
njega med všolanimi petnajstletniki v Sloveniji.
Metode. Uporabili smo podatke za všolane petnajstletnike iz slovenske podatkovne baze presečne raziskave Z 
zdravjem povezana vedenja v šolskem obdobju 2014. Za oblikovanje vzorca je bilo uporabljeno dvostopenjsko 
stratificirano vzorčenje. Anketiranje je potekalo v šolah s pomočjo spletne ankete. Vprašalnik obsega niz 
mednarodnih obveznih in opcijskih vprašanj ter nekaj nacionalnih vprašanj o različnih področjih z zdravjem 
povezanih vedenj mladostnikov.
Rezultati. O uporabi vsaj enega tobačnega ali s tobakom povezanega izdelka v zadnjih 30 dneh je poročalo 
25,2% všolanih petnajstletnikov, največ o uporabi cigaret (93,1% uporabnikov), vodnih pip (11,7%) ter cigar, 
cigarilosov in pip (9,4%). Izključna uporaba nekonvencionalnih izdelkov je nizka (5,1% uporabnikov). 85,5% 
uporabnikov kateregakoli izdelka je uporabljalo en izdelek, 48,5% izdelke z dodanimi okusi. Uporaba različnih 
tobačnih in s tobakom povezanih izdelkov, enega ali več izdelkov in izdelkov z dodanimi okusi se je razlikovala 
med spoloma.
Zaključek. Celovit program nadzora nad tobakom in preprečevanje rabe tobaka morata vključevati vse različne 
tobačne in s tobakom povezane izdelke.
ABSTRACT
Keywords: 
smoking, tobacco, 
adolescents
IZVLEČEK
Ključne besede: 
kajenje, tobak, 
adolescenti
*Corresponding author: Tel: ++ 386 1 2441 469; E-mail: helena.koprivnikar@nijz.si
10.1515/sjph-2017-0010 Zdr Varst 2017; 56(2): 74-81
74
UPORABA RAZLIČNIH TOBAČNIH IN S TOBAKOM POVEZANIH IZDELKOV 
Z OKUSOM ALI BREZ NJEGA MED PETNAJSTLETNIKI V SLOVENIJI
Koprivnikar H, Zupanic T. The use of different tobacco and related products, with and without flavours, among 15-year-olds in Slovenia. Zdr Varst 2017; 56(2): 74-81
© Nacionalni inštitut za javno zdravje, Slovenija. 
1 INTRODUCTION
Tobacco use is the leading preventable cause of death in 
Slovenia (1), and one of the leading risk factors in terms 
of attributable DALYs (2). Smoking is initiated mainly by 
adolescents; 84.7% of ever smokers in Slovenia, aged 35-
44, first smoked at 19 years of age or less (3). Although 
data on adolescent prevalence and trends of tobacco 
use in Slovenia is available (4-6), much less is known 
about adolescents’ use of different types of tobacco and 
related products. Different patterns and trends of use 
are present in different countries; in some the situation 
changed extensively in only a few years, which shows the 
importance of a regular surveillance of the use of these 
products (7, 8). In the United States of America (the USA), 
adolescents’ use of electronic cigarettes (e-cigarettes) 
and waterpipe surpassed cigarette use. In 2014, those 
aged 14-18 mostly used e-cigarettes (13.4%) and waterpipe 
(9.4%), followed by cigarettes (9.2%) and cigars (8.2%). 
E-cigarette use tripled and waterpipe use almost doubled 
from 2013 to 2014, while the use of cigarettes and cigars 
equivalently decreased (7). In Canada, where e-cigarettes 
and liquids with nicotine are not approved for sale, 2.6% 
of 15-19-year-olds reported current use of e-cigarettes 
in 2013 (9). In Poland, 29.9% of 15-19-year-olds used 
e-cigarettes in 2013-2014, the share increased from 5.5% in 
2010-2011 (8). In Finland, 2.0% of 12-18-year-olds in 2013 
reported the use of e-cigarettes more than 20 times in 
life (10), while in the United Kingdom, 3.0% of 11-16-year-
olds reported at least monthly use of e-cigarettes in 2014 
(11). Waterpipe is also gaining popularity among youth 
(7, 12). Current waterpipe use was reported by 10.2% of 
15-year-olds in Sweden in 2011 (13), 34.9% of 15-year-olds 
in Estonia in 2006 (14), 22.4% of 13-15-year-olds in Czech 
Republic, and 13.8% of 13-15-year-olds in Slovakia in 2011 
(15). Hand-rolled cigarettes are another popular product 
among youth (16, 17). In Croatia, 13% of 13-15-year-olds 
smoked hand-rolled cigarettes in 2011, in Slovakia 6.2% 
in 2011, and in Italy 8.2% in 2010 (15). In Canada in 2008-
2009, 24.2% of current smokers, aged 14-18 years, smoked 
hand-rolled cigarettes (16). Flavours in tobacco and 
related products increase the appeal of these products to 
youth (18). In Poland in 2009-2010, 12.3% of 15-19-year-
olds used flavoured cigarettes (19). In Canada, 50.0% of 
14-18-year-old users of any tobacco product reported the 
use of products with flavours in 2012/13, and in the USA, 
73.0% in 2014 (both countries have already introduced 
bans on flavours (excluding menthol) in selected tobacco 
products) (20, 21). Limited data also shows high shares 
of adolescents reporting current use of more than one 
tobacco or related product. In the USA in 2014, 51.3% of 
14-18-years-old users of any product used more than one 
product (7).
The latest data on the use of different tobacco and 
related products among Slovene youth is available from 
2011/2012 (15, 17). The most used products in 15-24 age 
group are manufactured cigarettes, hand-rolled cigarettes 
and waterpipe (92.7%, 18.0% and 3.9% of current users, 
respectively) (17). Among 13-15-year-olds, 23.8% used 
manufactured cigarettes, 10.9% hand-rolled cigarettes, 
9.8% waterpipe, 7.9% cigars, mini cigars or cigarillos, 
and 3.6% chewing tobacco or snus, while the study gives 
no data on the use of e-cigarettes (15). We found no 
published data on the use of flavoured products and more 
than one tobacco and related product among adolescents 
in Slovenia. The aim of this article is to present the most 
recent data on the use of different tobacco and related 
products (with and without flavours), and multiple 
tobacco and related products among 15-year-old students 
in Slovenia, and to explore differences in the use of these 
products by gender, age at smoking initiation, and the 
frequency of current smoking.
2 METHODS
Data for 15-year-old students were drawn from 2014 
Slovene database of the cross-sectional survey Health 
Behaviour in School-Aged Children, carried out on a 
nationally representative sample of 11-, 13- and 15-year-
old students attending schools. The basis for the sample 
were the data from the Ministry of education, science and 
sport about the enrolment and number of classes for the 
school year 2013/2014. The sample was drawn from the 
list of all relevant classes. The primary sampling unit was 
school class; classes were randomly selected. Stratified 
two-stage sampling was used. At the first stage, primary 
and secondary schools were selected, and at the second 
stage, among secondary schools, classes within different 
school programmes were selected (grammar school, 
4-year technical school, middle vocational school and 
lower vocational school). The survey was performed in 
schools with a self-administered web questionnaire from 
3rd to 14th February 2014. The final response rate (based 
on selected classes) was 92.7%. The sampling procedure 
met all international requirements on representativeness, 
so we did not decide to weight the data, despite minor 
differences in gender distribution of the sample in 
comparison to population gender distribution at the 
beginning of the school year. Survey methodology in detail 
is available elsewhere (4). The survey questionnaire 
included international compulsory, selected optional 
and national questions, all on a variety of topics related 
to youth health behaviour (questions on different 
demographic factors, eating habits, weight control and 
body image, physical activity, sedentary behaviour, risk 
behaviour, sexual health, violence and injuries, family 
culture, peer culture, health and wellbeing, school and 
social inequality), including tobacco and related products 
use. Questions on the use of different tobacco and related 
products, flavoured products, the number of cigarettes 
10.1515/sjph-2017-0010 Zdr Varst 2017; 56(2): 74-81
75
10.1515/sjph-2017-0010 Zdr Varst 2017; 56(2): 74-81
76
To estimate the number of tobacco and related products 
users among 15-year-old students in Slovenia, we used the 
data from Statistical office on the number of 15-year-old 
inhabitants of Slovenia in the first half of 2014 (18.532 
15-year-olds).
All analyses were conducted in IBM SPSS 21. We used the 
Chi-square test (X2) and correlation matrix to examine the 
association and correlation between selected variables; 
the significance level used was 0.05. If estimations are of 
lower accuracy, they are labelled with letters M (standard 
error of share on interval from 5% to 15%) and should be 
used with caution.
3 RESULTS
Our final sample comprised of 4.997 11-, 13- and 
15-year-old students and was representative of the 
whole population of 11-, 13- and 15-year-old students in 
Slovenia. The final response rate was 92.7%. Both genders 
were equally represented (51.0% female, 49.0% male). In 
our final sample, 68.3% of all students attended primary, 
12.9% grammar, 12.1% technical and 6.7% vocational 
schools. The sample of 15-year-old students, analysed 
in this study, consisted of 1.615 pupils, 46.1% male and 
53.9% female. 
Any tobacco or related product use during the last 30 days 
was reported by 25.2% of 15-year-old students (Table 1). 
The use of the majority of different types of tobacco and 
related products was related to gender.
smoked in the last 30 days, and age at first cigarette 
smoking were answered only by 15-year-old students. 
The use of different tobacco and related products was 
assessed by a national question, asking about the use of 
different products during the last 30 days (manufactured 
cigarettes, hand-rolled cigarettes, cigarillos, cigars, 
pipes, smokeless tobacco, e-cigarettes, waterpipe, 
others). For a part of our analysis, we merged products 
in two categories, namely conventional (cigarettes, 
cigarillos, cigars, pipes) and unconventional (smokeless 
tobacco, e-cigarettes, waterpipe, others) products. 
The use of flavoured products was assessed by a national 
question, asking about the use of products with flavours 
during the last 30 days (yes, I used only products with 
flavours; yes, I used products with and without flavours; 
no; I do not know). The share of those using flavoured 
products was calculated by combining the first two 
answers.
The number of cigarettes smoked during the last 30 days 
was assessed by an optional question (none; less than 
one a week; less than one a day; 1-5 a day; 6-10 a day; 
11-20 a day; more than 20 a day). The age at cigarette 
smoking initiation (more than one puff) was assessed by 
a compulsory question (never; 11 or earlier; 12; 13; 14; 
15 and 16 years of age or more). Both questions were 
used or adapted from the European School Survey Project 
on Alcohol and Other Drugs (ESPAD) survey. The data on 
current tobacco smoking was acquired by a compulsory 
question (every day; at least once a week, but not every 
day; less than once a week; I do not smoke). 
Table 1. The use of different tobacco and related products during the last 30 days among all 15-year-old students (n=1558) and users 
of any product (n=392).
*the sum of shares for each product exceeds the share of users of any product as one person can use more than one product
**shares for each product together exceed 100% as one person can use more than one product 
any product
manufactured cigarettes
hand-rolled cigarettes
waterpipe
cigarillos, cigars, pipes
e-cigarettes
smokeless tobacco
other
392
300
162
46
37
14
11
6
182
118
94
31
25
11
8
4
210
182
68
15
12
3
3
2
25.2%
19.3%
10.4%
3.0%
2.4%
0.9%
0.7%
0.4%
25.6%
16.6%
13.2%
4.4%
3.5%
1.5%
1.1%
0.6%
24.8%
21.5%
8.0%
1.8%
1.4%
0.4%
0.4%
0.2%
100.0%
76.5%
41.3%
11.7%
9.4%
3.6%
2.8%
1.5%
100.0%
64.8%
51.6%
17.0%
13.7%
6.0%
4.4%
2.2%
100.0%
86.7%
32.4%
7.1%
5.7%
1.4%
1.4%
1.0%
0,869
0.016
0.001
0.003
0.007
0.013
0.070
0.299
4670
3577
1927
556
445
167
130
74
n n n% 
among 
all 15-
year-old 
stu-
dents*
% 
among 
all 15-
year-old
male 
stu-
dents*
% 
among 
all 15-
year-old
female 
stu-
dents*
% 
among 
users**
% 
among 
users**
% 
among 
users**
Girls
p 
(diff. 
among 
genders 
for all 
15-year-old 
students)
Estimated 
number 
of users 
among 
15-year-olds 
of both  
genders
BoysBoth genders
10.1515/sjph-2017-0010 Zdr Varst 2017; 56(2): 74-81
77
Regardless of the type, cigarettes were used by 93.1% of 
all 15-year-old students that used any tobacco or related 
product, by 87.9% of boys and 93.3% of girls. The number 
of cigarettes smoked during the last 30 days (Figure 1) 
was not related to gender (p=0.455).
Table 2.
Table 3.
Figure 1.
The use of one or more tobacco and related products during the last 30 days among all 15-year-old students (n=1558) and 
users of any product (n=392).
The use of one or more tobacco or related products 
among 15-year-old students who are current smokers 
by the frequency of smoking tobacco.
The number of cigarettes smoked during the last 30 
days among 15-year-old students who are cigarette 
smokers (n=340).
one
two or more 
All
one 
two or more
Boys
one 
two or more
Girls
one 
two or more
335
57
144
38
191 
19 
21.5%
3.7%
20.3%
5.4%
22.5%
2.2%
85.5%
14.5%
168
29
76
22
92
7
 
85.3%
14.7%
77.6%
22.4%
92.9%
7.1%
74
15
27
9
47
6
 
83.1%
16.9%
75.0%M
25.0%M
88.7%
11.3%
0.643
0.756
0.372
79.1%
20.9%
91.0%
9.0%
3984
686
n n n% 
among all 
15-
year-old 
stu-dents*
% 
among all 
15-
year-old
male stu-
dents*
% 
among all 
15-
year-old
female 
stu-dents*
% 
among 
users**
% 
among 
users**
% 
among 
users**
Girls
Estimated 
number 
of users 
among 
15-year-olds 
of both  
genders
BoysBoth genders
p
Smoking tobacco 
weekly or more 
often
n n% %
Smoking tobacco  
less than weekly
Correlation matrix indicates weak negative correlation 
between the age at smoking initiation and the number 
of cigarettes smoked during the last 30 days (-0.219, 
p=0.01). It indicates that the number of cigarettes smoked 
during the last 30 days decreases by increasing the age at 
smoking initiation. 
Weekly or daily smoking is not related to the type 
of cigarettes used when comparing those using only 
manufactured cigarettes with those using hand-rolled 
cigarettes only or in combination with manufactured 
cigarettes (weekly smoking: p=0.052; daily smoking: 
p=0.092).
When considering manufactured and hand-rolled 
cigarettes as one product, the majority of 15-year-
old students that used any tobacco or related product 
(85.5%) used one product (Table 2); the use was related 
to gender (p=0.001). The most frequent combinations 
used were cigarettes with waterpipe or e-cigarette. Only 
one 15-year-old student used solely e-cigarettes, others 
used e-cigarettes and other products (12 of 14 e-cigarette 
users also smoked cigarettes). 15 of 46 waterpipe users 
used solely waterpipe, others used also other products 
(29 of 31 waterpipe users also smoked cigarettes).
The use of one or more tobacco and related products is 
not linked to the frequency of current smoking (Table 
3); however, it is associated with the age at smoking 
initiation (Table 4). 
M-less precise estimate 
10.1515/sjph-2017-0010 Zdr Varst 2017; 56(2): 74-81
78
Table 4.
Table 7.Table 6.
The use of one or more tobacco or related products 
among 15-year-old students who are ever smokers by 
the age of smoking initiation.
The use of flavoured tobacco and related products 
during the last 30 days among 15-year-old students 
who are current smokers by the frequency of 
smoking tobacco.
The use of flavoured tobacco and related products 
during the last 30 days among 15-year-old students 
and users of any product.
M-less precise estimate 
All
one 
two or more
Boys
one 
two or more
Girls
one 
two or more
All
Boys
Girls
Flavoured 
products
 
114
29
43
21
71
8
84
38
46
41
13
28
44.2%
40.0%M
48.4%M
50.0%M
39.4%M
57.1%M
189 75 114
 
79.7%
20.3%
67.2%M
32.8%M
89.9%M
10.1%M
48,5 40,3 55,9
 
213
26
95
16
118
10
 
89.1%
10.9%
85.6%
14.4%
92.2%
7.8%
p
p
p
First smoked 
cigarettes at 13 
or less
Smoking tobacco 
weekly or more 
often
All
n
n
n
n
n
n n
%
%
%
%
%
% %
First smoked 
cigarettes at 14 
or more
Smoking tobacco 
less than 
weekly
Boys Girls
0.011
0.379
0.951
0.321
0.004
0,002
0.566
A vast majority of 15-year-old students used conventional 
products (Table 5) and the use was not related to gender.
48.5% of 15-year-old students that used any product used 
products with flavours; the use was related to gender 
(Table 6), but not to the frequency of smoking (Table 7) 
or age at smoking initiation (Table 8).
Table 5. The use of conventional and unconventional products during the last 30 days among all 15-year-old students (n=1558) and 
users of any product (n=392).
*shares for each group together exceed the share of users as one person can use more products 
**shares for each product together exceed 100% as one person can use more products
conventional 
unconventional 
only unconventional 
372
66
20
167
44
15
205
22
5
23.9%
4.2%
1.3%
23.5%
6.2%
2.1%
24.2%
2.6%
0.6%
94.9%
16.8%
5.1%
 91.8%
24.2% 
8.2% 
97.6% 
10.5% 
0.2% 
0.763
<0.001
0.008
4429
778
241
n n n% 
among 
all 15-
year-old 
stu-
dents*
% 
among 
all 15-
year-old
male 
stu-
dents*
% 
among 
all 15-
year-old
female 
stu-
dents*
% 
among 
users**
% 
among 
users**
% 
among 
users**
Girls
p 
(diff. 
among 
genders 
for all 
15-year-old 
students)
Estimated 
number 
of users 
among 
15-year-olds 
of both  
genders
BoysBoth genders
10.1515/sjph-2017-0010 Zdr Varst 2017; 56(2): 74-81
79
Table 8. The use of flavoured tobacco and related products 
during the last 30 days among 15-year-old students 
who are ever smokers by the age of smoking 
initiation.
All
Boys
Girls
61
22
39
120
47
73
44.5%
36.7%M
50.6%M
50.6%
41.2%
59.3%
p
First smoked 
cigarettes at 13 
or less
n n% %
First smoked 
cigarettes at 14 
or more
0.255
0.559
0.228
4 DISCUSSION
In Slovenia, a vast majority of 15-year-old students 
that are users of any tobacco or related products use 
cigarettes (mostly manufactured cigarettes, but hand-
rolled cigarettes are also used substantially), followed 
by waterpipe and later by cigars, cigarillos and pipes. A 
vast majority therefore uses conventional products. Every 
sixth student uses unconventional products, whilst the 
exclusive use of unconventional product is low (mainly 
the exclusive use of waterpipe). There are significant 
differences in the use of different types of tobacco and 
related products between genders; girls are more likely 
to use manufactured cigarettes, while boys are more 
likely to use hand-rolled cigarettes, waterpipe, cigarillos, 
cigars and pipes, and e-cigarettes. Almost half of those 
that use any product – significantly more girls than boys – 
use products with flavours. Approximately every seventh 
15-year-old student who uses any product uses more 
than one product – significantly more boys than girls and 
significantly more those that started smoking cigarettes 
earlier. Most frequent combinations are cigarettes and 
waterpipe or cigarettes and e-cigarettes. 
Only limited comparisons on current use of different 
tobacco and related products with other studies are 
possible mainly due to different age groups, the range 
of monitored products, definitions of use and a limited 
number of published studies with nationally representative 
samples. When comparing our results to results of other 
available studies among youth in Slovenia, we find the 
same order of mostly used products, while shares differ 
(15, 17), which is most likely attributable to the difference 
in age groups of respondents, different year of the survey 
and methodological differences. The comparison with 
adult population in Slovenia shows lower shares of the use 
of manufactured cigarettes among 15-year-old students, 
while shares of the use of all other tobacco and related 
products are higher in students. The use of more than 
one product among adult population is around half of the 
use of more than one product among 15-year-old students 
(17). Adolescents are obviously prone to use different and 
unconventional products and more than one product. 
There is no data available to compare the use of flavoured 
products. The comparison of available data for similar 
age groups in other countries show that shares of the use 
of waterpipe, e-cigarette and unconventional products in 
general and more than one product are currently most 
likely lower in Slovenia (7-15, 22). 
The use of any product containing nicotine is problematic 
in adolescence, which is a sensitive developmental 
period of enhanced clinical vulnerability to nicotine. 
Adolescents are more likely than adults to become 
addicted to nicotine; nicotine acts as a gateway to 
other substances, affects brain development and has 
long-term negative consequences on cognition (22, 23). 
Adolescents that might never use conventional products 
are using unconventional products (8-11, 14). Those could 
act as a gateway to conventional tobacco products use 
by exposing youth to nicotine, and increase the overall 
number of youth initiating tobacco use. A limited number 
of longitudinal studies on waterpipe and e-cigarette 
ever users shows an increased risk of later conventional 
tobacco use initiation in adolescents (25-27). Flavoured 
products are tobacco industry’s innovation to increase 
the appeal of smoking in youth. They can promote youth 
smoking initiation and help young occasional smokers to 
become daily smokers by reducing or masking the natural 
harshness and taste of tobacco smoke (18). Young people 
who start smoking menthol cigarettes are at greater risk 
of progression to regular smoking and nicotine addiction 
than those who start smoking non-menthol cigarettes 
(28). Flavours are also one of the most important reasons 
for increasing the popularity of waterpipe (12) and 
e-cigarettes among youth (29). Hand-rolled cigarettes are 
popular among youth also due to lower prices compared 
to manufactured cigarettes (30, 31). Some studies 
indicate that youth using hand-rolled cigarettes are more 
likely heavier and more frequent smokers (16, 32), more 
addicted to nicotine and less likely considering quitting 
smoking, when compared to users of manufactured 
cigarettes (32). Our study does not show more weekly or 
more daily smoking among 15-year-old students that use 
hand-rolled cigarettes compared to those that use only 
manufactured cigarettes. Studies also indicate increased 
nicotine addiction in users of more than one product (33). 
All this underlines the importance of continuous 
surveillance, timely implementation and strict 
enforcement of all necessary measures to prevent 
children and adolescents from using all nicotine containing 
products, and the inclusion of all different products in 
prevention and cessation programmes. Products gaining 
popularity among youth are currently mostly unregulated 
or inadequately regulated, available to minors and/
or more affordable, also in Slovenia. The World Health 
Organization recommends regulation for waterpipe and 
e-cigarettes as for any other tobacco product (34, 35). 
10.1515/sjph-2017-0010 Zdr Varst 2017; 56(2): 74-81
80
Our research is the first in Slovenia to give insight into the 
use of different tobacco and related products, including 
e-cigarettes, flavoured products and the use of more than 
one product, on a representative sample of adolescents 
in schools. Our study has some limitations. The cross-
sectional design of the study does not allow for any 
conclusions on causality or chronology. Under-reporting 
of smoking and the use of different products is possible. 
We collected data only from 15-year-olds that attend 
school and we can generalize data only to the group 
of 15-year-old students. We did not ask about nicotine 
content of liquids for e-cigarettes and the use of tobacco 
(rather than herbal products) for waterpipe. Regardless 
of limitations, our study offers valuable data for tobacco 
control and prevention.
5 CONCLUSIONS
15-year-old students in Slovenia currently use mostly 
conventional products, very often they use products with 
flavours and mostly they use only one product. However, 
situation in other countries shows that a pattern of 
use can change quickly, especially due to non-existent 
or inadequate regulation of certain products or their 
characteristics, such as flavours. Regular surveillance, 
addressing all different products in prevention and 
cessation programmes, and timely and effective 
regulation, comparable for all different products, 
combined with strict enforcement are necessary for 
effective prevention of the use of all products containing 
nicotine among youth in Slovenia.
CONFLICT OF INTEREST
The authors declare that no conflict of interest exists.
FUNDING 
The study was financed by the Ministry of Health of the 
Republic of Slovenia.
ETHICAL APPROVAL
The study was approved by the Republic of Slovenia 
National Medical Ethics Committee on 29 October 2013 
under the number 139/10/13.
REFERENCES
1. World Health Organization. The European Health Report 2005: public 
health action for healthier children and populations. Copenhagen: 
WHO Regional Office for Europe, 2005.
2. GBD 2013 Risk Factors Collaborators. Global, regional, and national 
comparative risk assessment of 79 behavioural, environmental 
and occupational, and metabolic risks or clusters of risks in 188 
countries, 1990-2013: a systematic analysis for the Global Burden of 
Disease Study 2013. Lancet 2015; 386: 2287-323.
3. Koprivnikar H, Korošec A. Age at smoking initiation in Slovenia. Zdr 
Varst 2015; 54: 274-81.
4. Jeriček Klanšček H, Bajt M, Drev A, Koprivnikar H, Zupanič T, Pucelj 
V, editors. Z zdravjem povezana vedenja v šolskem obdobju med 
mladostniki v Sloveniji: izsledki mednarodne raziskave HBSC 2014. 
Ljubljana: Nacionalni inštitut za javno zdravje, 2015. 
5. Stergar E. Evropska raziskava o alkoholu in preostalih drogah med 
šolsko mladino 2011. Ljubljana: Klinični inštitut za medicino dela, 
prometa in športa, 2015.
6. Juričič M. Slovenia 2011 (Ages 13-15) Global Youth Tobacco Survey 
(GYTS) FACT SHEET. Available February 24, 2016 from: nccd.cdc.gov/
gtssdata/Ancillary/DownloadAttachment.aspx?ID=1168
7. Arrazole RA, Singh T, Corey CG, Husten CG, Neff LJ, Apelberg BJ. 
et al. Tobacco use among middle and high school students—United 
States, 2011-2014. MMWR 2015; 64: 381-5.
8. Goniewicz ML, Leigh NJ, Gawron M, Nadolska J, Balwicki L, McGuire 
C. et al. Dual use of electronic and tobacco cigarettes among 
adolescents: a cross-sectional study in Poland. Int J Public Health 
2016; 61: 189-97.
9. Czoli CD, Reid JL, Rynard VL, Hammond D. E-cigarettes in Canada 
- tobacco use in Canada: patterns and trends, 2015 edition, special 
supplement. Waterloo, ON: Propel Centre for Population Health 
Impact, University of Waterloo, 2015.
10. Kinnunen JM, Ollila H, El-Amin SE, Pere LA, Lindfors PL, Rimpelä 
AH. Awareness and determinants of electronic cigarette use among 
Finnish adolescents in 2013: a population-based study. Tob Control 
2015; 24: e264-70. 
11. Bauld L, MacKintosh AM, Ford A, McNeill A. E-cigarette uptake 
amongst UK youth: experimentation, but little or no regular use in 
nonsmokers. Nicotine Tob Res 2016; 18: 102-3. 
12. Akl EA, Ward KD, Bteddini D, Khaliel R, Alexander AC, Lotfi T. et al. 
The allure of the waterpipe: a narrative review of factors affecting 
the epidemic rise in waterpipe smoking among young persons 
globally. Tob Control 2015; 24(Suppl 1): i13-21. 
13. Galanti MR, Al-Adhami M. Use of a water pipe is not an alternative 
to other tobacco or substance use among adolescents: results from a 
national survey in Sweden. Nicotine Tob Res 2015; 17: 74-80. 
14. Pärna K, Usin J, Ringmets I. Cigarette and waterpipe smoking among 
adolescents in Estonia: HBSC survey results, 1994-2006. BMC Public 
Health 2008; 8: 392. 
15. Centers for Disease Control. Global Tobacco Surveillance System 
Data (GTSSData). Available December 4, 2015 from: http://nccd.
cdc.gov/gtssdata/Ancillary/DataReports.aspx?CAID=2. 
16. Leatherdale ST, Burkhalter R. Roll-your-own tobacco use among 
Canadian youth: is it a bigger problem than we think? BMC Public 
Health 2012; 12: 557. 
17. Lavtar D, Drev A, Koprivnikar H, Zorko M, Rostohar K, Štokelj 
R. Uporaba prepovedanih drog, tobaka in alkohola v Sloveniji 
2011–2012: metodologija raziskave in izbrani statistični podatki. 
Ljubljana: Nacionalni inštitut za javno zdravje, 2014.
10.1515/sjph-2017-0010 Zdr Varst 2017; 56(2): 74-81
81
18. Carpenter CM, Wayne GF, Pauly JL, Koh HK, Connolly GN. New 
cigarette brands with flavors that appeal to youth: tobacco marketing 
strategies. Health Aff (Millwood) 2005; 24: 1601-10.
19. Kaleta D, Usidame B, Szosland-Fałtyn A, Makowiec-Dąbrowska T. Use 
of flavoured cigarettes in Poland: data from the global adult tobacco 
survey (2009-2010). BMC Public Health 2014; 14: 127.
20. Corey CG, Ambrose BK, Apelberg BJ, King BA. Flavored tobacco 
product use among middle and high school students - United States, 
2014. MMWR Morb Mortal Wkly Rep 2015; 64: 1066-70.
21. Manske SR, Rynard VL, Minaker LM. Flavoured tobacco use among 
Canadian youth: evidence from Canada’s 2012/2013 youth smoking 
survey. Waterloo: Propel Centre for Population Health Impact, 2014.
22. Wang B, King BA, Corey CG, Arrazola RA, Johnson SE. Awareness 
and use of non-conventional tobacco products among U.S. students, 
2012. Am J Prev Med 2014; 47(2 Suppl 1): S36-52.
23. England LJ, Bunnell RE, Pechacek TF, Tong VT, McAfee TA. Nicotine 
and the developing human: a neglected element in the electronic 
cigarette debate. Prev Med; 49: 286-93. 
24. Yuan M, Cross SJ, Loughlin SE, Leslie FM. Nicotine and the adolescent 
brain. J Physiol; 593: 3397-412.
25. Jaber R, Madhivanan P, Veledar E, Khader Y, Mzayek F, Maziak 
W. Waterpipe a gateway to cigarette smoking initiation among 
adolescents in Irbid, Jordan: a longitudinal study. Int J Tuberc Lung 
Dis 2015; 19: 481-7. 
26. Soneji S, Sargent JD, Tanski SE, Primack BA. Associations between 
initial water pipe tobacco smoking and snus use and subsequent 
cigarette smoking: results from a longitudinal study of US adolescents 
and young adults. JAMA Pediatr 2015; 169: 129-36. 
27. Leventhal AM, Strong DR, Kirkpatrick MG, Unger JB, Sussman S, Riggs 
NR. et al. Association of electronic cigarette use with initiation of 
combustible tobacco product smoking in early adolescence. JAMA 
2015; 314: 700-7. 
28. Kong G, Morean ME, Cavallo DA, Camenga DR, Krishnan-Sarin S. 
Reasons for electronic cigarette experimentation and discontinuation 
among adolescents and young adults. Nicotine Tob Res 2015; 17: 847-
54. 
29. Nonnemaker J, Hersey J, Homsi G, Busey A, Allen J, Vallone D. 
Initiation with menthol cigarettes and youth smoking uptake. 
Addiction 2013; 108: 171-8. 
30. Gilmore AB, Tavakoly B, Hiscock R, Taylor G. Smoking patterns in 
Great Britain: the rise of cheap cigarette brands and roll-your-own 
tobacco. J Public Health (Oxf) 2015; 37: 78-88. 
31. Dunlop SM, Perez D, Cotter T. Australian smokers’ and recent quitters’ 
responses to the increasing price of cigarettes in the context of a 
tobacco tax increase. Addiction 2011; 106: 1687-95. 
32. Leatherdale ST1, Kaiserman M, Ahmed R. The roll-your-own cigarette 
market in Canada: a cross-sectional exploratory study. Tob Induc Dis 
2009; 5: 5.
33. Lee YO, Hebert CJ, Nonnemaker JM, Kim AE. Youth tobacco product 
use in the United States. Pediatrics 2015; 135: 409-15.
34. WHO Study Group on Tobacco Product Regulation. Waterpipe tobacco 
smoking: health effects, research needs and recommended actions 
for regulators, 2nd Edition. Geneva: World Health Organization, 
2015.
35. Conference of the Parties to the WHO Framework Convention on 
Tobacco Control. Electronic nicotine delivery systems. A Report by 
WHO, 1 September 2014.
THE DEVELOPMENT OF A CONSENSUS DEFINITION FOR HEALTHCARE 
IMPROVEMENT SCIENCE (HIS) IN SEVEN EUROPEAN COUNTRIES: 
A CONSENSUS METHODS APPROACH 
Brigita SKELA-SAVIČ1*, Rhoda MacRAE2, Manuel LILLO-CRESPO3, Kevin D ROONEY2,4
1Faculty of Health Care Jesenice, Spodnji Plavž 3, 4270 Jesenice, Slovenia
2Institute of Care and Practice Improvement, School of Health, Nursing and Midwifery, 
University of the West of Scotland, Hamilton, ML3OBA, Scotland
3Faculty of Health Sciences, University of Alicante, Spain, Carretera de San Vicente del Raspeig s/n 03690 
San Vicente del Raspeig, Alicante, Spain
4Anaesthesia & Intensive Care Medicine, Royal Alexandra Hospital, Corsebar Road, Paisley, PA2 9PN, Scotland
Received: Feb 28, 2016
Accepted: Aug 29, 2016
Original scientific article
Introduction. There is a limited body of research in the field of healthcare improvement science (HIS). Quality 
improvement and ‘change making’ should become an intrinsic part of everyone’s job, every day in all parts 
of the healthcare system. The lack of theoretical grounding may partly explain the minimal transfer of health 
research into health policy.
Methods. This article seeks to present the development of the definition for healthcare improvement science. 
A consensus method approach was adopted with a two-stage Delphi process, expert panel and consensus 
group techniques. A total of 18 participants were involved in the expert panel and consensus group, and 153 
answers were analysed as a part of the Delphi survey. Participants were researchers, educators and healthcare 
professionals from Scotland, Slovenia, Spain, Italy, England, Poland, and Romania. 
Results. A high level of consensus was achieved for the broad definition in the 2nd Delphi iteration (86%). The 
final definition was agreed on by the consensus group: ‘Healthcare improvement science is the generation of 
knowledge to cultivate change and deliver person-centred care that is safe, effective, efficient, equitable and 
timely. It improves patient outcomes, health system performance and population health.’
Conclusions. The process of developing a consensus definition revealed different understandings of 
healthcare improvement science between the participants. Having a shared consensus definition of healthcare 
improvement science is an important step forward, bringing about a common understanding in order to advance 
the professional education and practice of healthcare improvement science. 
Uvod. Skupno število raziskav na področju znanosti o uvajanju izboljšav v zdravstvu (Healthcare Improvement 
Science - HIS) je majhno. Vendar pa bi morala izboljševanje kakovosti in vpeljava sprememb postati neločljivo 
povezana z vsakodnevnim delom zdravstvenih delavcev na vseh področjih zdravstvenega sistema. Pomanjkanje 
teoretičnih izhodišč bi lahko delno pojasnilo nizko stopnjo prenosa rezultatov raziskav v zdravstveno politiko.
Metode. Namen članka je predstaviti postopek oblikovanja definicije za pojem znanost o uvajanju izboljšav 
v zdravstvu. Pri pristopu z metodami konsenza smo uporabili metodo Delfi, izvedeno v dveh krogih, panelne 
skupine strokovnjakov in tehnike konsenzne skupine. V skupini strokovnjakov in konsenzni skupini je sodelovalo 
18 oseb, v okviru metode Delfi smo analizirali 153 odgovorov. Sodelujoče osebe so bile raziskovalci, izobraževalci 
in zdravstveni delavci iz Škotske, Slovenije, Španije, Italije, Anglije, Poljske in Romunije.
Rezultati. V drugem krogu metode Delfi je bila dosežena visoka stopnja konsenza (86-odstotna) za široko 
definicijo. Konsenzna skupina je oblikovala naslednjo končno definicijo: »Znanost o uvajanju izboljšav omogoča 
ustvarjanje znanj za izvajanje sprememb ter zagotavljanje v pacienta usmerjene zdravstvene obravnave, ki 
je varna, zmogljiva, učinkovita, pravična in pravočasna. Izboljšuje pacientove izide zdravstvene obravnave, 
sistem nudenja zdravstvenih storitev in zdravje populacije.« 
Razprava. Postopek oblikovanja dogovorne definicije je razkril različno razumevanje pojma znanost o uvajanju 
izboljšav v zdravstvu med sodelujočimi. Oblikovanje skupne dogovorne definicije je pomemben korak naprej, 
saj pripomore k splošnemu razumevanju tega pojma in s tem tudi k spodbujanju strokovnega izobraževanja in 
izvajanju znanosti o uvajanju izboljšav v praksi.
ABSTRACT
Keywords: 
healthcare, 
improvements,  
science, consensus 
methods, definition, 
Europe
IZVLEČEK
Ključne besede: 
zdravstvo, izboljšave, 
znanost, metode 
konsenza, definicija, 
Evropa
*Corresponding author: Tel: ++ 386 4 5869 360; E-mail: bskelasavic@fzj.si
10.1515/sjph-2017-0011 Zdr Varst 2017; 56(2): 82-90
82
OBLIKOVANJE DOGOVORNE DEFINICIJE ZA POJEM ZNANOST O UVAJANJU 
IZBOLJŠAV V ZDRAVSTVU (HIS) V SEDMIH EVROPSKIH DRŽAVAH: 
PRISTOP Z METODAMI KONSENZA
Skela-Savič B, MacRae R, Lillo-Crespo M, Rooney KD. The development of a consensus definition for Healthcare Improvement Science (HIS) in seven European 
countries: a consensus methods approach. Zdr Varst 2017; 56(2): 82-90
© Nacionalni inštitut za javno zdravje, Slovenija. 
1 INTRODUCTION
Improving healthcare quality has become a priority over 
the past years (1). While much of quality improvement 
work is unscientific (2), the adoption of a more scientific 
approach to improvement could enhance the ability of 
health systems to provide high-quality care and use their 
resources optimally (3). Healthcare improvement science 
(HIS) represents ‘the combined and unceasing efforts of 
everyone – healthcare professionals, patients and their 
families, researchers, payers, planners and educators 
– to make the changes that will lead to better patient 
outcomes (health), better system performance (care) 
and better professional development (learning)’ (4, 
p. 2). In this respect, quality improvement and change 
making should become an essential part of healthcare 
professionals’ work at all levels of the healthcare system 
(5). Implementation science and quality improvement 
efforts share the ultimate goal of improving healthcare 
quality. However, there are differences. Quality begins 
with a specific problem in a given healthcare system, 
while implementation science begins with evidence-
based practice (EBP) that is not used effectively, then 
identifies and addresses quality issues at the level of 
providers, clinics, or the healthcare system (6). Topic 
reviewers reported a diversity of outcomes used for 
quality improvement evaluations, flaws in study design, 
and incomplete reporting (7).
The Health Foundation (8) developed an explanation of 
what improvement science should be. They defined it as 
‘building a knowledge base for improving health services 
and translating this knowledge into practice to deliver the 
best possible patient care’ (8, p. 7). Moreover, it has been 
stressed that the implementation science community 
should provide guidelines for reporting the use of 
theoretical frameworks within implementation studies 
and their efficacy (9). This is important because few 
hospital-based interventions are essentially theoretical 
(10). The Health Foundation definition (8, p. 7) states 
clearly that improvement science ‘requires a systematic, 
scientifically rigorous approach to close the gap between 
current and best practice.’
In the case of quality improvement, theory needs 
development, which draws from a better understanding 
of practice (11). Many quality improvement practitioners 
believe that quality improvement is implemented 
in complex adaptive systems. They sometimes feel 
limited by research findings that fail to recognize 
the complexity and the need for adaptation. Quality 
improvement practitioners could help researchers to 
see the larger patterns specific to complex systems 
and draw upon possible solutions (12). Organizational 
learning framework could be used to explore the factors 
influencing improvement feasibility (13). Organizations 
worldwide have adopted interprofessional education 
and interprofessional collaborative practice in an effort 
to improve healthcare delivery systems (14), which fits 
well with the Health Foundation definition (8, p. 7) that 
‘improvement science embraces disciplines across health 
services research, from sociology to statistics, psychology 
to health economics.’ The next generation of healthcare 
professionals will require new skills to make sure that 
quality improvement in health care is successful (15). 
For this reason, it is important how different healthcare 
professionals understand HIS.
1.1 The Aim
The article describes a part of the project ‘Improvement 
Science Training for European Healthcare Workers 
(ISTEW),’ which aimed to develop shared academic and 
practice-based programmes that would enable European 
institutions to build improvement capacity within their 
healthcare workforce. This article discusses a keystone 
component of the project, namely the development of a 
consensus definition for HIS, based on the interprofessional 
and multicultural consensus of healthcare professionals 
considered as experts from seven European countries.
2 METHODS
2.1 Research Design and Data Collection 
Research design comprised of consensus methods, 
including the nominal group technique, the Delphi 
process, and the consensus group technique. The aim of 
consensus methods is to determine the extent to which 
experts or lay people agree about a given issue. The 
term ‘agreement’ takes two forms, which need to be 
distinguished: firstly, the extent to which each respondent 
agrees with the issue under consideration (typically rated 
on a numerical or categorical scale) and, secondly, the 
extent to which respondents agree with each other (16). 
2.1.1 The Nominal Group Technique
The nominal group technique uses a highly structured 
meeting to gather information from relevant experts 
(usually 9-12 in number) about a given issue. It consists of 
two rounds, in which panellists rate, discuss, and then re-
rate a series of items or questions (16). The nominal group 
technique was used twice. First, we implemented two 
structured Skype meetings with research group members 
from seven European countries. The goal was to develop 
a working definition of HIS, to increase consensus amongst 
participants about the meaning and content of HIS, and to 
establish the level of development and understanding of 
HIS in each country. Expert-based knowledge, literature 
and comprehension across a core group of seven experts 
was used. After two meetings, a working definition was 
developed and tested in the Delphi process. Second, the 
nominal group technique was used on the same team in 
10.1515/sjph-2017-0011 Zdr Varst 2017; 56(2): 82-90
83
10.1515/sjph-2017-0011 Zdr Varst 2017; 56(2): 82-90
84
group technique by using qualitative and quantitative 
data from the first Delphi iteration. These reformulated 
items were presented in the second Delphi iteration. 
2.1.3 The Consensus Group Technique
Following two Delphi iterations and two nominal group 
techniques, a consensus group technique (a combination 
of a focus group and a public meeting) was employed. 
The main difference between consensus groups and focus 
groups is that in the latter, the moderator or analyst 
decides, whereas in consensus groups, the participants 
themselves negotiate and decide the findings (18). We 
used this technique at a three-day meeting of all ISTEW 
project members held in Bled (June 2014), Slovenia. The 
consensus group included 18 participants from seven 
European countries. We wanted to provide everyone 
in the group the opportunity to work collaboratively, 
to be involved in the decision-making processes, to 
be valued as experts, and for the group to become a 
forum for change (19). Consensus group members were 
discussing quantitative and qualitative data of the 2nd 
Delphi iteration and the suitability of the developed HIS 
definition. The aim was to arrive at a final definition of 
HIS.
2. 2 The Sample 
A non-random convenience sample was used for the Delphi 
process. Each project team invited 10–15 people they 
knew to be involved in HIS in their respective settings. 
‘Participants’ came from seven European countries 
(England, Italy, Poland, Romania, Scotland, Slovenia, and 
Spain) and from different healthcare areas. The total 
number of respondents in the 1st iteration was 87; of 
these, 66 chose to participate in the 2nd iteration. HIS 
fields in which the respondents were involved are shown 
in Table 1.
order to develop an extended questionnaire for the 2nd 
Delphi iteration. 
2.1.2 The Delphi Process
A Delphi study is a robust method that draws on expert 
opinions, and compares them with the combined opinions 
of other participating experts over several rounds, 
until a consensus on specified criteria is reached (16). 
A two-iteration Delphi technique was developed with 
participants from seven European countries. The Delphi 
technique enabled us to collect data from several 
domains related to healthcare in education, research, 
management and practice. Moreover, two questionnaires 
were designed for each iteration with quantitative and 
qualitative elements allowing us to achieve a merging 
of opinions from 87 respondents. The respondents were 
consulted in two anonymous Delphi rounds. A 5-point 
Likert scale (1–strongly disagree, 2–disagree, 3–neither 
agree nor disagree, 4–agree, 5–strongly agree) was used 
in both questionnaires to rate the importance of each 
HIS definition element and of the definition as a whole. 
Respondents could also offer suggestions for improving 
separate parts of the definition by replying to open-ended 
questions and providing HIS descriptions used in their 
countries. Furthermore, we were interested in obtaining 
some demographic data (the area of work, respondent 
connection with HIS and involvement in HIS) and data 
about the description of HIS in their countries. Reliability 
test results were good. Cronbach’s Alpha for statements 
(N=6) for 1st Delphi iteration was 0.867 and for statements 
(N=7) for 2nd Delphi iteration 0.843. 
Over several rounds, a specific definition element was 
included in the next version of the definition if >80% of 
respondents judged it as important or very important, and 
excluded if >50% judged it as not important or moderately 
important (17). The suggested reformulations of the 
second HIS definition were analysed with the nominal 
Table 1. Participants’ fields of work in the Delphi study.
1st iteration 
(n=87)
2nd iteration
(n=66)
37% (32)
27% (18)
29% (25)
39% (26)
23% (20)
23% (15)
7% (6)
5% (3)
4% (4)
6% (4)
Patient 
organisations
Public policyClinical practiceManagementEducation,
research
10.1515/sjph-2017-0011 Zdr Varst 2017; 56(2): 82-90
85
Ten members of the research group participated in 
the nominal group technique, and 18 members of the 
ISTEW project group participated in the consensus group 
technique. Members of both groups were experts in HIS 
from different professions.
2.3 Research Conduction and Ethical Approval
The research was conducted from November 2013 to 
September 2014. We started with the nominal group 
technique, first conducted in November 2013 and then 
in March 2014, followed by the Delphi process (the 
first and second iterations in December 2013 and April 
2014, respectively), and finally, the consensus group 
technique in June 2014.The research protocol was 
approved by the scientific ethical committee of the 
research coordinator, the Faculty of Health Care Jesenice 
(Slovenia). Partners also adhered to their ethical approval 
processes for research. Participants were assured about 
the anonymity and confidentiality of their collaboration. 
The questionnaires were available either in English or the 
language of the expert. Partners were responsible for 
translating the questionnaires, as required.
2.4 Data Analysis
Quantitative data analysis was carried out with descriptive 
statistics (Mean, Standard Deviation, Frequency, 
Percentage) and bivariate statistics (Chi-Square (p) 
– Kruskal-Wallis H nonparametric test, Independent-
samples t-test, Paired samples t-test, Pearson 
correlation, Spearman correlation) in the statistical 
software programme SPSS, v. 20.0. The level of statistical 
significance was set at p<0.05. 
Qualitative data was obtained from the open-ended 
questions of the first and second Delphi iterations. 
First, each partner team extracted the themes from the 
qualitative data from both Delphi iterations and exposed 
these themes in the nominal group technique and the 
consensus group technique. The software programme 
NVivo, v. 10.0 and content analysis were used. 
3 RESULTS 
3.1 Characteristics of Respondents in the Delphi Survey 
We asked respondents to classify themselves in three 
ways: those who worked in the field of HIS daily (this 
group we called ‘expert’), those who worked in the field 
of HIS occasionally (this group we called ‘occasional 
worker’), and those who were informed about HIS (this 
group we called ‘informed layman’). In the 1st Delphi 
iteration, significant differences were established 
between countries in identifying respondents as experts of 
Table 2. Respondents’ involvement in HIS and differences 
between countries.
‘Expert’
‘Occasional 
worker’
‘Informed 
layman’
21 
(28.0)
44 
(62.9)
32 
(48.5)
30
(45.5)
31
(47.0)
24
(26.7)
24.411
(<0.001)
13.687
(0.033)
2.614
(0.856)
21.181
(0.002)
11.172
(0.083)
15.048
(0.020)
1st Delphi iterationGroups of 
respondents
Yes
f (%)
Yes
f (%)
X2 (p) X2 (p) 
2st Delphi iteration
HIS (p<0.001), occasional workers (p=0.033) or informed 
laymen (p<0.001). The number of answers ‘Yes’ was 97; 
10 respondents recognized themselves in more than one 
way. In the 2nd Delphi iteration, significant differences 
were established between respondents in identifying HIS 
‘expert’ (p=0.002) and in classification as an ‘informed 
layman’ (p=0.020). The number of answers ‘Yes’ was 
85; 19 respondents recognized themselves in more than 
one way. Results are in Table 2. We had more experts 
and occasional workers in the second iteration, and less 
informed laymen.  
Comment: X2 (p) is answer ‘Yes’ between countries. 
A total of 50% of respondents agreed that their national 
bodies, commissions or government which include HIS 
play an important part in healthcare and social policy in 
their country (n=43, x2=37.502, p<0.001). A total of 25 
respondents (35.2%) agreed that HIS has an accepted 
status in healthcare or social organizations, and 29 
respondents (35.4%) reported that national scientific 
research in HIS has been conducted in their country. We 
did not collect this data in the 2nd iteration.
3.2 Results of the 1st Delphi Iteration
In the 1st iteration, we tested the definition of HIS 
which was developed with the nominal group technique. 
The definition is shown in Table 3. Results reveal a high 
level of agreement with the definition as a whole on a 
5-point scale. Respondents from Scotland expressed a 
significantly lower agreement with the definition as a 
whole, but with the highest level of standard deviation 
of their answers (M=2.83, SD=1.27, p<0.001) and also 
with its separate parts (a) (M=2.25, SD=1.22, p<0.001), 
(c) (M=2.67, SD=0.99, p<0.001), (d) (M=2.57, SD=1.78, 
p=0.029), except for parts (b) and (e). 
10.1515/sjph-2017-0011 Zdr Varst 2017; 56(2): 82-90
The agreement with the definition increased with the level 
of respondents’ expertise. Correlation analysis showed us 
that respondents who identified themselves as ‘experts’ 
were in positive correlation with the agreement with the 
definition as a whole (r=0.264, p=0.023). Respondents’ 
fields of work were not in correlation with the level of 
agreement with the definition as a whole. 
Qualitative responses of respondents (n=21) who rated 
themselves as experts were analysed. A review of 
responses to the question ‘Which areas are missing from 
the proposed definition?’ yielded 120 codes and 8 main 
categories, namely the content of the definition, the 
purpose of the definition, multidisciplinary approach, 
definition structure, healthcare professionals, negative 
consequences, innovative emergent field, and references. 
Word frequency of codes revealed that, in the opinions of 
HIS experts, the most frequent words to be included in 
Table 3. Results for the level of agreement with the definition as a whole and with its separate parts (a – e), and differences between 
countries in the 1st Delphi iteration (n=87).
Agreement with the definition as a whole, from the part a to the part e.
Healthcare Improvement Science (HIS) is the framework for achieving efficiency, efficacy and quality in 
health and social care (a).
HIS depends on knowledge from a wide range of sources, not just research (b).
HIS is a link (connection) between numerous variables in clinical work and management/leadership work, 
such as: multi-disciplinary approach, patient involvement, patient safety, total quality management, change 
management, personal involvement of healthcare/social workers, personal development and responsibility, 
team work/group work, connections between practice-theory-problems-research, etc. (c).
HIS is thus an umbrella term for all actions (practice, education, science and policy) that can lead to better 
health treatment outcomes (health), better system performance (care), better professional development 
(learning) and healthier communities (d).
HIS also demands flexibility and responsibility in our understanding, in our theories of knowledge, and in our 
use of research evidence in health care improvement strategy (e).
3.93
(0.861)
3.85
(1.070)
4.15
(0.958)
4.15
(0.907)
4.01
(1.101)
4.26
(0.767)
21.533 
(<0.001)
30.278 
(<0.001)
7.921
(0.244)
27.436 
(<0.001)
14.017 
(0.029)
7.319
(0.292)
Chi-Square 
(p)
M 
(SD)
Legend: M – central value of a discrete set of numbers calculated by the sum of the values divided by the number of values; SD – 
Standard deviation; Chi-Square (p) – Kruskal-Wallis H nonparametric test for several independent samples; a, b, c, d, e – parts of the 
whole definition.
Comment: X2 (p) is mean value between countries. 
the proposed definition are the following: Improvement 
(n=19), social (n=14), science (n=13), health (n=10), 
methods (n=10), healthcare (n=9), care (n=8), patient 
(n=8) and work (n=8).
3.3 Results of the 2nd Delphi Iteration
In the second iteration, we tested the definition of HIS 
developed with the nominal group technique, where 
the results from the first Delphi iteration were used. 
The definition is worded in Table 4. The average level of 
agreement with the second proposed definition of HIS was 
higher (M=4.30) compared to the first iteration (M=3.93) 
(t=-3.006, p=0.004). The average portion of agreement in 
the first iteration was 78.5% and in the second 86%. For 
separate parts of the definition, the greatest agreement 
was expressed with the part (b). 
86
10.1515/sjph-2017-0011 Zdr Varst 2017; 56(2): 82-90
87
Expertise in the field of HIS and respondents’ area of work 
did not significantly correlate with the agreement with 
HIS definition. Significant differences in the definition as 
a whole were established between partners (p<0.001). 
For example, respondents from Scotland expressed a 
significantly lower agreement with the definition as 
Table 4.
Table 5.
Results for the level of agreement with the definition as a whole and with its separate parts (a – f), and differences between 
countries in the 2nd Delphi iteration (N= 66).
Results of countries look for the level of agreement with the definition as a whole and with its separate parts (a – f) in the 
2nd Delphi iteration.
Agreement with the definition as a whole, from the part a to the part f
Healthcare Improvement Science (HIS) is focused on safety, effectiveness, efficiency, timeliness, equity and 
continuous patient-centred improvement in healthcare (a).
HIS depends on valid and reliable knowledge from a wide range of sources, in both the academic sector and 
service sector in healthcare (b).
HIS is multidisciplinary in its approach, drawing both on biomedical and on social sciences (c).
Different factors, such as methodological frameworks for improvement, knowledge of healthcare 
professionals, patient involvement and innovation, support organizational culture, are factors that interplay 
with the approach (d).
HIS is thus an umbrella term for any action (practice, education, science and policy) that can lead 
to improved healthcare outcomes for people, better health system performance and also healthier 
communities (e).
HIS also demands flexibility and responsibility in understanding, in theories of knowledge, and in the use of 
research evidence in health care improvement strategy (f).
Whole definition
Part a
Part b
Part c
Part d
Part e
Part f
4.30
(0.61)
64
65
65
66
66
66
66
4.20 
(0.45)
4.40 
(0.55)
4.20 
(0.84)
3.60 
(0.89)
3.20 
(0.84)
4.20 
(0.45)
4.20 
(0.45)
4.56 
(0.53)
4.56 
(0.53)
4.50  
(0.76)
4.44 
(0.53)
4.56 
(0.53)
4.44 
(0.53)
4.44 
(0.73)
4.33
(0.50)
4.67 
(0.50)
4.56  
(1.01)
4.67 
(0.50)
4.22 
(0.83)
4.11 
(0.93)
4.33 
(0.50)
4.71 
(0.49)
4.57 
(0.54)
4.86  
(0.38)
4.86 
(0.38)
4.57 
(0.54)
4.71 
(0.76)
4.86 
(0.38)
4.47 
(0.52)
4.20 
(0.86)
4.40 
(1.06)
4.40 
(0.91)
4.47 
(0.74)
4.53 
(0.52)
4.40 
(0.83)
3.50 
(0.53)
3.70 
(1.06)
4.09 
(1.04)
4.09 
(0.94)
3.36  
(1.43)
3.55 
(1.29)
3.45 
(0.93)
5.613 
(<0.001)
1.456 
(0.210)
0.676 
(0.669)
1.923 
(0.092)
3.473 
(0.005)
2.389 
(0.039)
3.648 
(0.004)
4.30
0.61
4.29
0.86
4.40
0.86
4.33
0.81
4.14
0.96
4.26
0.83
4.24
0.77
4.33
(0.50)
4.20
(1.23)
4.30
(0.48)
4.10
(0.88)
4.20
(0.63)
4.30
(0.48)
4.20
(0.42)
4.29
(0.86)
4.40
(0.86)
4.33
(0.81)
4.14
(0.96)
4.26
(0.83)
4.24
(0.77)
21.227 
(0.002)
7.727 
(0.259)
7.611
(0.268)
11.468 
(0.075)
14.904 
(0.021)
11.226 
(0.082)
16.761 
(0.010)
Chi-Square 
(p)
N PolandEngland Slovenia ScotlandM (SD) RomaniaItaly Spain Chi-
Square (p)
M 
(SD)
Legend: M – central value of a discrete set of numbers calculated by the sum of the values divided by the number of values; SD – 
Standard deviation; Chi-Square (p) – Kruskal-Wallis H nonparametric test for several independent samples; a, b, c, d, e – parts of the 
whole definition.
Comment: X2 (p) is mean value between countries. 
a whole (M=3.50, SD=0.53) and also with the part (f) 
(M=3.45, SD=0.93). Moreover, respondents from Scotland 
(M=3.36, SD=1.43) and England (M=3.20, SD=0.84) 
expressed a significantly lower agreement with the part 
(d). Detail results are in Table 5. 
10.1515/sjph-2017-0011 Zdr Varst 2017; 56(2): 82-90
88
3.4 Results of the Consensus Group Technique
The consensus group technique started with a discussion 
of 18 experts from institutions involved in the ISTEW 
project. It was moderated by the principal investigator 
and research fellow for the project, with notes being 
taken. First, the aim was to increase the understanding of 
the first and second Delphi iteration results and to present 
partners’ qualitative analysis results. The consensus group 
discussion yielded 14 key themes of the HIS definition, 
which were merged into 3 content categories, namely 
‘country differences knowledge’, ‘science and HIS,’ 
and ‘new dimension of HIS for practice’. The strongest 
extracted elements were knowledge, change, outcome, 
population, safety, efficiency, person-centred. 
All partners agreed that the HIS definition had to be short 
and concise. The emphasis of each definition part and 
the theme from qualitative data was discussed until a 
consensus was reached. Based on this, the final consensus 
on the definition of Healthcare Improvement Science is 
as follows:
‘Healthcare improvement science is the generation 
of knowledge to cultivate change and deliver person-
centred care that is safe, effective, efficient, equitable 
and timely. It improves patient outcomes, health system 
performance and population health.’
4 DISCUSSION 
HIS is incorporated into every phase of healthcare 
proceedings and is, consequently, a complex concept to 
define. Coining a definition that encompasses the multi-
faceted nature of HIS with all of its variables is very 
challenging, particularly because the use of the term 
HIS and its synonyms in the existing literature and other 
educational documents (grey literature) varies depending 
on the country and language. 
By applying consensus methods, the definition was 
developed gradually. According to the results of two 
Delphi iterations, a high consensus among participants 
was achieved. Consensus increased from the first iteration 
to the second, until 86% agreement was reached. It is 
very important that the definition was developed through 
interprofessional interactions. Only a third of respondents 
in the Delphi survey perceived themselves as experts in 
HIS, although the Delphi study coordinators identified 
them as such. This, we suggest, reflects the extent to 
which HIS is developed in the partner countries involved, 
as we found that HIS was not equally developed in all 
partner countries. Indeed, we found that the term itself 
is not used at all in Spain, and that the concept is not well 
developed in Romania, where the term implementation 
science with its different connotations tends to be 
more common. Respondents coming from countries 
where HIS was not broadly developed may have found it 
challenging to respond in an informed way. Respondents 
from Scotland, where HIS is a political priority, expressed 
a significantly lower level of consensus with the entire 
definition compared to non-English speaking respondents, 
but the level of agreement was still high at 70%. These 
differences may be related to a number of issues. Firstly, 
most respondents from Scotland defined themselves as 
experts in HIS. Secondly, in Scotland, HIS is a more crucial 
driver of healthcare institution policies compared to the 
other countries. Finally, research and accessibility of 
scientific and professional HIS literature is greater in the 
United Kingdom compared to other non-English native 
speaking countries (20).
The definition formed in two Delphi iterations was a 
broadened one. The consensus group technique greatly 
contributed to reaching a compromise between experts, 
so that a short and concise version of the definition was 
developed. The definition includes a wide range of key 
themes, such as knowledge for cultivating change, person-
centred care, improvement of patient outcomes, health 
system performance and population health. These themes 
are also included in definitions proposed by other authors 
(4, 8). The employed method for developing a definition 
leads to a better understanding of practice (11).
Only half of respondents agreed that HIS-related national 
bodies play an important role in health and social care 
policies in their own country. However, a closer look at 
the data reveals that the situation is not comparable 
across countries. Only one third of respondents felt that 
HIS has an appropriate status in their national healthcare 
system. As a result, HIS indicators are not developed well 
and HIS is an uncommon term in some countries. This 
is a major issue because, as Booth et al. (21) stated, 
changing population demographics, disease patterns and 
new research findings demand change in healthcare and 
require new ways for ensuring continuous improvement. 
Robson (22) believes that in making healthcare safer, 
one should join all levels of the profession in an effort to 
clearly work with their working environment and patient 
safety.
For a continuous improvement, an understanding and 
recognition of the need for HIS are crucial. Our definition 
underpins the need for developing interprofessional 
study programmes and other materials. Hall and Zierler 
(23) stressed that, with the growth of interprofessional 
education and practice in healthcare schools, faculty 
members have to assume new roles in designing and 
delivering interprofessional curricula. Senior leadership 
support is also vital in ensuring sustainable change, and it 
will be a key requirement in healthcare organisations and 
universities to make healthcare improvement a priority. 
10.1515/sjph-2017-0011 Zdr Varst 2017; 56(2): 82-90
89
5 LIMITATIONS 
The sample was not equally weighted for the nationalities 
of the HIS experts, and this may have impacted the Delphi 
process. Knowledge of the English language and familiarity 
with it may have also played a role in providing answers, 
in questionnaire translation, and in discussions within the 
nominal and consensus groups. In countries where English 
is not spoken as the official language, most partners had 
to first translate the questionnaire, then obtain answers 
from experts in their languages, and translate the 
responses back into English. This may have resulted in 
the loss of understanding and different interpretations. 
This study has only included seven European countries; 
consequently, it would be interesting to explore the ways 
in which HIS is understood, practised and taught across 
Europe as a whole.
6 CONCLUSION
A new consensus definition of HIS was developed in a 
European context. The consensus method approach was 
pertinent to unify the new concept from multidisciplinary, 
interprofessional and multicultural perspectives. Having a 
shared definition of HIS is important to boost healthcare 
culture and professional training processes. The definition 
enables higher education institutions in health and social 
care to address this gap in their curricula, and ensure that 
improvement science becomes a core competency for all 
healthcare graduates.
ACKNOWLEDGEMENTS
This article was written on behalf of the Improvement 
Science Training for European Healthcare Workers 
(ISTEW) project team. We would like to acknowledge the 
contributions of Professor Julita Sansoni, Dr Elisabetta 
Corvo, Dr Alan Taylor, Dr Katrina Ritters, Magdalena 
Glowacka, Dr Traian Mihaescu, Dr Radu Crisan, Dr Maria 
Josefa Cabanero Martinez, Barbara O’Donnell, Dr Joca 
Zurc and Miram Sanchez San Segundo.
CONFLICTS OF INTEREST
The authors report no conflicts of interest.
FUNDING 
The research was financed by the European Union funded 
ERASMUS Lifelong Learning Project, ISTEW: Improvement 
Science Training for European Healthcare Workers (Project 
No. 539194-LLP-1-2013-1-UK-ERASMUS-EQR). 
ETHICAL APPROVAL
The research protocol was approved by the Senate 
Committee for Science, Research and Development at 
the Faculty of Health Care Jesenice (FHCJ) in December 
2013. The serial number of Ethics Approval is 12/5 in 
the academic year 2013/2014. Partners also adhered to 
their ethical approval processes for research. Participants 
received written information about different aspects of 
the study; explained to them were their rights on voluntary 
participation and withdrawal from the study at any time, 
as well as their privacy and confidentiality rights. 
REFERENCES
1. Wandersman A, Alia KA, Cook B, Ramaswamy R. Integrating 
empowerment evaluation and quality improvement to achieve 
healthcare improvement outcomes. BMJ Qual Saf 2015; 24: 645–52. 
2. Auerbach AD, Landefeld CS, Shojania KG. The tension between 
needing to improve care and knowing how to do it. N Engl J Med 
2007; 357: 608–13.
3. Marshall M, Pronovost P, Dixon-Woods M. Promotion of improvement 
as a science.  Lancet 2013; 381: 419–21.    
4. Batalden PB, Davidoff F. What is ‘‘quality improvement’’ and how can 
it transform healthcare? Qual Saf Health Care 2007; 16: 2–3. 
5. Jones A, Williams A, Carson-Stevens A. Integrating quality 
improvement into pre-registration education. Nurs Stand 2013; 27: 
44-8.
6. Bauer MS, Damschroder L, Hagedorn H, Smith J, Kilbourne AM. An 
introduction to implementation science for the non-specialist. BMC 
Psychol 2015; 16: 32. 
7. McDonald KM, Schultz EM, Chang C. Evaluating the state of quality-
improvement science through evidence synthesis: insights from the 
closing the quality gap series. Perm J 2013; 17: 52-61. 
8. The Health Foundation, Improvement Science Fellowships. 
London: The Health Foundation, 2011. Available October 15, 2013 
from: http://www.renal.org/docs/default-source/what-we do/
Improvement_Science_Fellowships.pdf?sfvrsn=0 
9. Helfrich CD, Damschroder LJ, Hagedorn HJ, Daggett GS, Sahay A, 
Ritchie M, et al. A critical synthesis of literature on the promoting 
action on research implementation in health services (PARIHS) 
framework. Implement Sci 2010; 5: 82-101.
10. Conry MC, Humphries N, Morgan K, McGowan Y, Montgomery A, 
Vedhara K, Panagopoulou E, Mc Gee H. A 10 year (2000–2010) 
systematic review of interventions to improve quality of care in 
hospitals. BMC Health Serv Res 2012; 12: 1-16.
11. Øvretveit J, Leviton L, Parry G. Increasing the generalisability of 
improvement research with an improvement replication programme. 
BMJ Qual Saf 2011; 20: i87-i91. 
12. Leviton L. Reconciling complexity and classification in quality 
improvement research. BMJ Qual Saf 2011; 20: i28-29. 
13. Hovlid E, Bukve O, Haug K, Aslaksen AB, von Plessen C. Sustainability 
of healthcare improvement: what can we learn from learning theory? 
BMC Health Serv Res 2012; 12: 235. 
14. Dominguez DG, Fike DS, MacLaughlin EJ, Zorek JA. A comparison of 
the validity of two instruments assessing health professional student 
perceptions of interprofessional education and practice. J Interprof 
Care 2015; 29: 144–9.
15. Estrada CA, Dolansky MA, Singh MK, Oliver BJ, Callaway-Lane C, 
Splaine M, et al. Mastering improvement science skills in the new era 
of quality and safety: the Veterans Affairs National Quality Scholars 
Program. J Eval Clin Pract 2012; 18: 508–14. 
10.1515/sjph-2017-0011 Zdr Varst 2017; 56(2): 82-90
90
16. Jones J, Hunter D. Qualitative research: consensus methods for 
medical and health service research. BMJ 1995; 311: 376–80.
17. Minkman M, Ahans K, Fabbricotti I, Nabitz U, Huijsman R. A quality 
management model for integrated care: results of a Delphi and 
Concept Mapping study. Int J Qual Health Care 2009; 21: 66–75. 
18. List D. The Consensus group technique: a users’ manual. Wellington: 
Original Books, 2005.  
19. Race KE, Hotch DF. Rehabilitation program evaluation: use of focus 
groups to empower clients. Eval Rev 1994; 18: 730-40.
20. The Scottish Government. The healthcare quality strategy for NHS 
Scotland. Edinburg: The Scottish Government, 2010.
21. Booth BJ, Zwar N, Harris MF. Healthcare improvement as planned 
system change or complex responsive processes: a longitudinal case 
study in general practice. BMC Fam Pract 2013; 14: 51. 
22. Robson W. Eliminating avoidable harm: time for patient safety to 
play a bigger part in professional education and practice. Nurse Educ 
Today 2014; 34: e1–2. 
23. Hall WL, Zierler BK. Interprofessional education and practice guide 
No. 1: developing faculty to effectively facilitate interprofessional 
education. J Interprof Care 2015; 29: 3–7.
THE EFFECT OF AN EDUCATIONAL INTERVENTION IN FAMILY PHISICIANS 
ON SELF-RATED QUALITY OF LIFE IN PATIENTS WITH MEDICALLY 
UNEXPLAINED SYMPTOMS
Vojislav IVETIĆ1,2, Klemen PAŠIĆ2, Polona SELIČ3*
1University of Maribor, Faculty of Medicine, Department of Family Medicine, Taborska 8, 2000 Maribor, Slovenia
2SAVA MED, d.o.o., Cesta k Dravi 8, 2241 Spodnji Duplek, Slovenia
3University of Ljubljana, Faculty of Medicine, Department of Family Medicine, 
Poljanski nasip 58, 1000 Ljubljana, Slovenia
Received: May 9, 2016
Accepted: Sep 2, 2016
Original scientific article
Introduction. Medically unexplained symptoms (MUS) are very common in family medicine, despite being a poorly-
defined clinical entity. This study aimed to evaluate the effect of an educational intervention (EI) on self-rated quality 
of life, treatment satisfaction, and the family physician-patient relationship in patients with MUS. 
Methods. In a multi-centre longitudinal intervention study, which was performed between 2012 and 2014, patients 
were asked to rate their quality of life, assess their depression, anxiety, stress and somatisation, complete the 
Hypochondriasis Index, the Medical Interview Satisfaction Scale and the Patient Enablement Instrument for assessing 
the physician-patient relationship, before and after the EI. 
Results. The mean values before and after the intervention showed that after the EI, patients with MUS gave a lower 
(total) mean rating of their health issues and a higher rating of their quality of life, and they also had a more positive 
opinion of their relationship with the physician (p<0.05). However, there were no differences in the (total) rating of 
treatment satisfaction before and after the EI (p=0.423). Significant differences in the symptoms in patients with MUS 
before and after the intervention were confirmed for stress, somatisation and hypochondriasis (p<0.05). 
Conclusions. It could be beneficial to equip family physicians with the knowledge, skills and tools to reduce 
hypochondriasis and somatisation in MUS patients, which would improve patients’ self-rated health status.
Uvod. V družinski medicini so medicinsko nepojasnjena stanja (MNS) pogosta, vendar slabo opredeljena klinična 
entiteta; od 2,5 do 25% bolnikov, ki obiskujejo zdravnika, se pritožuje zaradi telesnih simptomov, za katere ni mogoče 
odkriti patofiziološkega vzroka. Namen študije je bil odkriti dejavnike, povezane s samooceno kakovosti življenja pri 
bolnikih z MNS v povezavi z edukativno intervencijo (EI), ki so ji bili podvrženi zdravniki.
Metode. K sodelovanju v multicentrični vzdolžni intervencijski raziskavi, ki je potekala od leta 2012 do leta 2014, 
je bilo povabljenih 90 zdravnikov družinske medicine, v sodelovanje jih je privolilo 63 (70% ali 7,5% vseh timov 
družinske medicine v Sloveniji). Po koncu prve faze je bila polovica zdravnikov (32 od 63) povabljena na dvodnevno 
usposabljanje s področja prepoznavanja in obravnave bolnikov z MNS. Bolniki so pred EI in po njej izpolnjevali 
vprašalnike o kakovosti življenja, depresiji, anksioznosti, stresu in somatizaciji, hipohondriji, zadovoljstvu in o 
partnerskem odnosu z zdravnikom.
Rezultati. Zdravniki so s sistematičnim vzorčenjem povabili k sodelovanju 1410 bolnikov, v sodelovanje je privolilo 
826 (58,58%) bolnikov, od tega 422 z izpolnjenimi kriteriji za MNS: (1) starost (18 do 80 let), (2) zdravnikov sum in (3) 
izpolnjeni točkovni in klinični kriteriji za MNS (prisotnost simptoma vsaj tri mesece, klinično pomembne težave pri 
bolniku, nezmožnost razlage simptoma s katerokoli znano telesno boleznijo). Povprečna starost bolnikov je bila 50,35 
leta ± 11,49 leta, izstopale so ženske (64,9 %) in osebe z osnovnošolsko (23,7%) in srednješolsko izobrazbo (29,1%). 
Brezposelnih je bilo 28,9% bolnikov. Bolniki z MNS so po intervenciji izkazovali v povprečju nižjo (skupno) oceno težav 
in višjo kakovost življenja ter boljše mnenje o partnerskem odnosu z zdravnikom (p<0,05), ne pa razlik v (skupni) oceni 
zadovoljstva z obravnavo pred intervencijo in po njej (p=0,423). Značilne razlike v simptomih pri bolnikih z MNS pred 
intervencijo in po njej smo potrdili v primeru stresa, somatizacije in hipohondrije (p<0,05).
Zaključki. Edukacija zdravnikov je pri bolnikih z MNS izboljšala samooceno težav z zdravjem in kakovosti življenja, 
vplivala je na znižanje stopnje simptomov stresa, somatizacije in znižala stopnjo hipohondrije ter izboljšala mnenje 
bolnikov o partnerskem odnosu z zdravnikom. Do sedaj v Sloveniji MNS kot ena od specifičnih vsebin dela v družinski 
medicini niso bila dovolj raziskana, tudi pristopi k obravnavi bolnikov z MNS še niso bile oblikovani. Zdravnike bi 
kazalo opremiti z znanji, veščinami in orodji za zmanjševanje hipohondrije in somatizacije pri bolnikih z MNS, saj bi 
na ta način tudi izboljšali samooceno zdravja bolnikov.
ABSTRACT
Keywords: 
medically unexplained 
symptoms, family 
medicine, educational 
intervention, quality 
of life, treatment 
satisfaction, family 
physician-patient 
relationship
IZVLEČEK
Ključne besede: 
medicinsko 
nepojasnjena stanja, 
družinska medicina, 
edukacijska 
intervencija, kakovost 
življenja, zadovoljstvo 
bolnikov, odnos med 
družinskim zdravnikom 
in bolnikom
*Corresponding author: Tel: ++ 386 31 379 707; E-mail: polona.selic@siol.net
10.1515/sjph-2017-0012 Zdr Varst 2017; 56(2): 91-98
91
VPLIV EDUKATIVNE INTERVENCIJE ZDRAVNIKOV NA SAMOOCENO 
KAKOVOSTI ŽIVLJENJA, ZADOVOLJSTVA Z OBRAVNAVO IN PARTNERSKEGA 
ODNOSA Z ZDRAVNIKOM DRUŽINSKE MEDICINE PRI BOLNIKIH Z MEDICINSKO 
NEPOJASNJENIMI STANJI
Ivetić V, Pašić K, Selič P. The effect of an educational intervention in family phisicians on self-rated quality of life in patients with medically unexplained symptoms. 
Zdr Varst 2017; 56(2): 91-98
© Nacionalni inštitut za javno zdravje, Slovenija. 
1 INTRODUCTION
Medically unexplained symptoms (MUS) are very common 
in family medicine despite being a poorly-defined clinical 
entity; approximately 2.5%-25% of patients who visit a 
family physician (FP) complain of physical symptoms for 
which no physical pathology can be found even after a 
number of examinations (1-2). The diagnosis of MUS still 
has a very negative connotation for patients (3). A recent 
Slovenian study (4) observed an 8.6% frequency of MUS 
in family medicine practice attendees, which shows that 
MUS is as much a public health care problem in Slovenia as 
anywhere else in the world (1, 2). Other Slovenian studies 
on the new project of model practices in family medicine 
did not show any more successful treatment of patients 
with MUS (5). 
Some studies show that depression and anxiety disorders 
are comorbid with medically unexplained conditions (6). 
In the literature, MUS are often described as equivalent 
to somatoform disorders, but studies show that only 
approximately 25% of MUS patients meet the criteria 
for one of the somatoform disorders according to the 
Diagnostic and Statistical Manual of Mental Disorders, 
4th Edition (DSM-IV-TR), and the 10th revision of the 
International Statistical Classification of Diseases and 
Related Health Problems (ICD-10) (7-9), which means that 
MUS is a problem which goes beyond the definition and 
criteria for somatoform disorders. Recent international 
studies emphasise the importance of one or more 
psychosocial stressors that could not be identified 
either by the patients or the physicians (10). Following 
an explanation of how such stressors can affect the 
occurrence of physical symptoms in patients, five areas 
were researched in detail, namely everyday stress and 
problems in life, prolonged effect/effects of childhood 
stress, physical manifestation of depression, post-
traumatic stress disorder (PTSD), and anxiety (11). 
According to the results (12), the pharmacological 
treatment of MUS is among the most common interventions. 
Also very common is the use of cognitive behavioural 
methods and other non-specific interventions. Cognitive 
behavioural approaches proved to be the most effective 
(12) and were confirmed (13) as being useful in reducing 
medically unexplained chronic fatigue in the paediatric 
population (aged from 11 to 18 years).
The aim of our study was to examine the effect of an 
educational intervention (EI) for FPs on the MUS patients’ 
self-rated quality of life, treatment satisfaction and 
relationship with the physician.
2 METHODS
2.1 Participants and Procedures
2.1.1 Participating Physicians
A total of 90 family medicine practices or FPs from across 
Slovenia were invited to participate in a multi-centre 
longitudinal intervention study. The recruitment of the 
physicians was carried out between January and March 
2012. The practices were selected by random sampling 
from the publicly available register of all Slovenian 
family medicine practices provided by the Institute of 
Public Health of the Republic of Slovenia (ZZZS) (14). This 
register enabled us to include in our study a population 
with a great socioeconomic diversity and different ethnic 
backgrounds. Out of 90 invited physicians, a total of 63 
(70%) decided to take part in the first phase of our study 
(7.5% of all family medicine teams in Slovenia) (14).
The FPs were mailed the Study Protocol and Questionnaires. 
After completing the first phase of the study, half of the 
physicians (32 of 63) were invited to attend a two-day 
training on the identification and treatment of patients 
with MUS. All the invited physicians attended the training 
and were taught about the basic principles of cognitive 
behavioural therapy (CBT), mental illness diagnostic 
criteria and a proper multidisciplinary approach toward 
patients with MUS. Finally, a Model of Careful Assessment 
(MCA) as an intervention in the treatment of patients with 
MUS was introduced and trained. In the continuation of 
the study, those physicians represented the intervention 
group. Every three months during a 12-month period, they 
were sent by post a clinical vignette of a patient with 
MUS, which was also used in the workshops, as well as 
a MCA. In this way, the intervention group participants 
were able to refresh their newly gained knowledge. The 
physicians from the control group were sent only the 
technical instructions for the conduct of the study.
In the second phase of the study, a total of 45 (23 from 
the intervention group and 22 from the control group) 
of the participating physicians decided to continue their 
participation, which represented 5.36% of all family 
medicine teams in Slovenia (Figure 1).
In physicians, attendance at the two-day training counted 
as inclusion criteria for intervention group, which in fact 
acted as an interventional variable when following up 
several characteristics in MUS patients.
10.1515/sjph-2017-0012 Zdr Varst 2017; 56(2): 91-98
92
10.1515/sjph-2017-0012 Zdr Varst 2017; 56(2): 91-98
93
2.2 Instruments
The FPs were asked to complete Questionnaire A 
(demographic information about the physician and 
practice), whereas the patients answered Questionnaire 
B, consisting of the PHQ-15 (Patient Health Questionnaire) 
for assessing the severity of non-specifi c somatic symptoms 
that have been shown to be highly associated with MUS 
diagnosis in the past four weeks (15), and three additional 
questions regarding the patients’ problems and the 
physicians’ clinical opinion (i.e., the symptom has been 
present for at least three months, the symptom has been 
causing clinically signifi cant problems for the patient, and 
the symptom cannot be explained by any known physical 
disease) (1). A PHQ-15 score of 15 or higher and positive 
answers to all three additional questions formed the criteria 
for the inclusion of patients with MUS. The patients were 
also asked to complete Questionnaire D, which included 
theEQ-5D-3L (EuroQoL 5D-3L) instrument rating quality of 
life (16), as well as the 4DSQ (Four-Dimensional Symptom 
Questionnaire) for assessing depression, anxiety, stress 
and somatisation (17), the WI (Whiteley Hypochondriasis 
Index) (18), the MISS-21 (Medical Interview Satisfaction 
Scale) for measuring patient satisfaction (19), and the 
PEI (Patient Enablement Instrument) for assessing the 
physician-patient relationship (20).
Questionnaires A and B were completed only in the fi rst 
phase of the study (0 months), and Questionnaire D was 
completed at inclusion into the study (0 months) and 12 
months later. The latest data are crucial for this study 
aim, providing 269 MUS patients´ refl ection on depression, 
anxiety, stress and somatisation, satisfaction, and also 
assessing the physician-patient relationship.
2.3 Statistical Analysis
Those parts of the questionnaire which were adapted 
from international questionnaires were assessed for their 
construct validity (using factor analysis) and measuring 
reliability. The reliability of all measures was exemplary 
(Cronbach’s alpha α>0.80). To analyse the differences in 
MUS patients before and after the EI, we used the paired 
sample t-test for numeric variables and the chi-square 
test for categorical variables. Statistical signifi cance for 
the entire analysis was set at a 5% risk level (p≤0.05).
The data were statistically processed using SPSS 21.0 
statistical software (IBM Corp., Grouponk, NY, USA). 
3 RESULTS
At the beginning of the recruitment process, there were 
422 patients in the group with MUS, and 404 patients in 
the group without MUS. 
2.1.2  Participating Patients
For three consecutive ‘typical’ working days, determined 
by the participating physicians themselves, patients aged 
between 18 and 80 years who came to the practice for any 
reason and who were suspected of MUS were invited by 
their physician to participate in this study. The physicians 
screened a total of 4,921 patients and invited 1,410 of 
them; 584 (41.42%) of the invited patients refused to 
take part in the study. The most common reason for non-
participation was lack of time. 
In the end, a total of 826 (58.58%) of patients decided 
to participate in the fi rst phase of the study, with 422 of 
these fulfi lling the criteria for MUS (patient age between 
18 and 80 years, suspicion of MUS, and fulfi lled criteria for 
MUS from Questionnaire B) and 404 patients not fulfi lling 
the criteria for MUS. The latter were not followed-up. 
In the second phase of the study (carried out between 
April 2012 and June 2013), 510 of the 1,410 initially invited 
patients (36.17%) decided to continue their participation; 
of those, 269 patients had MUS. Those 269 MUS patients 
were analysed after the 12-month period.
Figure 2 shows the participation of patients by the study 
phase.
Figure 1.
Figure 2.
A graphic presentation of the number of physicians 
participating in either phase of the study.
A graphic presentation of the number of patients 
participating in either phase of the study.
Table 1. The analysis of differences in the (total) rating of health issues, quality of life, treatment satisfaction and the physician-
patient relationship in patients with MUS, before and after the intervention.
Mean ± standard deviation
95% confidence interval
   lower limit
   upper limit
Mean ± standard deviation
95% confidence interval
   lower limit
   upper limit
Mean ± standard deviation
95% confidence interval
   lower limit
   upper limit
Mean ± standard deviation
95% confidence interval
   lower limit
   upper limit
8.51±1.40
8.38
8.65
44.81±11.927
43.67
45.95
3.72±0.400
3.68
3.76
2.69±0.48
2.65
2.74
8.16±1.45
7.98
8.33
48.21±11.722
46.80
49.62
3.71±0.304
3.68
3.75
2.39±0.53
2.33
2.45
0.000
0.000
0.423
0.000
BEFORE INTERVENTION
n=422
(Total) rating of health issues
Quality of life (self-rated health)
(Total) rating of treatment satisfaction
Opinion on the physician-patient relationship
AFTER INTERVENTION
n=269
P
10.1515/sjph-2017-0012 Zdr Varst 2017; 56(2): 91-98
94
Gender structure: The whole cohort consisted of 826 
patients; 62.2% were women and 37.8% were men. The 
number of women was higher in the MUS patient group 
(64.9%), compared to the group without MUS (59.4%). 
However, this relatively higher proportion of women in 
the MUS group was not statistically significant (χ2=2.679; 
df=1; p=0.102). 
Patients’ age: The mean age of patients was 49.40±12.29 
years. The mean age of patients with MUS was 50.35±11.49 
years, which is approximately two years older than the 
mean age of patients without MUS (48.41±13.01 years; 
p=0.040). 
Educational level and employment status: The MUS group 
included 23.7% of patients who were primary school 
graduates (primary school leaving age is 14 in Slovenia), 
whereas in the group without MUS there were 13.9% of 
patients with primary school only; secondary education 
was completed by significantly fewer patients in the MUS 
group (29.1%) as compared to 38.9% of patients without 
MUS (p=0.002). The majority of patients had completed 
vocational (31.7%) and secondary (33.9%) education; 
those were followed by primary education (18.9%), post-
secondary, higher or university education (14.4%), and 
Master’s or Doctoral degree (1.1%). Most patients were in 
employment (39.8%), followed by retired people (26.9%) 
and the unemployed (21.3%); 9.0% of patients indicated 
that they were self-employed or farmers, whereas 
3.0% were secondary school or university students. A 
statistically significant greater proportion of patients 
with MUS were unemployed (with MUS: 28.9%; without 
MUS: 13.4%) a smaller proportion of them were employed 
(with MUS: 34.1%; without MUS: 45.7%), and a smaller 
proportion of them had secondary school or university 
student status (with MUS: 1.7%; without MUS: 4.5%; 
p=0.000). 
Other characteristics: There were no statistically 
significant differences observed between the two groups 
of patients in terms of settlement size (p=0.8865). 
Patients with MUS differed from others by the number 
of persons in the household (p = 0.028). In other words, 
patients with MUS lived in significantly larger households 
(3.39±1.46 members vs. 3.17±1.45 members).
3.1 The Effect of EI on Self-Rated Quality of Life, 
MUS, Hypochondriasis, Patient Satisfaction, and the 
Physician-Patient Relationship
10.1515/sjph-2017-0012 Zdr Varst 2017; 56(2): 91-98
4 DISCUSSION 
The aim of our study was to investigate the effect of the 
EI in physicians on some factors associated with quality 
of life and the perception of the relationship with the 
selected physician in patients with MUS. The mean values 
before and after the intervention showed that after the 
EI, patients with MUS gave a lower (total) mean rating 
of their health issues and a higher rating of their quality 
of life, and they also had a more positive opinion about 
their relationship with the physician (p<0.05; Table 1). 
However, there were no differences in the (total) rating of 
treatment satisfaction before and after the EI (p=0.423) 
Table 2. The analysis of differences in the symptoms of MUS patients, and the hypochondriasis index before and after the 
intervention.
Mean ± standard deviation
95% confidence interval
   lower limit
   upper limit
Mean ± standard deviation
95% confidence interval
   lower limit
   upper limit
Mean ± standard deviation
95% confidence interval
   lower limit
   upper limit
Mean ± standard deviation
95% confidence interval
   lower limit
   upper limit
Mean ± standard deviation
95% confidence interval
   lower limit
   upper limit
3.92±3.270
3.61
4.24
6.53±5.325
6.02
7.04
19.57±7.216
18.88
20.26
19.49±5.728
18.94
20.04
45.31±9.732
44.38
46.24
n=260 (61.6%)
n=126 (29.9%)
n=367 (87.2%)
n=393 (93.1%)
n=422 (100%)
n=170 (63.2%)
n=77 (28.6%)
n=237 (88.1%)
n=248 (92.2%)
n=269 (100%)
3.85±3.100
3.48
4.22
6.36±5.145
5.74
6.97
18.64±6.701
17.83
19.44
18.54±5.585
17.87
19.21
44.16±8.785
43.11
45.22
0.051
0.558
0.000
0.000
0.000
BEFORE INTERVENTION
n=422
Depression
(Criterion: depression >2)
Anxiety
(Criterion: anxiety >8)
Stress
(Criterion: stress >10)
Somatisation
(Criterion: somatisation >10)
Whiteley Hypochondriasis Index
AFTER INTERVENTION
n=269
P
(Table 1). Significant differences in the symptoms in 
patients with MUS before and after the intervention were 
confirmed for stress, somatisation and hypochondriasis 
(p<0.05; Table 2). 
The study topic corresponds to the current public health 
care policies and needs in Slovenia and around the world 
(11, 22, 23). The EI for physicians was prepared as a 
combination of several different approaches to treating 
patients with MUS in accordance with recommendations 
in the literature (21), focusing on practical work in small 
groups and presenting the MCA tool (24). A German study 
from 2007 (25) investigated the effect of a one-session 
95
10.1515/sjph-2017-0012 Zdr Varst 2017; 56(2): 91-98
96
EI using a cognitive-behavioural approach, observing 
a reduced number of visits to practices and fewer 
somatisation symptoms at 6-month follow-up. In our 
study, the intervention`s impact was evaluated through 
patients’ self-rated quality of life, treatment satisfaction, 
and their assessment of the physician-patient relationship, 
as well as through somatisation, anxiety, depression and 
hypochondriasis (Table 1, Table 2). We could not find such 
criteria (as a whole) in the available body of literature. 
4.1 EI and Self-Rated Quality of Life
After the EI, patients with MUS gave a lower (total) mean 
rating of their health issues and a higher self-rating of 
their quality of life (Table 1). A more recent Dutch study 
(26), which also employed the EQ-5D questionnaire, 
observed that MUS have a greater effect on the quality 
of life in a younger population (aged less than 65 years). 
A Slovenian study from 2014 (16) found that – considering 
the results of the EQ-5D questionnaire – the presence of 
psychological symptoms is associated with lower quality 
of life. Klemenc Ketiš et al. (16) observed a correlation 
between factors identified using the EQ-5D questionnaire 
(the presence of chronic disease, pain, and anxiety and 
depression), which are independently associated with 
psychological symptoms. In addition, findings of other 
authors (27) confirm the correlation between psychological 
and unexplained symptoms, and lower quality of life, as 
measured by the EQ-5D questionnaire. We reached the 
same conclusion with regard to the patients’ quality of 
life, and we also confirmed the EI’s effect on patients 
with MUS in terms of their quality of life. 
4.2 EI and Depression, Anxiety, Stress, and Somatisation 
and Hypochondriasis
In our study, patients with MUS had a lower mean level 
of depression, anxiety, stress and somatisation after the 
EI, and this reduction in stress and somatisation was 
confirmed as statistically significant (Table 2). Similar to 
our study, the majority of other studies (12, 28) found 
that improved communication with the MUS patient is 
beneficial, as is the use of empathy in the physician-
patient relationship, or better acceptance of MUS patients 
as they are. 
The analysis of differences in hypochondriasis in patients 
with MUS before and after the EI (Table 2) confirmed 
that the intervention – i.e., capacity building in 
physicians – significantly reduced (but did not eliminate 
altogether) hypochondriasis in MUS patients. A recent 
German study from 2013 (29) analysed the effect of 
the EI (a multidisciplinary approach and the inclusion 
of a psychosomatic therapist in the education of family 
physicians) on the treatment of MUS patients, and found 
a decrease in the level of hypochondriasis, which is 
comparable to the findings of our study. 
4.3 EI and Patients’ Treatment Satisfaction
Statistical analysis did not reveal significant differences 
in the (total) rating of treatment satisfaction before and 
after the EI (Table 1). Similarly, a study by Jackson, J. et 
al. (30) found that there were no significant differences 
in the treatment satisfaction of patients with MUS 
compared to patients without MUS. A study from Sri Lanka 
(31) investigated the effect of an intervention on several 
factors, including patients’ treatment satisfaction, and 
showed that patient satisfaction increased after the 
EI (mainly the education of physicians in cognitive-
behavioural techniques). Differences in the findings of 
various authors and those of this study are probably a 
result of the use of different types of intervention and 
different questionnaires for analysing patient satisfaction; 
this is why the findings are difficult to compare. 
4.4 EI and the Patient-Physician Relationship
We found (Table 1) that intervention in patients with 
MUS significantly improved their opinion of the physician-
patient relationship. The results of our study match those 
of another study carried out in Scotland (32), which showed 
that intervention in MUS patients significantly improved 
their opinion of the physician-patient relationship, but 
failed to confirm significant differences in the (total) 
rating of treatment satisfaction before and after the EI. 
We could not find any data on the intervention’s effect 
on the physician-MUS patient relationship in comparable 
foreign studies (33, 34), which provides an opportunity 
for new research in this field, and for improving the 
quality of treatment of patients with MUS. 
4.5 Limitations of the Study 
The criteria for including patients in the group of patients 
with MUS are non-standardised, due to the non-existence 
of an accurate definition of MUS. For the purposes of this 
study, the inclusion criteria were based on a large review 
of the literature, and represented a combination of 
several factors to reduce this limitation. The acquisition 
of data excluded all physicians and patients who refused 
to participate in the study. 
The study included patients who had visited their 
physician for three ‘typical’ working days, coming to 
the practice for different reasons, many due to an acute 
disease or the worsening of a chronic disease. Because 
of this, frequent visitors to family medicine practices 
had a greater chance of being included in the study. This 
study limitation could have been eliminated if we had 
included random sampling of patients using a register of 
family medicine practice patients, but the participation 
of patients would have most probably been even lower. 
The decrease in the number of participating FPs and 
patients was relatively large by the end of the study, 
10.1515/sjph-2017-0012 Zdr Varst 2017; 56(2): 91-98
97
which is also reported by other researchers of MUS (12, 
35). We also failed to track possible patterns within the 
MUS patients group after one-year-period, and did not 
analyse characteristics of the fall-out patients.
The EI was relatively short (two days) and was a ‘one-
off.’ The EI attendees were probably more motivated to 
participate (a personal interest in this topic) than other 
physicians.
The success of the EI was at first limited to the observation 
of quality of life, treatment satisfaction and opinion 
of the physician-patient relationship. However, this 
limitation has been mitigated by the fact that quality of 
life was additionally evaluated based on the assessment 
of somatisation, depression, anxiety and hypochondriasis.
The success of the EI was followed up only over one-year-
period. Unfortunately, we do not have any follow-up data 
for the participating physicians and patients after one 
year. We also do not have any follow-up data about the 
incidence of MUS symptoms after EI, neither about the 
frequency of patients’ attendance at family clinics. 
In this study, we decided not to record the occurrence 
of acute diseases or the worsening frequency of chronic 
disease, which might have affected the self-rated quality 
of life in our patients. 
5 CONCLUSIONS
Education of physicians improved the self-rating of 
health issues and quality of life by patients with MUS. 
Moreover, it also helped to reduce symptoms of stress and 
somatisation, and decreased the level of hypochondriasis, 
while also improving patients’ opinion of the physician-
patient relationship. In Slovenia, MUS are still not 
sufficiently researched as a specific family medicine 
topic, and approaches to the treatment of patients with 
MUS have not yet been developed. Considering the results 
of our study, we find that it is necessary to equip FPs with 
knowledge, skills and tools for reducing hypochondriasis 
and somatisation in MUS patients. In this way, we could 
also improve their self-rated health status. In addition, 
the observed correlation with stress and anxiety highlights 
the necessity to manage anxiety in patients with MUS, 
and strengthen problem-centred strategies for managing 
stress.
CONFLICT OF INTEREST
The authors declare that they have no conflicts of interest. 
FUNDING
This study was partly supported by the Slovenian Research 
Agency, Research Programme Code P3-0339.
ETHICAL APPROVAL 
The study was approved by the National Medical Ethics 
Committee of the Republic of Slovenia, Decision No. 
45/05/11bis dated 25 May 2011. 
REFERENCES
1. Peveler R, Kilkenny L, Kinmonth AL. Medically unexplained physical 
symptoms in primary care: a comparison of self-report screening 
questionnaires and clinical opinion. J Psychosom Res 1997; 42: 245-
52.
2. Burton C. Beyond somatisation: a review of the understanding and 
treatment of medically unexplained physical symptoms (MUPS). Br J 
Gen Pract 2003; 53: 231-9.
3. Stone J, Wojcik W, Durrance D, Carson A, Lewis S, MacKenzie L, et 
al. What should we say to patients with symptoms unexplained by 
disease?: the “number needed to offend”. BMJ 2002; 325: 1449-50.
4. Ivetić V. Medicinsko nepojasnjena stanja v ambulanti družinske 
medicine: doktorska disertacija. Ljubljana: Medicinska fakulteta, 
2015. 
5. Poplas-Susič T, Švab I, Kersnik J. The project of model practices in 
family medicine in Slovenia. Zdrav Vestn 2013; 82: 635–47.
6. Schur EA, Afari N, Furberg H, Olarte M, Goldberg J, Sullivan PF, et 
al. Feeling bad in more ways than one: comorbidity patterns of 
medically unexplained and psychiatric conditions. J Gen Intern Med 
2007; 22: 818-21.
7. Olde Hartman T, Hassink-Franke L, Dowrick C, Fortes S, Lam C, 
van der Horst H, et al. Medically unexplained symptoms in family 
medicine: defining a research agenda. Proceedings from WONCA 
2007. Fam Pract 2008; 25: 266-71.
8. Svetovna zdravstvena organizacija. Mednarodna klasifikacija bolezni 
in sorodnih zdravstvenih problemov za statistične namene: MKB-
10. 10. revizija. Ljubljana: Inštitut za varovanje zdravja Republike 
Slovenije, 1995.
9. Smith RC, Gardiner JC, Lyles JS, Sirbu C, Dwamena FC, Hodges A, 
et al. Exploration of DSM-IV criteria in primary care patients with 
medically unexplained symptoms. Psychosom Med 2005; 67: 123-9.
10. Švab I, Torres-González F, Clarke D, Stern A, Edwards T, Ivbijaro G. A 
three session intervention for patients with co-morbid physical and 
mental health problems and medically unexplained symptoms. 16th 
Wonca Europe conference Family Medicine into the Future Blending 
Health & Cultures. Malaga, Spain, 2010.
11. Clarke DD. Solving medical mysteries: hidden stresses and 
unexplained symptoms. Zdr Varst 2016; 55: 152-4. 
12. Sumathipala A. What is the evidence for the efficacy of treatments 
for somatoform disorders?: a critical review of previous intervention 
studies. Psychosom Med 2007; 69: 889-900.
13. Chalder T, Tong J, Deary V. Family cognitive behaviour therapy for 
chronic fatigue syndrome: an uncontrolled study. Arch Dis Child 
2002; 86: 95-7.
10.1515/sjph-2017-0012 Zdr Varst 2017; 56(2): 91-98
98
14. Zavod za zdravstveno zavarovanje Slovenije. Število opredeljenih 
po starostnih skupinah pri aktivnih zdravnikih SA, OD+ŠD na dan 
30.11.2014. Available March 29, 2015 from: http://www.zzzs.si/
zzzs/internet/zzzs.nsf/o/6F2CAD56EE119706C125770B00390171
15. Kroenke K, Spitzer RL, Williams JB. The PHQ-15: validity of a new 
measure for evaluating the severity of somatic symptoms. Psychosom 
Med 2002; 64: 258-66.
16. Klemenc-Ketiš Z, Kuhar P, Kersnik J, Burazeri G, Czabanowska K. 
Self-assessment questionnaire for family doctors’ assessment of 
quality improvement competencies: a cross-cultural adaptation in 
Slovenia. Zdr Varst 2014; 53: 34-41.
17. Terluin B, van Marwijk HW, Ader HJ, de Vet HC, Penninx BW, Hermens 
ML, et al. The Four-Dimensional Symptom Questionnaire (4DSQ): a 
validation study of a multidimensional self-report questionnaire to 
assess distress, depression, anxiety and somatization. BMC Psychiatry 
2006; 6: 34.
18. Speckens AE, Spinhoven P, Sloekers PP, Bolk JH, van Hemert AM. A 
validation study of the Whitely Index, the Illness Attitude Scales, 
and the Somatosensory Amplification Scale in general medical and 
general practice patients. J Psychosom Res 1996; 40: 95-104.
19. Meakin R, Weinman J. The ‘Medical Interview Satisfaction Scale’ 
(MISS-21) adapted for British general practice. Fam Pract 2002; 19: 
257-63.
20. Howie JG, Heaney DJ, Maxwell M, Walker JJ, Freeman GK, Rai H. 
Quality at general practice consultations: cross sectional survey. BMJ 
1999; 319: 738-43.
21. Dowrick C. Understanding unexplained physical symptoms in primary 
care. Primary Care Mental Health 2005; 3: 215-9.
22. Svab I. Universality and uniqueness in family medicine. Eur J Gen 
Pract; 20: 91-2.
23. Ivetić V, Kersnik J, Klemenc-Ketis Z, Svab I, Kolsek M, Poplas-Susic 
T. Opinions of Slovenian family physicians on medically unexplained 
symptoms: a qualitative study. J Int Med Res 2013; 41: 705-15.
24. Walker EA, Unutzer J, Katon WJ. Understanding and caring for the 
distressed patient with multiple medically unexplained symptoms. J 
Am Board Fam Pract 1998; 11: 347-56.
25. Martin A, Rauh E, Fichter M, Rief W. A one-session treatment for 
patients suffering from medically unexplained symptoms in primary 
care: a randomized clinical trial. Psychosomatics 2007; 48: 294-303.
26. Arnold IA, Speckens AE, van Hemert AM. Medically unexplained 
physical symptoms: the feasibility of group cognitive-behavioural 
therapy in primary care. J Psychosom Res 2004; 57: 517-20.
27. Molarius A, Berglund K, Eriksson C, Eriksson HG, Linden-Bostrom M, 
Nordstrom E, et al. Mental health symptoms in relation to socio-
economic conditions and lifestyle factors-a population-based study 
in Sweden. BMC Public Health 2009; 9: 302.
28. Deary V, Chalder T, Sharpe M. The cognitive behavioural model 
of medically unexplained symptoms: a theoretical and empirical 
review. Clin Psychol Rev 2007; 27: 781-97.
29. 29. Schaefert R, Kaufmann C, Wild B, Schellberg D, Boelter R, Faber 
R, et al. Specific collaborative group intervention for patients with 
medically unexplained symptoms in general practice: a cluster 
randomized controlled trial. Psychother Psychosom 2013; 82: 106-
19.
30. Jackson J, Kincey J, Fiddler M, Creed F, Tomenson B. Differences 
between out-patients with physical disease and those with medically 
unexplained symptoms with respect to patient satisfaction, 
emotional distress and illness perception. Br J Health Psychol 2004; 
9: 433-46.
31. Sumathipala A, Hewege S, Hanwella R, Mann AH. Randomized 
controlled trial of cognitive behaviour therapy for repeated 
consultations for medically unexplained complaints: a feasibility 
study in Sri Lanka. Psychol Med 2000; 30: 747-57.
32. Howie JG, Heaney DJ, Maxwell M, Walker JJ. A comparison of a Patient 
Enablement Instrument (PEI) against two established satisfaction 
scales as an outcome measure of primary care consultations. Fam 
Pract 1998; 15: 165-71.
33. Olde Hartman TC, Woutersen-Koch H, Van der Horst HE. Medically 
unexplained symptoms: evidence, guidelines, and beyond. Br J Gen 
Pract 2013; 63: 625-6.
34. Hatcher S, Arroll B. Assessment and management of medically 
unexplained symptoms. BMJ 2008; 336: 1124-8.
35. Smith RC, Lyles JS, Gardiner JC, Sirbu C, Hodges A, Collins C, et 
al. Primary care clinicians treat patients with medically unexplained 
symptoms: a randomized controlled trial. J Gen Intern Med 2006; 
21: 671-7.
USING MOVIES IN FAMILY MEDICINE TEACHING: 
A REFERENCE TO EURACT EDUCATIONAL AGENDA
Zalika KLEMENC KETIŠ1,2,3*, Igor ŠVAB2
1University of Maribor, Faculty of Medicine, Department of Family Medicine, Taborska 8, 2000 Maribor, Slovenia
2University of Ljubljana, Faculty of Medicine, Department of Family Medicine, 
Poljanski nasip 58, 1000 Ljubljana, Slovenia
3Community Health Centre Ljubljana, Metelkova 9, 1000 Ljubljana, Slovenia
Received: Apr 6, 2016
Accepted: Sep 13, 2016
Original scientific article
Introduction. Cinemeducation is a teaching method where popular movies or movie clips are used. We aimed 
to determine whether family physicians’ competencies as listed in the Educational Agenda produced by the 
European Academy of Teachers in General Practice/Family Medicine (EURACT) can be found in movies, and to 
propose a template for teaching by these movies.
Methods. A group of family medicine teachers provided a list of movies that they would use in cinemeducation. 
The movies were categorised according to the key family medicine competencies, thus creating a framework 
of competences, covered by different movies. These key competencies are Primary care management, Person-
centred care, Specific problem-solving skills, Comprehensive approach, Community orientation, and Holistic 
approach.
Results. The list consisted of 17 movies. Nine covered primary care management. Person-centred care was 
covered in 13 movies. Eight movies covered specific problem-solving skills. Comprehensive approach was 
covered in five movies. Five movies covered community orientation. Holistic approach was covered in five 
movies.
Conclusions. All key family medicine competencies listed in the Educational Agenda can be taught using 
movies. Our results can serve as a template for teachers on how to use any appropriate movies in family 
medicine education.
Uvod. Cinemeducation je metoda poučevanja z uporabo filmov ali filmskih izsekov. Namen te raziskave je bil 
oceniti, ali filme lahko uporabimo pri poučevanju družinske medicine na osnovi kompetenc družinske medicine, 
ki jih je predlagala Evropske akademija učiteljev v družinski medicini (EURACT). 
Metode. Skupina učiteljev družinske medicine je predlagala seznam filmov, ki bi jih lahko uporabili v 
poučevanju družinske medicine. Dva učitelja družinske medicine sta filme pregledala in na osnovi vsebine 
ter prizorov iz filmov za vsak film določila seznam kompetenc družinske medicine, ki se lahko s pomočjo 
določenega filma učijo. Kompetence družinske medicine so vodenje primarne zdravstvene oskrbe, v osebo 
usmerjena zdravstvena oskrba, zmožnost reševanja specifičnih problemov, celostni pristop, usmerjenost v 
skupnost in celovito oblikovanje modelov. 
Rezultati. Končen seznam je obsegal 17 filmov. Devet jih je zajemalo kompteneco vodenja primarne 
zdravstvene oskrbe. Kompetenca v osebo usmerjene zdravstvene oskrbe je bila zajeta v 13. filmih. Osem filmov 
je vsebovalo kompetenco zmožnosti reševanja specifičnih problemov. Celovit pristop je bil zajet v petih filmih, 
prav tako tudi kompetenca usmerjenosti v skupnost. Celosten pristop je bil zajet v petih filmih.
Zaključek. Vse kompetence družinske medicine, ki jih je predlagal EURACT, se lahko učijo tudi s pomočjo 
filmov. Rezultati te raziskave lahko služijo kot predloga učiteljem družinske medicine glede načina uporabe 
filmov v izobraževanju iz družinske medicine.  
ABSTRACT
Keywords: 
family practice, 
medical education, 
narration, professional 
competences
IZVLEČEK
Ključne besede: 
družinska medicina, 
medicinska 
izobraževanja, 
narativna medicina, 
profesionalne 
kompetence
*Corresponding author: Tel: ++ 386 3 896 31 22; E-mail: zalika.klemenc@um.si
10.1515/sjph-2017-0013 Zdr Varst 2017; 56(2): 99-106
UPORABA FILMOV V POUČEVANJU DRUŽINSKE MEDICINE NA OSNOVI 
KOMPETENC DRUŽINSKE MEDICINE
Klemenc Ketiš Z, Švab I. Using movies in family medicine teaching: a reference to EURACT educational agenda. Zdr Varst 2017; 56(2): 99-106
99
© Nacionalni inštitut za javno zdravje, Slovenija. 
1 INTRODUCTION
Family medicine is a unique medical specialty, which does 
not only provide clinical care, but predominantly focuses 
on continuing, comprehensive and holistic care that 
is person-based (1). Family physician should therefore 
also possess special skills in order to be able to deliver 
appropriate care, such as communication skills.
The Educational Agenda produced by the European 
Academy of Teachers in General Practice/Family Medicine 
(EURACT) (2) has become a standard for developing family 
medicine curricula in Europe. This document is based 
on the European Definition of General Practice/Family 
Medicine (1). 
The European Definition of General Practice/Family 
Medicine(1) was developed following the need for an 
authoritative statement which would define both the 
discipline of family medicine and the tasks of the family 
physician, and relate them to the context of the health 
care system (1). It defines six main competencies of family 
medicine: primary care management, person-centred 
care, specific problem-solving skills, comprehensive 
approach, community orientation and holistic modelling. 
Primary care management includes the ability to manage 
primary contact with patients, dealing with unselected 
problems; to cover the full range of health conditions; 
to co-ordinate care with other professionals in primary 
care and with other specialists; to master effective and 
appropriate care provision and health service utilisation; 
to monitor, assess and improve quality and safety of 
care; to make available to the patient the appropriate 
services within the health care system; and to act as an 
advocate for the patient. Person-centred care includes 
the ability to adopt a person-centred approach in dealing 
with patients and problems in the context of the patient’s 
circumstances; to develop and apply the general practice 
consultation, so as to bring about an effective doctor-
patient relationship, with respect for the patient’s 
autonomy; to communicate, set priorities and act in 
partnership; to promote patient empowerment; and to 
provide longitudinal continuity of care as determined by 
the needs of the patient referring to continuing and co-
ordinated care management. Comprehensive approach 
includes the ability to manage multiple complaints and 
pathologies, both acute and chronic health problems in the 
individual, simultaneously; to promote health and well-
being by applying health promotion and disease prevention 
strategies appropriately; and to manage and co-ordinate 
health promotion, prevention, cure, care, palliation and 
rehabilitation. Community orientation includes the ability 
to reconcile health needs of individual patients with the 
community in which they live, in balance with available 
resources. Holistic approach includes the ability to use a 
bio-psycho-social model, taking into account cultural and 
existential dimensions. 
Based on these competencies that every family physician 
should possess, the Educational Agenda lists, in broad 
terms, educational objectives of family medicine at 
different levels of training (2). 
Usually, medical education is focused on studying 
diseases and their clinical management (3-5). Due to 
straightforwardness and rationality of such topics, they 
are usually relatively easy to teach using traditional 
teaching methods. On the other hand, medical education 
is also trying to teach more complex issues, namely 
comprehensive care, holistic care, multimorbidity, 
professionalism, ethical dilemmas, family functioning, 
communication in different situations, community 
orientation, etc. This requires adoption not only of 
knowledge and skills, but often also a change of attitudes 
of learners (4, 6), where traditional teaching methods are 
less likely to produce the expected results (6-9). 
Cinemeducation is a relatively new method of medical 
education where movies or movie clips are used. In 
recent years, also television series on medical issues 
are being used in this context (10). Cinema utilises sight 
and sound, which enhance learners’ ability of watching 
and listening. Moreover, movies stimulate discussions 
and reflections, which is a part of an active learning 
process (11). Cinemeducation is useful in teaching the 
bio-psycho-social-spiritual approach (12), especially the 
areas of communication, palliative care, ethical issues, 
professionalism, family dynamics and doctor-patient 
relationship (8, 11-14). This is achieved by engaging 
students in active learning, which is comparable to 
learning experienced during the actual consultations with 
patients (11). 
Therefore, cinemeducation seems a very useful method of 
teaching a complex area of family medicine. Nevertheless, 
there are practically no studies that would go beyond the 
mere description of the topics and movies (11).
The aim of this study was to determine whether family 
physicians’ competencies as listed in the Educational 
Agenda produced by the European Academy of Teachers 
in General Practice/Family Medicine (EURACT) (2), can be 
found in movies, and to propose a template for teaching 
by these movies. This exercise could then be repeated by 
different teachers interested in teaching family medicine 
by the use of cinemeducation. 
2 METHODS
We asked a group of 41 family medicine teachers to 
provide us with a list of movies that they would use in 
cinemeducation. Movies were defined as recordings of 
moving images that tell a story and that people watch 
on a screen or television. We only included fictional and/
or narrative movies and excluded documentaries and 
10.1515/sjph-2017-0013 Zdr Varst 2017; 56(2): 99-106
100
10.1515/sjph-2017-0013 Zdr Varst 2017; 56(2): 99-106
101
television series. The latter were excluded because we 
wanted to ensure homogeneity of the methodology. We 
used full-length movies, but there were no other requests 
regarding the content, language, or country of production 
of the movies.
In the next step, the authors independently viewed the 
suggested movies and wrote a short synopsis of each, in 
which they described their usefulness for family medicine 
education. The evaluation was based on the learning 
outcomes of family medicine education proposed in the 
Educational Agenda (2). The movies were then categorized 
according to the key family medicine competencies by 
both authors independently. In practice, it meant that the 
authors looked for the scenes or clips of a particular movie 
that would stimulate the discussion and/or reflection 
upon at least one competency. Both authors afterwards 
discussed the findings and came to a mutual agreement. 
There were no disagreements between authors regarding 
the categorisation of movies.
The selection of movies was finished when the learning 
outcome of every competence was covered by at least 
five movies.
3 RESULTS
The list consisted of 17 movies.
Most of the movies covered more than one key 
competence. Table 1 shows the list of movies and their 
categorisation according to competencies. Table 2 shows 
which competencies were covered by which movies. 
Table 1. The selected movies according to the category, key 
family medicine competency and key topic.
4 Months, 3 
Weeks and 2 Days 
(2007, Cristian 
Mungiu)
50/50 
(2011, Jonathan 
Levine)
Amour (2012, 
Michael Haneke)
As Good as It 
Gets (1997, 
James L. Brooks)
Doc Hollywood 
(1991, Michael 
Caton-Jones)
Dr. T & the 
Women (2000, 
Robert Altman)
Once Upon a 
Time Was I, 
Veronica (2012, 
Marcelo Gomez)
Hannah and Her 
Sisters (1996; 
Woody Allen)
The Intouchables 
(2011, Olivier 
Nakache & Eric 
Toledano)
Person-centred 
care
Community 
orientation
Person-centred 
care
Specific problem-
solving skills
Primary care 
management
Holistic approach
Person-centred 
care
Community 
orientation
Comprehensive 
approach
Primary care 
management
Person-centred 
care
Specific problem-
solving skills
Comprehensive 
approach
Primary care 
management
Person-centred 
care
Specific problem-
solving skills
Comprehensive 
approach
Primary care 
management
Primary care 
management
Specific problem-
solving skills
Comprehensive 
approach
Holistic approach
Person-centred 
care
Community 
orientation
Comprehensive 
approach
Social and cultural 
dimension
The effect of poverty on 
health care utilisation
Understanding the patient’s 
personality and life aims
An approach to patients 
according to prevalence and 
incidence of diseases in a 
community
Stroke
Holistic concept of care
Social and cultural 
dimension
The effect of poverty on 
health care utilisation
Multimorbidity
Obsessive-compulsive 
disorder 
Coordination of health 
services’ utilization
Continuous management of 
patients
Diagnosis and management 
of an emergency situation
The use of complementary 
and alternative medicine by 
patients
Common conditions
Continuous management 
of patients
Uncertainty in medicine
Simultaneous management 
of acute and chronic 
conditions
Gynaecological disorders
Depression, anxiety, 
somatoform disorders
Irrational use of 
investigations
Consultation in family 
medicine
Holistic approach to 
communication
Understanding the patient’s 
personality and life aims
Health care management of 
patients with special needs
Consultation in family 
medicine
Movie 
(year, director)
Movie 
(year, director)
The key family 
medicine 
competency
The key family 
medicine 
competency
The key topics in the 
corresponding 
competence
The key topics in the 
corresponding 
competence
10.1515/sjph-2017-0013 Zdr Varst 2017; 56(2): 99-106
102
3.1 Primary Care Management
Primary care management was covered in nine movies. 
They mainly focused on clinical topics or clinical dilemmas, 
and most of them narrated patients’ experiences and 
their points of view. Some of them also described the 
coordination of care.
3.2 Person-Centred Care
Person-centred care was covered in 13 movies. Those 
movies depicted the relationship between the doctor 
and patient, building a partnership, communication, and 
continuous management of patients.
3.3 Specific Problem-Solving Skills
Specific problem-solving skills were covered in eight 
movies. The movies portrayed specific techniques and 
problem solving skills of family physicians.
3.4 Comprehensive Approach
Comprehensive approach was covered in five movies. The 
movies depicted doctors dealing with multiple complaints 
of a patient, health promotion, prevention and lifestyle 
issues.
3.5 Community Orientation
Community orientation was covered in five movies. The 
movies in this category described the influence of a 
disease on a community and family, and vice versa.
3.6 Holistic Approach
Holistic approach was covered in five movies. The movies 
portrayed comprehensive solutions to patients’ problems 
that needed to include negotiations between biomedical 
solutions and social and cultural realities.
Steel Magnolias 
(1989, Herbert 
Ross)
What’s Eating 
Gilbert Grape? 
(1993, Lasse 
Hallström)
Wit (2001, 
Mike Nichols)
Stopped on Track 
(2011, Andreas 
Dresen)
The Death of Mr. 
Lazarescu (2005, 
Cristi Puiu)
The Doctor (1991, 
Randa Haines)
The King’s 
Speech (2010, 
Tom Hooper)
The Last King of 
Scotland (2006, 
Kevin McDonald)
Person-centred 
care
Primary care 
management
Comprehensive 
approach
Holistic approach
Person-centred 
care
Holistic approach
Person-centred 
care
Specific problem-
solving skills
Person-centred 
care
Comprehensive 
approach
Community 
orientation
Primary care 
management
Specific problem-
solving skills
Person-centred 
care
Community 
orientation
Specific problem-
solving skills
Primary care 
management
Holistic approach
Person-centred 
care
Person-centred 
care
Specific problem-
solving skills
Primary care 
management
Holistic approach
Understanding the patient’s 
personality and life aims
Diabetes 
Multimorbidity
Holistic approach to 
communication
Family-centred approach
Practical approach to 
holistic care
Communication
Partnership building
Ethics of history taking and 
clinical examination
Patients’ and their families’ 
understanding of a disease
House visit
Health care organization 
and its effect on patients’ 
management outcomes
Epidural haematoma, 
abdominal pain, alcoholism 
Coordination of emergency 
services
Diagnosis and management 
of emergency situation
Communication
Inequalities in health care 
provision
Step-wise decision-making
Cancer 
Ill physician
Person-centred 
communication
Partnership building
Social and cultural 
dimension
Communication
Partnership building
Diagnosis and management 
of an emergency situation
Epilepsy
Holistic concept of care
Movie 
(year, director)
Movie 
(year, director)
The key family 
medicine 
competency
The key family 
medicine 
competency
The key topics in the 
corresponding 
competence
The key topics in the 
corresponding 
competence
10.1515/sjph-2017-0013 Zdr Varst 2017; 56(2): 99-106
Table 1. The selected movies according to the category, key family medicine competency and key topic.
Primary are 
management
Person-centred 
care
Specific problem 
solving skills
To manage primary contacts with patients dealing 
with unselected problems
To adopt a person-centred approach in dealing 
with patients and problems in the context of the 
patient’s circumstances
To relate specific decision making processes to 
the prevalence and incidence of illness in a 
community
To selectively gather and interpret information 
from history-taking, physical examination and
Investigations, and apply it to an appropriate 
management plan in collaboration with the patient
To develop and apply the general practice 
consultation, so as to bring about an effective 
doctor-patient relationship, with respect for the 
patient’s autonomy
To communicate, set priorities and act in 
partnership
To provide longitudinal continuity of care as 
determined by the needs of the patient, referring 
to continuing and co-ordinated care management
To cover the full range of health conditions
To co-ordinate care with other professionals 
in primary care and with other specialists
To master effective and appropriate care 
provision and health service utilisation
To make available to the patient the appropriate 
services within the health care system
To act as an advocate for the patient
Doc Hollywood (1991, Michael Caton-Jones)
The Last King of Scotland (2006, Kevin McDonald)
The Death of Mr. Lazarescu (2005, Cristi Puiu)
Once Upon a Time Was I, Veronica (2012, Marcelo Gomez)
Steel Magnolias (1989, Herbert Ross)
What’s Eating Gilbert Grape? (1993, Lasse Hallström)
The Last King of Scotland (2006, Kevin McDonald)
4 Months, 3 Weeks and 2 Days (2007, Cristian Mungiu)
50/50 (2011, Jonathan Levine)
Intouchables (2011, Olivier Nakache & Eric Toledano)
Stopped on Track (2011, Andreas Dresen)
The Doctor (1991, Randa Haines)
Dr. T & the Women (2000, Robert Altman)
Doc Hollywood (1991, Michael Caton-Jones)
The Death of Mr. Lazarescu (2005, Cristi Puiu)
Amour (2012, Michael Haneke)
The Death of Mr. Lazarescu (2005, Cristi Puiu)
Wit (2001, Mike Nichols)
Wit (2001, Mike Nichols)
Steel Magnolias (1989, Herbert Ross)
As Good as It Gets (1997, James L. Brooks)
The Doctor (1991, Randa Haines)
The King’s Speech (2010, Tom Hooper)
50/50 (2011, Jonathan Levine)
Dr. T & the Women (2000, Robert Altman)
Doc Hollywood (1991, Michael Caton-Jones)
Amour (2012, Michael Haneke)
Doc Hollywood (1991, Michael Caton-Jones)
The Death of Mr. Lazarescu (2005, Cristi Puiu)
Once Upon a Time Was I, Veronica (2012, Marcelo Gomez)
Dr. T & the Women (2000, Robert Altman)
The Death of Mr. Lazarescu (2005, Cristi Puiu)
As Good as It Gets (1997, James L. Brooks)
The Doctor (1991, Randa Haines)
Once Upon a Time Was I, Veronica (2012, Marcelo Gomez)
The Death of Mr. Lazarescu (2005, Cristi Puiu)
The main 
competency
Learning outcome Movie (year, director)
103
10.1515/sjph-2017-0013 Zdr Varst 2017; 56(2): 99-106
104
4 DISCUSSION
To the best of the authors’ knowledge, no study has dealt 
with the use of movies in teaching key competencies 
of family medicine (2) so far. In 2007, an article 
about using movies to teach Accreditation Council for 
Graduate Medical Education (ACGME) competencies was 
published (15). The article gives examples of a movie 
clip appropriate for teaching each of the six ACGME 
competencies, namely: Patient care, medical knowledge, 
practice-based learning and improvement, system-based 
practice, professionalism and communication (16). The 
article additionally described proposed methods and 
content areas in order to create a framework for teaching 
by cinemeducation. Other articles about cinemeducation 
already published usually offer a narrative view from an 
author that had used movies or movie clips in teaching, 
or in some way report on the feedback from learners 
engaged in cinemeducation (11). 
In our study, similarly as in the aforementioned article 
(15), we used a theoretical framework as a basis for the 
Specific problem 
solving skills
Comprehensive 
approach
Community 
orientation
Holistic 
approach
To adopt appropriate working principles (e.g., 
incremental investigation), using time as a tool 
and to tolerate uncertainty
To manage multiple complaints and pathologies, 
both acute and chronic health problems in the 
individual, simultaneously
To reconcile health needs of individual patients 
and the community in which they live, in balance 
with available resources
To use a bio-psycho-social model, taking into 
account cultural and existential dimensions
To intervene urgently when necessary
To manage conditions which may present 
early and in an undifferentiated way
To make effective and efficient use of diagnostic 
and therapeutic interventions
To promote health and well-being by applying 
health promotion and disease prevention 
strategies appropriately
To manage and co-ordinate health promotion, 
prevention, cure, care, palliation and 
rehabilitation
The Doctor (1991, Randa Haines)
Dr. T & the Women (2000, Robert Altman)
Steel Magnolias (1989, Herbert Ross)
As Good as It Gets (1997, James L. Brooks)
Doc Hollywood (1991, Michael Caton-Jones)
As Good as It Gets (1997, James L. Brooks)
The Death of Mr. Lazarescu (2005, Cristi Puiu)
Intouchables (2011, Olivier Nakache & Eric Toledano)
The Doctor (1991, Randa Haines)
4 Months, 3 Weeks and 2 Days (2007, Cristian Mungiu)
The Death of Mr. Lazarescu (2005, Cristi Puiu)
Amour (2012, Michael Haneke)
What’s Eating Gilbert Grape? (1993, Lasse Hallström)
The Last King of Scotland (2006, Kevin McDonald)
The Doctor (1991, Randa Haines)
Steel Magnolias (1989, Herbert Ross)
Doc Hollywood (1991, Michael Caton-Jones)
The Last King of Scotland (2006, Kevin McDonald)
The Death of Mr. Lazarescu (2005, Cristi Puiu)
The Death of Mr. Lazarescu (2005, Cristi Puiu)
Dr. T & the Women (2000, Robert Altman)
The Doctor (1991, Randa Haines)
The Doctor (1991, Randa Haines)
Dr. T & the Women (2000, Robert Altman)
Hannah and Her Sisters (1996; Woody Allen)
Hannah and Her Sisters (1996; Woody Allen)
Steel Magnolias (1989, Herbert Ross)
Dr. T & the Women (2000, Robert Altman)
Stopped on Track (2011, Andreas Dresen)
As Good as It Gets (1997, James L. Brooks)
Dr. T & the Women (2000, Robert Altman)
The main 
competency
Learning outcome Movie (year, director)
10.1515/sjph-2017-0013 Zdr Varst 2017; 56(2): 99-106
105
selection of movies. As we wanted to test the template of 
EURACT Educational Agenda (2) on the case of education 
with movies, we used a pragmatic method of selection 
of movies, based on trainers’ opinions. The aim was not 
to make a reference list of movies that can be used in 
family medicine by cinemeducation, but to come up with 
a framework, which could be useful for teachers in their 
work. Therefore, many very good movies, widely used in 
education, were not listed.
This study showed that movies can be used in teaching 
all key family medicine competencies as described by the 
Educational Agenda (2).
Most of the movies included in our study covered more than 
one family medicine competence. These movies could be 
very useful in teaching family medicine, as the discipline 
itself is of a holistic and comprehensive nature. Topics, 
like family dynamics, holistic perception, comprehensive 
approach, continuous management of patients and 
person-centred approach are more difficult to teach when 
compared to clinical themes (7, 17, 18), and the use of 
movies could be a good addition to conventional teaching. 
It is understandable that movies cannot be used for 
teaching practical skills. However, as our study showed, 
they can be used to demonstrate their importance or to 
show the appropriate procedure.
Since family medicine is a very comprehensive discipline, 
it should not be taught by only one method. Our study 
aims to provide a template of how movies could be fitted 
into a framework of different teaching techniques of 
family medicine teaching. 
Cinemeducation in family medicine can be used in different 
ways, and traditional classroom setting is probably not 
the best one. Since a lot of teaching of family medicine 
is done through a process of mentoring, discussions about 
movies may be a useful method of encouraging discussions 
with young learners and elucidating dilemmas of different 
kinds (19, 20). Teaching with movies gives students enough 
time to reflect on a situation they have seen, discuss 
about the problems with a supervisor, and find a solution 
which could help them were they facing similar problems 
in actual clinical work (8). It is interesting that we have 
found no descriptions of such use of cinemeducation. It 
is possible that the potential use of cinemeducation in 
teaching family medicine in one-to-one setting has been 
left to tutors’ beliefs, without the use of a structured 
tool. The strength of this study is the use of a theoretical 
framework, such as the Educational Agenda (2). We chose 
not to perform the systematic search for appropriate 
movies, as we wanted to produce a model for teaching 
with movies that are available to teachers. Our list of 
movies serves only as a model for teaching and not as 
a proposed list of movies to be used in family medicine 
teaching. Teachers could use any movie (the content of 
which is appropriate for family medicine teaching) and 
employ it for teaching on the basis of EURACT Educational 
Agenda (2), together with their direction and moderation. 
The only criterion for the choice of an appropriate movie 
or movie clip is that it covers the teaching aims and that 
it stimulates discussion/reflection.
Our study has also some limitations. The first one is that 
only two teachers assessed the movies. Other teachers 
were only asked to provide a list of movies they thought to 
be appropriate for cinemeducation. The study would have 
a greater value if also those teachers provided analyses 
of suggested movies. Another limitation of the study 
pertains to the fact that we did not assess the usefulness 
of the movies for teaching, but were merely concerned 
with whether they are appropriate for teaching family 
medicine competencies according to the opinions of the 
teachers and authors. In addition, this study did not test 
the usefulness of this model in teaching, as no feedback 
from the teachers was sought. However, this study was 
only aimed at determining whether family physicians’ 
competencies can be found in movies, and proposing a 
template for teaching by means of these movies. 
We suggest that further studies assess the usefulness of 
this approach, and also examine where in the educational 
process movies could be successfully used.
5 CONCLUSIONS
This is the first attempt to use the EURACT teaching 
agenda to provide a systematic approach with an 
innovative teaching method. It suggested that 
cinemeducation can map across the different domains 
proposed in the Educational agenda, and vice versa. 
Therefore, any appropriate movie that covers at least 
one family medicine competence can be used in family 
medicine teaching. Our results can serve as a template 
for teachers on how to start selecting appropriate movies 
in the education of family medicine. Further studies are 
needed to determine the usefulness of such an approach 
in teaching and the extent of coverage of different areas 
of medicine in cinema.
CONFLICTS OF INTEREST
The authors declare that no conflicts of interest exist.
FUNDING
The study received no funding.
10.1515/sjph-2017-0013 Zdr Varst 2017; 56(2): 99-106
106
ETHICAL APPROVAL
The study did not include any persons, so we did not seek 
ethical approval.
REFERENCES
1. Allen J, Gay B, Crebolder H, Heyrman J, Svab I, Ram P. The European 
definition of general practice/family medicine. Evans P, editor: 
Wonca Europe, 2011.
2. Heyrman J. EURACT Educational Agenda. Leuven: European Academy 
of Teachers in General Practice, 2005.
3. Krzton-Królewiecka A, Svab I, Oleszczyk M, Seifert B, Smithson WH, 
Windak A. The development of academic family medicine in central 
and eastern Europe since 1990. BMC Fam Pract 2013; 14: 37.
4. Švab I. Education in medicine. Zdr Varst 2012; 51: 223-6.
5. Klemenc-Ketis Z, Kersnik J. Deficiency areas in decision making in 
undergraduate medical students. Adv Med Educ Pract 2014; 5: 223-7.
6. Kersnik J, Wensing M. Improving the quality of care for patients with 
chronic diseases: what research and education in family medicine 
can contribute. Eur J Gen Pract 2012; 18: 238-41.
7. Kersnik J. Learning and teaching to educate future GPs. Eur J Gen 
Pract 2012; 18: 199-200.
8. Klemenc-Ketis Z, Kersnik J. Using movies to teach professionalism to 
medical students. BMC Med Educ 2011; 11: 60.
9. Klemenc-Ketis Z, Kersnik J. New virtual case-based assessment 
method for decision making in undergraduate students: a scale 
development and validation. BMC Med Educ 2013; 13: 160.
10. Wong RY, Saber SS, Ma I, Roberts JM. Using television shows to teach 
communication skills in internal medicine residency. BMC Med Educ 
2009; 9: 9.
11. Darbyshire D, Baker P. A systematic review and thematic analysis of 
cinema in medical education. Med Humanit 2012; 38: 28-33.
12. Alexander M, Hall MN, Pettice YJ. Cinemeducation: an innovative 
approach to teaching psychosocial medical care. Fam Med 1994; 26: 
430-3.
13. Blasco PG, Moreto G, Roncoletta AFT, Levites MR, Janaudis MA. Using 
movie clips to foster learners’ reflection: improving education in the 
affective domain. Fam Med 2006; 38: 94-6.
14. Weber CM, Silk H. Movies and medicine: an elective using film to 
reflect on the patient, family, and illness. Fam Med 2007; 39: 317-9.
15. Alexander M, Pavlov A, Lenahan P. Lights, camera, action: using film 
to teach the ACGME competencies. Fam Med 2007; 39: 20-3.
16. Swing SR, Clyman SG, Holmboe ES, Williams RG. Advancing resident 
assessment in graduate medical education. J Grad Med Educ 2009; 
1: 278-86.
17. Wensing M, Kersnik J. Improving the quality of care for patients with 
chronic diseases: what research and education in family medicine 
can contribute. Eur J Gen Pract 2012; 18: 238-41.
18. Petek Ster M, Svab I, Ster B. Final year medical students’ 
understanding of family medicine. Acta Med Acad 2014; 43: 40-9.
19. Alfandre D, Rhodes R. Improving ethics education during residency 
training. Med Teach 2009; 31: 513-7.
20. Buxton M, Phillippi JC, Collins MR. Simulation: a new approach to 
teaching ethics. J Midwifery Womens Health 2015; 60: 70-4.
THE PROFILING OF UNIVERSITY OF LJUBLJANA STUDENTS ACCORDING 
TO THEIR MOTIVES FOR EXERCISE PARTICIPATION
Katja CERAR1, Miran KONDRIČ2*, Joško SINDIK3
1University of Ljubljana, Kongresni trg 12, 1000 Ljubljana, Slovenia
2University of Ljubljana, Faculty of Sport, Gortanova 22, 1000 Ljubljana, Slovenia
3Institute for Anthropological Research, Gajeva 32, 10000 Zagreb, Croatia
Received: Avg 15, 2015
Accepted: Sep 14, 2016
Original scientific article
Introduction. The main research objective is the analysis of the grouping of the students of the University 
of Ljubljana, with respect to the intensity of different types of exercise participation motives, their gender, 
discipline and year of study, level of physical activity, status of physical education class, organization of physical 
activities during study, and place of residence. 
Methods. Data were collected using personal data sheets during enrolling students at the University of 
Ljubljana. Students completed The Exercise Motivations Inventory (EMI-2), with additional data about socio-
demographic parameters. 
Results. The results reveal that the students could be grouped in three distinctive clusters, which can be very 
clearly explained in terms of the prevalence of exercise participation motives in general. The students grouped 
in the first cluster have the lowest average values (means) in all exercise participation motives. The students 
grouped in the second cluster have the profile with moderate means in all exercise participation motives, while 
the students grouped in the third cluster have the profile with the highest means in all exercise participation 
motives. 
Conclusion. The results indicate overall higher motivation for physical activity in men. All the sub-samples are 
different in their relevant features used in clustering (e.g., male students are dominant in life sciences, etc.), 
which provide a guide both for the explanation of the results obtained and for practical implications. 
Uvod. Glavni cilj raziskave je analiza razvrstitve udeležencev (študentov Univerze v Ljubljani) v skupine glede 
na intenzivnost njihovih motivov za sodelovanje pri športni vadbi na podlagi spola, vrste študija, letnika 
študija, stopnje telesne dejavnosti,  statusa predmeta športne vzgoje med študijem, organizacije športnih 
dejavnosti med študijem in kraja bivališča.
Metode. Podatki so bili zbrani na podlagi obrazcev, ki so vključevali osebne podatke. Le-ti so bili izpolnjeni 
ob študentovem vpisu na Univerzo v Ljubljani. Študenti so izpolnili vprašalnik o motivih za vadbo “Exercise 
Motivations Inventory” (EMI-2) z dodatnimi vprašanji, ki so obsegala 12 socio-demografskih parametrov.
Rezultati. Rezultati kažejo, da lahko študente razvrstimo v tri skupine, ki jih je mogoče zelo jasno ločiti glede 
na prevladujoč motiv za sodelovanje pri vadbi na splošno (močno, srednje in šibko izraženi motivi). Študenti, 
uvrščeni v prvo skupino, imajo najnižje povprečne vrednosti pri vseh motivih za vadbo. Študenti, uvrščeni 
v drugo skupino, imajo profil s srednjimi vrednostmi, medtem ko imajo študenti, uvrščeni v tretjo skupino, 
najvišje motive za vadbo.
Zaključek. Rezultati v splošnem kažejo na večjo skupno motivacijo fantov za športno dejavnost. Vendar pa 
se vsi pod-vzorci statistično razlikujejo glede na osebnostne lastnosti, ki smo jih uporabili pri razvrščanju v 
skupine (npr. moški študenti prevladujejo v naravoslovnih znanostih itn.). Ta podatek nas usmerja pri razlagi 
rezultatov in tudi pri njihovi praktični uporabi.
ABSTRACT
Keywords: 
cluster analysis, 
motivation, university 
students, Slovenia
IZVLEČEK
Ključne besede: 
analiza skupin, 
motivacija, študenti, 
Slovenija
*Corresponding author: Tel: ++ 386 1 520 77 37; E-mail: miran.kondric@fsp.uni-lj.si
10.1515/sjph-2017-0014 Zdr Varst 2017; 56(2): 107-114
PROFILIRANJE ŠTUDENTOV UNIVERZE V LJUBLJANI GLEDE NA NJIHOV 
MOTIV ZA SODELOVANJE PRI ŠPORTNI VADBI
Cerar K, Kondrič M, Sindik J. The profiling of University of Ljubljana students according to their motives for exercise participation. Zdr Varst 2017; 56(2): 107-114
107
© Nacionalni inštitut za javno zdravje, Slovenija. 
1  INTRODUCTION
Sports activity, regularly practiced, protects people 
against diseases and contributes to a better quality of life 
in the lifespan. In the curriculum of Physical Education 
(PE), children and young people could learn about the 
importance of movement, sport, fitness activities and a 
healthy lifestyle (1, 2). The course PE in higher education 
in Slovenia differs from the conceptions of those courses 
in primary and secondary schools, where PE is more clear 
defined in belonging curriculums.
The basic motives are supposed to be unconscious and 
they are hard to be recognized by the individual (3). Thus, 
weak motivation or lack of motivation appear when the 
individual is unaware of what drives and motivates him/
her (4, 5). The lack of enjoyment in sports activities in 
an individual is often influenced by selecting activities 
unsystematically, following trends and without setting 
goals (6).
The motivational factors of each individual in physical 
education and organized sports activities should be 
strengthened, in order to ensure that the effects of 
these activities would be transferred to a student’s 
life after graduation (7). However, the involvement in 
a sports activity during studies and later depends both 
on an individual’s motivational structure (interests, 
attitudes, motives), and on the available conditions and 
opportunities in a certain environment (8).
The main motivational factors for students to engage 
in sports activities are health pressures, enjoyment, 
physical fitness, stress management and well-being (9-
13). Gender differences in the motives for participating 
in sports activities are evident in several studies. Males 
and females differ in motives for participating in certain 
sports activities, having different interests in sport, 
training methods and unorganized sports activities (14). 
Gender differences are obvious in five motivational 
factors (affiliation, competition, weight management, 
rehabilitation and social recognition) (10). Primary 
factors which motivate men’s participation in sport are 
competition, social interaction and health, while for 
women, the most important motives are: social contacts, 
competition and health (15). Men are more often motivated 
by internal factors (power, competition, challenge, 
fitness), while women are more frequently motivated by 
external factors (weight management, appearance) (16, 
17). In all age groups, men show higher levels of general 
motivation and participation in sports than women do. For 
both genders, negative factors observed for non-engaging 
in sports activities are the lack of interest and limited 
time (15).
A current offer of sports activities at the University 
comprises of different sets of programmes, which differ in 
scope and in terms of quality, standards and purpose. The 
offer includes programmes of the Centre for University 
Sports (CUS), elective subjects of some members at 
the University of Ljubljana, programmes of competitive 
sports that take place under the auspices of the Sports 
Association of the University of Ljubljana (ŠZUL) and 
the Slovenian University Sports Association (SUSA), 
and programmes carried out under the auspices of the 
Institute for student sports at the Student Organisation of 
the University of Ljubljana (SOU sport). However, there 
is a strong negative trend in the proportion of students 
enrolled in regularly organized sports activities, since 
the start of the Bologna reform and the reorganization of 
sports activities at the University of Ljubljana. In other 
words, the abolition of compulsory Physical Education and 
the introduction of extracurricular sports activities at the 
University of Ljubljana have resulted in the drop in the 
share of students involved in organized sports activities 
on a regular basis (18). The University of Ljubljana offers 
organized sport activities that promote participation and 
socialization between groups and individuals, enhance 
physical fitness, and foster a spirit of fair play and 
sportsmanship. Each year, the content of activities varies 
according to the interests of students. 
In this study, Slovenian high school and university 
students are grouped according to the prevalence of their 
motives in exercise participation, in relation to several 
socio-demographic features, engagement in physical 
activities, and relevant factors linked with institutional 
forms of organizing students’ physical exercise. This 
taxonomisation should provide a platform for considering 
differences between educational programmes, adjusted 
to the characteristics of participants in certain taxa.
The profiling of students according to their motives 
for exercise participation is in line with their own and 
institutional efforts to enhance them. The insight in those 
profiles should provide the guidelines for a better and 
individually adjusted engagement of students in various 
physical exercise activities. The scope of this article is to 
offer a comprehensive description of those profiles.
The goal of this study is the analysis of the grouping of 
the participants according to the intensity of different 
types of motives for exercise participation, in relation to 
their gender, discipline of study, year of study, level of 
physical activity, status of physical education class during 
study, and organization of physical activities during study. 
The initial hypothesis was that students could be clearly 
categorized into several clusters (characteristic profiles) 
in terms of motives for exercise participation, according 
to abovementioned characteristics.
10.1515/sjph-2017-0014 Zdr Varst 2017; 56(2): 107-114
108
10.1515/sjph-2017-0014 Zdr Varst 2017; 56(2): 107-114
109
2 METHODS
2.1 Participants 
The sample of participants included 5,271 students of the 
University of Ljubljana, which represents a 9.8 percent 
sample of the University of Ljubljana students in the 
2010/2011 academic year. The age of the participants 
was between 19 and 42 years. Most students (25.6%) were 
enrolled in the first year of study. The data were collected 
by questionnaires distributed by e-mail to respondents. 
The data were collected on the basis of personal data 
sheets, completed when students were enrolling at the 
University of Ljubljana.
2.2 Procedure
We have disseminated questionnaires to 43,751 students 
of the University of Ljubljana (81.6% of all enrolled 
students). According to the instructions provided, 5,486 
of them returned the questionnaires. In the study we 
considered the responses of 5,271 students – 2,033 male 
(38.5%) and 3,238 females (61.5%). Improperly completed 
questionnaires (215) were excluded from further 
processing. An email open rate of 15-20% is considered 
‘good’, because we have expected the percentage of 
participants who respond to the survey to be even less 
than that. In other words, we emphasized that the 
participation in this research is voluntarily, and that drop 
out is acceptable even after a participant starts to fill in 
the questionnaire. 
2.3 Instruments
In the study, we use the EMI-2 questionnaire (Exercise 
Motivations Inventory; Markland and Hardy) (19), which 
is widely employed in several studies of motivation. The 
EMI-2 was developed for assessing participation motives, 
in order to examine issues, such as the influence of 
motives on exercise participation, their influence on the 
choice of activities undertaken, how affective responses 
to exercise may be influenced by reasons for exercising, 
and how involvement in sports activities might have a 
reciprocal influence on participation motives. The authors 
developed an instrument to examine questions concerning 
functional significance of exercise motives from the 
perspective of Deci and Ryan’s (20) self-determination 
theory. The EMI-2 comprises of 51 items that describe 
motives for participating in sports activities. A six-point 
Likert scale was used, in range from 0 (‘not true at all’) to 
6 (‘very true’). Moreover, we added 10 socio-demographic 
parameters to the EMI-2 (faculty, year of study, gender, 
age, engagement in sports activities, etc.). Factors that 
represent all scales of EMI-2 explained together 69.4% of 
the total variance (19). 
2.4 Data Analysis 
The data were analysed with the IBM SPSS Statistics (20.0) 
software. The basic descriptive characteristics were 
calculated (mean, standard deviation, the frequency 
of answers). Then, basic metric characteristics of the 
measuring instruments were determined (reliability 
and factor validity). The internal consistency of the 12 
subscales was generally acceptable, with Cronbach’s 
alpha reliability coefficients ranging from 0.63 to 0.90, 
while test-retest reliability coefficients over a 4-5-week 
period were ranged from 0.59 to 0.88 (19). In our research, 
reliability type internal consistency for the entire EMI-2 
was 0.80, while the reliabilities for certain scales were: 
Stress Management (4 items; α=0.84); Revitalization (3 
items; α=0.84); Enjoyment (4 items; α=0.93); Challenge 
(4 items; α=0.81); Social Recognition (4 items; α=0.85); 
Affiliation (4 items; α=0.88); Competition (4 items; α=0.94); 
Health Pressures (4 items; α=0.58); Ill-Health Avoidance 
(3 items; α=0.86); Positive Health (3 items; α=0.87); 
Weight Management (4 items; α=0.87); Appearance (4 
items; α=0.93); Strength & Endurance (4 items; α=0.82); 
Nimbleness (3 items; α=0.83). Chi-square test is used to 
test the differences between students according to gender 
and discipline of study (life and social sciences), with 
all other relevant factors in research (label criteria for 
classification according to clusters obtained in research). 
K-means – i.e., non-hierarchical clustering method – is 
applied using quantitative (ratio-type) variables only, 
which represent types of exercise participation motives. 
Initial cluster centres are automatically determined, while 
three-cluster solution is estimated the most suitable. 
After clustering, with respect to the abovementioned 
quantitative variables (exercise participation motives), 
cases are classified in relation to: gender (male/female), 
discipline of study, year of study (1st, 2nd,…year), the 
level of physical activity (according to time used for 
physical activity, in the range from <30 minutes to >1000 
minutes), status of physical education class during study 
(the status of PE program at the faculties: YES/NO) and 
organization of physical activities during study (available 
programmes of activities: YES/NO).
3 RESULTS 
3.1 Differences in Socio-Demographic Characteristics
We analysed the differences among students according 
to two socio-demographic areas (gender and discipline of 
study).
According to the discipline of study (life/social sciences), 
more (n=2202, or 76%) female students are studying social 
sciences, while more (n=1341, or 56%) male students 
study life sciences (Chi square=581.95; df=1; p<0.001). 
More students in the first years of study are studying 
10.1515/sjph-2017-0014 Zdr Varst 2017; 56(2): 107-114
110
social sciences (n=1761 or 61%) (Chi square=220.18; 
df=11; p<0.001). Students who are studying life sciences 
are more physically active (n=1316 or 55%) (Chi square= 
31.36; df=1; p<0.001). Students who are studying life 
sciences attended more classes of physical exercise 
(n=1442 or 61%) (Chi square=32.39; df=1; p<0.001), 
while students who are studying social sciences attended 
organised physical activities for longer periods of time 
(with median 360-420 minutes per week) (Chi square= 
27.88; df=11; p<0.01). Students who are studying life 
sciences and who have a stronger need for mandatory 
physical activities (n=1290, or 54%) (Chi square=7.77; 
df=1; p<0.01) spend longer periods of time participating 
in unorganised physical activities (with median 180-240 
minutes per week) (Chi square=50.11; df=11; p<0.01).
When analysing gender differences among students, 
according to the discipline of study (all faculties), the 
results revealed that more female students (n=2202, or 
76%) are engaged in social sciences (e.g., the Academy 
of music, the Academy of fine arts and design, the 
Biotechnical faculty, the Faculty of economics, the 
Faculty of architecture, the Faculty of social sciences, 
etc.) (Chi square=1419.34; df=22; p<0.001). Male students 
are prevalent at the Faculty of electrical engineering, the 
Faculty of civil and geodetic engineering, the Faculty of 
maritime studies and transport, the Faculty of computer 
and information science, and the Faculty of mechanical 
engineering (n=1341, or 56%). Female students are more 
often students (n=2367, or 74%) of lower years of study 
(Chi=36.66; df=11; p<0.001) and they are less physically 
active (n=1713, or 66%) during their studies than male 
students (Chi square=54.10; df=1; p<0.001). Male students 
attended more classes of physical exercise (n=1181, or 
58%) (Chi square=4.10; df=1; p<0.05) and spent longer 
periods of time in organised physical activities (with 
median 300-360 minutes per week) (Chi square=101.03; 
df=11; p<0.001). Male students with a greater need 
for mandatory physical activities (n=1097 or 54%) (Chi 
square=4.33; df=1; p<0.05)spent longer periods of time 
performing unorganised physical activities (with median 
180-240 minutes per week) (Chi square=215.01; df=11; 
p<0.001).
3.2 The Grouping of Students by their Motivations 
The results of cluster analyses reveal that the students 
could be grouped into three distinctive clusters, which 
can be very clearly explained in terms of the prevalence 
of motives for exercise participation in general, namely: 
strongly, moderately and weakly expressed motives (Table 
1). Students grouped in the first cluster have the lowest 
average values (means) in all exercise participation 
motives. The students grouped in the second cluster 
have the profile with moderate means in all exercise 
participation motives, while the students grouped in the 
third cluster have the profile with the highest means 
Table 1. Taxonomisation of Slovenian students’ motives for 
sport and exercise by gender (K-means clustering).
stress management
revitalisation
enjoyment
challenge
social recognition
affiliation
competition
health pressures
ill health avoidance
positive health
weight management
appearance
strength & endurance
nimbleness
Overall number of 
cases in each cluster
Males 
Females 
8.60
8.04
9.16
5.76
2.25
5.86
2.58
1.65
5.76
8.01
7.12
8.18
8.62
7.46
1084
432
652
2319
579
1740
1868
1022
846
15.19
12.77
15.94
11.84
4.31
8.93
3.11
2.81
10.34
12.96
12.88
14.99
15.09
11.97
16.56
13.40
17.88
15.98
11.77
14.49
13.34
4.80
10.86
13.27
13.21
15.68
16.86
12.85
Variables Clusters
3
strong 
motivation
2
moderate 
motivation
1
weak 
motivation
in all exercise participation motives (Table 1). The 
majority of students are classified into the second cluster 
(moderate motivation), while the minority of students 
are classified into the first cluster (weak motivation). 
Women are prevailing in the second cluster, while men 
are dominant in the third cluster (Table 1). Both male and 
female students are rarely represented in the first cluster. 
This grouping shows that male students in general have 
strongly emphasised all exercise participation motives, 
while both male and female students have moderately 
emphasised exercise participation motives (Table 1).
According to their discipline of study, the results indicate 
that students of life sciences are prevailing in the second 
(almost equally) and in the third cluster, while students of 
social sciences are dominant in the second cluster (Table 
2). Students of life and social sciences are very rarely 
represented in the first cluster. Students of life sciences 
in general have higher means in all exercise participation 
motives than students of social sciences, but all students 
are at least moderately physically active (Table 2). The 
most physically active students are found at the Faculty 
of economics, the Faculty of electrical engineering, the 
Faculty of civil and geodetic engineering, the Faculty of 
10.1515/sjph-2017-0014 Zdr Varst 2017; 56(2): 107-114
Table 3.
Table 2.
Taxonomisation of Slovenian students’ motives for 
sport and exercise by the year of study (K-means 
clustering).
Taxonomisation of Slovenian students’ motives 
for sport and exercise by the discipline of study 
(K-means clustering).
1st year, 1st level
2nd year, 1st level
3rd year, 1st level
4th year, 1st level
5th year, 1st level
6th year, 1st level
1st year, 2nd level
2nd year, 2nd level
1st year, 3rd level
2nd year, 3rd level
3rd year, 3rd level
advanced university 
student
Academy of theatre, 
film, radio and 
television
Academy of music
Academy of fine arts 
and design
Biotechnical faculty
Faculty of economics
Faculty of architecture
Faculty of social 
sciences
Faculty of electrical 
engineering
Faculty of pharmacy
Faculty of civil and 
geodetic engineering
Faculty of chemistry 
and chemical 
technology
Faculty of 
mathematics and 
physics
Faculty of maritime 
studies and transport
Faculty of computer 
and information 
science
Faculty of social work 
Faculty of mechanical 
engineering
Faculty of sport
Faculty of 
administration
Faculty of arts
Faculty of medicine
Faculty of natural 
sciences and 
engineering 
Faculty of education
Faculty of health 
sciences
Overall number of 
cases in each cluster
Overall number of 
cases in each cluster
Life sciences
Social sciences
268
193
139
93
10
2
60
42
19
24
15
219
6
5
5
45
133
27
97
66
25
36
32
35
8
64
33
70
7
49
234
27
52
23
5
1084
1084
492
592
2319
2319
943
1376
1868
1868
942
926
467
356
314
218
39
18
159
98
40
48
33
529
4
9
23
100
339
65
244
83
92
72
62
67
21
75
52
92
38
111
495
115
81
61
18
476
321
299
136
15
6
102
83
21
26
30
353
3
1
6
73
354
36
125
147
50
95
68
74
16
84
10
168
105
97
185
71
57
40
3
Variables
Variables
Clusters
Clusters
3
3
strong 
motivation
strong 
motivation
2
2
moderate 
motivation
moderate 
motivation
1
1
weak 
motivation
weak 
motivation
chemistry and chemical technology (the third cluster), 
while only the students from the Academy of theatre, 
film, radio and television are dominant in the first cluster.
When analysing the grouping of participants according to 
their year of study, the results indicate that only students 
in the 1st year, on the 1st level are prevailing in the third 
cluster, while students from all other years of study are 
relatively and absolutely dominant in the second cluster 
(Table 3).
Students who do sports during their studies are prevailing 
in the third (and almost equally in the second) cluster, 
while students who do not do sports during their studies 
are dominant in the second cluster (Table 4). Students 
who have and those who do not have physical education 
classes during their studies are prevailing in the second 
cluster (but those who have physical education as a class 
have relatively more cases grouped in the third cluster). 
Students who have mandatory physical education classes 
are dominant in the third cluster, while those who do 
not have them are prevailing in the second cluster. Both 
students who have and those who do not have organised 
physical activities during their studies are prevailing in 
the second cluster (but those who do not have organised 
physical activities have fewer cases grouped in the third 
cluster).
111
10.1515/sjph-2017-0014 Zdr Varst 2017; 56(2): 107-114
112
4 DISCUSSION
The main finding of the research is the fact that students 
from Slovenian universities could be grouped into three 
distinctive clusters, which can be explained in terms 
of the prevalence of motives for exercise participation 
in general, namely: Strongly, moderately and weakly 
expressed motives for physical exercise. The minority of 
students have weak motivation (the first cluster), while 
the majority of students have moderate motivation 
(grouped in the second cluster). In the third cluster (the 
strongest all exercise participation motives) are grouped 
students who are mostly male, students of life sciences, 
very physically active during their studies and who have 
mandatory physical education classes during the studies. 
Stronger motivation in males is argued in the majority 
of previous studies (21). However, there are just a few 
opposite results: Women showed stronger motivation for 
participation in sports activities, comparing to men in the 
study of Gill et al. (22), while in some studies, gender 
appears as non-important for the type of motivation in 
males and females (23). The analysis of the differences in 
their important socio-demographic characteristics (gender 
and the discipline of study) revealed that students who 
study social sciences are more often females, students 
from lower years of study, who are less physically active, 
have fewer classes of physical exercise, and lower need 
for mandatory physical activity. Students who study life 
sciences showed the opposite trend. Gender differences 
are statistically significant in the discipline of study (the 
choice of universities). Relatively more female students 
are in lower years of the study, less physically active, 
have less classes of physical exercise, attend organised 
and unorganised physical activities for shorter periods of 
time, and demonstrate lower need for mandatory physical 
activity. Trends in male students for same variables are 
just opposite. The results of grouping of the participants 
could be mostly explained in terms of a higher level of 
general motivation and a higher level of participation in 
sports activities in men than in women, while the lack of 
interest in sports compared to other activities that request 
less effort and time could be reasons for non-engaging in 
sports activities(15). For example, younger generations 
probably prefer other types of fun and spending free time 
differently. On the other hand, women can be described 
as expressive, socially sensitive, and genuinely interested 
in other people (24, 25); they demonstrated greater 
perceptiveness, empathy, and adaptability than men (26), 
showing higher social and emotional intelligence (27). 
This could be reflected in reported higher motivation in 
women to stay in good shape than men (28), and women 
also prefer team sports to individual ones (29).
In fact, the analysis of the differences in both socio-
demographic characteristics (gender and the discipline of 
study) indicated that the interaction between gender and 
higher (in men) or lower (in women) interest in physical 
activity in general, is also reflected in the choice of study 
(men more often choose life sciences, women social 
sciences). The most reasonable explanation could be that 
male students have higher level of general motivation 
and higher level of participation in sports activities than 
female students (30). However, the intensity of certain 
types of motivational factors depends on gender, the 
availability of physical exercise (i.e., specific programmes 
at certain universities), types of sports, and the level 
of sport engagement (in terms of sports excellence). 
In other words, there are differences in motivational 
factors between athletes who participate in competitive 
sports and those participating in all types of sports (31). 
The results of several studies reflect gender differences 
in the motives for participating in sports activities, 
found in motives for participating in certain sports 
activities, interest in sports, training methods (14), in 
five motivational factors (10, 15), internal factors and 
external factors (16, 17, 32). Hence, stimulating the 
students of social studies to be more engaged in sports 
could be ‘translated’ into stimulating female students to 
be more physically active (women are dominant in social 
studies). In the study conducted in Croatia (33), one 
of suggested ways to stimulate women’s participation 
in sports is to hire women in various positions in sports 
as a solution (coaches, judges, tournament officials, 
members of sports club management, etc.). Adequate 
educational courses for with abovementioned positions 
of women and better media coverage of women in sport 
are prerequisites for a better inclusion of women in sport 
(33).
The advantage of the research is the examination of a 
current (actual), large and representative sample of 
students at the University of Ljubljana, with selected 
socio-demographic features, relevant for the purpose 
of this research. The limitations of the research arise 
from differences according to gender and discipline of 
study, which are reflected in groupings of participants. 
Moreover, in spite of a large sample of participants, 
only students from the University of Ljubljana were 
included in the research. Differences in students’ 
relevant features used in clustering (e.g., male students 
are dominant in life sciences, etc.), provide a guide for 
practical implications. Information about the groupings 
of participants according to their exercise participation 
motives, offer a framework for planning educational 
programmes at the universities, as well as for health 
prevention programmes. It has to be pointed out that 
in Ljubljana, more and more sports events and sports 
activities are organised, even for free, and that in the 
last years, also many students participate in such events 
and activities. Moreover, many people are in the parks, 
running, skating or biking on the tracks, which gives more 
credit to the daily physical activity of students. In a wider 
10.1515/sjph-2017-0014 Zdr Varst 2017; 56(2): 107-114
113
framework, as for the Ljubljana citizens, exercises (such 
as aerobics, Pilates and other recreational activities) are 
offered for very acceptable costs, out of PE contents 
and offers.  Therefore, it is possible that this study does 
not reflect the ‘real’ situation about sports engagement 
amongst the University of Ljubljana students (including 
only PE lessons at the faculties).     
The recommendation for future research can involve using 
more sophisticated statistical methods (for example, 
general linear model, with two-factorial or three-factorial 
MANOVA), as well as extending this investigation to 
students of different universities and faculties in different 
cities in Slovenia, or to the international level.
5 CONCLUSION
Slovenian university students could be grouped into three 
distinctive clusters. These clusters could be explained 
in terms of the prevalence of motives for exercise 
participation in general (strongly, moderately and 
weakly expressed motives). Male students, students who 
are studying life sciences, who are sport active during 
their studies, and those who have mandatory physical 
education classes during their studies are grouped in 
the third cluster, with the highest level of all exercise 
participation motives. To use the results obtained for 
practical purposes (designing particular programmes), 
it is important to consider the differences in socio-
demographic variables. From a social standpoint, sports 
are a powerful tool that brings students together and 
creates a sense of community. They develop connections 
that bond together students from all the faculties. These 
relationships are essential for emotional and physical 
health throughout the entire period of the study. 
CONFLICTS OF INTEREST
The authors declare that no conflicts of interest exist.
FUNDING 
There was no relevant financial interest in this article. 
ETHICAL APPROVAL
The study was conducted in accordance with the code of 
Ethics of the World Medical Association (Declaration of 
Helsinki).
REFERENCES
1. Afsanepurak SA, Seyed Hossini RN, Seyfari MK, Fathi H. Analysis of 
motivation for participation in sport for all. Inte Res J Applied Basic 
Sci 2012; 3: 790-5.
2. Škof B. Get in motion - for health and happiness: how to improve 
exercise capacity of Slovenian youth. Ljubljana: Faculty of Sport, 
Institute for Sport, 2010.
3. Tušak M. Motivation and sport: key to success. Ljubljana: Faculty of 
Arts, University of Ljubljana, Psychology Department, 1999. 
4. Peralta M, Martins J, Marques A, Correia C. Correlates of motivation 
to practice physical activity among students from Portuguese military 
college. Brit J Sport Med 2013; 47: e4.
5. Turkmen M. Investigation of the relationship between academic and 
sport motivation orientations. Middle-East J Sci Res 2013; 16: 1008-
14.
6. Fortes PC, Rodrigues G, Tchantchane A. Investigation of academic 
and athletic motivation on academic performance among university 
students. IAC S IT Press 2011; 2: 181-5.
7. Buckworth J, Nigg C. Physical activity, exercise, and sedentary 
behavior in college students. J Am Coll Health 2004; 53: 28-34.
8. Sindik J, Furjan Mandić G, Schiefler B, Kondrič M. Differences in the 
structure of motivation for participation in sport activities among 
sport students in three countries. Kinesiologia Slovenica 2013; 19: 
14-31.
9. Kondrič M, Sindik J, Furjan-Mandić G, Schiefler B. Participation 
motivation and student`s physical activity among sport students in 
three countries. J Sport Sci Med 2013; 12: 10-8.
10. Santos Legnani RF, Guedes DP, Legnani E, Cordeiro Barbosa Filho V, 
de Campos W. Motivational factors associated with physical exercise 
in college students. Revista Brasileira Ciências Esporte 2011; 33: 3.
11. Ebben W, Brudzynski L. Motivations and barriers to exercise among 
college students. J Exercise Physiology 2008; 11: 5.
12. Spray CM, Wang CKJ, Biddle SJH, Chatzisarantis NLD. Understanding 
motivation in sport: an experimental test of achievement goal and 
self determination theories. Eur J Sport Sci 2006; 6: 43-51.
13. Wang CKJ, Stuart JH, Biddle JHS. Intrinsic motivation towards sports 
in Singaporean students: the role of sport ability beliefs. Health 
Psychol 2003; 8: 515-23.
14. Shao-hua J, Mei-ting Q, Rui-guang C, Qiong-xia C. Research on 
gender difference of consciousness and behavior of physical exercise 
among college students in Hebei Province. J Beijing Sport University 
2006; 13: 42-4.
15. Sirard JR, Pfeiffer KA, Pate RR. Motivational factors associated with 
sports program participation in middle school students. J Adolescent 
Health 2006; 38: 696-703.
16. Egli T, Bland HW, Melton BF, Czech DR. Influence of age, sex, and race 
on college students’ exercise motivation of physical activity. J Am 
Coll Health 2011; 59: 399-406. 
17. Guedes DP, Santos Legnani RF, Legnani E. Exercise motives in college 
students and associated factors. Revista Brasileira Educação Física 
Esporte 2012; 26: 4.
18. Kolar E, Cerar K, Piletič S, Svetlik I, Kugovnik O. Analysis of some 
aspects of the sports activity of University of Ljubljana students. 
Šport 2009; 57: 20-7.
19. Markland D, Hardy L. The Exercise motivations inventory: preliminary 
development and validity of a measure of individuals’ reasons for 
participation in regular physical exercise. Pers Indiv Differ 1993; 15: 
289-96.
20. Deci EL, Ryan RM. The general causality orientations scale: self-
determination in personality. J Res Pers 1985; 19: 109-34.
21. Chen W. Chinese and American college students’ motives for 
participation in physical activities. Percept Motor Skill 1998; 87: 
1463-70.
10.1515/sjph-2017-0014 Zdr Varst 2017; 56(2): 107-114
114
22. Gill DL. Gender differences in competitive orientation and sport 
participation, Int J Sport Psychol 1988; 19: 145 -59. 
23. Gernigon C, Bars HL. Achievement goals in aikido and judo, 
comparative study among beginner and experienced practitioners. J 
Appl Sport Psychol 2000; 12: 168-79.
24. Eagly AH. Sex differences in social behavior: a social-role 
interpretation. Hillsdale, NJ: Erlbaum, 1987.
25. Eagly AH, Johnson BT. Gender and leadership style: a meta-analysis. 
Psychological Bulletin 1990; 108: 233-56.
26. Argyle M. The psychology of interpersonal behavior. Harmondsworth, 
UK: Penguin, 1990.
27. Schutte N, Malouff J, Hall E, Haggerty D, Cooper J, Golden D, 
Dornheim I. Development and validation of a measure of emotional 
intelligence. Pers Indiv Differ 1998; 25: 167-77.
28. Smith BL, Handley P, Eldredge DA. Sex differences in exercise 
motivation and body-image satisfaction among college students. 
Percept Motor Skill 1998; 86: 723-32.
29. Bowman M, Flower N, Machuga J, Morris M, Pasternak A, Raudenbush 
B. Motivation differences between group and individual athletic 
teams participation in intercollegiate and intramural sport. J Sport 
Exercise Psy 2001; 23: S29. 
30. Winberg R, Gershon T, Alex M, Susa J, Mark A, Robert G, Gerald F. 
Motivation for youth participation in sport and physical activity: 
relationships to culture, self-reported activity levels, and gender. J 
Sport Psychol 2000; 31: 321-46. 
31. Cindy H, Linder K. Situational state balances and participation 
motivation in youth sport. Brit J Educ Psychol 2006; 76: 369-84.
32. Guedes DP, Santos Legnani RF, Legnani E. Reasons for physical 
exercise practice in university students according to body mass 
index. Rev Bras Ativ Fis Saúde 2012; 17: 270-4.
33. Sindik J, Kozjak Mikić Z, Dodigović L, Čorak S. Analysis of the relevant 
factors for the engaging women in various sports in Croatia. Monten J 
Sports Sci Med 2016; 5: 17-28.
EVALUATION OF NATIONAL FOOD AND NUTRITION POLICY IN ALBANIA
Ehadu MERSINI
1*
, Jolanda HYSKA
1
, Genc BURAZERI
1
1’Terres des Hommes’, Health for all project, Rr. “Faik Konica”, Villa 19, P.O. Box 7426, 1010 Tirana, Albania 
Received: Nov 20, 2015
Accepted: Sep 19, 2016
Original scientific article
Introduction. The paper aims to describe the progress that has been made in the implementation of the 
Albanian food and nutrition policy since 2003, so as to consider its impacts to date, and to identify strategic 
priorities/critical areas and priorities for Albania’s future policy on improving the national food and nutrition 
situation.
Methods. In 2011-2012, an expert group applied an intersectoral participatory approach to evaluate the 
implementation of Food and Nutrition Action Plan 2003-08 in Albania. The experts employed the quantitative 
method, using a 9 question logical assessment matrix to measure the achievements of the individual goals of 
the Plan, and a qualitative tool for the interview of an interdisciplinary sample of 68-key informants-persons 
operating in public health nutrition, food safety and food availability related subfields, from a wide range of 
pertinent institutions and stakeholders. 
Results. The quantitative and qualitative assessment revealed that the implementation process has faced 
serious barriers linked to the design of the plan, which did not accurately anticipate a theoretical framework, 
or structured methods for its implementation. Other impeding factors included the lack of institutional/
infrastructure support, lack of intersectoral coordination and motivation, as well as insufficient capacities 
and know-how. Intersectoral response to the multifaceted nature of double burden of malnutrition is of key 
importance to improve nutritional wellbeing and health outcomes in Albania. 
Conclusion. Participatory approaches that involve all relevant sectors and actors in the development, 
monitoring and evaluation of the implementation of public health policies based on comprehensive action-
oriented assessments are promising and should be further supported.
Uvod. Namen članka je opis napredka pri izvajanju prehranske politike v Albaniji od leta 2003, pregled vplivov 
prehranske politike do danes ter prepoznavanje strateških prednosti in kriznih področij za prihodnjo politiko 
Albanije pri izboljševanju prehranskega stanja v državi.
Metode. Strokovna skupina je med letoma 2011 in 2012 izvedla medsektorski pristop s sodelovanjem za 
ovrednotenje izvajanja Akcijskega načrta za hrano in prehrano 2003–2008 (Food and Nutrition Action Plan 
2003-08) v Albaniji. Strokovnjaki so uporabili kvantitativno metodo z uporabo logične ocenjevalne matrice z 
devetimi vprašanji, ki vrednoti  dosežke posameznih ciljev v Načrtu ter kvalitativno orodje za namen intervjuja 
interdisciplinarnega vzorca 68 ključnih oseb, ki delujejo na področju javnega zdravja, prehrane, varnosti živil 
in podpodročjih, ki so povezana z razpoložljivostjo hrane – širok izbor med ustreznimi ustanovami in deležniki.
Rezultati. Kvantitativno in kvalitativno vrednotenje je razkrilo, da so se med postopkom izvedbe pojavile 
resne ovire, ki so povezane z oblikovanjem načrta, ki ni ustrezno predvideval teoretičnih okvirjev ali 
strukturiranih metod za izvedbo. Ostali oviralni dejavniki so bili tudi pomanjkanje podpore s strani institucij 
in infrastrukture, pomanjkanje medsektorske koordinacije in motivacije kot tudi nezadostne kapacitete in 
znanje. Medsektorski odziv na večstransko naravo dvojnega bremena pri neustrezni prehranjenosti je ključna 
zadeva pri izboljševanju prehranske blaginje in zdravstvenih izidov v Albaniji.
Zaključek. Pristopi s sodelovanjem, ki vključujejo vse pomembne dejavnike in akterje pri razvoju, nadzoru 
in ovrednotenju izvedbe politike javnega zdravja na podlagi celostne in akcijske ocene, so obetavni in jih je 
treba podpirati tudi v prihodnje.
ABSTRACT
Keywords: 
Albania, food, nutrition 
policy, implementation, 
assessment
IZVLEČEK
Ključne besede: 
Albanija, hrana, 
prehranska politika, 
izvedba, ovrednotenje
*Corresponding author: Tel: ++ 385 5 694 089 031; E-mail: ehadmersini@yahoo.com
10.1515/sjph-2017-0015 Zdr Varst 2017; 56(2): 115-123
OVREDNOTENJE NACIONALNE PREHRANSKE POLITIKE V ALBANIJI
Mersini E, Hyska J, Burazeri G. Evaluation of national food and nutrition policy in Albania. Zdr Varst 2017; 56(2): 115-123
115
© Nacionalni inštitut za javno zdravje, Slovenija. 
1 INTRODUCTION
There has been a significant progress in food and 
nutrition policy development across Europe in the past 
two decades. Despite this progress, most countries in 
the European region have, nevertheless, not achieved 
nutrition and dietary goals (1). Overall, a number of 
developments in global policy, research and guidance have 
contributed continuously to strengthening and supporting 
the implementation of food and nutrition policies across 
the countries (2-6). 
However, the implementation of the policies continues 
to be a major challenge due to lack of funds, political 
commitment and coordination. Hence, more support 
should be given to the implementation and evaluation 
of policies, and a shift towards stronger environmental 
approaches is needed (7).
Following the endorsement of the First Action Plan 
for Food and Nutrition Policy for the WHO European 
Region 2000–2005 (3), an intersectoral working group 
with representatives from health, agriculture, tourism, 
education, environment, economy, labour and social 
affairs, finance, as well as other governmental institutions, 
developed the Albanian Action Plan for Food and Nutrition 
2003-2008 (AAPFN), signed by the Prime Minister in July 
2003 (8). The adoption of the AAPFN 2003-2008 by the 
government provided the basis for political commitment 
and regulatory entry point for all public institutions to 
translate the policy into actions in the field of food and 
nutrition in Albania. However, there is little published 
evidence about this event. The AAPFN aimed at reducing 
the level of foodborne diseases, protecting and promoting 
health, and putting public health at the heart of food 
policy.
The purpose of this paper is to describe the progress 
that has been made in the implementation of the AAPFN 
since 2003 – what has been achieved and what remains 
to be done – to consider its impacts to date, and to 
identify strategic priorities/critical areas and priorities 
for Albania’s future policy on improving the food and 
nutrition situation in Albania.
2 METHODS
The evaluation expert group applied an intersectoral 
participatory approach involving, in addition to the 
review of existing documents and data, qualitative 
and quantitative methods. The same methodology was 
applied as for the evaluation of the implementation of the 
Slovenian Food and Nutrition Policy 2005-2010 (ReNPPP 
2005-10) (9).
The intersectoral expert group employed the quantitative 
method using a 9 question logical assessment matrix 
(Table 1), with nine evaluations grades (Table 2) (9), to 
measure the achievements of the individual (re-shaped) 
goals of the AAPFN 2003-2008 
10.1515/sjph-2017-0015 Zdr Varst 2017; 56(2): 115-123
116
Table 1.
Table 2.
A nine question logical assessment matrix (9).
Criteria for evaluation (9).
Q 1: Has there been enough evidence produced to set priorities 
for the individual goal?
Q 2: Were appropriate measures proposed for the individual 
goal?
Q 3: Were proposed measures adopted?
Q 4: Were adopted measures implemented?
Q 5: Were social inequalities tackled by proposed measures?
Q 6: Was at least 50% of the target population reached by the 
measure?
Q 7: Were sufficient financial resources provided for the 
implementation?
Q 8: Were strategic aims of the FNAP in line with 
implementation goals?
Q 9: Were implementation activities in line with the individual 
goal?
Considerable success (+)   (4.8-5)
Considerable/moderate success   (4.3-4.7)
Moderate success (+/0)    (3.8-4.2)
Moderate/little success    (3.3-3.7)
Little success (0)     (2.8-3.2)
Little/minimal success    (2.3-2.7)
Minimal success (0/-)   (1.8-2.2)
Minimal/no success    (1.3-1.7)
No success (-)    (1.2 and less)
The National Institute of Public Health conducted the 
qualitative assessment exercising a tool (9) with 36 closed 
questions with open space to express opinions, for the 
interview of an interdisciplinary sample (9) of 68-key 
informants-persons operating in public health nutrition, 
food safety and food availability related subfields, from a 
wide range of pertinent institutions and stakeholders. In 
total, 60 interviews were carried out.
2.1 Data Analysis 
The intersectoral expert group estimated the achievement 
of individual goals calculating the average score for each 
goal (9) of the AAPFN 2003-08, as per the abovementioned 
criteria.
10.1515/sjph-2017-0015 Zdr Varst 2017; 56(2): 115-123
117
The processing of 60 completed questionnaires used a 
specially designed database and translated the questions 
or fields in the questionnaire into 87 variables during the 
analysis. Statistical Package for Social Sciences (SPSS, 
version, 17.0) was used for the analysis of quantitative 
data, whereas most of the analysis concerning the 
qualitative information applied horizontal qualitative 
techniques. 
The expert group prepared a detailed analysis of the 
assessments conducted, recommendations included. 
One large intersectoral group discussed and agreed upon 
the recommendations for the future nutrition policy in 
Albania. 
The authors aggregated the findings of this process. 
2.2 Validation 
The ‘methodology’ workshop (9), including a training 
session on quantitative methodology, was held in October 
2011. Workshops sharing results were held in March and 
July 2012.
3 RESULTS
The APFN 2003-2008 addressed a food safety pillar through 
the fulfilment of four (reshaped) goals and a nutrition 
pillar through the fulfilment of eleven (reshaped) goals 
(Table 3). 
Table 3. Estimation by pillars1.
FOOD SAFETY 
(3.8)
1. Harmonising the 
legislation with 
the EU (4.8)
2. The 
improvement 
of food safety 
control and the 
establishment of 
SS (3.6)
3. Strengthening 
food control 
laboratories (2.7)
4. Strengthening 
technical 
capacities (4.1)
NUTRITION (4.2)
1. The prevention 
of malnutrition 
among women: 
adolescence 
pregnancy 
lactation (4.1)
2. Preserving 
the tradition of 
breastfeeding, and 
timely initiation 
of complementary 
feeding (4.3)
5.0
5.0
5.0
5.0
5.0
5.0
5.0
5.0
3.1
4.5
3.0
2.5
2.5
4.6
3.7
4.3
3.8
5.0
3.0
2.0
4.0
3.9
3.7
4.1
3.3
4.5
3.0
1.8
3.8
3.7
3.7
3.1
NA
NA
NA
NA
NA
4.4
5.0
4.9
NA
NA
NA
NA
NA
3.4
3.0
3.9 
3.3
4.5
3.0
1.8
4.0
3.4
3.5
4.0
4.3
5.0
4.2
3.0
5.0
4.9
4.7
5.0
4.3
5.0
4.2
3.0
5.0
4.8
4.7
5.0
Research 
data
available, 
with
priorities 
Q1 (5.0)
Measures 
were
Imple-
mented Q4
(3.5)
Measures 
were
Proposed 
Q2 (3.9)
Social 
inequalities
were 
tackled by 
the
measure Q5 
(4.4)
Measures 
were
Adopted Q3 
(3.8)
At least 50% 
of the 
Target 
Population 
was 
reached by 
the
Goals are in 
line with 
The 
strategic 
aim of 
Sufficient 
financial
Resources 
were
Provided Q7 
(3.4)
Imple-
mented
Activities 
are in line
with the 
goals Q9 
(4.6)
1 Results are presented in the template generated for the evaluation of the implementation of the Slovenian Food and Nutrition Policy 
2005-2010 (ReNPPP 2005-10) (9).
10.1515/sjph-2017-0015 Zdr Varst 2017; 56(2): 115-123
118
3. The evaluation 
of the mode of 
nutrition (4.0)
4.The evaluation 
of the nutritional 
status of the 
population (4.9)
5. Determining 
main risk factors 
for nutrition-
related diseases 
(4.3)
6. The eradication 
of iodine 
deficiency in 
the Albanian 
population (4.7)
7. Anaemia 
prevalence 
assessment and 
the prevention of 
IDA (4.0)
8. The 
improvement of 
oral health (3.8)
9.The 
establishment of a 
national network 
for M&E of the 
nutritional status 
of the population 
(4.2)
10. Managing 
the nutritional 
education, 
informing the 
population on a 
healthy nutrition 
and healthy 
lifestyle (3.7)
(i) The 
compilation of 
Recommendations 
for a healthy 
nutrition and 
healthy lifestyle 
(3.9)
(ii) The 
preparation of 
inter-sectorial 
promotional 
interventions (4.1)
5.0
5.0
5.0
5.0
5.0
5.0
5.0
5.0
5.0
5.0
5.0
5.0
5.0
5.0
4.6
4.4
5.0
3.9
4.3
4.1
4.0
5.0
4.2
5.0
3.3
3.3
3.2
3.3
3.7
3.7
3.1
5.0
4.0
4.9
3.1
2.7
4.0
2.9
2.7
3.7
2.6
5.0
4.6
4.9
4.9
3.7
5.0
3.4
4.0
4.1
2.9
4.6
2.8
4.1
3.6
3.0
2.5
3.0
3.3
3.9
3.3
4.3
3.2
3.9
2.2
3.8
3.5
2.3
2.3
3.1 
5.0
5.0
5.0
4.9
5.0
4.4
5.0
4.9
5.0
4.7
5.0
5.0
5.0
4.9
4.3
4.4
5.0
5.0
5.0
5.0
Research 
data
available, 
with
priorities 
Q1 (5.0)
Measures 
were
Imple-
mented Q4
(3.5)
Measures 
were
Proposed 
Q2 (3.9)
Social 
inequalities
were 
tackled by 
the
measure Q5 
(4.4)
Measures 
were
Adopted Q3 
(3.8)
At least 50% 
of the 
Target 
Population 
was 
reached by 
the
Goals are in 
line with 
The 
strategic 
aim of 
Sufficient 
financial
Resources 
were
Provided Q7 
(3.4)
Imple-
mented
Activities 
are in line
with the 
goals Q9 
(4.6)
10.1515/sjph-2017-0015 Zdr Varst 2017; 56(2): 115-123
(iii) Strengthening 
intellectual 
potential on 
nutrition (3.2)
11.The 
improvement of 
legislation for the 
improvement of a 
nutrition-related 
health status (4.7) 
5.0
5.0
3.2
5.0
2.6
3.8
2.4
4.5
2.0
4.6
1.8
4.5
1.6
4.5
5.0
5.0
5.0
5.0
Research 
data
available, 
with
priorities 
Q1 (5.0)
Measures 
were
Imple-
mented Q4
(3.5)
Measures 
were
Proposed 
Q2 (3.9)
Social 
inequalities
were 
tackled by 
the
measure Q5 
(4.4)
Measures 
were
Adopted Q3 
(3.8)
At least 50% 
of the 
Target 
Population 
was 
reached by 
the
Goals are in 
line with 
The 
strategic 
aim of 
Sufficient 
financial
Resources 
were
Provided Q7 
(3.4)
Imple-
mented
Activities 
are in line
with the 
goals Q9 
(4.6)
Food security component was not addressed specifically 
compared to two other pillars of this Plan, namely: food 
safety and nutrition. Thus, it neither elaborates specific 
goals and measures nor does it identify the resources 
needed to implement any action related to food security. 
Regarding the adequacy and consistency of the way 
this policy document addresses food safety pillar, the 
evaluation showed the highest level of availability of 
evidence to set priorities for the respective goals, the 
food safety implementation goals aligned with strategic 
aims and the same as the implementation activities with 
the individual goals, but small appropriateness related 
to the measures proposed for the food safety individual 
goals. 
The evaluation revealed that the food safety measures 
proposed by the Plan were adopted moderately, and that 
the food safety measures adopted were implemented 
partially successfully. The financial resources for their 
implementation, quantified through perceptions of key 
informants, were evaluated as partially sufficient. 
With regard to the implementation of food safety 
individual goals, the evaluation revealed that ‘the 
improvement and harmonization of food safety legislation 
with the European Union’ was implemented successfully; 
moreover, ‘strengthening food safety technical staff 
capacities at the central and regional level’ was 
implemented moderately successfully; in addition, ‘the 
improvement of food safety control system and the 
establishment of a surveillance system’ was implemented 
partially successfully, whereas the goal of ‘strengthening 
and increasing the level of food control laboratories, in 
health, agriculture and food systems’ was implemented 
very partially successfully. 
Table 4. Findings from the logical evaluation matrix2.
The number of goals
Average assessment
No success (%)
Minimal/no success (%)
Minimal (%)
Little/Minimal (%)
Little (%)
Moderate/little (%)
Moderate (%)
Considerable/moderate (%)
Considerable (%)
TOTAL
1
2
3
4
5
6
7
8
9
4
Moderate
0
0
0
25
0
25
25
0
25
100%
11
Moderate
0
0
0
0
0
9
46
36
9
100%
FOOD SAFETY NUTRITION
The evaluation found the overall implementation of 
the food safety strategic goals of the AAPFN 2003-08 
moderately successful (Table 4).
Regarding the adequacy and consistency of the way the 
AAPFN 2003-08 documents address nutrition pillar, the 
evaluation found a very high level of the availability of 
evidence to set priorities for relative goals, nutrition 
implementation goals very aligned with strategic aims 
and the same as the implementation activities with 
the individual goals. The evaluation revealed that the 
measures for nutrition individual goals were appropriately 
proposed, and that social inequalities were tackled 
by them sufficiently. The evaluation revealed that the 
nutrition measures proposed by the Plan were adopted 
moderately, that the nutrition measures adopted were 
2 Results are presented in the template generated for the evaluation of the implementation of the Slovenian Food and Nutrition Policy 
2005-2010 (ReNPPP 2005-10) (9).
119
10.1515/sjph-2017-0015 Zdr Varst 2017; 56(2): 115-123
120
partially implemented, that the coverage by them of at 
least 50% of target population was partially achieved, 
and that the financial resources provided for their 
implementation were partially sufficient. 
With regard to the implementation of nutrition individual 
goals, the evaluation revealed that ‘the evaluation 
of the nutritional status of the population” was likely 
to be implemented very successfully, whereas the 
goals of ‘preserving the tradition of breastfeeding, 
exclusive breastfeeding and providing timely initiation 
of complementary feeding, ‘determining main risk 
factors for nutrition-related diseases, ‘the eradication 
of iodine deficiency in the Albanian population,’ and ‘the 
improvement and strengthening of legislation for the 
improvement of health status due to a better nutrition’ 
were considered to be implemented successfully.
‘Managing the nutritional education, informing the 
population on a healthy nutrition and healthy lifestyle’ 
was estimated to be implemented partially successfully.
The evaluation found the overall strategic nutrition 
goals of the AAPFN 2003-08 implemented ‘moderately’ 
successfully (Table 4). 
Regarding knowledge sharing process of the AAPFN 
2003-08 after its adoption, only 55% of key informants 
interviewed had a good or relatively good knowledge 
about the APFN and, among them, the majority worked 
in central institutions and came from the health sector, 
compared to agriculture, education, culture, tourism, 
youth, sport, work, social welfare, finances and civic 
society sectors, producers and private enterprises. The 
lack of the systematic exchange of information among 
different management levels within a sector and among 
different sectors was identified as the main cause of not 
having enough knowledge on the Plan. 
Regarding the extent of the influence the AAPFN had on 
improving the situation, respondents thought that the 
Plan had contributed more to the increased consumption 
of fruits and vegetables than to the reduction of food-
borne diseases and obesity. 
There seemed to be an almost complete consensus among 
key informants interviewed regarding the successfulness 
of the implementation of three main nutrition goals, 
namely: (i) support and promotion of breastfeeding; (ii) 
established health education programs for pregnant and 
postpartum women, and (iii) improved availability of 
health beneficial foods and healthy nutrition. 
The main types of implemented actions mentioned by key 
informants included informing and educating the media, 
consumers, whole population, mothers, school-aged 
children, teaching staff, private enterprises for specific 
issues (such as iodized salt) and health professionals in 
particular; the enlargement of the network of baby-
friendly maternity-hospitals; the development of Albanian 
Guidelines for a Healthy Nutrition; population nutritional 
status monitoring with the focus on the childbearing-age 
of women and children; preparing bylaws and improving 
legislation in the field of food and nutrition along with the 
harmonization of the respective legislation with the EU 
framework; food safety capacity building and food safety 
inspection and monitoring activities.
Regarding the employment of the participatory 
multisectoral approach during the implementation of 
AAPFN activities, the respondents from the Ministry of 
Health and Ministry of Agriculture, Food and Consumer 
Protection, in 72% and 63%, respectively, declared that they, 
in general, frequently collaborate with other ministries. 
This collaboration is likely to be more between these two 
ministries than between each of them and others. There 
seem to be somehow more positive opinions related to 
the level of collaboration with the Ministry of Education 
and Science, while regarding other ministries, according 
to key informants, there seems to be consistently scarce 
or absent collaboration. Collaboration with the Ministry 
of Finance and the Ministry of Culture, Tourism, Youth and 
Sports is reported to be at very low levels. 
Regarding other institutions, the respondents’ 
collaboration is likely to be better with the public 
health authorities, media, NGOs, health centres and 
education institutions, and less frequent with professional 
institutions, faculties, food industries, while respondents 
report that their collaboration with chambers of commerce 
is insignificant. A significant number of respondents think 
that the AAPFN has not been very effective in supporting 
the communication, especially with the public and other 
actors outside the public system (target groups, civil 
society organisations, etc.). It seems that the plan had 
been mostly effective in facilitating communication 
among professionals from the same sector.
Difficulties or barriers encountered during the 
implementation of the Plan, identified by a large number 
of respondents, were related to: 
• Designing the plan: Most of the objectives of the 
plan were relatively clear and feasible, but detailed 
descriptions of the actions, the responsible actors, 
implementers and the time frame were often missing; 
in some cases, objectives and measures were very 
general and not measurable; estimated costs for the 
implementation and prioritisation of actions were 
missing too; the plan was also lacking an integrated 
plan for monitoring and the evaluation of the 
effectiveness of proposed objectives and activities.
• Time and energy was lost due to frictions among 
different sectors and institutions.
• There was a shortage of institutional and 
infrastructural support, in particular, there was lack 
of a national body responsible for the implementation 
10.1515/sjph-2017-0015 Zdr Varst 2017; 56(2): 115-123
121
and coordination of activities of the nutrition policy 
and insufficient capacities and ‘know how’ at the 
central and local level.
• There were difficulties related to the population’s 
nutrition habits and resistance to behavioural 
changes.
Among factors which could help the implementation of 
food and nutrition policies, the key informants consider: 
firstly, the awareness of professionals and the public; 
secondly, the collaboration with other sectors; and 
thirdly, financial support.
The prioritised recommendations/actions for the 
future nutrition policy in Albania, according to the key 
informants, were as follows:
• Exercising regular inspection measures over the 
energy and nutritional value of school and nursery 
school meals. 
• Integrating healthy nutrition-related topics into 
school curricula.
• Installing water fountains in schools and nursery 
schools. 
• Integrating measures in the field of nutrition with 
measures in the field of physical activity. 
• The establishment of a cross-sector body in the field 
of food supply/food safety/nutrition.
• The preparation of specific measures to limit the 
trend of obesity.
• Increasing the encouragement of farmers to sell food 
in the local environment.
• Restricting the marketing of unhealthy foods to 
children.
• Supporting measures to improve the diet of socially 
disadvantaged groups. 
• Reducing the taxation of fruits and vegetables. 
4 DISCUSSIONS 
The AAPFN 2003-08 was the main strategic policy, providing 
the basis for political commitment, and enabling public 
health and non-health institutions to transfer policy into 
action in the fields of food and nutrition in Albania. This 
Plan created and supported a policy environment that 
placed more emphasis on food safety and nutrition than 
on food security. In this sense, the Plan provides only a 
vague guidance for integrating some aspects of food 
security in developmental programs, while it does not 
point out activities at the macro and specific level for 
respective sectors and institutions. There is a lack of 
actions addressing the macro environment (food supply 
sector, urban planning and transport) through accessibility, 
affordability and availability. 
Despite the overall progress over the last decade in 
improving the food and nutrition situation, legal and 
institutional frameworks, research and evaluation 
activities, and public awareness, Albania is still facing 
multiple nutrition-related problems. Albania is currently 
facing the double burden of malnutrition, including high 
rates of stunting (19 %) and overweight among children 
under 5 years (22%), disparities in health and nutrition 
status and micronutrient deficiencies (10), and a burden 
of Non Communicable Diseases (NCD) with an increasing 
trend (11). A current analysis revealed that actions were 
not being implemented on a regular basis and nationally, 
and that they were not tailored to the current dynamics 
of the food system (price, availability and accessibility 
of food) and marketing pressure that addresses the 
demand for food in a completely different direction from 
what the dietary guidelines indicate (7). This cannot be 
achieved by the health sector on its own; it requires the 
involvement of different sectors of the government as 
well as different stakeholders in the society (12). The 
AAPFN 2003-08 did not create a supportive environment to 
facilitate the fulfilment of health and nutrition objectives 
with the contribution of all sectors. The information 
on the implementation and effectiveness of specific 
interventions of the AAPFN 2003-08 was not available. 
The analysis showed that the document includes neither 
detailed implementation plans and respective financial 
implications nor mechanisms and tools to evaluate the 
implementation. 
Regardless of the engagement of different stakeholders 
in the APFN 2003-08 formulation, the lack of effective 
collaboration among different sectors and institutions 
acted as a barrier for the implementation of this plan. 
The APFN 2003-08 did not clearly define the responsible 
ministries and task performers, which could support 
the implementation of actions, and it did not explicitly 
foresee the establishment of any specific administrative 
structure/body responsible for the implementation and 
coordination of activities of this food and nutrition policy. 
The presence of a national coordination body, such 
as a food and nutrition council, allows governments to 
develop, implement, monitor and evaluate nutrition 
policies, guidelines and action plans (7).
The evaluation methodology (9) used in Albania, helped 
us to show a similar multifaceted nature of the food and 
nutrition as in Slovenia, where the methodology was 
developed. So, the key common findings were the need 
for (i) an intersectoral response, involving the design 
and implementation of food and nutrition policies; (ii) 
systematically including evaluation and monitoring 
mechanisms and tools in the policy document; (iii) having 
in place a food and nutrition policy intending to achieve 
its primary objectives through influencing other public 
policy measures.
10.1515/sjph-2017-0015 Zdr Varst 2017; 56(2): 115-123
122
4.1 Study Limitations 
This analysis was limited to the AAPFN 2003-2008 
document and interviewing of 68 key informants, and 
additional information obtained from recent publications, 
various web-sites of national health ministries and health 
agencies. Although the sample of key informants had 
a multisectoral representation centrally and locally, 
it remains a purposive sample that does not allow 
extrapolating the findings on knowledge over the AAPFN 
2003-2008 in all relevant executives and professionals 
in the country. However, the main purpose of the 
assessment was to help analyse the appropriateness of 
the plan designation and gaps or weaknesses during its 
implementation through a multisectoral participatory 
identification of key deficiencies and relevant actions to 
address them. 
5 CONCLUSIONS
The comprehensive, participatory assessment of the 
Action Plan for Food and Nutrition (APFN) 2003-2008 
created an evidence base for the development of the 
new food and nutrition policy more responsive to food 
and nutrition situation in Albania. The institutional 
intersectoral response to the multifaceted nature of a 
double burden of malnutrition is essential to improve 
nutritional wellbeing and health outcomes in Albania. 
The main lesson drawn from this process was that 
‘policy makers should develop implementation strategies 
that explicitly take account of financial, managerial 
and technical aspects of the policy (capacity) and the 
anticipated resistance and support from all the actors in 
the subsystem within and outside government, to avoid 
the gap between policy expectation and reality’ (13).
Albania case study has shown that in spite of a very 
demanding evaluation of the national policy, the 
methodology used in this study (9) can be transposed 
from one country (Slovenia) to another (Albania). It has 
the ‘potential’ to be used to evaluate the implementation 
of other public policies in other contexts and sectors 
with some respective adaptations. Therefore, this study 
is a contribution to future endeavours to advance the 
methodologies for the evaluation of the implementation 
of complex national policies.
CONFLICTS OF INTEREST
None declared. 
FUNDING
This evaluation was performed in the framework of the 
Joint Programme on Nutrition 2010-2013, funded by the 
Spanish Millennium Development Goals Achievement 
Fund and implemented by the Albania Ministry of Health 
(MoH) and Ministry of Agriculture, Food and Consumer 
Protection (MoAFCP), with support from the World Health 
Organisation (WHO), the United Nations Children’s Fund 
(UNICEF) and the Food and Agriculture Organisation of the 
United Nations (FAO). The preparation of this paper was 
not financially supported.
ETHICAL APPROVAL
A verbal informed consent from the Directory of Public 
Health at the Albania Ministry of Health allowed us to 
conduct the assessment in question with an intersectoral 
working group, consisting of representatives from the 
Ministry of Health and Ministry of Agriculture, Food and 
Consumer Protection, while the WHO provided technical 
assistance with the assessment methodology. 
REFERENCES
1. World Health Organization. Comparative analysis of nutrition 
policies in WHO  European region. Copenhagen: WHO Regional Office 
for Europe, 2006.  Available April 20, 2014 from: http://www.euro.
who.int/document/Nut/instanbul_conf_%20ebd02.pdf.
2. Food and Agriculture Organization of the United Nations.  World 
declaration and plan of action for nutrition. Rome: FAO, 1992. 
Available April 21, 2014 from: http://whqlibdoc.who.int/hq/1992/
a34303.pdf.
3. World Health Organization. The first action plan for food and 
nutrition policy, WHO European region 2000–2005. Copenhagen: 
WHO Regional Office for Europe, 2000.  Available April 20, 2014 from: 
http://www.euro.who.int/Document/E72199.pdf.
4. World Health Organization. World Health Assembly Resolution 
WHA57.17 on the global strategy on diet, physical activity and 
health. Geneva: WHO, 2004. Available April 20, 2014 from: http://
www.who.int/gb/ebwha/pdf_files/WHA57/A57_9-en.pdf.
5. World Health Organization. European charter on counteracting 
obesity. Copenhagen: WHO Regional Office for Europe, 2006. 
Available April 20, 2014 from: http://www.euro.who.int/Document/
E89567.pdf.
6. World Health Organization. WHO European Action Plan for food and 
nutrition policy 2007-2012. Copenhagen: WHO Regional Office for 
Europe, 2008. Available May 21, 2014 from: http://www.euro.who.
int/__data/assets/pdf_file/0017/74402/E91153.pdf.
7. Trübswasser U, Branca F. Nutrition policy is taking shape in Europe. 
Public Health Nutr 2009; 12: 295–306.
8. Albania Council of Ministers.  National action plan for food and 
nutrition 2003-2008. Tirana: Council of Ministers, 2003. Official 
Gazette/Fletorja zyrtare 2003; 60: 2538.
10.1515/sjph-2017-0015 Zdr Varst 2017; 56(2): 115-123
123
9. Gabrijelcic Blenkus M, Gregoric M, Ivanusa M. Evaluating the 
implementation of the Resolution on the national program of the 
food and nutrition policy 2005-2010 (ReNPPP 2005-10): report for 
the Ministry of Health. Ljubljana: National Institute of Public Health 
Slovenia, 2010. Available August 15, 2016 from: http://www.nijz.
si/sites/www.nijz.si/files/uploaded/slovenefnap2005-10evaluation_
shortenversion_dec2011_final.pdf
10. Institute of Statistics, Institute of Public Health (Albania), ICF 
Macro. Albania demographic and health survey 2008–2009. Tirana: 
Institute of Statistics, Institute of Public Health, ICF Macro, 2010. 
Available April 20, 2014 from: http://www.measuredhs.com/pubs/
pdf/FR230/FR230.pdf
11. World Health organization. Non communicable diseases country 
profiles: Albania. Geneva: World Health Organization, 2011. 
Available April 20, 2014 from: http://www.who.int/nmh/countries/
alb_en.pdf.
12. World Health Organization. The challenge of obesity in the WHO 
European region and the strategies for response. Copenhagen: WHO 
Regional Office for Europe, 2007: Chapters 16 and 18.  Available April 
20, 2014 from: http://www.euro.who.int/document/E90711.pdf
13. Buse K, Mays N, Walt G. Making health policy. London: Open 
University Press, 2012; 146.
LONGEVITY IN SLOVENIA: PAST AND POTENTIAL GAINS IN LIFE EXPECTANCY 
BY AGE AND CAUSES OF DEATH
Aleša LOTRIČ DOLINAR1, Petra DOŠENOVIĆ BONČA1, Jože SAMBT1*
1University of Ljubljana, Faculty of Economics, Kardeljeva ploscad 17, 1000 Ljubljana, Slovenia
Received: Jun 27, 2016
Accepted: Dec 30, 2016
Original scientific article
Introduction. In Slovenia, longevity is increasing rapidly. From 1997 to 2014, life expectancy at birth increased 
by 7 and 5 years for men and women, respectively. This paper explores how this gain in life expectancy at birth 
can be attributed to reduced mortality from five major groups of causes of death by 5-year age groups. It also 
estimates potential future gains in life expectancy at birth. 
Methods. The importance of the five major causes of death was analysed by cause-elimination life tables. The 
total elimination of individual causes of death and a partial hypothetical adjustment of mortality to Spanish levels 
were analysed, along with age and cause decomposition (Pollard). 
Results. During the 1997–2014 period, the increase in life expectancy at birth was due to lower mortality from 
circulatory diseases (ages above 60, both genders), as well as from lower mortality from neoplasms (ages above 
50 years) and external causes (between 20 and 50 years) for men. However, considering the potential future gains 
in life expectancy at birth, by far the strongest effect can be attributed to lower mortality due to circulatory 
diseases for both genders. If Spanish mortality rates were reached, life expectancy at birth would increase by 
more than 2 years, again mainly because of lower mortality from circulatory diseases in very old ages.  
Discussion and conclusions. Life expectancy analyses can improve evidence-based decision-making and allocation 
of resources among different prevention programmes and measures for more effective disease management that 
can also reduce the economic burden of chronic diseases.
Uvod. V Sloveniji se življenje hitro podaljšuje. V obdobju od leta 1997 do leta 2014 se je pričakovano trajanje 
življenja ob rojstvu podaljšalo za 7 let za moške in za 5 let za ženske. V članku ugotavljamo, koliko je k temu 
povišanju v Sloveniji do leta 2014 prispevalo zniževanje umrljivosti po petih glavnih skupinah vzrokov smrti 
in po posameznih starostnih razredih. Poleg tega v članku ocenjujemo tudi potencialno bodoče podaljšanje 
pričakovanega trajanja življenja ob rojstvu za Slovenijo. 
Metode. Uporabili smo skrajšane tablice umrljivosti s petletnimi starostnimi razredi. Razčlenitev podaljševanja 
pričakovanega trajanja življenja ob rojstvu v posameznih  starostnih razredih smo analizirali po prirejeni 
Pollardovi metodi. Znotraj posameznega starostnega razreda smo posamezni skupini glavnih vzrokov smrti 
(bolezni obtočil, novotvorbe, zunanji vzroki smrtnosti, bolezni dihal, bolezni prebavil) pripisali enak relativni 
vpliv na povišanje pričakovanega trajanja življenja, kot ga ima ta vzrok smrti na znižanje stopnje smrtnosti 
v tistem starostnem razredu. Potencialen prispevek posameznega vzroka smrti na podaljšanje pričakovanega 
trajanja življenja v prihodnosti smo analizirali s tablicami smrtnosti z izločenim posameznim vzrokom smrti. 
Ker je popolna izločitev posameznega vzroka smrti nerealistična, smo upoštevali tudi hipotetično prilagoditev 
stopenj smrtnosti na raven Španije, ki ima najdaljše pričakovano trajanje življenja ob rojstvu v EU in je po 
dejavnikih tveganja za glavne skupine vzrokov smrti podobna Sloveniji. 
Ugotovitve. V obdobju 1997-2014 je imelo največji vpliv na podaljševanje pričakovanega trajanja življenja 
zmanjševanje smrtnosti zaradi bolezni obtočil v starosti nad 60 let pri obeh spolih. Pri moških je močno vplivala 
tudi nižja smrtnost zaradi novotvorb v starosti nad 50 let ter zaradi zunanjih vzrokov smrti v starosti med 20 
in 50 let. Pri analizi potencialnega povišanja pričakovanega trajanja življenja smo ugotovili, da bi lahko največ 
pridobili z zniževanjem smrtnosti zaradi bolezni obtočil. Do podobnega sklepa pripelje prilagoditev stopenj 
smrtnosti v Sloveniji na ravni, ki jih ima Španija. Pričakovano trajanje življenja ob rojstvu bi bilo v tem primeru 
višje za več kot 2 leti, pri čemer bi največ razlike ponovno prispevali najvišji starostni razredi z nižjo smrtnostjo 
zaradi bolezni obtočil in novotvorb, precej pa tudi moški vseh starosti zaradi zunanjih vzrokov. 
Razprava in sklep. Analiza pričakovanega trajanja življenja po vzrokih smrti in starosti omogoča z dokazi 
podprto odločanje o razporejanju resursov med različne preventivne programe in programe obvladovanja bolezni.
ABSTRACT
Keywords: 
longevity, life tables, 
mortality, causes  
of death, cause 
elimination, age 
decomposition,  
potential gains in life 
expectancy,  
life expectancy at  
birth, Slovenia
IZVLEČEK
Ključne besede: 
dolgoživost, tablice 
umrljivosti,  
umrljivost, vzroki  
smrti, razčlenjevanje  
po starosti,  
potencialno  
podaljšanje  
življenjskega 
pričakovanja,  
Slovenija
*Corresponding author: Tel: ++ 386 1 5892 630; E-mail: joze.sambt@ef.uni-lj.si
10.1515/sjph-2017-0016 Zdr Varst 2017; 56(2): 124-130
124
DOLGOŽIVOST V SLOVENIJI: PRETEKLO IN PRIHODNJE PODALJŠEVANJE  
ŽIVLJENJSKEGA PRIČAKOVANJA PO STAROSTI IN VZROKIH SMRTI
Lotrič Dolinar A, Došenović Bonča P, Sambt J. Longevity in Slovenia: past and potential gains in life expectancy by age and causes of death. 
Zdr Varst 2017; 56(2): 124-130
© Nacionalni inštitut za javno zdravje, Slovenija. 
1 INTRODUCTION
In developed countries, a major outcome of fertility 
decline and increased longevity is population ageing that 
holds important implications for labour markets, social 
security, healthcare systems, and related developmental 
strategies. Currently, however, many countries are 
undergoing a different demographic transition (1). 
Given that in developed countries the biggest gains in 
life expectancy are realised at older ages (1, 2), such 
countries are examining these gains’ policy implications.
The literature addressing gains in life expectancy can 
be placed in three broad groups. The first comprises 
numerous studies of life expectancy gains arising from 
specific medical interventions and innovations. The 
second group studies the relationship between risk 
factors and life expectancy but, given the complexity 
of this relationship, either the impact of a specific risk 
factor on different causes of death is investigated (3-5), 
or different risk factor modifications for a specific disease 
are observed (6-7). The third group investigate the impact 
of the major causes of death on life expectancy, often 
combined with decomposition by age (8-11). 
This paper contributes to the third group of papers. The 
key motivation for this research is to build on the results 
for Slovenia shown in recent literature (10), where only 
aggregate estimations are provided. In 2015, a very 
influential paper looking at global, regional and national 
age-sex specific all-cause and cause-specific mortality 
was published (11). While studying the epidemiological 
convergence across countries, the decomposition of life 
expectancy showed the prominent role of reductions in 
age-standardised death rates for cardiovascular diseases 
and cancers in high-income regions, and reductions in 
child deaths from diarrhoea, lower respiratory infections, 
and neonatal causes in low-income regions. Because 
Slovenia is only briefly mentioned, our paper’s main 
goal is to provide a more detailed analysis of the main 
triggers for past and future potential improvements 
in life expectancy at birth. More specifically, we try 
to confirm that the mortality improvement pattern – 
whereby the biggest gains are achieved in older ages 
due to fewer cardiovascular diseases and cancers – also 
holds for Slovenia, implying that Slovenia, too, is in the 
“cardiovascular revolution” stage of epidemiological 
transition (12).
This paper therefore investigates longevity improvement 
in Slovenia between 1997 and 2014, for which detailed 
data are available. During this period, life expectancy 
at birth e0 increased by 7.0 and 5.0 years for men and 
women, respectively. Assuming the mortality pattern 
from 2014, e0 equals 78.2 years for men and 84.1 years 
for women. The e0 increase was particularly fast between 
2004 and 2014, that is 4.6 years for men and 3.3 years for 
women. As also shown by Figure 1, the increase is more 
pronounced for men. Figure 1 reveals the increase in e0 in 
Slovenia was larger than in the EU-28 area, respectively 
2.9 and 2.1 years for men and women. 
10.1515/sjph-2017-0016 Zdr Varst 2017; 56(2): 124-130
125
Eurostat’s latest population projections for Slovenia 
assume a further rise in e0 (14). Technically, e0 is most 
strongly influenced by preventing infant mortality. 
However, infant mortality is already very low in 
developed countries, and significant improvements are 
no longer possible. On the other hand, preventing deaths 
at older ages has less impact on e0, but there is greater 
room for improvement as gains in e0 may be made by, for 
instance, controlling disease risk factors and preventing, 
postponing, or slowing down the progression of a disease. 
Analyses for developed countries show the e0 increases in 
the last decades were mainly due to lower mortality in 
older age groups (1, 2, 10). 
As future gains in e0 are chiefly attributed to improved 
responses to noncommunicable diseases and changing 
social and environmental determinants of health (15), we 
can expect gains in e0 will continue to be made in higher age 
groups. A better understanding of potential improvements 
in e0 is growing in importance, because life expectancy 
gains concentrated at the end of life can unsettle any 
economy and may have even stronger negative effects in 
countries like Slovenia, having experienced a rapid rise in 
life expectancy and notable and rapid population ageing, 
requiring prompt policy and societal changes.
We estimate improvements in e0 due to reduced mortality 
from various groups of causes of death made between 
1997 and 2014, and potential improvements, using Spain, 
with the highest e0 among all EU countries in 2014, as a 
benchmark. We also look for differences between groups 
of causes of death in age terms, in which the strongest 
gains in e0 are possible. By decomposing potential gains 
Figure 1. Life expectancy at birth for men (M) and women 
(W), 2004–2014, Slovenia (SI) and EU-28 (13).
10.1515/sjph-2017-0016 Zdr Varst 2017; 56(2): 124-130
126
in e0 by age, we thus show whether these gains are 
potentially greater in working ages, or when people are 
mostly retired.
2 METHODS
In this study, we employ life tables that are used to 
describe age-specifi c mortality rates of a population. 
A life table is a demographic model that builds on age-
specifi c mortality rates of an actual population in a given 
year (16-18). Age specifi c mortality rates are converted 
into probabilities of dying, which are applied to a cohort 
of newborn children (a round number, usually 100,000, is 
assumed). The dying-off process is observed through life-
table functions: from each age to the next, the population 
is decremented according to the fi xed age-specifi c 
mortality probabilities until all members have died (16). 
If we divide the number of years to be lived by all cohort 
members during their life (the “total number of person-
years”) by 100,000, we obtain life expectancy at birth e0. 
It shows the expected longevity if the mortality pattern 
remains unchanged. Thus, e0 is not affected by the age 
distribution and is therefore comparable geographically 
and across time. We use abridged period life tables for 
5-year age groups (16-18) due to data availability. We use 
the 1997–2014 period data from the National Institute of 
Public Health (NIPH) (19) and Eurostat (20).
2.1 Potential Gains in Life Expectancy
By modifying the mortality pattern, we can estimate 
potential gains in e0 (8), attained by changing various 
determinants of life expectancy. According to WHO 
(15), much of the gain in e0, in developed countries, 
can be achieved by tackling noncommunicable diseases 
and changing social and environmental determinants of 
health. 
In Slovenia, the fi ve major groups of causes of death 
(according to ICD-10) in the last decades have been, 
namely: diseases of the circulatory system, neoplasms, 
external causes of morbidity and mortality, diseases of 
respiratory system, and diseases of digestive system 
(19). These fi ve groups combined represented 90% of all 
deaths in Slovenia in 2014. All other causes of death with 
a notably smaller share are allocated to “Other causes of 
death”. The analysed groups of causes’ contributions to 
the total mortality rate are shown in Figure 2.
Figure 2 shows a substantial drop in mortality rates. The 
improvement in e0 due to reduced cause-specifi c mortality 
can be calculated using different approaches, involving 
either complete or partial elimination of causes of death. 
The obtained potential gain in e0 reveals the impact of 
each cause of death.
Cause-eliminated life tables answer the hypothetical 
question of what a life-table cohort’s mortality would be 
if a particular cause of death were completely eliminated 
(21). Compared to the master life table, the number of 
deaths is now set to 0 for the analysed cause, whereas 
it remains unaltered for all other causes. For persons 
who actually died from the analysed cause of death, it 
is thus assumed they would still be alive and as healthy 
as their peers in the same age group, leading to lower 
death rates and higher e0. In reality, eradicating a certain 
disease is likely to result in increased mortality due to 
other diseases, especially in older ages, when people 
often have more than one illness (“competing risks”) (18). 
However, by constructing cause-eliminated life tables 
no interdependencies among the causes of death are 
assumed. The assumption of independence can hardly be 
avoided until more is known about the interdependency 
of various causes of death (8). Also, when the causes are 
distant (disease) categories, the correlation has a very 
small effect on the results, and therefore the assumption 
of independence among causes of death may be considered 
acceptable (9).
Because entirely eliminating a cause of death is an 
unrealistic assumption, we analyse the effects of a partial 
reduction (8) in mortality. We assume the probability 
of dying for Slovenia could fall to the level of a better 
performing comparable country. We estimate the 
potential gains in e0 by assuming the probability of dying 
in Slovenia would decrease to the Spanish levels from 
2014. Spain was selected because, in recent years, it 
has had the highest e0 among EU-28 countries (13). Ediev 
(22) argues that, although future gains will become ever 
harder to achieve when approaching the biological limit 
on the length of life, in the future, e0 could rise to above 
Figure 2. Age-standardised, cause-specifi c mortality rates 
for analysed groups of causes of death (according 
to ICD-10) for men (M) and women (W) in 1997 
and 2014, Slovenia (standard age structure: total 
population 2014) (19, 20).
10.1515/sjph-2017-0016 Zdr Varst 2017; 56(2): 124-130
127
95 and even close to 100 years, with up to 40% of a cohort 
becoming centenarians. Besides having the highest e0, 
Spain is comparable with Slovenia regarding risk factors, 
such as smoking and physical activity (23), the two key 
risk factors (7, 12) for the two most important groups 
of causes of death, namely: circulatory diseases and 
neoplasms. Relevant data are available from the Spanish 
national statistical institute (INE) (24).
2.2 Age- and Cause-Specifi c Decomposition
To investigate which age groups exhibit the strongest 
gains in e0, a decomposition by age is conducted. The 
most commonly applied decomposition methods include 
the Arriaga (25) and Pollard (26) approaches. Although 
mathematically equivalent (26), the various discrete 
approximations give somewhat different results. Arriaga’s 
approach underestimates the contribution of older age 
groups (21), but with the Pollard’s method, the total 
of partial contributions does not precisely add up (27) 
because of approximate formulae derived from the 
continuous approach. The Pollard’s method also requires 
detailed life tables for age groups above 85 years in order 
to measure the last open-ended age group’s contribution.
In this paper, the correct summation of e0 differences 
where standard life-table function notations are used 
with 1 and 2 denoting different points in time, different 
countries, or any other different populations.
To this age decomposition we also apply a procedure (29) 
for simultaneous decomposition by cause of death. This 
technique requires mutually exclusive and exhaustive 
causes of death. Here we assume an individual cause of 
death’s contribution to the e0 change in each age group 
is proportional to the mortality change arising from that 
cause of death in the total mortality rate change for the 
same age group.
3 RESULTS
Partial contributions of decreased mortality by individual 
age groups and different groups of causes of death to the 
total increase in e0 between 1997 and 2014 are presented 
in Table 1.
across age groups is ensured by using a different weighting 
formula for Pollard’s approach suggested by Pressat (28):
Table 1. Contributions of age groups and different groups of causes of death to the 1997–2014 increase in life expectancy at birth e0 
(in years) by gender in Slovenia.
Explanation of abbreviations: Circ. – diseases of the circulatory system, Neopl. – neoplasms, Ext. – external causes of morbidity and 
mortality, Resp. – diseases of respiratory system, Dig. – diseases of digestive system, Other – all other causes of death not included in 
previous fi ve groups.
0
1–4
5–9
10–14
15–19
20–24
25–29
30–34
35–39
40–44
45–49
50–54
55–59
60–64
65–69
70–74
75–79
80–84
85+
All ages
0.29
0.05
0.09
0.08
0.09
0.22
0.23
0.17
0.24
0.37
0.46
0.54
0.55
0.74
0.79
0.66
0.67
0.43
0.38
7.03
0.20
0.06
0.00
0.01
0.07
-0.02
0.04
0.04
0.11
0.17
0.20
0.23
0.27
0.30
0.49
0.53
0.71
0.78
0.80
5.01
0.01
0.00
0.06
0.05
0.10
0.18
0.18
0.11
0.10
0.15
0.11
0.11
0.08
0.08
0.05
0.02
0.04
0.01
0.05
1.47
0.02
0.01
0.01
0.00
0.04
-0.01
0.02
0.01
0.05
0.03
0.04
0.03
0.03
0.03
0.03
0.04
0.01
0.05
0.00
0.44
0.00
0.02
0.00
0.00
0.00
0.01
0.00
0.04
0.02
0.09
0.12
0.19
0.21
0.25
0.34
0.36
0.37
0.27
0.18
2.45
0.01
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.01
0.03
0.01
0.05
0.07
0.13
0.20
0.29
0.43
0.45
0.34
2.02
0.02
0.00
0.00
0.00
0.00
0.00
0.00
0.01
0.00
0.01
0.01
0.03
0.04
0.04
0.12
0.08
0.13
0.14
0.12
0.76
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.01
0.02
0.02
0.03
0.07
0.09
0.10
0.21
0.56
0.00
0.01
0.00
0.02
-0.01
0.02
0.00
0.00
0.05
0.04
0.12
0.13
0.12
0.24
0.19
0.12
0.09
-0.04
-0.02
1.08
0.01
0.03
-0.01
0.02
0.00
-0.01
0.01
0.02
0.03
0.06
0.08
0.05
0.06
0.04
0.11
0.03
0.05
0.02
0.05
0.65
0.00
0.01
0.01
0.00
0.00
0.00
0.01
0.02
0.04
0.05
0.06
0.05
0.06
0.08
0.04
0.04
0.00
0.01
-0.01
0.45
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.01
0.03
0.03
0.04
0.07
0.05
0.04
0.02
0.04
0.03
0.00
0.35
0.27
0.01
0.03
0.00
0.00
0.01
0.03
0.00
0.03
0.03
0.04
0.05
0.03
0.05
0.06
0.04
0.04
0.04
0.06
0.81
0.17
0.02
0.01
0.00
0.03
0.00
0.00
0.01
0.02
0.02
0.04
0.06
0.02
0.02
0.08
0.08
0.08
0.13
0.20
0.98
All 
causes
All 
causes
Resp. Resp.Neopl. Neopl.Other OtherCirc. Circ.Dig. Dig.Ext. Ext.
1997–2014
change in 
e0 (years)
Men Women
Age 
group
10.1515/sjph-2017-0016 Zdr Varst 2017; 56(2): 124-130
128
Table 3.
Table 2.
Age- and cause-specific decomposition of potential gains in life expectancy (PGLE) at birth e0 (in years) if Slovenian 
mortality levels are changed to Spanish ones in 2014, by gender.
Potential gains in life expectancy (PGLE) at birth e0 (in years) if one group of causes of death is completely eliminated for 
persons born in 2014, Slovenia, by gender.
Explanation of abbreviations: see Table 1.
Explanation of abbreviations: see Table 1.
0
1–4
5–9
10–14
15–19
20–24
25–29
30–34
35–39
40–44
45–49
50–54
55–59
60–64
65–69
70–74
75–79
80–84
85+
All ages
All Ages
-0.11
-0.02
0.00
0.01
0.04
0.01
0.03
0.07
0.10
0.03
0.02
0.08
0.17
0.29
0.23
0.32
0.17
0.30
0.28
2.05
-0.06
-0.03
0.01
0.00
-0.01
0.05
-0.01
0.02
0.02
-0.01
0.02
0.02
0.09
0.20
0.18
0.29
0.26
0.36
0.61
2.01
0.00
-0.01
0.00
0.00
0.04
0.04
0.05
0.06
0.06
0.03
0.05
0.06
0.05
0.04
0.04
0.06
0.03
0.05
0.05
0.70
1.42
0.00
0.00
0.00
0.01
0.00
0.03
0.00
0.01
0.02
-0.01
0.01
0.01
0.00
0.01
0.00
0.02
0.03
0.02
0.14
0.33
0.67
0.00
0.00
0.00
0.00
0.00
-0.01
0.00
-0.01
0.01
-0.02
-0.01
0.00
0.03
0.11
0.15
0.15
0.17
0.26
0.56
1.39
5.74
0.00
0.00
0.00
0.00
0.00
-0.01
0.00
0.00
-0.01
-0.01
0.02
0.00
0.01
0.02
0.08
0.14
0.20
0.40
1.46
2.31
10.98
0.00
0.00
0.00
0.00
0.00
0.00
0.00
-0.01
-0.01
-0.01
-0.01
-0.02
-0.02
-0.03
-0.03
-0.02
-0.06
-0.05
-0.09
-0.37
0.69
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
-0.01
-0.01
-0.01
-0.01
0.00
-0.01
-0.02
-0.02
-0.12
-0.23
0.59
0.00
-0.01
0.00
0.00
-0.01
-0.02
-0.01
0.00
0.01
0.01
-0.02
0.01
0.03
0.08
0.02
0.13
0.08
0.16
0.12
0.58
4.75
0.00
-0.01
0.01
-0.01
0.00
0.01
0.00
0.00
0.01
0.02
0.00
0.02
0.07
0.12
0.11
0.16
0.13
0.15
0.13
0.91
3.73
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.01
0.01
0.02
0.01
0.03
0.05
0.04
0.03
0.02
0.03
0.01
-0.02
0.21
0.71
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.01
0.00
0.01
0.03
0.01
0.01
0.01
0.01
-0.02
0.07
0.49
-0.10
0.00
0.00
0.02
0.01
0.00
-0.01
0.02
0.02
0.00
0.00
0.01
0.04
0.06
0.02
-0.02
-0.07
-0.13
-0.33
-0.46
1.46
-0.05
-0.01
0.00
0.00
-0.01
0.02
0.00
0.01
0.00
-0.01
-0.02
-0.01
0.01
0.03
-0.03
-0.02
-0.09
-0.20
-0.99
-1.38
1.30
All 
causes
All 
causes
Resp.
Resp.
Resp.
Resp.
Neopl.
Neopl.
Neopl.
Neopl.
Other
Other
Other
Other
Circ.
Circ.
Circ.
Circ.
Dig.
Dig.
Dig.
Dig.
Ext.
Ext.
Ext.
Ext.
PGLE 
(years)
2014
PGLE 
(years)
Men
Men
Women
Women
Age 
group
Group of
causes
Past improvements in e0 indicate the mortality patterns 
have been changing. The likely underlying reasons are 
living conditions and national income, patient awareness 
and empowerment in treatment options etc. (30, 31). By 
modifying the mortality pattern, we can assess potential 
gains in e0 from improving such determinants of life 
expectancy. One option is to estimate potential gains in e0 
by completely eliminating one individual cause of death 
at a time (Table 2).
As shown by Table 2, e0 could still be improved. However, 
given that complete elimination of individual causes of 
death is unrealistic, we focus more (presenting results 
also by age groups) on partial potential gains in e0. We 
assume that Slovenian mortality rates would change to 
Spanish levels in 2014 for each age group and each group 
of causes of death, resulting in e0 of 80.2 and 86.1 years 
for men and women, respectively (Table 3).
10.1515/sjph-2017-0016 Zdr Varst 2017; 56(2): 124-130
129
If current mortality rates for Slovenia were to change to 
the levels of Spain in 2014, the largest contribution to 
longer e0 would stem from lower mortality from circulatory 
diseases for both genders, followed by neoplasms for 
women and external causes and neoplasms for men. For 
both genders, the biggest improvement in e0 would arise 
from the highest age groups due to circulatory diseases, 
neoplasms and digestive system diseases. For deaths due 
to external causes, similar potential gains in e0 could be 
achieved in different age groups for men. 
Table 3 reveals negative values of potential gains in e0 for 
other causes of death and respiratory diseases, meaning 
that in 2014, Slovenian mortality – due to respiratory 
diseases and other causes of death (the latter is also 
clearly seen in the first year of life) – was already lower 
than Spanish. The large negative values in “Other causes 
of death” for higher age groups, especially for women, are 
due to lower mortality from causes of death characteristic 
of old age (such as diabetes, senility, osteoporosis, etc.). 
According to available data (19, 24), mortality from these 
causes is much higher in Spain than in Slovenia. We also 
computed the results for Sweden in 2014, whereby the 
“Other” group again has a highly negative value of -1.36 
years for women and -0.46 years for men. We suspect 
that decreasing mortality from circulatory diseases and 
neoplasms results in higher mortality from other causes of 
death. However, comparing the e0 for Spain in 2014 with 
the e0 for Spain in 2006 (when e0 was almost the same as 
in Slovenia in 2014), the value is positive (0.35 years for 
men and 0.26 years for women) for the “Other” group 
and also for respiratory diseases. The peculiarly negative 
results for these two groups of causes of death in Slovenia 
need further investigation – whether mortality in Slovenia 
is actually lower, or whether there are problems with 
underreporting (32).
By combining Tables 2 and 3, we can assess what shares 
of Slovenian 2014 potential gains in e0 would be realised 
if Spanish mortality levels had been reached. Regarding 
three of the major groups of causes of death – circulatory 
diseases, neoplasms and external causes – much can be 
done. For external causes, about one-half of the potential 
gain in e0 (Table 2) could actually be realised by achieving 
Spanish mortality rates from 2014 (50% (=0.71/1.42) for 
men and 48% (=0.32/0.67) for women). 
 
4 DISCUSSION AND CONCLUSIONS
In Slovenia, life expectancy at birth e0 has been zooming 
in the last decades, particularly between 2004 and 2014. 
Our analysis confirms that, in Slovenia, most of the 
improvement in e0 resulted from lower mortality in older 
age groups due to better prevention and treatment of 
circulatory diseases and neoplasms, which is in line with 
(1, 2, 10, 11). This paper also estimates possible future 
gains in e0 that could be achieved by reducing mortality 
from major groups of causes of death. Social change and 
health education generally are expected to continue 
improving longevity, and even have a stronger role than 
medical treatments themselves (33), also because people 
will reach old age in better health (34). 
Estimated potential gains in e0 for Slovenia using Spain 
(i.e., the country with the highest e0 in the EU-28) as a 
benchmark reveal that mortality related to respiratory 
diseases and newborn care is already lower in Slovenia. 
This calls for further investigation. However, there is 
room for improvement in the area of circulatory diseases, 
especially at older ages. Spain also has considerably lower 
mortality due to neoplasms (particularly in older ages), as 
well as due to external causes for men in almost all age 
groups above 20 years.
The results presented in this paper identify which causes 
of death- and age-related improvements in mortality 
produce the highest potential gains in e0, thereby 
indicating certain priority areas for Slovenia. Identifying 
the best policy actions, however, is beyond this paper’s 
scope, even though it clearly shows challenging and 
unavoidable policy responses are required to prevent 
negative increased longevity impacts on economic 
sustainability. Future gains in e0, that will continue to 
be realised at later ages can be expected to lead to a 
lower labour force participation time span as a proportion 
of life expectancy at birth, unless there is a significant 
rise in labour force participation rates across middle and 
older ages. As Eggleston and Fuchs (1) note, lengthier 
retirement lives are inconsistent with continued rises in 
per capita income, unless there are notable increases 
in savings, investment and productivity. Improving 
productivity and increasing ability to work later in life 
will play a central role. Investments in public health 
and medical technologies that improve quality of life, 
assessing the value of innovation by considering not only 
costs of care, but also productivity gains and prevention, 
timely detection and effective management of chronic 
diseases, will help alleviate these diseases’ economic 
burden in the context of growing longevity.
CONFLICTS OF INTEREST
The authors declare that no conflicts of interest exist.
FUNDING
None.
10.1515/sjph-2017-0016 Zdr Varst 2017; 56(2): 124-130
130
ETHICAL APPROVAL
All data used in this study were provided by NIPH, INE and 
Eurostat. All personal data were anonymised.
REFERENCES
1. Eggleston KN, Fuchs VR. The new demographic transition: most gains 
in life expectancy now realised late in life. J Econ Perspect 2012; 
26: 137-56.
2. Canudas-Romo V, Schoen R. Age-specific contribution to changes in 
the period and cohort life expectancy. Demogr Res 2005; 13: 63-82.
3. Lee IM, Shiroma EJ, Lobelo F, Puska P, Blair SN, Katzmarzyk PT. 
Effect of physical inactivity on major non-communicable diseases 
worldwide: an analysis of burden of disease and life expectancy. 
Lancet 2012; 380: 219-29.
4. Renteria EJP, Forman D, Soerjomataram I. The impact of cigarette 
smoking on life expectancy between 1980 and 2010: a global 
perspective. Tob Control 2016; 25: 551-7.
5. Machado de Rezende LF, de Sa TH, Mielke GI, Yukari Kodaira Viscondi 
J, Rey-López JP, Totaro Garcia LM. All-cause mortality attributable 
to sitting time analysis of 54 countries worldwide. Am J Prev Med 
2016; 51: 253-63.
6. Jenko Pražnikar Z, Pražnikar J. The effects of particulate matter air 
pollution on respiratory health and on the cardiovascular system. Zdr 
Varst 2012; 51: 190-9. 
7. Tsevat JI, Weinstein MC, Williams LW, Tosteson AN, Goldman L. 
Expected gains in life expectancy from various coronary heart 
disease risk factor modifications. Circulation 1991; 83: 1194-201.
8. Tsai SP, Lee ES, Hardy RJ. The effect of a reduction in leading causes 
of death: potential gains in life expectancy. Am J Public Health 1978; 
68: 966-71.
9. Conti S, Farchi G, Masocco M, Toccaceli V, Vichi M. The impact of 
the major causes of death on life expectancy in Italy. Int J Epidemiol 
1999; 28: 905-10.
10. Klenk J, Keil U, Jaensch A, Christiansen MC, Nagel G. Changes in life 
expectancy 1950-2010: contributions from age- and disease-specific 
mortality in selected countries. Popul Health Metr 2016; 14: 20.
11. Naghavi M, Wang HD, Lozano R, Davis A, Liang XF, Zhou MG et al. 
Global, regional, and national age-sex specific all-cause and cause-
specific mortality for 240 causes of death, 1990-2013: a systematic 
analysis for the Global Burden of Disease study 2013. Lancet 2015; 
385: 117-71.
12. Bongaarts J. Trends in causes of death in low-mortality countries: 
implications for mortality projections. Popul Dev Rev 2014; 40: 189-
212.
13. Eurostat. Life expectancy by age and sex. Available 5 April 2016, 
from: http://ec.europa.eu/Eurostat/data/database (Database 
by themes – Population and social conditions – Demography and 
migration – Mortality) 
14. EUROPOP population projections 2013. Available 8 June 2016, from: 
http://ec.europa.eu/Eurostat/data/database?node_code=proj 
15. World Health Organization. WHO health statistics 2015. Geneva: 
WHO, 2015.
16. Poston DL, Bouvier LF. Population and society – an introduction to 
demography. Cambridge: Cambridge University Press, 2013.
17. Malačič J. Demografija – teorija, analiza, metode in modeli. 
Ljubljana: Ekonomska fakulteta, 2006.
18. Kintner HJ. The life table. In: Swanson DA, Siegel JS. The methods 
and materials of demography. San Diego: Elsevier Science, 2003: 
301-40.
19. National Institute of Public Health Slovenia. Internal data on deaths 
by age and causes of death 1997-2014. On request.
20. Eurostat. Population on 1 January by age and sex. Available 4 
November 2015, from: http://ec.europa.eu/Eurostat/data/
database (Database by themes – Population and social conditions – 
Demography and migration – Population)
21. Beltrán-Sánches H, Preston SH, Canudas-Romo V. An integrated 
approach to cause-of-death analysis: cause-deleted life tables and 
decompositions of life expectancy. Demogr Res 2008; 19: 1323. 
22. Ediev DM. Life expectancy in developed countries is higher than 
conventionally estimated: implications from improved measurement 
of human longevity. J Popul Ageing 2011; 4: 5-32.
23. Eurostat. Smoking of tobacco products by sex, age and educational 
attainment level. Available 19 November 2016, from: http://
ec.europa.eu/eurostat/data/database (Database by themes – Health 
– Health determinants – Tobacco consumption)
24. INE – Instituto Nacional de Estadística. Death according to cause 
of death 2014. Available 12 April 2016, from: http://www.ine.
es/dynt3/inebase/index.htm?type=pcaxis&path=/t15/p417/
a2014&file=pcaxis&L=1
25. Arriaga EE. Measuring and explaining the change in life expectancies. 
Demography 1984; 21: 83-96.
26. Pollard JH. On the decomposition of changes in expectation of life 
and differentials in life expectancy. Demography 1988; 25: 265-76.
27. Ponnapalli KM. A comparison of different methods for decomposition 
of changes in expectation of life at birth and differential in life 
expectancy at birth. Demographic Res 2005; 12: 141-72.
28. Pressat R. Contribution des écarts de mortalité par âge à la différence 
des vies moyennes. Population 1985; 4: 765-70.
29. Arriaga EE. Changing trends in mortality decline during the last 
decades. In: Ruzicka L, Wunsch G, Kane P. Differential mortality. 
Oxford: Clarendon Press, 1995: 105-30.
30. Bilas V, Franc S, Bošnjak M. Determinant factors of life expectancy 
at birth in the European Union countries. Collegium Antropol 2014; 
38: 1-9.
31. Lichtenberg FR. The impact of pharmaceutical innovation on 
premature mortality, cancer mortality, and hospitalization in 
Slovenia, 1997–2010. Appl Health Econ Health Policy 2015; 13: 207-
22.
32. Fleming DM, Schellevis FG, Van Casteren V. The prevalence of known 
diabetes in eight European countries. Eur J Public Health 2004; 14: 
10-4.
33. Ridsdale B, Gallop A. Mortality by cause of death and by socio-
economic and demographic stratification 2010. ICA 2010 conference. 
Available 13 January 2016, from: http://www.actuaries.org/EVENTS/
Congresses/Cape_Town/Presentations/Life%20Insurance%20
(IAALS)/183_PPT_Ridsdale.pdf
34. Vaupel JW. Biodemography of human aging. Nature 2010; 464: 536-
42.
ASSESSMENT OF THE ASSOCIATION BETWEEN DENTATE STATUS AND 
SELF-RATED GENERAL HEALTH
Martin RANFL1*, Lijana ZALETEL-KRAGELJ2
1National Institute of Public Health, Regional unit Murska Sobota, 
Ulica arhitekta Novaka 2b, 9000 Murska Sobota, Slovenia
2University of Ljubljana, Faculty of Medicine, Department of Public Health, 
Zaloška 4, 1000 Ljubljana, Slovenia
Received: Mar 4, 2016
Accepted: Jan 5, 2017
Original scientific article
Objective. Aiming at preparing the basis for evidence-based dental public health policy making in Slovenia, the 
objective of the study was to assess the strength of association between oral health status measured by the 
number of missing teeth and self-rated health (SRH).
Methods. The study was designed as a pooled individual-level data study from four national cross-sectional 
studies carried out in the period 2001-2012, based on CINDI Health Monitor methodology. Altogether, 34,412 
participants were included. A logistic regression model with poor SRH as observed outcome and the number of 
teeth as explanatory factor (adjusted for selected biologic, socio-economic and health factors) was proposed.
Results. In the sample, women represented 55.7% and men 44.3%, median age was 45 years. Persons with 
more missing teeth more likely rated their health as poor. The association was persistent even when different 
confounding variables were included in the model. In the group with 1-5 missing teeth, in comparison to the 
group with none missing teeth, OR was 1.23 (p=0.049), whereas for the group with 6-10 missing teeth, OR was 
1.32 (p=0.019); for the group with >10 missing teeth, but not all, OR was 1.77 (p<0.001), and for the group with 
all missing teeth, OR was 2.19 (p<0.001). 
Conclusion. Study results showed clear association of SRH with dentate status, which confirms the oral-general 
health connection. This indicates the need for the development of proper dental public health policies for 
better oral health, and presents a new view on the importance of preserving teeth.
Namen. Z namenom priprave podlage za pripravo na dokazih temelječe politike ustnega javnega zdravja v 
Sloveniji je bil cilj raziskave ocena povezanosti stanja ustnega zdravja, merjenega s samoporočanim številom 
manjkajočih zob in samoocene lastnega zdravja.
Metode. Zasnovo raziskave je predstavljala analiza združenih podatkov na individualni ravni, pridobljenih v 
štirih zaporednih nacionalnih presečnih raziskavah v Sloveniji, izvedenih v obdobju 2001-2012 po metodologiji 
CINDI Health Monitor. Skupno je bilo v analizo vključenih 34.412 udeležencev. Ženske so predstavljale 55,7% 
in moški 44,3%, mediana starosti je bila 45 let. V modelu logistične regresije je bila opazovana spremenljivka 
samoocena lastnega zdravja kot slabega, pojasnjevalna spremenljivka pa samoporočano število manjkajočih 
zob (upoštevani izbrani biološki, socialno-ekonomski in z zdravjem povezani moteči dejavniki). 
Rezultati. Osebe z več manjkajočimi zobmi imajo višje obete za to, da ocenjujejo svoje zdravje kot slabo, 
kar velja tudi po prilagoditvi za nekatere biološke, socialne in zdravstvene dejavnike. Razmerje obetov 
za nizko samooceno lastnega zdravja v primerjavi s skupino brez manjkajočih zob je bilo pri osebah z 1-5 
manjkajočimi zobmi 1,23 (p=0,049), za skupino s 6-10 manjkajočimi zobmi 1,32 (p=0,019), za skupino z več kot 
10 manjkajočimi zobmi 1,77 (p<0,001) in za skupino z vsemi manjkajočimi zobmi 2,19 (p<0,001).
Zaključek. Raziskava je pokazala jasno povezanost med samoocenjenim splošnim zdravjem in zobnim 
statusom, kar potrjuje povezanost ustnega in splošnega zdravja. Rezultati tudi nakazujejo potrebo po razvoju 
primernih politik ustnega javnega zdravja za boljše ustno zdravje in hkrati nakazujejo tudi nov pogled na 
pomen ohranitve zob.
ABSTRACT
Keywords: 
self-rated health,  
oral health, missing 
teeth, preserved teeth, 
public health policy
IZVLEČEK
Ključne besede: 
samoocena zdravja, 
ustno zdravje, 
manjkajoči zobje, 
ohranjeni zobje, 
javnozdravstvene 
politike
*Corresponding author: Tel: ++ 386 2 5302 139; E-mail: martin.ranfl@nijz.si
10.1515/sjph-2017-0017 Zdr Varst 2017; 56(2): 131-139
131
OCENA POVEZANOSTI MED ZOBNIM STATUSOM IN SAMOOCENO 
SPLOŠNEGA ZDRAVJA
Ranfl M, Zaletel-Kragelj L. Assessment of the association between dentate status and self-rated general health. Zdr Varst 2017; 56(2): 131-139.
© Nacionalni inštitut za javno zdravje, Slovenija. 
1 INTRODUCTION
Mouth with teeth and oral tissues play an important role 
in human lives. Compartmentalisation of oral health has 
been replaced by the belief that it is an integral part of 
general health and has an influence quality of life. To 
maintain and improve oral health, it is essential to carry 
out hygiene measures and perform dental check-ups 
regularly. Retention of functional, aesthetic and natural 
dentition of not less than 20 teeth throughout life is World 
Health Organization’s (WHO’s) goal, actually a milestone 
on the road to retention of all natural teeth (1). It is based 
on the evidences that masticatory ability is related to the 
number of teeth, and that this ability is impaired when 
the patient has less than 20 well-distributed teeth (2); in 
addition, it is in agreement with short dental arch (SDA) 
concept (3). Regular dental check-ups are also important, 
because multiple systemic diseases and imbalances 
have different signs in oral cavity, and sometimes these 
manifestations are disease-specific (4). Oral-systemic 
connection is bidirectional, as studies show association of 
periodontal disease with cardiovascular diseases, diabetes 
mellitus, and pregnancy outcomes (5). 
Trying to understand the potential oral-systemic 
association, it is important to consider a wider look 
on health, which – besides biomedical status – takes 
into account also the patient’s ability to perform daily 
activities (6). This broader multidimensional view is 
used in the concept of health surveys which encompass 
traditional clinical assessment and also the individual’s 
subjective assessment of health status impact on his 
or her own wellbeing and daily functioning (7). Health-
related quality of life (HRQoL) with the inclusion of the 
patient’s perspective represents measurement tools 
with a more holistic approach to health. It is affected 
by the individual’s physical health, psychological state, 
personal beliefs, social relationships, and the relationship 
to salient features of the individual’s environment (8). On 
the same theoretic base, measurements of Oral Health-
Related Quality of Life (OHRQoL) was developed (9).
One similar and simpler measurement, which is a part 
of self-rated quality of life tools, is self-rated health 
(SRH). It represents personal and subjective perception 
of one’s own health, and it could be biased according to 
social desirability, expectations and relative deprivation 
(10). Researchers have explored different influences 
and correlations between self-perceived health status 
and characteristics of one’s social environment (11). 
The importance of SRH can be explained by simplicity 
and the fact that researchers confirmed SRH has an 
independent effect on mortality (12-14), morbidity (15) 
and hospitalizations (16). Self-assessment of health 
is a widely used method in epidemiology, and can be 
assessed through different questionnaires. Most widely 
used are single-item indicators that differ in the number 
of available answers, but, in general, they represent a 
parallel assessment of subjective health (10). A study in 
Slovenia using SRH showed that PSRH is associated with 
multimorbidity and unhealthy life-style (17). Subjective 
evaluation of health is used also in econometric analyses 
and health technology assessment (18).
Multiple biological mechanisms are connecting oral 
diseases to systemic health (5). Tooth loss represents the 
main consequence of persistent or past oral diseases, 
injuries, or compromised possibilities of dental healthcare 
utilization. It can affect someone’s appearance, psychical 
state and well-being (19). But does it influence self-rated 
general health, which is connected to other confirmed 
negative health outcomes? This was the main question we 
wanted to answer with our study. 
Aiming at preparing the basis for evidence-based policy 
making in the field of dental public health in Slovenia, 
the objective of the study was to assess the strength 
of association between oral health status measured 
by the number of missing teeth and self-rated health 
(SRH), controlled for some characteristics of one’s social 
environment and major health problems.
2 METHODS
The study was designed as a pooled individual-level 
data study from four cross-sectional studies, based on 
the methodology of the WHO Countrywide Integrated 
Non-Communicable Disease Intervention (CINDI) Health 
Monitor database (CHM) in Slovenia (CHMS). The surveys 
were conducted in 2001, 2004, 2008 and 2012, and the 
number of participants invited was 15379, 15297, 15963, 
and 16000, respectively, in the frame of CINDI Slovenia. 
Based on the CHM Core Questionnaire (20, 21), Slovene self-
administered postal questionnaire was created. In 2012, 
also the possibility for online responses existed. Different 
approaches, including extensive media campaigns, a 
lottery with prizes enhancing healthy behaviour, and up 
to two reminder letters, were used.
SRH by participants was the observed outcome. It was 
measured through a single question: “How would you 
assess your present state of health?” Five-level Likert-like 
scale with answers “very good,”“good,”“fair,”“poor” and 
“very poor” was used. For the purpose of the analysis, the 
new variable PSRH was created, in which participants who 
rated their health as poor or very poor were pooled in a 
group of interest (PSRH: 0=no, 1=yes).
Dentate status as explanatory factor of interest was 
self-assessed by the question: “How many teeth are you 
missing?” Predefined answers were: 0 - none, 1 – 1 to 5, 
2 - 6 to 10, 3 - more than 10 but not all, 4 – all teeth are 
missing/I have dentures. 
10.1515/sjph-2017-0017 Zdr Varst 2017; 56(2): 131-139
132
10.1515/sjph-2017-0017 Zdr Varst 2017; 56(2): 131-139
133
Confounders gender, age, educational level, type of work 
and self-classified social class were assessed. Additionally, 
information on self-reported persistence of diagnosed 
diseases, self-confirmed pain in the last 30 days, and 
admission to the hospital in the past twelve months was 
included. Ages of the participants were recoded from the 
reported year of birth and arranged into five categories, 
starting with 25-29, then three 10-year categories, 30-
39, 40-49 and 50-59, and the last category from 60 to 
64 years (the participants aged 65-74 were excluded 
from the analysis). In the assessment of the education 
level, participants were able to choose one out of seven 
categories (1 - incomplete primary, 2 - primary, 3 - 
vocational, 4 - secondary, 5 - college, 6 - university and 
7 - postgraduate), which represented their highest level 
of education achieved. For the needs of the analysis, the 
last two categories were combined into one. The question 
about the type of work consisted of 10 different categories 
of self-classification. For most judicious use of this data, 
we combined them into 4 categories, according to work 
characteristics. These categories were, namely: 1 - heavy 
work (agriculture, farming, forestry, industry, mining 
and construction), 2 - administrative and intellectual 
work (work in office, light physical work, services, higher 
management, research, development, and students), 
3 - housekeeping (housekeeper and pensioners) and 4 
- unemployed (at the time of the study). Self-reported 
social class was assessed by the question: “In your option, 
which social class do you belong to?” Participants could 
choose one of the answers: 1 – lower, 2 – labour, 3 – 
middle, 4 – upper-middle, and 5 - upper); for the needs 
of the analysis, upper-middle class and upper class were 
combined. Self-reported diseases of participants were 
assessed by the question: “Do you have any of conditions, 
confirmed by a physician?” Participants chose between 
answers: 1 - No, 2 - Yes, it was confirmed in the last year, 
and 3 – Yes, it was confirmed more than a year ago. During 
statistical analysis the last two answers were combined 
into a single category. From the list of questions, six 
diseases, which have impact on daily activities, were 
included into the analysis, namely: myocardial infarction, 
angina pectoris, heart failure, cerebrovascular insult, 
back illness, and rheumatism or arthritis. The participants 
were categorized into categories according to the number 
of confirmed conditions (1 - without any of the specified 
conditions, 2 - one of the specified conditions, 3 - more 
than one of the specified conditions). We also included 
the information on some pain symptoms in the last month 
(chest pain during physical activity, back pain, shoulder/
neck pain, joint pain, headache and toothache). It was 
assessed by the question: “Have you had any of the 
following symptoms or complaints during the last 30 
days?” Participants could choose between two answers: 
1 - Yes and 2 - No. The answers were again combined 
into three groups depending on the number of reported 
problems (1 - with none of the problems, 2 - with one 
of the problems, 3 - with more of the problems). The 
information on hospital admissions was included as 
well. Data were assessed from the question: “How many 
times, during the last twelve months, were you admitted 
into the hospital?”. Participants answered the question 
with the number of admissions. For the purpose of the 
analysis, answers were aligned into groups according to 
the number of admissions: none, one time and multiple 
times. Finally, the year of the survey was included in the 
analysis as a confounder.
The association between PSRH and the number of missing 
teeth as explanatory variable, adjusted for confounders, 
was assessed univariately, using chi-square tests. The 
association was assessed multivariately, using binary 
multiple logistic regression (LR). The dummy variables 
were created for explanatory and confounding variables, 
using the simple method. In all statistical tests, p≤0.05 
was considered significant. The IBM SPSS for Windows 
Version 21.0 (SPSS Inc., Chicago, IL., USA) software was 
used.
3 RESULTS
In the pooled sample, there were 34412 participants, 
aged 25-64 (2001: 9034, 2004: 8528, 2008: 7352, 2012: 
9498), whose questionnaires were eligible for analysis. 
Response rate was 62.9% in 2001, 57.4% in 2004, 49.0% in 
2008, and 59.6% in 2012. There was a slight predominance 
of females, but participants were equally distributed 
across age groups – median age was 45 years. Further 
characteristics are presented in Table 1. 
10.1515/sjph-2017-0017 Zdr Varst 2017; 56(2): 131-139
134
SRH was reported by 34085/34412 participants (99.0%), 
among them very good SRH was reported by 10.7%, 
good SRH was reported by 41.9%, 38.3% reported fair 
SRH, 7.5% poor SRH, and 1.4% reported very poor SRH. 
The prevalence of very good and good SRH was higher in 
people with less missing teeth, whereas the prevalence 
of fair, poor and very poor SRH was higher in persons with 
more missing teeth. The association between variables 
was highly significant (p<0,001).
The prevalence of PSRH was 9.0% (3076/34085). 
The question about dentate status was answered by 
34041/34412 participants (98.9%). About two thirds of 
them had 5 or less missing teeth. After cross-matching, 
both questions were adequately answered by 33908/34412 
(98.5%). The estimates of the prevalence of PSRH in each 
category of dentate status are presented in Table 2. The 
prevalence of PSRH is rising with increasing number of 
missing teeth. The differences were highly statistically 
significant (p<0.001). In Table 2 also estimates of 
prevalence of PSRH according to different socio-economic 
and health characteristics are presented, along with the 
results of univariate statistical analysis.
Complete data for LR analysis were available for 
24862/34412 participants (72.2%). The results of the 
logistic regression model showed a statistically significant 
association between PSRH and dentate status, when 
this relationship was adjusted to several confounders. 
Significance of Hosmer-Lemeshow test of the model was 
p=0.249; the model explained 30.9% of variance. The OR 
were rising with a higher number of missing teeth and 
were significant for all groups of people with missing 
teeth, in comparison with persons with no missing teeth. 
Detailed results are presented in Table 3, and were robust 
to regrouping participants according to age categories 
(10-year categories) and the type of work (separating 
pensioners and housekeepers). 
Table 1. Characteristics of participants taking part in the 
study of pooled individual-level data from four cross-
sectional studies in Slovenia, from 2001-2012.
Gender
Age (years)
Educational 
level
Type of work
Social class
44.3%
55.7%
11.4%
24.0%
26.3%
26.5%
11.8%
4.7%
15.0%
26.4%
29.6%
8.6%
15.7%
14.4%
50.0%
28.8%
6.8%
3.1%
36.9%
48.7%
10.3%
0.9%
15258
19154
3585
7527
8251
8333
3695
1604
5088
8960
10065
2917
5318
4776
16546
9510
2234
1002
11899
15678
3333
297
Men
Women
25-29
30-39
40-49
50-59
60-64
Incomplete primary
Primary
Vocational
Secondary
College
University
Heavy work
Administrative/intellectual 
work
Housekeeping
Unemployed (job seeker)
Lower
Labour
Middle
Upper-middle
Upper
Characteristic Category %N
10.1515/sjph-2017-0017 Zdr Varst 2017; 56(2): 131-139
135
Table 2. Estimates of the prevalence of poor self-rated health (PSRH) according to dentate status and selected socio-economic and 
health-related factors in a study of pooled individual-level data from four cross-sectional studies in Slovenia, from 2001-
2012.
Legend: Ntot=the total number of respondents, NPSRH=the number of participants with poor self-rated health, Ncat=the number of 
respondents within the category.
Missing teeth
Gender
Age (years)
Educational level
Type of work
Social Class
Admission to 
hospital
History of 
health problems 
in the last 30 days
Confirmed 
health problem 
by physician
Year
33908
34085
31090
33649
32771
31936
31818
33787
33865
34085
None
1 to 5
6 to 10
More than 10, but not all
All teeth - wear denture
Men
Women
25-29
30-39
40-49
50-59
60-64
Incomplete primary
Primary
Vocational
Secondary
College
University
Heavy work
Administrative/intellectual work
Housekeeping
Unemployed (job seeker)
Lower
Labour
Middle
Upper-middle/Upper
No
Once
Multiple times
None
One
More than one
None
One
More than one
2001
2004
2008
2012
206
1178
550
763
353
1373
1703
92
305
759
1082
433
419
853
864
614
130
124
562
681
1263
427
332
1513
804
113
1911
595
336
226
958
1857
718
1778
472
850
796
701
729
6126
16638
4763
4518
1863
15104
18981
3565
7472
8154
8237
3662
1587
5039
8860
9991
2895
5277
4734
16408
9425
2204
990
11800
15544
3602
27493
3291
1034
7247
17818
8722
21062
10804
999
9009
8321
7302
9453
3.4%
7.1%
11.5%
16.9%
18.9%
9.1%
9.0%
2.6%
4.1%
9.3%
13.1%
11.8%
26.4%
16.9%
9.8%
6.1%
4.5%
2.3%
11.9%
4.2%
13.4%
19.4%
33.5%
12.8%
5.2%
3.1%
7.0%
18.1%
32.5%
3.1%
5.4%
21.3%
3.4%
16.5%
47.2%
9.4%
9.6%
9.6%
7.7%
<0.001
0.705
<0.001
<0.001
<0.001
<0.001
<0.001
<0.001
<0.001
<0.001
Risk factor Category N
tot
N
cat
pN
PSRH
N
PSRH
/ N
cat 
(%)
10.1515/sjph-2017-0017 Zdr Varst 2017; 56(2): 131-139
136
Social Class
Admission to 
hospital
History of 
health problems 
in the last 30 days
Confirmed 
health problem 
by physician
Year
Middle
Lower
Labour
Upper-middle/Upper
No
Once
Multiple times
None
One
More than one
None
One
More than one
2012
2001
2004
2018
1.00
3.97
1.49
1,06
1.00
2.55
4.12
1.00
1.20
3.83
1.00
2.94
6.93
1.00
1.53
1.65
1.53
3.18
1.31
0.83
2.23
3.38
0.98
3.12
2.60
5.58
1.31
1.41
1.31
4.96
1.70
1.36
2.92
5.02
1.47
4.70
3.32
8.60
1.78
1.94
1.80
<0.001
<0.001
0.640
<0.001
<0.001
0.075
<.001
<0.001
<0.001
<0.001
<0.001
<0.001
Legend: CI=confidence interval, OR=odds ratio.
Table 3. Results of logistic regression analysis of the association of dentate status withthe prevalence of poor self-rated health (PSRH) 
in a study of pooled individual-level data from four cross-sectional studies in Slovenia, from 2001 to 2012.
Missing teeth
Gender
Age (years)
Educational level
Type of work
None
1 to 5
6 to 10
More than 10, but not all
All teeth - wear denture
Men
Women
25-29
30-39
40-49
50-59
60-64
University
Incomplete primary
Primary
Vocational
Secondary
College
Administrative/intellectual work
Heavy work
Housekeeping
Unemployed (job seeker)
1.00
1.23
1.32
1.77
2.19
1.00
1.24
1.00
1.23
1.71
1.72
1.35
1.00
2.55
1.71
1.41
1.47
1.11
1.00
1.37
1.21
2.34
1.00
1.05
1.40
1.64
1.11
0.92
1.29
1.29
0.97
1.86
1.29
1.08
1.14
0.81
1.16
1.02
1.96
1.51
1.67
2.25
2.91
1.38
1.64
2.26
2.29
1.87
3.50
2.27
1.84
1.89
1.52
1.60
1.42
2.80
0.049
0.019
<0.001
<0.001
<0.001
0.163
<0.001
<0.001
0.074
<0.001
<0.001
0.011
0.003
0.519
<0.001
0.027
<0.001
Risk factor Category 95% CI for OR limits
N
cat
pOR
N
PSRH
/ N
cat 
(%)
10.1515/sjph-2017-0017 Zdr Varst 2017; 56(2): 131-139
137
4 DISCUSSION
Results of our study showed that there exist differences in 
the prevalence of PSRH between groups according to the 
number of missing teeth. People with more missing teeth 
will more likely rate their general health as poor. Results 
are highly significant. The biggest likelihood of rating 
one’s own health as poor was for people with all of their 
teeth lost, or for people who wore dentures. We could 
try to explain this by lost functionality. This relationship 
was significant even when some health problems and 
socioeconomic characteristics, known to influence 
SRH (11), were taken into account. This indicates the 
importance of oral health, reflected through the number 
of missing teeth in self-evaluation of health, and also 
addresses the general belief that oral health represents 
an integral part of general health.
A lower number of teeth represent some kind of a 
functional limitation. The literature review about SDA 
concept states that dentition comprised of anterior teeth 
and premolar region fulfils the requirements of functional 
dentition (22). This means that no more than 8-12 teeth 
should be missing. We showed that the association exists 
even with less teeth lost, when functionality should 
not be compromised. Lost functionality is not the only 
possible link. This could be risk factors that are associated 
with oral diseases, which cause tooth loss, and other non-
communicable diseases (smoking, alcohol consumption). 
Systemic effects of periodontal disease, one of the main 
reasons for tooth loss in adults, can also represent biologic 
plausibility (5). The drop in prevalence of PSRH in 2012 
is somewhat strange in the light of economic crisis, but 
these results were also observed in some other studies 
(23). It is possible that the effect of economic crisis has 
not yet been expressed in poorer SRH, or that its rise is 
the consequence of changes in community and personal 
conceptualisation of health. 
We showed the association of health issues with PSRH. 
This also included former hospitalisation. The association 
between hospitalization and PSRH rises with a higher 
number of admissions, which may reflect possibilities 
of more serious diseases. Even PSRH is associated with 
hospitalisation (16), we also believe that it could be 
interpreted as a sign of a more serious disease that could 
be associated with PSRH. In the oldest age group, a drop 
in the prevalence of PSRH was observed. This could be 
attributed to changes in personal concepts of health, but 
other researchers also confirmed improving SRH with age 
when adjusted to some functional disabilities (24).
The question arises whether SRH is a proper measurement 
tool in the context of oral health. Studies show small, but 
significant association between oral and general health-
related quality of life, suggesting that the functioning of 
the mouth or body could be seen as a link between these 
concepts (25). Masticatory performance is significantly 
correlated to the number of missing teeth, and it is not in 
association with the age of the subjects (26). This means 
that in context of our research, where we study the 
influence of the number of missing teeth, which definitely 
represents functional limitation, general health measures 
could be used.
Even though tooth loss has not been put in association with 
SRH until now, studies that connect tooth loss with some 
health outcomes exist. Researchers confirmed associations 
of tooth loss and disease risk, disease development and 
increased mortality (27-30). We have shown that a greater 
number of missing teeth implies bigger odds for PSRH. We 
know that PSRH is a good predictor of mortality among 
patients with chronic heart failure too (14). This point to 
a possible underlying mechanism between SRH and health 
outcome, which involves teeth and should be a matter of 
further research.
The study has some limitations. Firstly, it is related to the 
cross-sectional study design that limits direct conclusions 
on causality. Another limitation is the self-assessment of 
data, where more socially desirable answers may be chosen 
and the final sample may consist mainly from persons 
more prone to participate. Some of these limitations 
were addressed by other authors, and they believe that 
they did not affect the study findings to a great extent 
(11, 31). Another limitation may arise from the fact 
that participants might not be sure about the number of 
missing teeth or might misunderstand the question about 
the presence of confirmed diseases by their physician. 
Cohort study with a clinical assessment of oral health 
status and longitudinal design could overcome some of 
the aforementioned limitations. Additionally, one can 
argue that regarding the sample structure, some overlaps 
in participants across the surveys were possible. However, 
only about 6% of adult population, aged 25-64 years, was 
invited to participate in each survey. Consequently, the 
probability of inclusion of the same person in all studies 
is very small. One can also argue against selected age 
groups; however, the same classification/grouping was 
used by other authors, who analysed CHMS data (11). 
Although some arguments against a single-item question to 
assess SRH can be raised, this simple measure represents 
a comprehensive screening tool for the patient’s health 
status (32).
Nonetheless, the study has some important strengths. 
The most important one is that, although Slovenia is a 
small European country, the results could represent a 
contribution to dental public health in a wider context, 
since, according to our best knowledge, there does not 
yet exists any study that would analyse the relationship 
between PSRH and dentate status. Additionally, results 
were obtained on a relatively large sample.
Because PSRH is associated with a greater number of 
missing teeth, which is indeed associated with increased 
mortality, tooth loss could be another useful predictor 
of health complications. Results showed that oral health 
has an important influence on the SRH, even when known 
confounders are taken into account. This shows that oral 
and systemic health should not be treated separately, and 
reflects the importance of oral health itself. All possible 
measures must be taken to preserve a higher number of 
natural teeth. This also points out to the need for greater 
concern about oral health status of the population and 
more extensive research on the field of public oral health. 
That will make possible to develop and implicate proper 
strategies and programmes, to advocate oral health, and 
to get wider support as well as raise awareness about the 
importance of oral health in the community. Oral health 
education for all population groups would help preserve 
higher number of teeth, and it will possibly have positive 
effects on SRH. 
As far as the future research in the field is concerned, 
it would be worth placing the question about SROH and 
other questions related to oral health in future CHMS 
surveys. That would help to clarify possible associations 
between oral and general health and help to make further 
decisions on our way to better health and wellbeing in 
general. 
5 CONCLUSION
PSRH is associated with a higher number of missing 
teeth. Because PSRH is connected with negative health 
outcomes, this association should not be neglected. 
Preserving natural teeth should be considered a global 
goal for better oral and general health.
CONFLICTS OF INTEREST
The authors declare that no conflicts of interest exist.
FUNDING
The study was performed in the frame of regular tasks of 
the National Institute of Public Health of Slovenia.
ETHICAL APPROVAL
The research protocol was approved for each survey by 
the Ethical Committee of the Republic of Slovenia.
 
REFERENCES
1. World Health Organization. Recent advances in oral health, WHO 
Technical Report Series No.826. Available Apr 8, 2015 from: http://
whqlibdoc.who.int/trs/WHO_TRS_826.pdf.
2. Sarita PT, Witter DJ, Kreulen CM, Van’t Hof MA, Creugers NH. Chewing 
ability of subjects with shortened dental arches. Community Dent 
Oral Epidemiol 2003; 31: 328-34.
3. Kaeyser AF. Shortened dental arches and oral function. J Oral Rehabil 
1981; 8: 457-62.
4. Islam NM, Bhattacharyya I, Cohen DM. Common oral manifestations 
of systemic disease. Otolaryngol Clin North Am 2011; 44: 161-82.
5. Kim J, Amar S. Periodontal disease and systemic conditions: a 
bidirectional relationship. Odontol 2006; 94: 10–21.
6. Garcia RI, Henshaw MM, Krall EA. Relationship between periodontal 
disease and systemic health. Periodontol 2000. 2001; 25: 21–36.
7. Ware JE. Measures for new era of health assessment. In: Stewart 
AL, Ware JE Jr. Measuring functioning and wellbeing: the medical 
outcomes study approach. Durham, NC: Duke University Press, 1992; 
3-11.
8. The WHOQOL Group. Measuring quality of life. Available Apr 8, 2015 
from: http://www.who.int/mental_health/media/68.pdf
9. Sischo L, Broder HL. Oral health-related quality of life: what, why, 
how, and future implications. J Dent Res. 2011; 90: 1264-70.
10. Eriksson I, Undén AL, Elofsson S. Self-rated health: comparisons 
between three different measures: results from a population study. 
Int J Epidemiol 2001; 30: 326-33.
11. Farkaš J, Pahor M, Zaletel-Kragelj L. Self-rated heath in different 
social classes of Slovenian adult population: nationwide cross-
sectional study. Int J Public Health 2011; 56: 45-54.
12. Mossey JM, Shapiro E. Self-rated health: a predictor of mortality 
among the elderly. Am J Public Health 1982; 72: 800-8.
13. Idler, EL, Benyamini, Y. Self-rated health and mortality: a review 
of twenty-seven community studies. J Health and Soc Behav 1997; 
38: 21–37.
14. Farkas J, Nabb S, Zaletel-Kragelj L, Cleland JG, Lainscak M. Self-
rated health and mortality in patients with chronic heart failure. Eur 
J Heart Fail 2009; 11: 518-24.
15. Latham K, Peek CW. Self-rated health and morbidity onset among 
late midlife U.S. adults. J Gerontol B Psychol Sci Soc Sci 2013; 68: 
107-16.
16. Wolinsky FD, Culler SD, Callahan CM, Johnson RJ. Hospital resource 
consumption among older adults: a prospective analysis of episodes, 
length of stay, and charges over a seven-year period. J Gerontol 
1994; 49: S240-52.
17. Petek D, Kersnik J. Evaluation of self-rated health–information on 
patient’s unmet needs? Zdr Varst 2014; 53: 179-87.
18. Rupel V, Ogorevc M. The EQ-5D health states value set for Slovenia. 
Zdr Varst 2012; 51: 124-40.
19. Saintrain MV, de Souza EH. Impact of tooth loss on the quality of life. 
Gerontology 2012; 29: e632-6.
20. World Health Organization. Protocol and guidelines: Countrywide 
Integrated Non-communicable Diseases Intervention (CINDI) 
Program. Available Apr 8, 2015 from: http://whqlibdoc.who.int/
hq/1994/EUR_ICP_CIND_94.02_PB04.pdf 
21. Prattala R, Helasoja V, Laaksonen M, Laatikainen T, Nikander P, 
Puska P. CINDI health monitor: proposal for practical guidelines. 
Available Apr 8, 2015 from: http://thl32kk.lib.helsinki.fi/bitstream/
handle/10024/78046/2001b14.pdf?sequence=1.  
22. Kanno T, Carlsson GE. A review of the shortened dental arch concept 
focusing on the work by the Käyser/Nijmegen group. J Oral Rehabil 
2006; 33: 850-62.
23. Parmar D, Stavropoulou C, Ioannidis JP. Health outcomes during the 
2008 financial crisis in Europe: systematic literature review BMJ 
2016; 354: i4588.
10.1515/sjph-2017-0017 Zdr Varst 2017; 56(2): 131-139
138
10.1515/sjph-2017-0017 Zdr Varst 2017; 56(2): 131-139
139
24. Jylhä M, Guralnik JM, Balfour J, Fried LP. Walking difficulty, walking 
speed, and age as predictors of self-rated health: the women’s 
health and aging study. J Gerontol A Biol Sci Med Sci 2001; 5: M609-
17.
25. Kieffer JM, Hoogstraten J. Linking oral health, general health, and 
quality of life. Eur J Oral Sci 2008; 116: 445-50.
26. Ikebe K, Matsuda K, Kagawa R, Enoki K, Yoshida M, Maeda Y et al. 
Association of masticatory performance with age, gender, number 
of teeth, occlusal force and salivary flow in Japanese older adults: 
is ageing a risk factor for masticatory dysfunction? Arch Oral Biol 
2011; 56: 991-6.
27. Hung HC, Joshipura KJ, Colditz G, Manson JE, Rimm EB, Speizer FE et 
al. The association between tooth loss and coronary heart disease in 
men and women. J Public Health Dent 2004; 64: 209-15.
28. Joshipura KJ, Hung HC, Rimm EB, Willett WC, Ascherio A. Periodontal 
disease, tooth loss, and incidence of ischemic stroke. Stroke 2003; 
34: 47-52.
29. Elter JR, Champagne CME, Offenbacher S, Beck JD. Relationship of 
periodontal disease and tooth loss to prevalence of coronary heart 
disease. J Periodontol 2004; 75: 782-90.
30. Abnet CC, Qiao YL, Dawsey SM, Dong ZW, Taylor PR, Mark SD. Tooth 
loss is associated with increased risk of total death and death from 
upper gastrointestinal cancer, heart disease, and stroke in a Chinese 
population-based cohort. Int J Epidemiol 2005; 34: 467-74.
31. Molarius A, Tegelberg A, Ohrvik J. Socio-economic factors, lifestyle, 
and headache disorders - a population-based study in Sweden. 
Headache 2008; 48: 1426-37.
32. Jylha M. What is self-rated health and why does it predict mortality?: 
towards a unified conceptual model. Soc Sci Med 2009; 69: 307-16.
THE HISTORY OF PUBLIC HEALTH USE OF FLUORIDES IN CARIES 
PREVENTION 
Tea ŠKET1, Andreja KUKEC1, Rok KOSEM2, Barbara ARTNIK1*
1University of Ljubljana, Faculty of Medicine, Department of Public Health, Zaloška 4, 1000 Ljubljana, Slovenia
2University Medical Centre Ljubljana, Dental Clinic, Hrvatski trg 6, 1000 Ljubljana, Slovenia
Received: Feb 2, 2016
Accepted: Nov 15, 2016
Review article
Aim. The aim of our study was to chronologically analyse various public health measures of fluoride use in caries 
prevention.
Methods. We systematically searched the PubMed database on the preventive role of fluorides in public health, 
published from 1984 to 2014. The search process was divided into four steps, where inclusion and exclusion 
criteria were defined. Qualitative methodology was used for the article analysis. In the research process, the 
described forms of F use, diversity of the described F agents, and the observed population group were analysed.
Results. In our systematic review, 40 relevant reviews were revealed. Fluorides have been used in many 
different forms, but only a few studies showed their significant role in public health. Water fluoridation was 
the most important public health measure. In the recent decades, the number of studies on topical fluorides is 
constantly rising. The most extensively described topical forms of fluorides are professionally applied fluoride 
agents and fluoride toothpaste for home-use. The use of fluoride containing toothpaste in caries prevention is a 
safe and successful public health measure (PHM) if their use is widespread, and it is recommended for all. The 
results on other topical forms of fluorides are insufficient to be suggested as an important PHM.
Conclusions. The role of fluorides in public health prevention has changed in accordance with the knowledge 
about the fluoride cariostatic mechanism. Previously the most important pre-eruptive effect of fluorides was 
supplemented by the post eruptive effect. Abundant evidence exists to show the effectiveness of systemic and 
topical fluorides.
Namen. Namen naše raziskave je bil kronološko analizirati različne načine javnozdravstvene uporabe fluoridov 
pri preventivi kariesa.
Metode. Sistematično smo pregledali pregledne članke v bibliografski bazi PubMed, ki so bili objavljeni v 
obdobju od leta 1984 do leta 2014. Iskalni proces je potekal v štirih fazah, v katerih smo določili vključitvene 
in izključitvene kriterije za določitev člankov za končno analizo. Za analizo člankov smo uporabili kvalitativno 
metodologijo. V procesu analiziranja smo se osredotočili na različne oblike in načine uporabe fluoridov, jih 
opisali ter opredelili uporabo fluoridov in pomen za populacijo, v kateri so bili uporabljeni.
Rezultati. V našem sistematičnem pregledu smo analizirali 40 preglednih člankov. Opisane so številne oblike 
uporabe fluoridov, vendar le nekatere raziskave poudarjajo javnozdravstveni pomen. Fluoridiranje pitne vode 
je najpomembnejši javnozdravstveni ukrep. Povečalo se je število raziskav, ki proučujejo topikalne fluoride. 
Največ raziskav opisuje profesionalne pripravke s fluoridi in kreme za zobe s fluoridi za domačo uporabo. 
Uporaba kreme za zobe s fluoridi je varen in uspešen javnozdravstveni ukrep v preventivi kariesa, zato ga 
priporočamo za celotno populacijo. Rezultati o drugih oblikah topikalnih fluoridov so nezadostni, zato jih ni 
mogoče označiti kot pomembne javnozdravstvene ukrepe.
Zaključki. Vloga fluoridov se je spremenila v skladu z znanjem o njihovem delovanju. V preteklosti 
najpomembnejšemu preeruptivnemu delovanju se je pridružilo razumevanje o posteruptivnem učinku 
fluoridov. Obstajajo jasni dokazi o učinkovitosti tako sistemskih kot topikalnih fluoridov.
ABSTRACT
Keywords: 
fluorides, public  
health dentistry,  
caries prevention,  
epidemiology
IZVLEČEK
Ključne besede: 
fluoridi, javno 
zobozdravstvo, 
preventiva kariesa, 
epidemiologija
*Corresponding author: Tel: ++ 386 1 5437 540; E-mail: barbara.artnik@mf.uni-lj.si
10.1515/sjph-2017-0018 Zdr Varst 2017; 56(2): 140-146
140
ZGODOVINA JAVNOZDRAVSTVENE UPORABE FLUORIDOV V PREVENTIVI
KARIESA
Šket T, Kukec A, Kosem R, Artnik B. The history of public health use of fluorides in caries prevention. Zdr Varst 2017; 56(2): 140-146.
© Nacionalni inštitut za javno zdravje, Slovenija. 
1 INTRODUCTION
Caries is a widespread oral disease (1). The prevention 
of caries with fluoride/s (F) has been proven to be an 
effective public health measure (PHM), and it is considered 
to be one of the ten greatest achievements of PH in the 
20th century (2).
Fluorides are salts of the chemical element fluorine (3). 
People can be exposed to F in various ways: through air, 
soil, water, or beverages (1).
Black and McKay first recognised the preventive effect 
of F in Colorado Springs at the beginning of the 20th 
century (4). In 1931, Churchill identified a higher 
concentration of F in Colorado Spring’s water (5). This 
discovery encouraged Dean to perform several population 
interventions with water fluoridation (WF) to prevent 
caries at the population level (6). His work served as a 
basis for decades of population caries prevention by 
using WF. Systemic F had been widely recommended 
until the 1970s, when the new concept of understanding 
caries and F anticariogenic action was introduced (7). It 
was established that F controls caries mainly through its 
topical effect (3, 8). The topical mechanism is achieved 
either with systemic F or with topical F. In contrast, WF 
is a major factor in preventing pit and fissure caries, the 
most common site of tooth decay (9, 10).
According to the described historical facts, F are still 
divided into two groups according to their form of action, 
namely: systemic and topical (local). Systemic F (F water, 
milk or salt, F supplements as tablets, drops or lozenges) 
are ingested and incorporated into tooth enamel during 
tooth development. They also provide some topical 
protection of already erupted teeth as a reservoir of F 
in oral mucosa and saliva (12). Topical F (F toothpaste, 
mouthwashes, varnishes, gels, foams, slow-release F 
devices) reduce demineralisation of the enamel, promote 
remineralisation, and disable the metabolism of the 
bacteria in dental plaque (8, 11).
After nearly 100 years, F remains an important research 
subject. The availability of many different forms of F 
in the modern developed world and their cumulative 
preventive effect are among the most urgent topics in 
dentistry.
The aim of our study was to chronologically analyse 
various PH F uses in caries prevention.
2 METHODS 
2.1 Search Strategy 
We systematically searched the PubMed database on the 
preventive role of F in PH, published from 1984 to 2014. 
The search was conducted in February 2015, and revised 
in September 2015. 
The search process consisted of four steps (Figure 1), 
namely: 
1. A basic search with included Medical Subject Headings 
(MESH): fluorides, dentistry, community health. We 
specified the search by reviewing the articles that 
also included the following words in the title or 
abstract: caries and preventive or prevention (Step 
1). 
2. The first screen was performed using the following 
three inclusion criteria: full text availability; written 
in English; narrative review or systematic review or 
meta-analysis as the type of the articles (shorted: 
reviews) (Step 2). 
3. In the second screen, the article selection was 
narrowed down by reading abstracts from the 
selected reviews in the first search (Step 3). 
4. After the second screen selection, three assessors 
independently read full-text articles and confirmed 
or declined the inclusion. Twenty-eight reviews 
met all our inclusion criteria, and further data 
were extracted and analysed. The hand-search of 
the bibliography resulted in the inclusion of twelve 
reviews (Step 4). 
2.2 Data Description 
Qualitative methodology was used for the article analysis. 
In the first step, the relevant articles were extracted in 
the form of a specifically designed table: the described 
forms of F use (systemic, topical or both), diversity of the 
described F agents, and observed population group. In the 
second step, we summarised the main results on various 
uses of F as preventive agents over the previous 30 years.
3 RESULTS 
3.1 Search Strategy 
According to the described search strategy, 40 relevant 
articles were read in full (Figure 1).  
We only included articles with the main theme 
of the preventive/community health role of F. 
We excluded reviews about the mechanism of 
action, F-containing materials, dental fluorosis, 
and the effect of F on bone remodelling, caries 
management with other chemical products or 
sealants. 
Since none of the seven reviews published before 
1984 met the criteria for inclusion, we determined 
the observed window of time as being from 1984 
until the present. 
a.
b.
10.1515/sjph-2017-0018 Zdr Varst 2017; 56(2): 140-146
141
10.1515/sjph-2017-0018 Zdr Varst 2017; 56(2): 140-146
142
In terms of content, articles were analysed according to 
the described F preventive agent(s) (Table 1).
3.2 Analysis by Periods of Observation 
We arranged the reviews into three periods according to 
publication dates (Figure 2).  
Table 1.
Figure 1.
Figure 2.
Figure 3.
Results of the reviews by year of publication and 
described form of the use of F preventive agents 
(PubMed, 1984-2014). 
Legend: NR – narrative review; SR – systematic review; MA – 
meta-analysis; W – water fluoridation, T – fluoridated tablets, 
drops, lozenges; S – fluoridate salt; M – fluoridate milk; 
I – ingested fluorides in food, drinks or as dental agents; P – 
toothpaste with fluorides; V – fluoride varnishes; G – fluoride 
gels; R – fluoride mouthwashes/mouth rinses; D – slow-release 
fluoride devices; To – topical fluorides in general; Co – 
combination of various fluoride agents.
According to the described search strategy, 
40 relevant articles were read in full (Figure 1).  
The number and substantive set of reviews about 
the preventive role of fluoride according to the 
publication date (PubMed, 1984-2014).
The number of reviews on systemic fluorides in the 
observed periods according to the form of use and 
publication time (F – fluoride/s).
Sheiham, 1984 
Hargreaves, 1990 
Horowitz, 1990 
Manji, 1990 
Lewis, 1994 
Levy, 1994 
Johnston, 1994 
O’Mullane, 1994 
Bowen, 1995 
Limeback, 1999 
Marinho, 2002 
Marinho, 2002 
Marinho, 2003 
Marinho, 2003 
Twetman, 2003 
Warren, 2003 
Axelsson, 2004 
Fejerskov, 2004 
Marinho, 2004 
Marinho, 2004 
Petersson, 2004 
Twetman, 2004 
Yeung, 2005 
Bonner, 2006 
Griffin, 2007 
Pizzo, 2007 
Ismail, 2008 
Kumar, 2008 
Pessan, 2008 
Espelid, 2009 
Marinho, 2009 
Toumba, 2009 
Carvalho, 2010 
Walsh, 2010 
Wong, 2011 
Lam, 2012 
Petersen, 2012 
Marinho, 2013 
Chong, 2014 
Moyer, 2014 
• 
• 
• 
• 
• 
• 
 
• 
• 
• 
 
 
 
 
 
• 
 
• 
 
 
 
 
 
 
• 
• 
 
• 
 
 
 
 
 
 
 
• 
• 
 
 
• 
• 
 
 
• 
 
 
 
• 
 
 
 
 
 
 
 
• 
 
 
 
 
 
 
 
 
 
 
 
• 
 
• 
 
 
 
 
 
 
• 
 
 
 
 
 
 
 
 
• 
 
• 
 
 
 
 
 
 
 
• 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
• 
 
 
 
 
• 
 
 
 
 
 
 
 
 
• 
 
 
 
 
 
 
• 
 
 
• 
 
• 
 
 
• 
• 
• 
 
• 
• 
 
 
 
 
 
 
 
 
 
 
 
 
• 
 
 
 
 
 
 
 
• 
 
 
 
 
 
• 
 
 
• 
 
 
 
 
• 
• 
 
 
 
 
 
 
• 
 
 
 
 
 
 
 
 
 
 
 
 
 
• 
 
 
 
 
 
 
• 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
• 
 
 
• 
 
• 
 
 
 
 
 
 
 
 
 
• 
 
• 
 
 
 
 
 
 
 
 
• 
• 
 
• 
 
 
 
 
 
• 
• 
 
 
• 
 
 
 
 
 
 
 
• 
 
 
 
 
 
 
 
• 
 
 
 
 
 
 
• 
 
 
 
 
 
 
• 
• 
 
 
 
 
 
 
 
 
 
• 
 
 
 
 
 
 
 
• 
 
 
 
• 
 
• 
 
 
 
 
 
 
 
 
 
 
 
 
• 
 
 
• 
 
 
• 
• 
• 
• 
 
 
 
 
 
 
 
 
 
• 
 
 
 
• 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
• 
 
 
 
 
• 
• 
 
 
 
 
 
 
 
 
 
 
 
• 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
• 
 
 
 
 
• 
 
• 
• 
 
 
 
 
 
 
• 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
• 
 
• 
• 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
NR 
NR 
NR 
NR 
NR 
NR 
NR 
NR 
NR 
NR 
MA 
MA 
MA 
MA 
MA 
NR 
SR 
NR 
MA 
MA 
MA 
SR 
SR 
SR 
SR 
SR 
SR 
NR 
SR 
SR 
SR 
SR 
SR 
MA 
SR 
NR 
NR 
MA 
SR 
SR 
First author, 
year of 
publication 
Review 
type 
W S I V R ToT M P G D Co
Nineteen different F interventions were observed in the 
selected reviews. The representation by their described F 
agents in caries prevention is shown in Figure 3 (systemic 
use) and Figure 4 (topical use).
10.1515/sjph-2017-0018 Zdr Varst 2017; 56(2): 140-146
143
In Figure 5, we arranged the reviews according to the 
observed population. Eight reviews did not specify the 
group of interest according to age. In terms of using 
different F interventions, systemic F were observed in 
all age groups, but for the analysis of topical effects in 
adults and the elderly, the data were too limited or non-
existent.
3.3 Results of Contents by Periods 
3.3.1 The Period between 1984 and 1993 
In the first observation period, only four relevant reviews 
were published. None of them focused on the topical 
F only and among the topical F agents; only Sheiham 
described toothpaste and mouth rinse (13). Hargreaves 
(14) and Manji (15) mentioned the possibility of topical 
F, but also warned about their insufficiently researched 
effects. All four reviews describe WF as an effective PHM 
of prevention of caries with 27-35% decline in caries 
prevalence in the exposed population, in comparison to 
the unexposed (13-16). The use of topical F (toothpaste, 
mouthwash) was reported to reduce the incidence of 
caries, especially during teeth eruption (13, 15).
Horowitz reported a lack of evidence on the combined 
use of systemic and topical F (16). He also reported that 
the use of topical products had helped to produce caries 
reduction in areas without WF and, thus, diminished the 
absolute differences in caries prevalence between WF 
and non-WF areas (16).
Nevertheless, Hargreaves emphasised the lack of 
knowledge in recommending different F agents according 
to the age and development of the individual for maximum 
effect with minimum risk (14).
3.3.2 The Period between 1994 and 2003 
Between 1994 and 2003, twelve relevant reviews were 
published. Four described systemic F; six described 
topical F; and the remaining two described both forms 
of use.
Levy (17) presented a new concept of prevention with F: 
the concept of optimal intake of F. Limeback reported 
that extensive use of systemic F might have masked the 
topical action of systemic F (18). Moreover, he wrote that 
the topical action could be even more important than the 
systemic effect (18). 
More than 50 years of experience with water fluoridation 
showed a reduction in the burden of caries among the 
exposed population in many countries (19). Differences 
in DMFS (decayed, missed, filled surfaces) were smaller 
than in the past; therefore, Lewis estimates WF as a 
relevant PHM in poor communities, but unnecessary 
in countries with low prevalence of caries (20). As an 
alternative, considerably less known or studied systemic F 
interventions were proposed (F supplements, fluoridated 
salt, milk or sugar) (17, 19-22). 
Studies on F supplements revealed significant limitations: 
poor patient compliance and inappropriate prescription 
(17, 22). 
When designing prevention programmes or individual 
counseling regarding F, the total intake of F should be 
taken into account, which also means taking into account 
F ingested with food, drink and oral care products 
(accidental) (17, 22). 
During this period, several authors described the 
effectiveness of topical uses of F. Gels were the favourite 
choice of the operators (23). Their estimated efficiency 
was 21-26% in a population unexposed to other F 
treatments (24). The prevented fraction (PF) of F varnishes 
and F mouthwashes was high in primary and permanent 
deciduous teeth, although no significant association was 
found between the estimates of D(M)FS PFs and baseline 
caries or background exposure to F (25-27). 
The reviews and meta-analysis confirmed that the effect 
of fluoride toothpaste is higher with higher F concentration 
and frequency of use (21, 28, 29). Moreover, Twetman 
found only limited evidence for anti-caries differences 
between low-fluoride and standard toothpaste (28). 
To summarise, they all concluded that the preventive 
action of F toothpaste when brushing every day in young 
permanent teeth was based on strong evidence (28, 29).
 
Figure 4.
Figure 5.
The number of reviews on topical fluorides in the 
observed periods, according to the form of use and 
publication time (F – fluoride/s).
The distribution of the reviews about the preventive 
role of F according to the observed population 
group.
10.1515/sjph-2017-0018 Zdr Varst 2017; 56(2): 140-146
144
3.3.3 The Period between 2004 and 2014 
We identified 21 reviews published in the last decade. 
Systemic F were described in four reviews, topical F in 
eleven reviews, and seven reviews included both. 
In spite of rising numbers of evidence-based reviews on the 
importance of the topical effect of F, systemic F remained 
an important anti-cariogenic measure, especially in terms 
ofcost-effectiveness (30). In 2006, over 300 million people 
in 39 countries lived in WF-supplied areas (31). 
F milk and F salt had shown some preventive effect, 
but they are difficult to prescribe in correctdoses, and 
scientific evidence for assessing their effect is insufficient 
to draw conclusions (30, 32-35). 
Nevertheless, of all systemic F, the safety and efficiency 
of F supplements seem to be an issue of some debate 
among professionals (36). Reduction in caries increment 
differs a lot between different age groups, and in terms of 
the duration of action, proper supervision and motivation.
The most popular form of professionally applied F agents 
in the reviews was F varnishes. The lack of evidence-
based studies makes it impossible to determine whether 
the concentration in the varnish affects its prophylactic 
activity (37-39). There is a moderate benefit in preventing 
future caries with F varnish application in all children 
starting at the age of eruption of primary teeth (40, 41). 
F varnish is appealing for PH programmes, because it is 
easily incorporated (30, 37). 
The two reviews mentioned F tooth-mousse and F gels as 
alternative forms of professionally applied F agents (33, 
38). 
In countries where F toothpasteis wide spread, it is the 
most relevant and accessible method of topical F (38, 
42). More than 500 million people around the world use F 
toothpaste, and its effectiveness is well documented (34, 
38, 42). 
The efficiency of F toothpaste is increased with supervised 
brushing, a higher concentration of F, frequent use, and 
in higher-risk patients (29, 30, 33, 43). The evidence of 
the combined use of F toothpaste and another F agent 
remains limited; however, simultaneous use results in 
caries reduction compared with F toothpaste use alone 
(34, 38, 44-46). In comparison with gels or mouthwashes, 
toothpaste appears to have similar effectiveness for 
caries prevention in children (46). 
Weekly rinsing under supervision has been proved to be 
effective and even an alternative to operative applied or 
unsupervised home use of F (30, 33). For the relatively 
new F agents, slow-release F devices, only a few studies 
are available, but they have shown a caries reduction 
capacity (47-50).
Although children and adolescents are still usually the 
population of interest, Griffin found that fluorides are 
effective among all adults, and the implementation of 
fluoride programmes should also serve this population 
(51). 
 
4 DISCUSSION 
The prevention of caries with F has been a significant PH 
intervention since it started with the implementation of 
F in drinking water. In our study, we have presented the 
shift from the traditional to the modern understanding of 
F use in caries prevention. 
Since 1984, when the first review of literature about 
PH fluoride prevention was conducted, fluorides have 
remained one of the milestones of caries prevention. 
Nevertheless, the recommendations and PH programmes 
have changed, and the number of published reviews about 
F prevention in the field of PH is constantly increasing.
In the first observed decade (1984-1993), the main points 
of interest were systemic F. In the second decade (1994-
2003), only half of the reviews were about systemic F. In 
the latest decade, only a minority of the reviews focused 
on systemic F, with half of the reviews during 2004-
2014, describing topical F as a PHM. Most of the research 
on F prevention was done in groups of children and 
adolescents. It is thought that F use in caries prevention 
is also an important measure among adults and especially 
among the elderly. 
In 2004, Fejerskov described the shift from “caries 
resistance concept” to understanding caries to be a 
multifactorial disease, following the ecological shift in 
the tooth-surface biofilm (7). 
At first, only WF seemed to be a major PHM, but in 
recent decades, topical F, especially widespread and very 
efficient F toothpaste, are considered as possible PHM, 
especially in populations with low caries increment. 
Although all the reviews agree that community WF is 
safe and effective in caries prevention, its role has 
been combined with the use of topical F as PHM. WF is 
cost saving and still recommendedby the US Community 
Preventive Services Task Force (41) and recent guidelines 
(52–54). Even in an era with a widespread availability 
of fluoride from other sources, studies prove that water 
fluoridation continues to be effective in reducing dental 
decay (55, 56). WF is believed to be a major factor in 
preventing pit and fissure caries, the most common 
site of tooth decay (9, 10). Maximal caries-preventive 
effects of water fluoridation are achieved by exposure 
to optimal fluoride levels both pre- and post-eruptively 
(55). Moreover, the latest Cochrane review on WF from 
10.1515/sjph-2017-0018 Zdr Varst 2017; 56(2): 140-146
145
2015, concluded that there is insufficient information 
to determine the effect of stopping water fluoridation 
programmes on caries prevalence (57). 
The post-eruptive effect became more important with 
the decreasing severity of caries attack (58). Efforts of 
the PH preventive programme for oral health should be 
directed towards the maintenance of a permanent low 
concentration of fluoride in the mouth in as many people 
as possible (3). Even the current guidelines recommend 
fluoride toothpaste as the basic fluoride regimen for the 
majority of European communities (52).
In Slovenia, the level of F in the water supply is below 
0.2 ppm (59), which is below the level of significance 
for WF. In addition, artificial fluoridation has never been 
implemented. In the 1970s, the topical application of a 
2% sodium fluoride in school-age children (7–15 years) 
was introduced, which was changed, in the 1980s, to the 
application of aminofluoride gel and the widespread use 
of toothpaste with fluoride (60). Today, preventive dental 
care in Slovenia follows updated guidelines on the use 
of F in Europe (52). We wish to (re-)establish a modern, 
scientific-based collective approach to F use in caries 
prevention, which is currently, unfortunately, based on 
the commitment of individuals or smaller professional 
groups, working in dental (prevention) offices. 
Our article presents the F use in caries prevention through 
the perspective of PH. Independently of the described 
form of F use, we highlighted the most significantfindings 
about all forms of F use in order to give the reader the best 
perspective on the field. A comprehensive knowledge on 
fluorides is described. At the same time, the comprehensive 
approach to our topic is also a restriction. We chose to 
limit our study to review articles; consequently, we could 
not assess the strength of evidence-based facts for each 
form of use. 
Our review showed that the use of F in caries prevention 
is very active. It is important to improve the evidence 
ability for different forms of use, and to research the 
cumulative effects of using different forms of F. The effect 
of different F agents should be measured in various ages 
and social environments. Furthermore, the need for the 
systematic use of F in societies with low caries prevalence 
should be scientifically assessed. More research should 
also be done in the older population groups, not focusing 
only on children and adolescents. 
 
5 CONCLUSION 
In contrast to traditional systemic forms of F as PHMs, 
nowadays, the use of topical F is becoming the primary 
approach. As the operative treatment of caries is very 
expensive, and F has proven to be efficient in caries 
reduction, it remains recommended as the standard 
for caries prevention. At the same time, it is necessary 
to promote oral health also at other levels (good oral 
hygiene, healthy nutrition, regular check-ups, being 
included in oral preventive programmes). 
CONFLICT OF INTEREST 
The authors declare that no conflicts of interest exist. 
FUNDING 
None. 
ETHICAL APPROVAL 
Review articles were studied. 
REFERENCES 
1. Petersen PE. The world oral health report 2003: continuous 
improvement of oral health in the 21st century - the approach of 
the WHO Global Oral Health Programme. Community Dent Oral 
Epidemiol 2003; 31(Suppl 1): 3-23. 
2. Centers for Disease Control and Prevention. Recommendations for 
using fluoride to prevent and control dental caries in the United 
States. MMWR Morb Mortal Wkly Rep 2001; 50: 1-42. 
3. WHO Expert Committee on Oral Health Status and Fluoride Use. 
Fluorides and oral health; WHO Technical Report Series No. 846. 
Geneva: WHO, 1994. 
4. Black GV, McKay FS. Original communications - mottled teeth: an 
endemic developmental imperfection of the enamel of the teeth 
heretofore unknown in the literature of dentistry. Dental Cosmos 
1916; 58: 129-56. 
5. Churchill HV. Occurrence of fluorides in some waters of the United 
States. J Ind Eng Chem 1931; 23: 996-8. 
6. Arnold FA Jr, Likins RC, Russell AL, Scott DB. Fifteenth year of the 
Grand Rapids fluoridation study. J Am Dent Assoc 1962; 65: 780-5. 
7. Fejerskov O. Changing paradigms in concepts on dental caries: 
consequences for oral health care. Caries Res 2004; 38: 182-91. 
8. Buzalaf MA, Pessan JP, Honório HM, ten Cate JM. Mechanisms of 
action of fluoride for caries control. Monogr Oral Sci 2011; 22: 97-
114. 
9. Singh KA, Spencer AJ. Relative effects of pre- and post-eruption 
water fluoride on caries experience by surface type of permanent 
first molars. Community Dent Oral Epidemiol 2004; 32: 435-46. 
10. Van Eck AAJM. Pre-and post-eruptive effect of fluoridated drinking 
water on dental caries experience. Utrecht: University of Utrecht; 
1987. 
11. Fejerskov O, Ekstrand J, Burt BA, editors. Fluoride in dentistry. 
Copenhagen: Munksgaard, 1996.  
12. Rošin-Grget K, Peroš K, Sutej I, Bašić K. The cariostatic mechanisms 
of fluoride. Acta Med Acad 2013; 42(Suppl 2): 179-88. 
13. Sheiham A. Changing trends in dental caries. Int J Epidemiol 1984; 
13: 142-7. 
14. Hargreaves JA. Water fluoridation and fluoride considerations for the 
future supplementation. J Dent Res 1990; 69: 765-70. 
15. Manji F, Fejerskov O. Dental caries in developing countries. J Dent 
Res 1990; 96: 733-41. 
10.1515/sjph-2017-0018 Zdr Varst 2017; 56(2): 140-146
146
16. Horowitz HS. The future of water fluoridation and other systemic 
fluorides. J Dent Res 1990; 69: 760-4. 
17. Levy SM. Review of fluoride exposures and ingestion. Community 
Dent Oral Epidemiol 1994; 22: 173-80. 
18. Limeback H. A re-examination of the pre-eruptive and post-eruptive 
mechanism of the anticaries effects of fluoride: is there any anti-
caries benefit from swallowing fluoride? Community Dent Oral 
Epidemiol 1999; 27: 62-71. 
19. O’Mullane DM. Systemic fluorides. Adv Dent Res 1994; 8: 181-4. 
20. Lewis DW, Banting DW. Water fluoridation: current effectiveness and 
dental fluorosis. Community Dent Oral Epidemiol 1994; 22: 153-8. 
21. Bowen WH. Are preventive programs sufficient for the needs of 
tomorrow? Adv Dent Res 1995; 9: 77-81. 
22. Warren JJ, Levy SM. Current and future role of fluorides in nutrition. 
Dent Clin North Am 2003; 47: 225-43. 
23. Johnston DW. Current status of professionally applied topical 
fluorides. Community Dent Oral Epidemiol 1994; 22: 159-63. 
24. Marinho VCC, Higgins JP, Sheiham A, Logan S. Fluoride gels for 
preventing dental caries in children and adolescents. Cochrane 
Database Syst Rev 2002; (1): CD002280. 
25. Marinho VCC, Higgins JP, Logan S, Sheiham A. Fluoride varnishes 
for preventing dental caries in children and adolescents. Cochrane 
Database Syst Rev 2002; (3): CD002279. 
26. Twetman S, Petersson L, Axelsson S, Dahlgren H, Holm A-K, Kallestal 
C et al. Cariespreventive effect of sodium fluoride mouthrinses: a 
systematic review of controlled clinical trials. Acta Odontol Scand 
2004; 62: 223-30. 
27. Marinho VCC, Higgins JPT, Logan S, Sheiham A. Fluoride mouthrinses 
for preventing dental caries in children and adolescents. Cochrane 
Database Syst Rev 2003; (3): CD002284. 
28. Twetman S, Axelsson S, Dahlgren H, Holm A, Källestål C, Lagerlöf F 
et al. Caries preventive effect of fluoride toothpaste: a systematic 
review. Acta Odontol Scand 2003; 61: 347-55. 
29. Marinho VCC, Higgins JP, Logan S, Sheiham A. Fluoride toothpastes 
for preventing dental caries in children and adolescents. Cochrane 
Database Syst Rev 2003; (4): CD002782. 
30. Kumar JV, Moss ME. Fluorides in dental public health programs. Dent 
Clin North Am 2008; 52: 387-401. 
31. Pizzo G, Piscopo MR, Pizzo I, Giuliana G. Community water 
fluoridation and caries prevention: a critical review. Clin Oral 
Investig 2007; 11: 189-93. 
32. Espelid I. Caries preventive effect of fluoride in milk, salt and 
tablets: a literature review. Eur Arch Paediatr Dent 2009; 10: 149-56. 
33. Petersen PE, Phantumvanit P. Perspectives in the effective use of 
fluoride in Asia. J Dent Res 2012; 91: 119-21. 
34. Marinho VCC. Cochrane reviews of randomized trials of fluoride 
therapies for preventing dental caries. Eur Arch Paediatr Dent 2009; 
10: 183-91. 
35. Yeung CA, Hitchings JL, Macfarlane TV, Threlfall AG, Tickle M, 
Glenny A-M. Fluoridated milk for preventing dental caries. Cochrane 
Database Syst Rev 2005; (3): CD003876. 
36. Ismail AI, Hasson H. Fluoride supplements, denal caries and fluorosis: 
a systematic review. J Am Dent Assoc 2008; 139: 1457-68. 
37. Carvalho DM, Salazar M, Heliosa de Olivera B, Silva Freire Coutinho 
E. Fluoride varnishes and caries incidence decrease in preschool 
children: a systematic review. Rev Bras Epidemiol 2010; 13(Suppl 
1): 1-11. 
38. Lam A, Chu C. Caries management with fluoride agents. N Y State 
Dent J 2012; 78: 29-36. 
39. Petersson LG, Twetman S, Dahlgren H, Norlund A, Holm AK, 
Nordenram G et al. Professional fluoride varnish treatment for caries 
control: a systematic review of clinical trials. Acta Odontol Scand 
2004; 62: 170-6. 
40. Marinho VCC, Worthington HV, Walsh T, Clarkson JE. Fluoride 
varnishes for preventing dental caries in children and adolescents. 
Cochrane Database Syst Rev 2013; (7): CD002279. 
41. Moyer VA. Prevention of dental caries in children from birth through 
age 5 years: US Preventive Services Task Force recommendation 
statement. Pediatrics 2014; 13: 1102-11. 
42. Wong MCM, Clarkson J, Glenny A-M, Lo ECM, Marinho VCC, Tsang BWK 
et al. Cochrane reviews on the benefits/risks of fluoride toothpastes 
J Dent Res 2011; 90(Suppl 5): 573-9. 
43. Walsh T, Worthington HV, Glenny A-M, Appelbe P, Marinho VCC, Shi 
X. Fluoride toothpastes of different concentrations for preventing 
dental caries in children and adolescents. Cochrane Database of Syst 
Rev 2010; (1): CD007868. 
44. Marinho VCC, Higgins JPT, Sheiham A, Logan S. Combinations of 
topical fluoride (toothpastes, mouthrinses, gels, varnishes) versus 
single topical fluoride for preventing dental caries in children and 
adolescents. Cochrane Database Syst Rev 2004; (1): CD002781. 
45. Axelsson S, Söder B, Nordenram G, Petersson LG, Dahlgren H, 
Norlund A et al. Effect of combined caries-preventive methods: a 
systematic review of controlled clinical trials. Acta Odontol Scand 
2004; 62(Suppl 3): 163-9. 
46. Marinho V, Higgins J, Sheiham A, Logan S. One topical fluoride 
(toothpastes, or mouthrinses, or gels, or varnishes) versus another 
for preventing dental caries in children and adolescents. Cochrane 
Database Syst Rev 2004; (1): CD002780. 
47. Pessan JP, Al-Ibrahim NS, Buzalaf MAR, Toumba KJ. Slow-release 
fluoride devices: a literature review. J Appl Oral Sci 2008; 16: 238-
46. 
48. Toumba KJ, Al-Ibrahim NS, Curzon MEJ. A review of slow-release 
fluoride devices. Eur Arch Paediatr Dent 2009; 10: 175-82. 
49. Chong LY, Clarkson JE, Dobbyn-Ross L, Bhakta S. Slow-release fluoride 
devices for the control of dental decay. Cochrane Database Syst Rev 
2014; (11): CD005101. 
50. Bonner BC, Clarkson JE, Dobbyn L, Khanna S. Slow-release fluoride 
devices for the control of dental decay. Cochrane Database of Syst 
Rev 2006; (4): CD005101. 
51. Griffin SO, Regnier E, Griffin PM, Huntley V. Effectiveness of fluoride 
in preventing caries in adults. J Dent Res 2007; 86: 410-5. 
52. European Academy of Paediatric Dentistry. Guidelines on the use of 
fluoride in children: an EAPD policy document. 2009; 10: 129-35. 
53. Australian Research Centre for population Oral Health. The use of 
fluorides in Australia: guidelines. Aust Dent J 2006; 51: 195-9. 
54. Canadian Dental Association. CDA position on use of fluorides in 
caries prevention, 2012. Available Jan 10, 2016 from: https://www.
cda-adc.ca/_files/position_statements/fluoride.pdf. 
55. Newbrun E. Systemic benefits of fluoride and fluoridation. J Public 
Health Dent 2004; 64: 35-9. 
56. Brunelle JA, Carlos JP. Recent trends in dental caries in U.S. children 
and the effect of water fluoridation. J Dent Res 1990; 69(Spec Iss): 
723-7. 
57. Iheozor-Ejiofor Z, Worthington HV, Walsh T, O’Malley L, Clarkson JE, 
Macey R et al. Water fluoridation for the prevention of dental caries. 
Cochrane Database of Syst Rev 2015; (6): CD010856. 
58. Groeneveld A, van Eck AAMJ, Backer Dirks O. Fluoride in caries 
prevention: is the effect pre- or post-eruptive? J Dent Res 1990; 69: 
751-5. 
59. NLZOH. Monitoring pitne vode - rezultati januar 2015 - avgust 
2015. Available Nov 10, 2015 from: http://www.mpv.si/rezultati [in 
Slovene]. 
60. Artnik B. Javnozdravstveni pogled na uporabo fluoridov v skrbi za 
boljše ustno zdravje. In: Juričič M, Mugoša J, editors. 100 let šolske 
medicine na Slovenskem: 1909-2009. Zbornik prispevkov. Ljubljana: 
Sekcija za šolsko in visokošolsko medicino, 2009: 186-97. 
National Institute of Public Health
Trubarjeva 2, 1000 Ljubljana, Slovenia
zdrav.var@nijz.si
http://www.nijz.si/sl/nijz/revija-zdravstveno-varstvo 
++386 1 2441 543
INSTRUCTIONS FOR AUTHORS
Journal: Zdravstveno varstvo (ZV) ISSN 0351-0026 (print edition) / Slovenian Journal of Public Health (SJPH)
ISSN 1854-2476 (electronic edition)
Slovenian Journal of Public Health publishes internationally oriented articles on the broad area of public health and 
encourages interdisciplinary approach to public health. It focuses on all specific issues in public health especially 
in Central and South East Europe, i.e. primary care, prevention of communicable and noncommunicable diseases, 
health promotion, environmental and occupational health, organization and management in public health, social and 
economical aspects of public health. 
The journal publishes original invited editorials, research papers, study protocols, and systematic reviews in English 
language only.
Instructions are in accordance with the Uniform Requirements for Manuscripts Submitted to Biomedical Journals. 
Complete instructions are published in N Engl J Med 1997; 336: 309-15 and in Ann Intern Med 1997; 126: 36-47 and on 
the URL address: http://www.icmje.org.
Editorial Office accepts only articles, that have not been published elsewhere and are not being considered for 
publication in other journals. Parts of the article, summarized after other sources (especially illustrations and tables) 
should include the author’s and publisher’s permission for reproduction.
ETHICAL GUIDELINES 
Submission of a manuscript to ZV/SJPH implies that all authors have read and agreed to its content. Research 
involving human subjects (including human material or human data) that is reported in the manuscript must have 
been performed in accordance with the Declaration of Helsinki of 1975, as revised in 2000 and must have been 
approved by an appropriate ethics committee. A statement detailing this, including the name of the ethics committee 
and the reference number where appropriate, must appear at the end of all manuscripts reporting research on human 
subjects. If a study has been granted an exemption from requiring ethics approval, this should also be detailed at 
the end of the manuscript. The authors must explain the rationale for their approach, and demonstrate that the 
institutional review body explicitly approved the doubtful aspects of the study. Further information and documentation 
to support this should be made available to editors on request. Manuscripts may be rejected if the editors consider 
that the research has not been carried out within an ethical framework. In rare cases, the editors may contact the 
ethics committee for further information. 
For all research involving human subjects, informed consent to participate in the study should be obtained from 
participants (or their parent or guardian in the case of children) and a statement to this effect should appear in the 
manuscript. 
For all articles that include information or images relating to individual participants, written informed consent for the 
publication of these images must be obtained from the participant (or their parent or guardian in the case of children) 
and a statement to this effect should be included in the manuscript. These documents must be made available to 
editors if requested, and will be treated confidentially. 
For research carried out on animals, authors are encouraged to comply with the “Animal Research: Reporting In Vivo 
Experiments” (ARRIVE) guidelines and must comply with local or institutional ethics approval requirements on the 
care and use of animals for research. A statement detailing such ethics approval and/or guidelines must be included 
in the manuscript. Relevant information should be included in the article as outlined in the guidelines. 
Authors who have done the project with the support of a company, should indicate this in the statement at the end 
of the manuscript. 
ZV/SJPH requires authors to declare any competing financial or other interest in relation to their work. All competing 
interests that are declared must be listed at the end of the manuscript. 
Required statements at the end of the manuscript are:  
CONFLICTS OF INTEREST (The authors declare that no conflicts of interest exist.) 
FUNDING (The study was financed by ... ) 
ETHICAL APPROVAL (Received from the ... )
PLAGIARISM DETECTION 
ZV/SJPH publisher is a member of the CrossCheck plagiarism detection initiative. In cases of suspected plagiarism 
CrossCheck report is available to the editors of the ZV/SJPH to detect instances of overlapping and similar text in 
submitted manuscripts. CrossCheck is a multi-publisher initiative allowing screening of published and submitted 
content for originality.  
MANUSCRIPT SUBMISSION 
We recommend the use of video instructions for authors. The ZV/SJPH welcomes submissions in electronic form to 
the Web-based peer review system, Editorial Manager at http://www.editorialmanager.com/sjph/ to receive all 
submissions and no longer accepts submissions by e-mail or post. The Editorial Office will only accept the post 
delivery of the Authorship and Copyright Transfer Agreement which requires the authors’ signatures. Kindly send 
them together with the electronic submission of the manuscript to the address: National Institute of Public Health, 
Zdravstveno varstvo, Trubarjeva 2, 1000 Ljubljana, Slovenia. 
Please register in Editorial Manager as ‘author’. The first registration will require the entry of the author’s data. 
All further entries will only require the entry of login data, which will be sent to your e-mail address after the first 
registration in the system. 
After a successful login, complete all the required structured fields. Confirm the statement that your contribution 
has not yet been published or considered for publication in any other journal, and that the contribution was read 
and approved by other authors, and that if the contribution includes experiments on humans or animals, these were 
performed in accordance with the principles of the Helsinki-Tokyo Declaration or in accordance with the ethical 
principles. 
Complete the data on the author and co-authors accurately and in full. Provide the corresponding author (with full 
address, telephone number and e-mail address), who will communicate with the Editorial Office and other authors. 
The language of the manuscript is English. Original studies and reviews must include Slovenian language translations 
of the title, the abstract and key words. English speaking authors submit all the text only in English, obligatory 
secondary abstract is the same as the first abstract (please repeat it in English). 
A special field for translation is provided only for the second language (Slovenian language) version of the abstract; 
other data must be written in the requested fields bilingually. The first abstract is always in English (limit 250 words), 
the secondary abstract is in Slovenian language with limitation of 400 words (extended abstract). English speaking 
authors submit all the text only in English! 
After the entry of the structured data, please submit the attachment - the manuscript (from the Introduction part 
onward), which may not include author’s data already written in the structured fields. The name of the file may not 
include the author’s personal data or titles of institutions included in the preparation of the manuscript. Include 
graphic and image material in the text where it should be positioned. Submit only one document (one attached file). 
Please use Line counter in the Word program. 
When submitting the manuscript, follow the instructions provided by the system; you can also use ‘Editorial Manager’s 
Tutorial for Authors’. 
The system works best with the latest version of Acrobat. 
If you have difficulties submitting your manuscript, please contact the journal office at zdrav.var@nijz.si. 
Our Web-based system provides full electronic capabilities not only for submission, but also for peer review and 
status updates. It also speeds manuscript turnaround and provides global access for authors, reviewers, and editors. 
Authors, reviewers, and editors receive automatic e-mail messages from Editorial Manager when significant events 
occur. 
We give some detailed instructions in the continuation. 
Manuscript should be written in Word for Windows word processor. Contribution should be typed with double-spaced 
with margins of at least 25 mm. 
Original scientific articles should be divided into following headings: 1 lntroduction, 2 Methods, 3 Results, 4 Discussion 
and 5 Conclusions (this is also the structure of the abstract). Other types of articles and systematic review articles 
can be designed differently, but the division in headings and subheadings should be clearly evident from the size of 
characters in the titles. Headings and subheadings should be numbered decadally by standard SIST ISO 2145 and SIST 
ISO 690 (e. g. 1, 1.1, 1.1.1 etc.). 
LENGTH
Required length for invited editorial is 250 to 1000 words and for research article 2000 to 4500 words with tables and 
references.
TITLE AND AUTHORS
The title of the article should be short and concise, descriptive and not affirmative (statements are not allowed 
in the title). Names of authors with concise academic and professional degrees and full adress of the department, 
institution or clinic where the work has been performed should be cited. Authors should be qualified for authorship. 
They should contribute to the conception, design, analysis and interpretation of data, and they should approve the 
final version of the contribution.
ABSTRACT AND KEY WORDS
The abstract of the original scientific article should be structured (Introduction, Methods, Results, Conclusions) and 
of no more than 250 words (Slovenian language abstracts are limited to 400 words). The abstract should be written 
in third person. The abstract of a original scientific article should state the purpose of the investigation, basic 
procedures, main findings together with their statistical significance, and principal conclusions. 3 - 10 key words 
should be cited for the purpose of indexing. Terms from MeSH - Medical Subject Headings listed in Index Medicus 
should be used. The abstract should normally be written in one paragraph, only exceptionally in several. The author 
propose the type of the artlice, but the final decision is adopted by the editor on the base of the suggestions of the 
professional reviewers. 
REFERENCES 
Each mentioning of the statements or findings by other authors should be supported by a reference. References 
should be numbered consecutively in the same order in which they appear in the text. Reference should be cited at 
the end of the cited statement. References in text, illustrations and tables should be indicated by Arabic numerals in 
parentheses ((1), (2, 3), (4-7) etc). References, cited only in tables or illustrations should be numbered in the same 
sequence as they will appear in the text. Avoid using abstracts and personal communications as references (the latter 
can be cited in the text). The list of the cited literature should be added at the end of the manuscript. Literature 
should be cited accordingto the enclosed instructions that are in accordance with those used by U. S. National Library 
of Medicine in Index Medicus. The titles of journals should be abbreviated according to the style used in Index Medicus 
(complete list on the URL address: http://www.nlm.nih.gov). List the names of all authors, if there are six authors or 
more, list the first six authors than add et al.
EXAMPLES FOR LITERATURE CITATION 
example for a book: 
1. Premik M. Uvod v epidemiologijo. Ljubljana: Medicinska fakulteta, 1998.
example for the chapter in a book: 
2. Goldberg BW. Population-based health care. In: Taylor RB, editor. Family medicine. 5th ed. New York: Springer, 
1999: 32-6.
example for the article in a journal: 
3. Barry HC, Hickner J, Ebell MH, Ettenhofer T. A randomized controlled trial of telephone management of suspected 
urinary tract infections in women. J Fam Pract 2001; 50: 589-94.
example for the article in a journal with no author given: 
4. Anon. Early drinking said to increase alcoholism risk. Globe 1998; 2: 8-10. 
example for the article in a journal with organization as author: 
5. Women’s Concerns Study Group. Raising concerns about family history of breast cancer in primary care 
consultations: prospective, population based study. Br Med J 2001; 322: 27-8.
example for the article from journal volume with supplement and with number:
6. Payne DK, Sullivan MD, Massie MJ. Women’s psychological reactions to breast cancer. Semin Oncol 1996; 23(Suppl 
2): 89-97.
example for the article from collection of scientific papers:
7. Hickner J, Barry HC, Ebell MH, Ettenhofer T, Eliot R, Sugden K et al. Suicides and non-suicidal deaths in Slovenia: 
molecular genetic investigation. In: 9th European Symposium on Suicide and Suicidal Behaviour. Warwick: 
University of Oxford, 2002: 76.
example for master theses, doctor theses:
8. Bartol T. Vrednotenje biotehniških informacij o rastlinskih drogah v dostopnih virih v Sloveniji: doktorska 
disertacija. Ljubljana: Biotehniška fakulteta, 1998.
example for electronic sources:
9. Mendels P. Textbook publishers extend lessons online. Available Sept 23, 1999 from:  
http://www.nytimes.com/library/tech/99/09.
TABLES 
Type on the place in the text where they belong. Tables should be composed by lines and columns which intersect in 
fields. Number tables consecutively. Each table should be cited in the text and supplied with a brief title. Explain all 
the abbreviations and non-standard units in the table.  
ILLUSTRATIONS 
Illustrations should be professionally drawn. When preparing the illustrations consider the black-and-white print. 
Illustration material should be prepared in black-and-white (not in color!). Surfaces should have no tone-fills, 
hatchings should be chosen instead (in case of bar-charts, so called pie-charts or maps). In linear graphs the individual 
lines should also be separated by various kinds of hatching or by different markers (triangles, asterisks ... ), but not 
by color. Graphs should have white background (i. e. without background). 
Letters, numbers or symbols should be clear, even and of sufficient size to be still legible on a reduced illustration. 
Freehand or typewritten lettering in the illustration is unacceptable. 
Each figure should be cited in the text. 
Accompanying text to the illustration should contain its title and the necessary explanation of its content. Illustration 
should be intelligible also without reading the manuscript. Ali the abbreviations from the figure should be explained. 
The use of abbreviations in the accompanying text to the illustration is unacceptable. Accompanying texts to 
illustrations should be written in the place of their appearing in the text. 
lf the identity of the patient can be recognized on the photograph, a written permission of the patient for its 
reproduction should be submitted.  
UNITS OF MEASUREMENT 
Should be in accordance with International System of Units (SI).
 
ABBREVIATIONS 
Avoid abbreviations, with the exception of internationally valid signs for units of measurement. Avoid abbreviations 
in the title and abstract. The full term for which an abbreviation stands should precede its first use in the text, 
abbreviation used in further text should be cited in parentheses.
EDITORIAL WORK
The received manuscript is submitted by the editor to three international professional reviewers. After the reviewing 
process, the contribution is sent to the author for approval and consideration of corrections. The final copy is than 
again submitted to the Editorial Office. During the editorial process, the secrecy of the contribution content is 
guaranteed. All the articles are language edited. Author receives in consideration also the first print, but at this stage 
corrigenda (printing errors) only are to be considered. Proofreading should be returned in three days, otherwise it is 
considered that the author has no remarks. 
The journal office strives for rapid editorial process. Authors should adhere to the deadlines set by them in letters; 
otherwise it may happen that the article will be withdrawn from the editorial process. 
Any appeal of the authors deals the Editorial Board of the ZV/SJPH. 
When the manuscript is accepted for publication, the author must assigns copyright ownership of the material to the 
National Institute of Public Health as a publisher. Any violation of the copyright will be legally persecuted. 
ZV/SJPH does not have article processing charges (APCs) nor article submission charges. 
The author receives one copy of the print issue in which the article is published.
Nacionalni inštitut za javno zdravje
Trubarjeva 2, 1000 Ljubljana, Slovenija
zdrav.var@nijz.si
http://www.nijz.si/sl/nijz/revija-zdravstveno-varstvo 
++386 1 2441 543
NAVODILA AVTORJEM
Revija: Zdravstveno varstvo (ZV) ISSN 0351-0026 (tiskana izdaja) / Slovenian Journal of Public Health (SJPH)
ISSN 1854-2476 (elektronska izdaja)
Navodila so v skladu z Uniform Requirements for Manuscripts Submitted to Biomedical Journals. Popolna 
navodila so objavljena v N Engl J Med 1997; 336: 309-15 in v Ann Intern Med 1997; 126: 36-47 in na spletni strani 
http://www.icmje.org.
ETIČNI STANDARDI
Uredništvo sprejema v obdelavo le članke s širšo mednarodno javnozdravstveno tematiko, ki še niso bili in ne bodo 
objavljeni drugje. Dele članka, ki so povzeti po drugi literaturi (predvsem slike in tabele), mora spremljati dovoljenje 
avtorja in založnika prispevka, da dovoli naši reviji reprodukcijo. 
Oddan rokopis morajo prebrati vsi avtorji in se z njegovo vsebino strinjati. 
Raziskave na ljudeh (vključno s človeškimi materiali in osebnimi podatki) morajo biti izpeljane v skladu s Helsinško 
deklaracijo in potrjene s strani nacionalne etične komisije. V izjavi na koncu rokopisa morajo avtorji podati izjavo o 
etiki raziskav na ljudeh, ki mora vsebovati ime etične komisije in referenčno števiko obravnave. Poročanje o raziskavah 
na ljudeh brez potrdila etične komisije zahteva dodatno razlago v poglavju o metodah dela. Na zahtevo Uredništva je 
avtor dolžan predložiti vso dokumentacijo o obravnavi raziskovalne etike njegovega rokopisa. Uredništvo si pridržuje 
pravico, da kontaktira etično komisijo.
Prav tako morajo avtorji, ki poročajo o ljudeh ali posredujejo javnosti njihovo slikovno gradivo, pridobiti dovoljenja 
vseh sodelujočih, da se z vključitvijo v raziskavo strinjajo (v primeru otrok so to starši ali skrbniki). Izjavo o pridobitvi 
teh dovoljenj morajo avtorji podati v poglavju o metodah dela. Uredništvo si pridržuje pravico vpogleda v to 
dokumentacijo.
Raziskave na živalih morajo biti izpeljane v skladu z navodili “Animal Research: Reporting In Vivo Experiments” 
(ARRIVE) in potrjene s strani nacionalne etične komisije. V poglavju o metodah dela in v izjavi na koncu rokopisa 
morajo avtorji podati izjavo o etiki raziskav na živalih z veljavno številko dovoljenja.
V izjavi na koncu rokopisa morajo biti zapisani morebitni finančni ali drugi interesi farmacevtske industrije ali 
proizvajalcev opreme ter inštitucij, povezani z objavo v ZV/SJPH. 
Avtorji morajo na koncu rokopisa zapisati sledeče izjave: 
CONFLICTS OF INTEREST (The authors declare that no conflicts of interest exist.) 
FUNDING (The study was financed by ... ) 
ETHICAL APPROVAL (Received from the… ali opis etičnega vidika raziskave)
PLAGIATI 
Kadar uredništvo ugotovi, da je rokopis plagiat, se rokopis takoj izloči iz uredniškega postopka. Plagiatorstvo 
ugotavljamo s programom za odkrivanje plagiatov CrossCheck plagiarism detection system. 
ELEKTRONSKA ODDAJA PRISPEVKA 
Priporočamo uporabo videoposnetka z navodili za avtorje. Prispevke oddajte v elektronski obliki s pomočjo spletne 
aplikacije Editorial Manager, ki se nahaja na spletnem naslovu http://www.editorialmanager.com/sjph/. V uredništvo 
sprejemamo po pošti le še Izjave o avtorstvu in avtorskih pravicah, ki zahtevajo lastnoročni podpis. Prosimo, da jih 
pošljete hkrati z elektronsko oddajo prispevka na naslov: Nacionalni inštitut za javno zdravje, za revijo Zdravstveno 
varstvo, Trubarjeva 2, 1000 Ljubljana. 
V spletno uredniško aplikacijo se prijavite kot ‘avtor’. Prva prijava zahteva vnos podatkov o avtorju, vse naslednje 
prijave pa le še vnos podatkov za prijavo, ki jih na svoj elektronski naslov prejmete po prvi prijavi v sistem.
Po uspešni prijavi izpolnite vsa zahtevana strukturirana polja. Potrdite izjavo, da vaš prispevek še ni bil objavljen ali 
poslan v objavo kakšni drugi reviji, da so prispevek prebrali in se z njim strinjajo vsi avtorji, da so raziskave na ljudeh 
oz. živalih opravljene v skladu z načeli Helsinško-Tokijske deklaracije oz. v skladu z etičnimi načeli. 
Avtorji, ki v objavo pošiljate raziskovalno delo, opravljeno s pomočjo nekega podjetja, to navedite na koncu rokopisa 
v izjavi o financiranju. 
Polje ‘Comments’ je namenjeno vašim dodatnim razlagam, navedete lahko tudi predlog recenzentov z imeni, nazivi, 
e-naslovi in zaposlitvijo.
Podatke o avtorju in soavtorjih vnesite kar se da natančno in popolno. Naveden naj bo korespondenčni avtor (s polnim 
naslovom, telefonsko številko in elektronskim naslovom), ki bo skrbel za komunikacijo z uredništvom in ostalimi 
avtorji. 
Jezik prispevka je angleščina. Objavljamo izvirne znanstvene članke, sistematične pregledne znanstvene članke, 
metodologije raziskav in vabljene uvodnike. Pri izvirnih, metodoloških in sistematičnih preglednih znanstvenih 
prispevkih morajo biti naslov, izvleček in ključne besede prevedeni tudi v slovenščino.  
Naslov, ključne besede in izvleček se oddajajo dvojezično v angleščini in slovenščini v strukturirana polja. Posebno 
polje za zapis v drugem jeziku obstaja le za izvleček, preostale podatke vnesite v obeh jezikih v ustrezno isto polje. 
Prvi izvleček je vselej v angleškem jeziku (do 250 besed - sistem vam besede sproti šteje), drugi pa v slovenskem 
jeziku (razširjen izvleček - do 400 besed).
Po vnosu strukturiranih podatkov oddajte še priponko - rokopis (od 1 Uvod naprej), ki ne sme zajemati podatkov, ki ste 
jih vnesli že pred tem v strukturirana polja, zlasti ne podatkov o avtorjih. Ime datoteke ne sme vključevati avtorjevih 
osebnih podatkov, prav tako ne imen ustanov, vključenih v pripravo rokopisa. Grafično in slikovno gradivo je kot ves 
rokopis v angleškem jeziku. Vključite ga v besedilo na mesto, kamor le-to sodi in ga opremite z naslovom. Oddate 
torej le en sam dokument, eno priponko. V Wordu uporabite možnost Postavitev strani/Številke vrstic (tako bo na 
robu vsake vrstice dokumenta dodana številka vrstice).
Pri oddaji sledite napotkom, ki vam jih ponuja sistem, pomagate pa si lahko tudi z ‘Editorial Manager’s Tutorial for 
Autors’.
Sistem najbolje deluje, če uporabljate zadnjo različico Acrobata.
Če pri oddajanju rokopisa naletite na nepremostljive težave, se za pomoč obrnite na naslov uredništva: 
zdrav.var@nijz.si.
V nadaljevanju podajamo še nekaj natančnejših napotkov.
ROKOPIS
Besedila naj bodo napisana z urejevalnikom Word for Windows 97-2003. Robovi naj bodo široki najmanj 25 mm. 
Znanstveni članki naj imajo naslednja poglavja: uvod, metode, rezultati, razpravljanje in zaključek. Uvodniki in 
sistematični pregledni članki so lahko zasnovani drugače, vendar naj bo razdelitev na poglavja in podpoglavja jasno 
razvidna iz velikosti črk naslovov. Poglavja in podpoglavja naj bodo številčena dekadno po standardu SIST ISO 2145 in 
SIST ISO 690 (npr. 1, 1.1, 1.1.1 itd.). 
DOLŽINA PRISPEVKOV
Zahtevana dolžina prispevka je za vabljen uvodnik od 250 do 1000 besed, za znanstveni članek (originalni, metodološki 
ali sistematični pregledni) pa od 2000 do 4500 besed s slikovnim gradivom in literaturo vred.
NASLOV IN AVTORSTVO
Naslov v angleškem in slovenskem jeziku naj bo kratek in natančen, opisen in ne trdilen (povedi v naslovih niso 
dopustne). Navedena naj bodo imena piscev z natančnimi akademskimi in strokovnimi naslovi ter popoln naslov 
ustanove, inštituta ali klinike, kjer je delo nastalo. Avtorji morajo izpolnjevati pogoje za avtorstvo. Prispevati morajo 
k zasnovi in oblikovanju oz. analizi in interpretaciji podatkov, rokopis morajo intelektualno zasnovati oz. ga kritično 
pregledati, strinjati se morajo s končno različico rokopisa. Samo zbiranje podatkov ne zadostuje za avtorstvo.
IZVLEČEK IN KLJUČNE BESEDE
Izvleček v angleškem in slovenskem jeziku naj bo pri znanstvenem in metodološkem članku strukturiran in naj ne bo 
daljši od 250 besed v angleščini in 400 besed v slovenščini, izvlečki ostalih člankov so lahko nestrukturirani. Izvleček 
naj vsebinsko povzema in ne le našteva bistvene vsebine dela. Izogibajte se kraticam in okrajšavam. Napisan naj bo 
v 3. osebi. 
Izvleček znanstvenega članka naj povzema namen dela, osnovne metode, glavne izsledke in njihovo statistično 
pomembnost ter poglavitne sklepe (struktura IMRC - Introduction, Methods, Results, Conclusions).
Navedenih naj bo 3-10 ključnih besed, ki nam bodo v pomoč pri indeksiranju. Uporabljajte izraze iz MeSH - Medical 
Subject Headings, ki jih navaja Index Medicus. 
KATEGORIJA PRISPEVKA 
Kategorijo prispevka predlaga z vnosom v ustrezno polje avtor sam, končno odločitev pa sprejme urednik na 
osnovi predlogov recenzentov. Objavljamo izvirne znanstvene članke, metodološke članke, sistematične pregledne 
znanstvene članke in vabljene uvodnike.
REFERENCE 
Vsako navajanje trditev ali dognanj drugih morate podpreti z referenco. Reference naj bodo v besedilu navedene po 
vrstnem redu, tako kot se pojavljajo. Referenca naj bo navedena na koncu citirane trditve. Reference v besedilu, 
slikah in tabelah navedite v oklepaju z arabskimi številkami ((1), (2, 3), (4-7)). Reference, ki se pojavljajo samo v 
tabelah ali slikah, naj bodo oštevilčene tako, kot se bodo pojavile v besedilu. Kot referenc ne navajajte izvlečkov in 
osebnih dogovorov (slednje je lahko navedeno v besedilu). Seznam citirane literature dodajte na koncu prispevka. 
Literaturo citirajte po priloženih navodilih, ki so v skladu s tistimi, ki jih uporablja ameriška National Library of 
Medicine v Index Medicus. Uporabljajte numerično citiranje. Imena revij krajšajte tako, kot določa Index Medicus 
(popoln seznam na naslovu URL: http://www.nlm.nih.gov). 
Navedite imena vseh avtorjev, v primeru, da je avtorjev šest ali več, navedite prvih šest avtorjev in dodajte et al.
 
PRIMERI ZA CITIRANJE LITERATURE 
primer za knjigo: 
1. Premik M. Uvod v epidemiologijo. Ljubljana: Medicinska fakulteta, 1998.
2. Mahy BWJ. A dictionary of virology. 2nd ed. San Diego: Academic Press, 1997.
primer za poglavje iz knjige: 
3. Urlep F. Razvoj osnovnega zdravstva v Sloveniji zadnjih 130 let. In: Švab I, Rotar-Pavlič D, editors. Družinska 
medicina. Ljubljana: Združenje zdravnikov družinske medicine, 2002: 18-27.
4. Goldberg BW. Population-based health care. In: Taylor RB, editor. Family medicine. 5th ed. New York: Springer, 
1999: 32-6.
primer za članek iz revije: 
5. Barry HC, Hickner J, Ebell MH, Ettenhofer T. A randomized controlled trial of telephone management of suspected 
urinary tract infections in women. J Fam Pract 2001; 50: 589-94.
primer za članek iz revije, kjer avtor ni znan: 
6. Anon. Early drinking said to increase alcoholism risk. Globe 1998; 2: 8-10. 
primer za članek iz revije, kjer je avtor organizacija: 
7. Women’s Concerns Study Group. Raising concerns about family history of breast cancer in primary care 
consultations: prospective, population based study. Br Med J 2001; 322: 27-8.
primer za članek iz suplementa revije z volumnom in s številko:
8. Shen HM, Zhang QF. Risk assessment of nickel carcinogenicity and occupational lung cancer. Environ Health 
Perspect 1994; 102(Suppl 2): 275-82.
9. Payne DK, Sullivan MD, Massie MJ. Women’s psychological reactions to breast cancer. Semin Oncol 1996; 23(Suppl 
2): 89-97.
primer za članek iz zbornika referatov:
10. Sugden K, Kirk R, Barry HC, Hickner J, Ebell MH, Ettenhofer T et al. Suicides and non-suicidal deaths in Slovenia: 
molecular genetic investigation. In: 9th European Symposium on Suicide and Suicidal Behaviour. Warwick: 
University of Oxford, 2002: 76.
primer za magistrske naloge, doktorske disertacije in Prešernove nagrade:
11. Bartol T. Vrednotenje biotehniških informacij o rastlinskih drogah v dostopnih virih v Sloveniji: doktorska 
disertacija. Ljubljana: Biotehniška fakulteta, 1998.
primer za elektronske vire:
12. Mendels P. Textbook publishers extend lessons online. Available Sept 23, 1999 from:  
http://www.nytimes.com/library/tech/99/09.
TABELE 
Tabele v angleškem jeziku naj bodo v besedilu prispevka na mestu, kamor sodijo. Tabele naj sestavljajo vrstice in 
stolpci, ki se sekajo v poljih. Tabele oštevilčite po vrstnem redu, vsaka tabela mora biti citirana v besedilu. Tabela naj 
bo opremljena s kratkim angleškim naslovom. V legendi naj bodo pojasnjene vse kratice, okrajšave in nestandardne 
enote, ki se pojavljajo v tabeli. 
SLIKE 
Slike morajo biti profesionalno izdelane. Pri pripravi slik upoštevajte, da gre za črno-beli tisk. Slikovno gradivo naj 
bo pripravljeno: 
• črno-belo (ne v barvah!);
• brez polnih površin, namesto tega je treba izbrati šrafure (če gre za stolpce, t. i. tortice ali zemljevide);
• v linijskih grafih naj se posamezne linije prav tako ločijo med samo z različnim črtkanjem ali različnim 
označevanjem (s trikotniki, z zvezdicami...), ne pa z barvo;
• v grafih naj bo ozadje belo (tj. brez ozadja).
Črke, številke ali simboli na sliki morajo biti jasni, enotni in dovolj veliki, da so berljivi tudi na pomanjšani sliki. Ročno 
ali na pisalni stroj izpisano besedilo v sliki je nedopustno. 
Vsaka slika mora biti navedena v besedilu. Besedilo k sliki naj vsebuje naslov slike in potrebno razlago vsebine. 
Slika naj bo razumljiva tudi brez branja ostalega besedila. Pojasniti morate vse okrajšave v sliki. Uporaba okrajšav v 
besedilu k sliki je nedopustna. Besedila k slikam naj bodo napisana na mestu pojavljanja v besedilu. 
Fotografijam, na katerih se lahko prepozna identiteta bolnika, priložite pisno dovoljenje bolnika. 
MERSKE ENOTE 
Naj bodo v skladu z mednarodnim sistemom enot (SI).
 
KRATICE IN OKRAJŠAVE 
Kraticam in okrajšavam se izogibajte, izjema so mednarodno veljavne oznake merskih enot. V naslovih in izvlečku 
naj ne bo kratic. Na mestu, kjer se kratica prvič pojavi v besedilu, naj bo izraz, ki ga nadomešča, polno izpisan, v 
nadaljnjem besedilu uporabljano kratico navajajte v oklepaju.
UREDNIŠKO DELO 
Prispelo gradivo z javnozdravstveno tematiko mednarodnega pomena posreduje uredništvo po tehnični brezhibnosti 
v strokovno recenzijo trem mednarodno priznanim strokovnjakom. Recenzijski postopek je dvojno slep. Po končanem 
uredniškem delu vrnemo prispevek korespondenčnemu avtorju, da popravke odobri in upošteva. Popravljen čistopis 
vrne v uredništvo po spletni aplikaciji Editorial Manager. Sledi jezikovna lektura, katere stroške krije založnik. Med 
redakcijskim postopkom je zagotovljena tajnost vsebine prispevka. Avtor dobi v pogled tudi prve, t. i. krtačne odtise, 
vendar na tej stopnji upoštevamo samo še popravke tiskarskih napak. Krtačne odtise je treba vrniti v treh dneh, sicer 
menimo, da avtor nima pripomb. 
V uredništvu se trudimo za čim hitrejši uredniški postopek. Avtorji se morajo držati rokov, ki jih dobijo v dopisih, sicer 
se lahko zgodi, da bo članek odstranjen iz postopka. 
Morebitne pritožbe avtorjev obravnava uredniški odbor revije. 
Za objavo članka prenese avtor avtorske pravice na Nacionalni inštitut za javno zdravje kot založnika revije (podpiše 
Pogodbo o avtorstvu in avtorskih pravicah).  Kršenje avtorskih in drugih sorodnih pravic je kaznivo.
Prispevkov ne honoriramo in tudi ne zaračunavamo stroškov uredniškega postopka. 
Avtor dobi izvod tiskane revije, v kateri je objavljen njegov članek.