A QUESTIONNAIRE FOR RATING HEALTH-RELATED QUALITY OF LIFE 
Slobodan M. JANKOVIĆ1, Nataša BOGAVAC-STANOJEVIĆ2*, Iva MIKULIĆ3,  
Sebija IZETBEGOVIĆ4, Ivana ILIČKOVIĆ5, Dušanka KRAJNOVIĆ6, Enra SULJIC 
MEHMEDIKA4, Refet GOJAK4, Admir MEHIČEVIĆ4, Belma GAZIBERA4, Nevena 
MAHMUTBEGOVIĆ4, Milorad STOJADINOVIĆ1, Nikola JANKOVIĆ7, Sanja K. MILJKOVIĆ8, 
Slađana POPOVIĆ2, Bojan BLIDAREVIĆ9, Snežana MUGOŠA10, Zorana ĐORĐEVIĆ11
1University of Kragujevac, Faculty of Medical Sciences and Clinical Center Kragujevac, Serbia
2University of Belgrade, Faculty of Pharmacy, Department of Medical Biochemistry, 11000 Belgrade, Serbia
3Division of Clinical Pharmacology, Department of Medicine, University Hospital Centre, Zagreb, Crotia
4Clinical Center University of Sarajevo, 71000 Sarajevo, Bosnia & Herzegovina
5Evropa Lek Pharma, Regulatory, Pharmacovigilance and Quality Department, 81000 Podgorica, Montenegro
6University of Belgrade, Faculty of Pharmacy, Department of Social Pharmacy and Pharmaceutical Legislation,  
11000 Belgrade, Serbia 
7Pharmacy Facility Lekofarma, Pharmacy FLOS, 15000 Šabac, Serbia
8Hemofarm A.D., 26300 Vršac, Serbia
9Medical Society for Rational Therapy, 34000 Kragujevac, Serbia
10Faculty of Pharmacy, University of Montenegro, Podgorica, Montenegro
11Clinical Center, Kragujevac, Serbia
Received: Mar 17, 2021
Accepted: Jul 26, 2021
Original scientific article
*Corresponding author: Tel. + 381 641 671 516; E-mail: naca@pharmacy.bg.ac.rs
10.2478/sjph-2021-0035 Zdr Varst. 2021;60(4):260-268
260
VPRAŠALNIK ZA MERJENJE KAKOVOSTI ŽIVLJENJA,  
POVEZANEGA Z ZDRAVJEM
© National Institute of Public Health, Slovenia. 
Janković SM et al. A questionnaire for rating health-related quality of life. Zdr Varst. 2021;60(4):260-268. doi: 10.2478/sjph-2021-0035.
ABSTRACT
Keywords: 
health-related 
quality of life, 
generic instrument, 
questionnaire, 
psychometric 
properties
IZVLEČEK
Ključne besede: 
z zdravjem povezana 
kakovost življenja, 
splošni inštrumenti, 
vprašalniki, 
psihometrične lastnosti
Background. Translations of instruments for measuring quality of life developed in certain, mostly more 
developed, parts of the world usually do not cover regionally specific aspects of health-related quality of life, 
even after transcultural validation. The aim of this study was to develop and validate a reliable questionnaire 
in Serbian, Croatian, Bosnian, and Montenegrin languages suitable for measuring health-related quality of life 
in adults. 
Methods. The study was of a cross-sectional type, assessing the reliability and validity of a newly developed 
questionnaire for measuring health-related quality of life (HRQoL) in adults residing in western Balkan states 
(WB-HRQoL). It was conducted on a sample of 489 adults from Serbia, Croatia, Bosnia & Herzegovina, and 
Montenegro, with a mean age of 52.2±14.4 years and a male/female ratio of 195/294 (39.9%/60.1%). 
Result. The definitive version of the WB-HRQoL scale with 19 items showed very good reliability, with 
Cronbach’s alpha 0.905. The scale was temporally stable, and satisfactory results were obtained for divergent 
and convergent validity tests. Exploratory factorial analysis brought to the surface four domains of health-
related quality of life, namely the physical, psychical, social, and environmental. 
Conclusion. The WB-HRQoL scale is a reliable and valid generic instrument for measuring HRQoL that takes into 
account the cultural specifics of the western Balkan region.
Uvod: Prenos instrumentov za merjenje kakovosti življenja, razvitih v nekaterih, večinoma razvitejših delih 
sveta, zahteva transkulturno validacijo. Kljub temu tovrstni instrumenti običajno ne zajemajo regionalno 
specifičnih vidikov kakovosti življenja, povezanih z zdravjem. Namen študije je bil razviti in potrditi zanesljiv 
vprašalnik v srbskem, hrvaškem, bosansko-hrvaško-srbskem in črnogorskem jeziku, primeren za merjenje 
kakovosti življenja, povezanega z zdravjem odraslih.
Metode: V presečni študiji smo ocenili zanesljivost in veljavnost novo razvitega vprašalnika za merjenje 
kakovosti življenja, povezanega z zdravjem (HRQoL) pri odraslih, ki prebivajo v državah zahodnega Balkana 
(WB-HRQoL). V študijo smo vključili 489 odraslih iz Srbije, Hrvaške, Bosne in Hercegovine ter Črne gore s 
povprečno starostjo 52,2 ± 14,4 let in razmerjem med moškimi in ženskami 195 : 294 (39,9 : 60,1 %).
Rezultat: Dokončna različica lestvice WB-HRQoL z 19 elementi je pokazala zelo dobro zanesljivost (koeficient 
Cronbach alfa 0,905). Lestvica je bila stabilna, tako za divergentne kot konvergentne teste veljavnosti smo 
pridobili zadovoljive rezultate. Raziskovalna faktorska analiza je pokazala štiri področja zdravstvene kakovosti, 
povezane z zdravjem: fizično, psihično, socialno in okoljsko. 
Zaključek: Lestvica WB-HRQoL je zanesljiv in veljaven generični instrument za merjenje HRQoL, ki upošteva 
kulturne posebnosti regije zahodnega Balkana
10.2478/sjph-2021-0035 Zdr Varst. 2021;60(4):260-268
261
1 BACKGROUND
The World Health Organization (WHO) defines quality of 
life as “an individual’s perception of their position in life 
in the context of the culture and value systems in which 
they live and in relation to their goals, expectations, 
standards and concerns “(1). It is essential for physicians 
to understand a disease’s effects on their patients’ quality 
of life, in order to choose optimal therapeutic method 
and follow the patients’ response to this method. There 
are already a number of generic health-related quality 
of life (HRQoL) instruments, developed and validated in 
various countries and languages, which could be used to 
measure the effects of almost any disease on quality of 
life.  Among the most frequently used are the following 
generic instruments (2): (i) the Medical Outcomes Study 
36-Item Short Form; (ii) the Nottingham Health Profile; 
(iii) the Sickness Impact Profile; (iv) the Dartmouth 
Primary care Cooperative Information Project – COOP 
Charts; (v) the Quality of Well-Being – QWB Scale; (vi) 
the Health Utilities Index; (vii) WHOQOL-BREF, and (viii) 
EuroQol Instrument (EQ-5D). Each of the mentioned 
instruments has its advantages and disadvantages that 
are more or less apparent in various circumstances (3). 
Some of these instruments were translated into Serbian, 
Croatian, Bosnian, or Montenegrin and then transculturally 
validated. However the translations of instruments 
developed in other, mostly more developed, parts of the 
world never cover all aspects of quality of life specific for 
western Balkan nations, implying the need to develop of 
new, regionally specific, generic instrument for measuring 
HRQoL (4). A review of the cross-cultural utility of the 
most frequently used instruments showed that translated 
and validated versions are rarely equivalent to the original 
instrument in all aspects: conceptual, item, semantic, 
operational, and measurement equivalence (5).
Although divided into four independent states (Serbia, 
Croatia, Bosnia & Herzegovina, and Montenegro), the 
western Balkan nations that speak languages created 
after the dissolution of the pluricentric Serbo-Croatian 
(or Croato-Serbian) to Serbian, Croatian, Bosnian, and 
Montenegrin share many cultural elements and easily 
understand both the language and cultural context of their 
neighbours (6). We still do not have an original generic 
instrument for measuring HRQoL that was developed in 
Serbian, Croatian, Bosnian, or Montenegrin and that could 
cover all aspects of HRQoL that are recognized as such 
(physical, mental, social function, self-integrity, safety, 
harmony, and spiritual well-being) by inhabitants of the 
western Balkans (6-8). Such an instrument would be of 
great help to medical researchers, practicing physicians, 
and patients from the region, and could lead to more 
precise estimates of quality of life. 
The aim of this study was to develop and validate a 
reliable questionnaire in Serbian, Croatian, Bosnian, and 
Montenegrin suitable for measuring HRQoL in adults.
2 MATERIALS AND METHODS
2.1 Design
The study was designed as multicentric and cross-
sectional. 
2.2 Construction and validation of the new questionnaire
The new questionnaire (WB-HRQoL) was developed 
following the guidelines set by Robert F. DeVellis (8), 
through eight steps: (A) HRQoL in adults was determined 
as object of measurement, being one of the main 
treatment outcomes in both clinical research and clinical 
practice; (B) the item pool with completely new items 
was generated during two brainstorming sessions, one 
week apart; only the authors participated in the sessions 
and creation of the item pool, taking care to cover the 
domains of quality of life encompassed in the majority 
of other generic instruments: physical, psychical, social 
and environmental (members of the group that generated 
the items had the following profiles: two specialists 
of clinical pharmacology, one specialist of  general 
surgery, a professor of Biochemistry, a professor of Social 
Pharmacy, five internal medicine specialists, a specialist 
of infectious diseases, a specialist of epidemiology, three 
community pharmacists, a specialist of anaesthesiology 
and reanimation, a specialist of clinical pharmacy and a 
general practitioner); (C) the Likert’s scale was chosen 
as format of measurement, with the following offered 
answers to statements reflecting quality of life: “I do not 
agree completely”, “I do not agree partially”, “I neither 
agree, nor do not agree”, “I agree partially”, and “I agree 
completely” (the answers were rated from 1 [“I do not 
agree completely”] to 5 [“I agree completely”]); (D) the 
initial pool of items was revised and corrected by the 
three-member expert committee composed of a general 
surgeon, an internal diseases specialist, and a clinical 
pharmacology specialist (these profiles were chosen to 
broadly cover both surgical and non-surgical aspects of 
health, since the scale was intended for use primarily 
at the secondary and tertiary health care levels); (E) in 
order to determine the level of respondents’ socially 
desirable behaviour, one validation item was included in 
the questionnaire that read “I always try to be good to 
other people.”; (F) the initial pool of items was tested 
in each study centre on 3-5 PhD students for clarity and 
comprehension, and after the pilot and a few minor 
changes final versions of the questionnaire were copied 
in Serbian, Croatian, Bosnian, and Montenegrin (the 
investigators from respective language areas adapted the 
items’ formulation to their language preferences), and 
10.2478/sjph-2021-0035 Zdr Varst. 2021;60(4):260-268
262
prepared for reliability testing (Supplementary file); (G) 
evaluating the items  and (H) optimizing the questionnaire 
length were then made as described below. A visual 
analogue scale for assessing HRQoL was also offered to 
the study patients for validation purposes.
Cognitive status of the study subjects was assessed 
by the Mini-Cog screening test (9). For the purpose of 
convergent validation of the WB-HRQoL the study subjects 
were offered short form (26 items) of the World Health 
Organization’s generic questionnaire for measuring HRQoL 
(WHOQoL BREF) in Bosnian (10), as well as Visual Analogue 
Scale (VAS) with a ruler from 0 to 100. Divergent criteria 
validation was made by the 10-item Emotional Regulation 
Questionnaire (ERQ) in Serbian (minimally adapted for 
Croatian, Bosnian, and Montenegrin), which measures two 
strategies of emotional regulation: cognitive reappraisal 
and emotion suppression (11). Permissions to use these 
supplementary questionnaires were granted before the 
start of the study.  
2.3 Data collection 
The final Serbian, Croatian, Bosnian, and Montenegrin 
versions of the both new (WB-HRQoL) and supplementary 
(Mini-Cog, WHOQoL BREF and ERQ) questionnaires were 
tested for reliability on convenient sample of adults 
who were healthy, outpatients, or hospital patients, 
respectively, at the five study centres: Belgrade, 
Kragujevac (Serbia), Zagreb (Croatia), Sarajevo (Bosnia 
& Herzegovina) and Podgorica (Montenegro). The 
surveys took place from September 2019 to July 2020. 
The inclusion criteria were full consciousness, literacy, 
and age over 18. The exclusion criteria were pregnancy, 
lactation, major psychiatric diseases (mood disorders, 
psychoses, mental retardation), psychotropic drugs 
abuse, chronic alcoholism, and emergency conditions. The 
sample of the subjects was of consecutive nature, i.e. all 
subjects who came into the contact with an investigator 
on the survey day (satisfied inclusion and did not have 
exclusion criteria) were offered the questionnaires. 
During the first encounter one copy the questionnaires 
was completed by the investigators who were questioning 
the study subjects, and another copy was given to the 
subjects to complete it next day by themselves, at their 
homes, and send it back to the investigators. The second 
encounter was 15-30 days later, and on that occasion both 
HRQoL and supplementary scales were completed by the 
investigators who were questioning the patients, in order 
to test for temporal stability. The study was approved 
by the Ethics Committees of Clinical Center Kragujevac, 
Serbia (No 01-19-3041, date 19 July 2019.), Clinical Hospital 
Center, Zagreb, Croatia (No 02/21 AG, date 02 July 2019.), 
Clinical Center of Sarajevo University, Sarajevo, Bosnia & 
Herzegovina (No 10-01-58803, date 18 November 2019.), 
Medical Faculty, University of Montenegro, Podgorica, 
Montenegro (No 2240/3, date 15. November 2019.), and 
University of Belgrade – Faculty of Pharmacy, Belgrade, 
Serbia (No 1122/2, date 11 July 2019.). The patients 
signed an informed consent before the study onset and 
were treated with due respect and care, according to the 
principles stated in Declaration of Helsinki.
2.4 Data analysis
The reliability of the WB-HRQoL was tested threefold. 
First, internal consistency was assessed by calculating 
Cronbach’s alpha for the whole questionnaire. Second, 
the questionnaire was randomly split to two halves with 
the same number of questions, and Cronbach’s alpha for 
each of the parts was calculated. Based on the alphas 
of both parts, the number of questions, and the average 
correlation between questions in the two parts, the 
Spearman-Brown coefficient for the questionnaire as a 
whole was calculated by the “prediction” formula (12). 
Third, for each question the mean score and its variances 
were calculated, in order to check for the ability to 
measure the whole extent of possible answers.
Exploratory factorial analysis of the questionnaire was 
made in order to discover principal factors (13). Principal 
axis factoring (14) groups the items of a scale to a smaller 
number of factors describing majority of the variance 
of the responses to the scale items. Factors covering 
maximal variance are kept, and others are discarded. The 
amount of variance covered by each factor is measured 
by its eigenvalue. Assumptions of the factorial analysis 
were tested on the sample by the Kaiser-Meyer-Olkin 
measure of sampling adequacy and by the Bartlett’s test 
of sphericity. Then, the factors were extracted at first 
without rotation, conditional on Eigenvalues >1.0, and with 
the Scree-plot. Second, the referent axes were rotated 
obliquely by the Promax method, and another extraction 
was made, using the same criteria as for the unrotated 
solution. The extracted factors were than given names. 
The calculations were performed by the SPSS statistical 
software, version 18.0.
Content validity of the questionnaire was evaluated by 
an independent panel of three experienced clinicians at 
Clinical Center Kragujevac, Serbia: a general surgeon, an 
internal diseases specialist, and a clinical pharmacology 
specialist.  
The criterion validity was tested by three methods: (1) 
comparing the WB-HRQoL score with the Visual Analogue 
Scale (VAS), (2) comparing the WB-HRQoL score with 
the WHOQoL BREF domains, 1st and 2nd item scores 
(convergent validity testing), and (3) comparing the WB-
HRQoL score with the score of the Emotional Regulation 
Questionnaire (ERQ) (divergent validity testing). The 
correlations between scores of the questionnaires and/
or VAS values were calculated. The calculations were 
10.2478/sjph-2021-0035 Zdr Varst. 2021;60(4):260-268
263
performed by the SPSS statistical software, version 18.0. 
External validity was tested by comparing the WB-HRQoL 
scores between study subjects having at least one chronic 
disease and the healthy subjects. The comparison was 
made by Mann-Whitney U test.
Temporal stability of the WB-HRQoL results was tested 
by second session of completing questionnaires by the 
investigators who repeatedly interviewed the study 
subjects 15-30 days after the first encounter. The study 
subjects were scheduled for the second encounter at the 
end of the first one.
3 RESULTS
The WB-HRQoL questionnaire that was composed of 20 
questions and, after the pilot and minor adjustments, it 
was tested on the sample of 489 study subjects: mean age 
52.2±14.4 years, mean body weight 77.5±15.5 kg, male/
female ratio 195/294 (39.9%/60.1%), education: elementary 
school or less/high school/university=67/240/182 
(13.7%/49.1%/37.2%), living alone/in a family=188/301 
(38.4%/61.6%). There were 278 (56.9%) study subjects 
with at least one chronic noncommunicable disease in the 
sample, and 211 (43.1%) healthy participants. The following 
habits were recorded in the study sample: active smoking/
stopped smoking 150/78 (30.7%/16.0%), drinking alcohol 
195 (39.9%), and drinking coffee 409 (83.6%). The average 
total Mini-Cog score in the sample was 4.0±1.3; there 
were 431 (88.1%) participants with Mini Cog total score ≥3 
(compatible with unimpaired cognition). A Mini-Cog score 
may range from 0 (maximum cognitive impairment) to 5 
(no cognitive impairment at all). 
3.1 Reliability testing
After testing original 20 items from the questionnaire, 
and reviewing results of correlation matrix, mean values, 
variance, skewness and kurtosis of response distributions 
for each of the items, 1 item was removed, leaving final 
version of the WB-HRQoL questionnaire with 19 items. The 
removed item had a low correlation with other items (the 
Spearman’s correlation coefficients were between -0.15 
and 0.15) and with the total score of the remaining 19 
items (the correlation coefficient was 0.092). Cronbach’s 
alpha of the final version with 19 items was 0.905 when 
the scale was rated by the investigators. The intraclass 
correlation coefficient (single measures) was 0.301 
(95% confidence interval 0.273–0.333; F=9.627, df1=486, 
df2=9234, p<0.001). Mean values of responses, standard 
deviations, skewness, and kurtosis for each item are 
shown in the Table 1. 
I do not feel any pain (Q1)
I can perform any physical activity 
without limitation (Q2)
I fall asleep easy and I sleep long enough; 
when I wake up, I feel rested (Q3)
I take care of myself completely (Q4)
My physical condition is excellent (Q5)
I am always in a good mood (Q6)
I feel upset very rarely (Q7)
I feel good in my skin (Q8)
Life is beautiful (Q9)
The world is beautiful (Q10)
My family relations are excellent (Q11)
I am doing excellently at my job (Q12)
I regularly meet my friends and 
enjoy their company (Q13)
I may say that my sex life is very good (Q14)
My relations with colleagues at job are good (Q15)
I am content with my finances (Q16)
I feel completely safe (Q17)
I easily adapt to environmental temperature (Q18)
I do not have problem with breathing 
where I live or work (Q19)
3.350
3.572 
3.523 
4.482
3.382
3.631
3.631
4.040
4.169
3.883
4.306
4.016
4.047 
3.668
4.207
3.511
4.130
3.656
4.190
-.426
-.579 
-.468 
-1.067
-1.350
-.887
-1.547
-1.070
-1.137
-.711
-1.221
-.606
-1.124 
-.708
-1.308
-.426
-.579
-.468
-1.067
1.376
1.311 
1.297 
.964
1.267
1.063
1.128
1.119
1.087
1.145
1.008
1.091
1.040 
1.333
.948
1.199
1.055
1.251
1.124
-1.210
-.774 
-.710 
3.607
-.857
-.373
-.667
.352
1.143
.023
1.783
.763
.816 
-.622
1.626
-.479
.499
-.516
.716
Table 1. Mean values, standard deviation, skewness and kurtosis of responses to items of WB-HRQoL.
The responses are rated from 1 to 5 on a Likert scale (1 [“I do not agree completely”] and 5 [“I agree completely”]); Q – question
Mean response SkewnessStandard deviation KurtosisIndependent variables
10.2478/sjph-2021-0035 Zdr Varst. 2021;60(4):260-268
264
aspects of quality of life are described by the questions 
12, 13, and 15, which belong to the fourth factor. The 
four-factor structure is common to other generic HRQoL 
instruments, due to conceptual similarity of four facets of 
health: physical, psychic, social life, and life in relation to 
environment (15).
3.3 Validity
The questionnaire’s construct validity was assessed and 
endorsed by the panel of experts; a few questions were 
slightly re-phrased by the panel.
Non-parametric correlations between scores of the WB-
HRQoL scale (when it was rated by investigators and by 
the patients themselves) and scores of the ERQ scale 
(when it was rated by investigators and by the patients 
themselves) were calculated to test the divergent criterion 
validity of the WB-HRQoL. Non-parametric correlations 
between scores of the WB-HRQoL scale (when it was rated 
by investigators and by the patients themselves), scores 
of the domains, the 1st and 2nd item of the WHOQoL 
BREF scale (when it was rated by investigators and by 
the patients themselves), and the VAS score were used 
to test convergent criterion validity of the WB-HRQoL. 
Non-parametric correlation was chosen due to non-
normal distribution of majority of the scores. Spearman’s 
correlation coefficients are shown in the Multi-trait, 
multi-method matrix (Table 3). The Spearman correlation 
After dividing the questionnaire by the split-half method 
the Spearman-Brown coefficient for the questionnaire 
as a whole was calculated by the Spearman-Brown 
“prediction” formula, and its value was 0.861. Since 
the Spearman-Brown coefficient did not drop below 0.7 
after the split-half method, the satisfactory reliability of 
the questionnaire is further confirmed (theoretically this 
coefficient may take any value between 0 and 1). When 
the scale was rated by the patients themselves (at the 
first encounter), Cronbach’s alpha was 0.900.
3.2 Factor analysis
The principal axis factoring method was used for 
exploratory factor analysis. The Kaiser-Meyer-Olkin 
measure of sampling adequacy was 0.918 and the 
Bartlett’s test of sphericity was significant (p<0.001). Using 
oblique rotation (Promax), four factors were extracted, 
explaining in total 46.7% of variance. The first factor bears 
4.946 eigenvalues (11.96% of variance), the second 4.818 
(11.66% of variance), the third 5.644 eigenvalues (13.66% 
of variance), and the fourth 3.889 eigenvalues (9.41% of 
variance). The rotated pattern matrix is shown in the 
Table 2. The items 1-3, 5, and 14 belong to the first factor, 
which reflects physical aspects of quality of life. The items 
9-11 belong to factor 2, which describes environmental 
aspects of quality of life, and the items 4, 6-8, and 16-19 
describe the psychical aspects of quality of life. Social 
Q1
Q2
Q3
Q4
Q5
Q6
Q7
Q8
Q9
Q10
Q11
Q12
Q13
Q14
Q15
Q16
Q17
Q18
Q19
.743
.991
.531
.544
.304
 
.291
.377
.565
.391
.573
.767
.587
.450
 
.926
.709
.495
 
.703
.234
.688
Table 2. The rotated pattern matrix of the WB-HRQoL scale. 
An item belongs to the factor where its loading is listed. *Insignificant loadings are not listed for the sake of clarity; Q – question
Factor 1
(Physical aspect of 
quality of life)
Factor 3
(Psychical aspects 
of quality of life)
Factor 2
(Environmental aspects 
of quality of life)
Factor 4
(Social aspects of 
quality of life)
Item
10.2478/sjph-2021-0035 Zdr Varst. 2021;60(4):260-268
265
parametric correlation coefficient, and its administration 
does not assume normal distribution of data within the 
variables.
WB-HRQoL 
score, rated by 
investigators
VAS scale
WB-HRQoL score, 
rated by patients
WHOQoL Bref, 
1st item, rated 
by investigators
WHOQoL Bref, 
2nd item, rated 
by investigators
WHOQoL 
Bref, Physical 
health, rated by 
investigators
WHOQoL Bref, 
Psychic health, 
rated by 
investigators
WHOQoL Bref, 
Social relations, 
rated by 
investigators
WHOQoL Bref, 
Environment, 
rated by 
investigators
WHOQoL Bref, 
1st item, rated 
by patients
WHOQoL Bref, 
2nd item, rated 
by patients
WHOQoL Bref, 
Physical health, 
rated by patients
WHOQoL Bref, 
Psychic health, 
rated by patients
WHOQoL Bref, 
Social relations, 
rated by patients
WHOQoL Bref, 
Environment, 
rated by patients
ERQ score, rated 
by investigators
ERQ score, rated 
by patients
1.000 
 
.537**
.857** 
.519** 
 
.524** 
 
.709** 
 
 
.714** 
 
 
.641** 
 
 
.576** 
 
 
.518** 
 
.448** 
 
.620** 
 
.593** 
 
.616** 
 
.583** 
 
.264** 
.310**
 
 
1.000
.490** 
.523** 
 
.575** 
 
.595** 
 
 
.521** 
 
 
.426** 
 
 
.237** 
 
 
.524** 
 
.583** 
 
.580** 
 
.411** 
 
.371** 
 
.259** 
 
-.005 
-.068
 
 
1.000 
.477** 
 
.503** 
 
.650** 
 
 
.632** 
 
 
.639** 
 
 
.549** 
 
 
.607** 
 
.517** 
 
.656** 
 
.656** 
 
.693** 
 
.646** 
 
.269** 
.293**
 
 
 
1.000 
 
.539** 
 
.483** 
 
 
.571** 
 
 
.466** 
 
 
.456** 
 
 
.669** 
 
.483** 
 
.411** 
 
.412** 
 
.410** 
 
.482** 
 
.115* 
.128*
 
 
 
 
 
1.000 
 
.589** 
 
 
.429** 
 
 
.376** 
 
 
.282** 
 
 
.483** 
 
.769** 
 
.594** 
 
.344** 
 
.345** 
 
.261** 
 
.022 
.036
 
 
 
 
 
 
 
1.000 
 
 
.689** 
 
 
.555** 
 
 
.442** 
 
 
.501** 
 
.529** 
 
.869** 
 
.588** 
 
.441** 
 
.381** 
 
.116* 
.156**
 
 
 
 
 
 
 
 
 
 
1.000 
 
 
.592** 
 
 
.564** 
 
 
.503** 
 
.375** 
 
.558** 
 
.840** 
 
.530** 
 
.545** 
 
.228** 
.201**
 
 
 
 
 
 
 
 
 
 
 
 
 
1.000 
 
 
.476** 
 
 
.486** 
 
.367** 
 
.431** 
 
.489** 
 
.829** 
 
.508** 
 
.119** 
.205**
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1.000 
 
 
.433** 
 
.247** 
 
.299** 
 
.457** 
 
.413** 
 
.843** 
 
.334** 
.363**
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1.000 
 
.571** 
 
.527** 
 
.551** 
 
.565** 
 
.515** 
 
.079 
.113
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1.000 
 
.588** 
 
.400** 
 
.427** 
 
.345** 
 
-.010 
.077
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1.000 
 
.591** 
 
.471** 
 
.386** 
 
.062 
.099
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1.000 
 
.572** 
 
.574** 
 
.215** 
.205**
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1.000 
 
.562** 
 
.146* 
.225**
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1.000 
 
.312** 
.367**
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1.000 
.832**
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1.000
Table 3. Multi-trait, multi-method correlation matrix (non-parametric Spearman’s coefficients).
**. p<0.01; *. p<0.0
WB-
HRQ-oL 
score, 
rated by 
investi-
gators 
VAS 
scale 
WB-
HRQoL 
score, 
rated by 
patients
WHO-
QoL 
Bref, 1st 
item, 
rated by 
investi-
gators
WHO-
QoL 
Bref, 
2nd 
item, 
rated by 
investi-
gators
WHO-
QoL 
Bref, 
Physical 
health, 
rated by 
investi-
gators
WHO-
QoL 
Bref, 
Psychic 
health, 
rated by 
investi-
gators
WHO-
QoL 
Bref, 
Social 
rela-
tions, 
rated by 
investi-
gators
WHO-
QoL 
Bref, 
Environ-
ment, 
rated by 
investi-
gators
WHO-
QoL 
Bref, 1st 
item, 
rated by 
patients
WHO-
QoL 
Bref, 
2nd 
item, 
rated by 
patients
WHO-
QoL 
Bref, 
Physical 
health, 
rated by 
patients
WHO-
QoL 
Bref, 
Psychic 
health, 
rated by 
patients
WHO-
QoL 
Bref, 
Social 
rela-
tions, 
rated by 
patients
WHO-
QoL 
Bref, 
Environ-
ment, 
rated by 
patients
ERQ 
score, 
rated by 
investi-
gators
ERQ 
score, 
rated by 
patients
Item
coefficient is a measure of the strength and direction of 
association between two variables, based on the rank 
of individual values instead of actual values; it is non-
When the WB-HRQoL scores were compared between 
healthy study subjects and those having at least one 
chronic disease, they were significantly higher in the 
former: 79 [16] vs. 73 [19] (Mann Whitney U test, p<0.001). 
This confirms the instrument’s discriminative ability.  
3.4 Temporal stability
The WB-HRQoL scale showed excellent temporal stability: 
when the rating (by the investigators) was repeated on the 
same patients 15-30 days later, the correlation between 
the scores (Spearman’s coefficient) was 0.842 (p<0.001). 
Cronbach’s alpha after the repeated rating was 0.898.
4 DISCUSSION 
4.1 Main finding of this study 
The definitive version of the WB-HRQoL scale with 19 
items showed very good reliability, with Cronbach’s 
alpha at 0.905. The scale was also temporally stable, 
and satisfactory results were obtained for divergent 
and convergent validity tests. Factor analysis showed 
four domains of HRQoL, physical, psychical, social, and 
environmental. This questionnaire can help identify 
patients with a low quality of life, and it can be used for 
better therapy monitoring and improving quality of life.
4.2 What is already known on this topic 
Physicians need insight into the effects of a disease on 
their patients’ quality of life to choose the best therapeutic 
method and follow the patient’s response thereto. There 
is no original generic instrument for measuring HRQoL 
developed in Serbian, Croatian, Bosnian, or Montenegrin 
that could cover all aspects of regional understanding 
of HRQoL. However, the translations of instruments 
developed in other, mostly more developed, parts of the 
world never cover all aspects of quality of life specific for 
western Balkan nations. 
4.3 What this study adds
Physical domain is present in almost all generic instruments 
for measuring HRQoL, and is valued even in circumstances 
when other domains are put aside, like emergency care 
situations (16). Key components of the physical aspect of 
quality of life are absence of pain and other discomforts, 
and the ability to have autonomous physical activity (17). 
The physical domain of our questionnaire has items that 
cover these key components, but also has two additional 
items covering quality of sleep and sex life, present in 
other quality of life scales like the WHOQoL BREF, too 
(10). Because of the major influence that items measuring 
pain, discomfort, and ability to move or work have on 
quality of life, some instruments minimize the effects 
of sleep and sex life (18); this was not the case with our 
10.2478/sjph-2021-0035 Zdr Varst. 2021;60(4):260-268
266
questionnaire, which showed that in this domain questions 
about influence of sleep, sex life, mobility, and absence 
of pain or discomfort are equally important (similar 
loadings), underlining the importance of all these aspects 
in regional context.  The mean values of the responses 
on items reflecting physical aspects of quality of life in 
our study were close to mean of the range of possible 
answers, variance was of acceptable size, and skewness 
and kurtosis suggested minor (but significant) detachment 
from normal distribution (see Table 1). Such results show 
good discriminative ability of the items from physical 
domain, and suggest their capacity to capture both mild, 
moderate, and severe decrease in health-related quality 
of life (8). 
Main items within the psychic domain are addressing 
mood, anxiety, and overall feeling of balance, well known 
facets of mental health. However, it is interesting that 
this domain encompassed also certain items that are 
traditionally linked to environmental aspect: adjustment 
to environmental temperature, air, and feeling of financial 
or safety in general. This is probably caused by specificity 
of western Balkan countries, which are extremely low-trust 
societies, where inhabitants either lack or have minimal 
trust in their governments and fellow citizens (19, 20). In 
such cultural contexts people do not expect that their 
environment will be taken care of by institutions or other 
individuals, therefore HRQoL will depend on how they 
feel about their personal ability to cope with supposedly 
polluted air, inappropriate ambient temperature, crime, or 
financial instability. In other words, individual perception 
of the environmental aspect of quality of life will depend 
more on resiliency and mood than on the real situation in 
the external world.   
Citizens of western Balkan societies frequently have a 
feeling of isolation (both personal and national) and are 
reluctant to admit their psychological difficulties, because 
they are aware of their stigmatizing potential (21). Such 
an attitude leads to perception of the world and life as 
something independent of the person itself (actually 
environmental), i.e. they principally see the world and life 
in general as beautiful or not, according to the prevailing 
public opinion. Accordingly, the items “life is beautiful 
“and “the world is beautiful “in our questionnaire slipped 
from psychical (what was expected when the items were 
constructed) to the environmental domain. Traditionally, 
also the social item “family relations” was attached 
to the environmental domain, probably reflecting 
emerging feeling of alienation within families; however, 
this speculation has yet to be confirmed. Three items 
within the social domain are oriented towards relations 
with friends and colleagues at work. The participants 
perceive two main areas of social interaction – work and 
entertainment.
From demographic and data about habits within the study 
sample we could see that smoking, drinking alcohol and 
coffee were widespread and could have influenced the 
study participants’ perception of HRQoL. Although the 
process of Europeanization is taking place in countries 
of Western Balkans, and has already led to a decrease 
in alcohol consumption (22, 23), these habits are still 
following patterns from the past and making up one of the 
target population’s important specifics, further stressing 
the need for region-specific instruments for measuring 
HRQoL.
Although the convergent criteria validation of the WB-
HRQoL showed very good results (see Table 3), divergent 
validation with the ERQ was less successful, since scores of 
the WB-HRQoL and ERQ showed a weak, but still significant 
correlation. However, significant correlation was also 
found between scores of the social and environmental 
domains of the WHOQoL BREF and that of the ERQ, while 
VAS, the 1st and 2nd items of the WHOQoL BREF were 
not correlated with the ERQ score. Probably the choice 
of ERQ for divergent validation purpose was not the best 
option; the ERQ rates how the control a person has over 
the expression of their emotions, which could affect social 
and environmental relations, and therefore quality of life. 
Since the WB-HRQoL does have social and environmental 
domains, a certain degree of correlation with the ERQ 
score is inevitable. 
4.4 Limitations of this study
The main limitation of this study was use of convenience 
instead of a random sample for validating the questionnaire, 
which reduced potential to generalize conclusions to the 
whole studied populations. Second, in order to validate 
a scale, it is beneficial to include participants having 
whole spectrum of a phenomenon, which is measured, if 
possible, confirmed by a “gold standard”. However, the 
“gold standard “for HRQoL is still lacking, creating a space 
for uncertainty.”
5 CONCLUSION 
The WB-HRQoL scale is a reliable and valid generic 
instrument for measuring HRQoL, including the four 
aspects: physical, psychical, social, and environmental. 
It could be used not only for research purposes, but for 
following individual patients’ quality of life, which is 
one of the important outcomes of treatment in clinical 
practice.
CONFLICT OF INTEREST
None of the authors has any conflict of interest in regards 
to the content of this manuscript.
FUNDING
This study was financially supported in part by grant 
No 175007 from the Ministry of Education, Science and 
Technological Development, Republic of Serbia.
ETHICAL APPROVAL
The study was approved by the Ethical Committees of the 
authors’ institutes. 
ACKNOWLEDGEMENTS
The authors of this study are grateful to the authors of 
the Mini-Cog, WHOQoL BREF, and ERQ questionnaire for 
their permission to use them for screening and validation 
purposes.
REFERENCES
1. The World Health Organization Quality of Life assessment (WHOQOL): 
position paper from the World Health Organization. Soc Sci Med. 
1995;41(10):1403-9. doi: 10.1016/0277-9536(95)00112-k.
2. Coons SJ, Rao S, Keininger DL, Hays RD. A comparative review of generic 
quality-of-life instruments. Pharmacoeconomics. 2000;17(1):13-35. 
doi: 10.2165/00019053-200017010-00002.
3. Longworth L, Yang Y, Young T, Mulhern B, Hernández Alava M, Mukuria 
C, et al. Use of generic and condition-specific measures of health-
related quality of life in NICE decision-making: a systematic review, 
statistical modelling and survey. Health Technol Assess Winch Engl. 
2014;18(9):1-224. doi: 10.3310/hta18090.
4. Matanov A, Giacco D, Bogic M, Ajdukovic D, Franciskovic T, Galeazzi 
GM, et al. Subjective quality of life in war-affected populations. BMC 
Public Health. 2013; 13:624. doi: 10.1186/1471-2458-13-624.
5. Schmidt S, Bullinger M. Current issues in cross-cultural quality of life 
instrument development. Arch Phys Med Rehabil. 2003;84(4, Suppl 
2):S29-34. doi: 10.1053/apmr.2003.50244.
6. Bugarski R. Language, identity and borders in the former Serbo-
Croatian area. J Multiling Multicult Dev. 2012;33(3):219–35. 
7. Kuspinar A, Mayo NE. A review of the psychometric properties of 
generic utility measures in multiple sclerosis. Pharmacoeconomics. 
2014;32(8):759-73. doi: 10.1007/s40273-014-0167-5.
8. Kagawa-Singer M, Padilla GV, Ashing-Giwa K. Health-related quality of 
life and culture. Semin Oncol Nurs. 2010;26(1):59-67. doi: 10.1016/j.
soncn.2009.11.008.
9. DeVellis RF. Scale development, theory and applications. 2nd ed. 
London: SAGE publications, 2003. 
10. Borson S, Scanlan JM, Chen P, Ganguli M. The Mini-Cog as a screen for 
dementia: validation in a population-based sample. J Am Geriatr Soc. 
2003;51(10):1451-4. doi: 10.1046/j.1532-5415.2003.51465.x.
11. Skevington SM, Lotfy M, O’Connell KA, WHOQOL Group. The World 
Health Organization’s WHOQOL-BREF quality of life assessment: 
psychometric properties and results of the international field trial: 
a report from the WHOQOL group. Qual Life Res Int J Qual Life 
Asp Treat Care Rehabil. 2004;13(2):299-310. doi: 10.1023/B:QU
RE.0000018486.91360.00.
12. Popov S, Janičić B, Dinić B. Validation of the Serbian adaptation of the 
Emotion Regulation Questionnaire (ERQ). Primenj Psihol. 2016;9(1):63-
81. 
10.2478/sjph-2021-0035 Zdr Varst. 2021;60(4):260-268
267
13. Streiner DL, Norman GR. Health Measurement Scales - a practical 
guide to their development and use. 1st ed. Oxford: Oxford University 
Press, 2008. 
14. Badia X, Arribas F, Ormaetxe JM, Peinado R, de Los Terreros MS. 
Development of a questionnaire to measure health-related quality of 
life (HRQoL) in patients with atrial fibrillation (AF-QoL). Health Qual 
Life Outcomes. 2007;5:37. doi: 10.1186/1477-7525-5-37.
15. Hooper D. Exploratory factor analysis. In: Chen H, ed. Approaches to 
quantitative research – theory and its practical application: a guide to 
dissertation students. Cork, Ireland: Oak Tree Press, 2012. 
16. Coons SJ, Rao S, Keininger DL, Hays RD. A comparative review of generic 
quality-of-life instruments. Pharmacoeconomics. 2000;17(1):13-35. 
doi: 10.2165/00019053-200017010-00002.
17. Dresden SM, McCarthy DM, Engel KG, Courtney DM. Perceptions 
and expectations of health-related quality of life among geriatric 
patients seeking emergency care: a qualitative study. BMC Geriatr. 
2019;19(1):209. doi: 10.1186/s12877-019-1228-6.
18. Connell J, O’Cathain A, Brazier J. Measuring quality of life in mental 
health: are we asking the right questions? Soc Sci Med. 2014;120:12-20. 
doi: 10.1016/j.socscimed.2014.08.026
19. Vozel D, Steiner N, Božanić Urbančič N, Mladenov D, Battelino S. 
Slovenian cross-cultural adaptation and validation of health-related 
quality of life measures for chronic otitis media (COMQ-12), vertigo 
(DHI, NVI) and tinnitus (THI). Zdr Varst. 2020;59(3):120-27. doi: 
10.2478/sjph-2020-00016. 
20. Krastev I. The Balkans: democracy without choices. J Democr. 
2002;13(3):39-53. 
21. Dimitrov M. Balkan countries most vulnerable to ‘fake’ news, report. 
Balkan Insight. Accessed Aug 25, 2020 at: https://balkaninsight.
com/2018/03/30/report-balkan-countries-most-vulnerable-to-
adverse-effects-of-fake-news-03-29-2018/. 
22. Jones L, Kafetsios K. Assessing adolescent mental health in war-
affected societies: the significance of symptoms. Child Abuse Negl. 
2002;26(10):1059-80.  
23. Tresa E, Benmarhnia T, Clemens T, Burazeri G, Czabanowska K. 
Europeanization process impacts the patterns of alcohol consumption 
in the Western Balkans. Eur J Public Health. 2018;28(3):516-521. doi: 
10.1093/eurpub/ckx175.
10.2478/sjph-2021-0035 Zdr Varst. 2021;60(4):260-268
268