Zdrav Vestn 2009; 78: 365-70
365
Research article/Raziskovalni prispevek
REPEATABILITY AND RELIABILITY OF A NEW THERMOTACTILE QUANTITATIVE SENSORY TESTING ALGORITHM
PONOVLJIVOST IN ZANESLJIVOST NOVEGA ALGORITMA TERMOTAKTILNEGA KVANTITATIVNEGA SENZORIČNEGA TESTIRANJA
Maruša Klopčič, Miroljub Jakovljevič
University of Ljubljana, Faculty of Health Studies, Physiotherapy department, Poljanska cesta 26 a,
1000 Ljubljana
Abstract
Background
Subjects and methods
Results
Conclusions
Key words
Thermotactile quantitative sensory testing (TQST), as a part of quantitative sensory testing (QST), has been used for quantifying perceived thermal sensory thresholds. The suggested TQST algorithm (method of alternate level stimuli - MALS) tries to overcome the weaknesses of the two most often applied algorithms/methods (thermal sensory limen, method of levels).
The study included30 healthy subjects, 16 men and 14 women. On average, they were 24 years old (SD 2years), 178.1 high (SD 8.8 cm) and weighed 74.8 kg (SD 16.4 kg). Warm sensory perceived thresholds (WSPT), cool sensory perceived thresholds (CSPT) and inter-threshold interval (II) were obtained for further data analysis. Every subject underwent a complex of three measurements twice. For comparison of experimental conditions, independent samples t-test and one-way analysis of variance (ANOVA) were used. Intraclass correlation coefficient (ICC [2.1]), coefficient of variation (CV) and minimal detectable change (MDC95) were used to assess repeatability and reliability of the proposed TQST algorithm.
ICCfor all measures within each measurement series (repeatability) was above 0.45 and CV did not exceed 50 %. When comparing both series (reliability), ICC was above 0.30 for all except two measurements and CV did not exceed 50 %. The difference between two measures within a series was in all cases lower than MDC95 The same was observed when comparing both series. Altogether, 180 measurements of thermotactile sensory perceived thresholds were made and their average duration was 7.1 min (SD 2.3 min).
Good repeatability and poor reliability of MALS was observed. The causes of poor reliability can be explained with various factors. Short duration of a single measurement implies promising application of the suggested algorithm for clinical purposes.
perceived thermal sensory threshold; method of alternate level stimuli; intraclass correlation coefficient; coefficient of variation; minimal detectable change
Izvleček
Izhodišča Termotaktilno kvantitativno senzorično testiranje (TKST) se uporablja za kvantifikacijo
zaznanega temperaturnega senzoričnega praga. Predlagani algoritem TKST (metoda izmeničnih nivojskih dražljajev - MIND) skuša odpraviti slabosti dveh najbolj pogosto uporabljenih metod in njihovih algoritmov (metoda toplotne senzorične meje, metoda nivojev).
Corresponding author / Avtorica za dopisovanje:
Maruša Klopčič, Physiotherapy department, Faculty of Health Studies, University of Ljubljana, Poljanska cesta 26 a, 1000 Ljubljana
366
Zdrav Vestn 2009; 78
Preiskovanci in metode
Rezultati
V razi skavo je bilo vključenih 30 zdravih preiskovancev, 16 moških in 14 žensk, povprečno starih 24 let (SD 2 leti), povprečno visokih 178,1 cm (SD 8,8 cm) in povprečno težkih 74,8 kg (SD 16,4). Za analizo podatkov so bile pridobljene naslednje spremenljivke: topel senzorični zaznani prag (TSZP), hladen senzorični zaznani prag (HSZP) in znotrajpražni interval (ZI). Vsak preiskovanec je dvakrat opravil sklop treh meritev. Za analizo eksperimentalnih pogojev sta bila uporabljena t-test za odvisne vzorce in enosmerna analiza variance (ANOVA), za ugotavljanje ponovljivosti in zanesljivosti algoritma pa intraklasni korela-cijski koeficient (IKK (2,1)), koeficient variacije (KV) in najmanjša zaznana sprememba (NZS9S).
Znotraj posameznih serij merjenja (ponovljivost) so bili IKK za vse meritve nad 0,45, KV pa niso presegali 50 %. Med obema serijama (zanesljivost) so bili IKK za vse meritve, razen dveh, nad 0,30, KV pa niso presegali 50 %. Razlika med primerjanimi vrednostmi znotraj obeh serij je v vseh primerih manjša od pridobljene vrednosti nZS95 Enako velja pri primerjavi obeh serij merjenja. V celoti je bilo izvedenih 180 meritev termotaktilnih senzoričnih zaznanih pragov in njihovo povprečno trajanje je znašalo 7,1 min (SD 2,3 min).
Zaključki
Rezultati so pokazali dobro ponovljivost in slabšo zanesljivost MIND. Vzroki slabše zanesljivosti MIND so pojasnjeni z različnimi dejavniki. Kratek čas trajanja posamezne meritve pri uporabi MIND predstavlja dober obet za uporabo predlaganega algoritma v klinične
namene.
Ključne besede   zaznani temperaturni prag; metoda izmeničnih nivojskih dražljajev; intraklasni koeficient korelacije; koeficient variacije; najmanjša zaznana sprememba
Introduction
Quantitative sensory testing (QST) is a concept encompassing the entire measurement procedure including the devices that register the perceived sensory thresholds (thermal, tactile, vibratory). QST is psychophysical in nature,1, 2 the process is noninvasive,3 and requires cooperation of the patient. Thermotactile quantitative sensory testing (TQST), as a part of QST, allows assessment of unmyelinated and slightly myelinated sensory nerve fibers, which cannot be detected by other forms of neurophysiological tests.4, 5 Algorithms for testing have been developed with a purpose of accelerating the determination of sensory thresholds.3 The methods used for the determination of thermotactile sensory thresholds can be divided into two basic categories: those that involve reaction time (RT) in the measurement, and those that do not. Among the later, the two main methods are the method of limits and the method of levels. Both have their advantages and disadvantages. We have designed an algorithm in order to overcome the existing weaknesses, called the method of alternate level stimuli (MALS), which seeks to eliminate the influence of reaction time and reduce the total time of testing. MALS is a combination of the two most commonly used algorithms. Hence, within a single measurement, we can measure warm and cool perceived thresholds and obtain interthreshold interval (warm-cold difference limen6) by means of alternately applying warm, cold or null stimulus to the subject.
Methods
The study involved healthy volunteers of both sexes, who signed a statement of voluntary participation. The
study was approved by the National Medical Ethics Committee of the Republic of Slovenia.
Experimental conditions
Measurements were made in a quiet and enclosed space with the possibility of adjusting the air temperature. All measurements were performed by the same examiner (M.K.). Experimental conditions were monitored with a combined device for measuring temperature and humidity in the room (see the Measurement instruments subsection). Total duration of each measurement was also measured by the TQST software.
Experimental procedure
Prior to implementation of MALS, subjects spent at least 15 minutes in the room where the testing was carried out in order to adapt to the ambient temperature. In the meantime, body weight and height of individuals was measured, and then the subject waited for further measurements in a recumbent position. After the fifteen-minute period, a probationary period followed with the intention to familiarize the individual with the testing algorithm and all accompanying instructions. After the probationary stimuli to the subject, as recommended by Gruener and Dyck,7 MALS was performed three times in sequence. MALS was started with any type of stimuli (cold, warm or null stimulus) , since that does not affect the course of the algorithm. If the first stimulus was not false, it was 3° C above or below the temperature of the skin of the tested site. The subject was given a sign on when the stimulus was supposed to appear and then he/she had to choose among four possible answers: »warm«, »cold«, »no feeling« and »a feeling, but I do not know what«. Irrespective of the
Klopčič M, Jakovljevič M. Repeatability and reliability of a new thermotactile quantitative sensory testing algorithm
367
answer, the nature of the next stimulus changed (if the first stimulus is warm, then the next one has to be cool and vice versa, whereby randomly a null stimulus can appear, i.e., without temperature change). After each stimulus, the thermode temperature returns to its initial value. As soon as a new change in stimulus type takes place (e.g., going back to warm stimuli), the intensity depends on the previous answer. If the previous answer was »yes« (whichever the modality), a reduction in intensity of stimulus by one half was made; if the previous answer was »no«, the stimulus intensity increased by one half. After the »I do not know what I feel« reply, the same modality was repeated with the same stimulus intensity. The same stimulus intensity within the same modality was also repeated in case the subject answered that he/she felt the opposite modality. In the way described above, same-modality stimulus intensity decreased and increased until the difference between the last two stimuli of the same modality was 0.2° C.
Subjects were lying on their back throughout testing. Sensitivity for thermotactile stimuli was measured on the volar side of the forearm of the dominant upper limb (the location measured from the medial epi-condil of the forearm towards the styloid process of the radius), where the thermode was attatched with an elastic band.
The following variables were obtained for data analysis: warm sensory perceived thresholds (WSPT), cool sensory perceived thresholds (CSPT) and interthresh-old interval (II). Data analysis also included calculation of the average duration of the implementation of the algorithm.
Measurement instruments
To implement the proposed algorithm, we used a thermal stimulator (Mak Elektronik, Škofja Loka, Slovenia) with a square-shaped thermode of size 4x4 cm and a personal computer with associated software, including the dedicated tSensy application.
Statistical methods
Statistical analyses were performed using SPSS for Windows 10.0 software (SPSS Inc., Chicago, IL, USA). Descriptive statistics were calculated for all variables. To assess repeatability and reliability of the TQST algorithm, we used intraclass correlation coefficient (ICC; Model 2, Type 1),8-11 coefficient of variation (CV)11,12 and minimal detectable change (MDC95).1L 13-19 For testing the differences in duration of each measurement, one-way ANOVA was applied to compare the individual measurements within the two series, and t-test for dependent samples was applied to compare the corresponding measurements between the two series and the average values of both series.
Results
The study included 30 healthy subjects, 16 men and 14 women. In 27 subjects, the right upper limb was the dominant one, while two men and one woman were
left-handed. On average, the subjects were 24 years old (SD 2 years), 178.1 cm high (SD 8.8 cm) and weighed 74.8 kg (SD 16.4 kg). The average location of the thermode between the two selected points, measured from the medial epicondil of the forearm, was 16.0 cm (SD 2.0). The values of the coefficients of repeatability/ reliability are presented in Tables 1 and 2.
Table 1. Intraclass correlation coefficient (ICC) and coefficient of variation (CV) to determine repeatability and reliability of MALS.
Tab. 1. Intraklasni koeficienti korelacije (IKK) in koeficienti variacije (KV) za ugotavljanje ponovljivosti in zanesljivosti MIND.
WSPT CSPT II
TSZP HSZP ZI
"Measurements	ICC	CV (%)	ICC	CV (%)	ICC	CV (%)
"Meritve	IKK	KV (%)	IKK	KV (%)	IKK	KV (%)
Repeatability / Ponovljivost merjenja						
11-12	0,76	34,9	0,53	32,1	0,77	28,4
11-13	0,76	32,7	0,53	42,3	0,59	33,8
12-13	0,81	18,3	0,48	34,5	0,80	17,8
21-22	0,77	19,8	0,74	25,1	0,78	18,1
21-23	0,61	34,1	0,71	29,3	0,71	22,7
22-23	0,77	23,0	0,62	35,4	0,76	20,8
Reliability / Zanesljivost merjenja						
11-21	0,81	23,7	0,37	41,7	0,68	31,6
12-22	0,13	28,6	0,37	38,2	0,43	25,8
13-23	0,26	32,1	0,31	44,7	0,48	31,8
1-2	0,45	36,1	0,35	48,7	0,54	33,6
"Meas urements: the first number stands for the series; the second number stands for consecutive measure.
* Meritve: prva številka označuje serijo, druga številka označuje zaporedno meritev.
Table 2. Minimal detectable change (MDC9^ to determine repeatability and reliability of MALS.
Tab. 2. Najmanjša zaznana sprememba (NZS^ za ugotavljanje ponovljivosti in zanesljivosti MIND.
	WSPT		CSPT		II	
	TSZP		HSZP		ZI	
"Measurements	MDC95	Diff.	MDC95	Diff.	MDC95	Diff.
"Meritve	NZS95	Razl.	NZS95	Razl.	NZS95	Razl.
Repeatability / Ponovljivost merjenja						
11-12	0,53	0,11	0,95	0,04	1,55	0,06
11-13	0,31	0,18	0,97	-0,05	0,85	0,14
12-13	0,38	0,07	1,08	-0,09	1,42	0,08
21-22	0,27	0,13	0,23	-0,01	0,48	0,14
21-23	0,20	0,10	0,32	0,02	0,48	0,22
22-23	0,18	0,04	0,33	0,02	0,48	0,08
Reliability / Zanesljivost merjenja						
11-21	0,38	0,04	0,53	-0,29	0,77	0,32
12-22	0,43	0,06	0,65	-0,33	1,26	0,40
13-23	0,13	0,09	0,76	-0,22	0,56	0,40
1-2	0,70	0,07	0,95	-0,28	1,41	0,37
" Measurements: the first number stands for the series; the second number stands for consecutive measure
**MDC95 < difference (Diff.) between two measurements.
" Meritve: prva številka označuje serijo, druga številka označuje zaporedno
meritev.
"" NZS95 < Razlika dveh meritev
Time required for data acquisition was shortened by repeating the measurements. In total, 180 measurements of perceived thermotactile sensory thresholds were carried out. Their average duration was 7.1 min (SD 2.3 min), which means that during this time
368
Zdrav Vestn 2009; 78
warm and cool sensory thresholds were obtained. The average duration of each measurement in the first series was 0.7 minutes longer than in the second series. This difference was statistically significant (p = 0.002).
Discussion
Repeatability of the method of alternate level stimuli
Within both individual series, good repeatability of MALS was found for obtaining WSPT and interthresh-old interval within both series and CSPT within the second series. The values of ICC in these cases were higher than 0.60 and the value of the difference between two measurements was in all cases less than MDC95. We can therefore deduce a 95 % probability that there were no actual differences between the two compared measurements. The third measure of repeatability, CV, ranged between 17.8 % and 42.3 % in all cases. Some authors21 oppose an arbitrarily defined CV acceptance limit (e.g., CV = 10 %) and they advocate that CV should be as low as possible to even talk about good repeatability/reliability. On the other hand, various studies examining repeatability/ reliability of TQST algorithms used arbitrary values of CV to define acceptable repeatability or reliability of measurements. Most authors6, 20 consider the value of CV up to about 40.0 % as an indicator of good repeatability/reliability, and following such definition the TQST algorithm appears to be suitable for clinical use. It seems that despite the initial unfavorable impression about repeatability of MALS, in terms of CV we can talk about good repeatability of our TQST algorithm. The value of CV further showed that the measurements of WSPT and interthreshold interval were more repeatable than CSPT, since CV for CSPT was higher by about 10 percentage points. Poor repeatability of CSPT compared to WSPT and interthreshold interval was confirmed with the values of ICC in the first series of measurements of cool sensory thresholds, which vary around 0.50.
Reliability of the method of alternate level stimuli
Comparison of the corresponding measurements between two series showed better reliability of MALS in obtaining WSPT in comparison with CSPT. The literature review by Chong and Cros6 indicates that different studies reached different conclusions on which perceived thermal threshold (warm or cold) acquired with TQST algorithm proved to be most reliable. Our proposed TQST algorithm proved performed the most reliably in obtaining the interthresh-old interval. However, in general, poor reliability of MALS was found through comparison of individual measurements of both series. ICC values (0.13-0.81) mainly demonstrates poor reliability, especially since the values tend to be below 0.50. Values of MDC95 did not show any differences between the corresponding measurements. With CV, according to the existing lit-
erature,9, 10 we can speak about good reliability of our TQST algorithm, though the CV values were up to 40 %. Similarly, weak reliability of the algorithm was found in comparing the average value of the two series, where the value of one reliability coefficient was low (ICC < 0.50) while the remaining two coefficients of reliability either did not show any change between the two series of measurements (MDC95) or their values were within acceptable limits (CV < 40 %). The values of the coefficients of reliability for interthreshold interval showed the relatively the highest reliability of MALS, but the absolute value of one of the indicators of reliability was still low (ICC = 0.54).
Causes of poor reliability of MALS can be sought in a variety of factors. The factors could be roughly divided into those resulting from the conditions of measurement, the measuring instrument, the proposed TQST algorithm and the differences that emerge between the different coefficients of reliability. It seems that one of the factors affecting reliability of the algorithm is the so-called learning effect.6, 20, 21 Even though the algorithm tries to eliminate it (null stimulus,2, 7, 21 starting the algorithm with a random stimulus), it was observed both in the subjects and in the examiner. They both had a predetermined concept of the algorithm and hence imposed their thoughts and feelings onto the procedure. Despite instructions to the contrary, the subjects were persistently trying to »decipher« the path of the algorithm and were thus constantly burdened by »correctness« of their responses. The examiner was the one who controlled the switch and adjusted the form of the stimulus to the subject's response, even though the AAEM22 recommends both the examiner and the subject to be unaware of the results of the previous measurement in order to achieve reliable results. In our case, the examiner was therefore the one who was primarily burdened with the expectations from previous measurements because of knowing the answers and being the one who pressed the switch to initiate the excitation while gaving the subject a sound signal at the same time. In such a system, training of the examiner is a very important factor, as recommended by the literature.6, 20 On the other hand, the literature23 recommends pre-testing of the subjects before the actual implementation of the measurements at least twice to eliminate the effect of learning. In our case, pre-testing was implemented only once before the actual testing and even then just before the first performance measurement. Our system is a relatively complicated »mixture« of a machine and two people, and as such under the influence of various factors that may have an adverse impact on the reliability of MALS. Such problem might be resolved or at least minimised by removing one person from that system, namely the examiner, which is nowadays readily available in technologically more advanced systems (CASE systems - ComputerAssisted Sensory Examination24-26). In addition to learning and related effects, the shortcomings of the software (the tSensy application) should also be mentioned, which does not allow the application of null stimuli. This does not have a direct
Klopčič M, Jakovljevič M. Repeatability and reliability of a new thermotactile quantitative sensory testing algorithm
369
affect on reliability of MALS since null stimuli have to be »simulated« by the examiner, but it certainly does not contribute to improving the quality of measurement.
Regarding statistical methods, there is known disagreement over proper use and selection of different reliability coefficients. After performing the statistical analysis, we noticed divergent implications from different indicators of repeatability and reliability. Especially in the case of reliability of the TQST algorithm, ICC should probably take priority over the remaining two repeatability/reliability indicators (CV, MDC95). Similar preferential use of ICC has been advocated27 when dealing with reproducibility and reliability of another measurement procedure. It can be seen that in our data, ICC and CV perform in sintony (the lower the value of ICC, the higher the value of CV), while MDC95 did not show significant differences between the two measurements even in the cases where the other two measures showed poor reliability. Meaningful usage of MDC95 is also questionable because it depends on a second coefficient of reliability,13, 19 in our case the ICC.
Average duration of a single measurement
The average duration of each measurement using MALS was 7.1 minutes. Claus and associates6, 20 found that implementation of forced-choice method in one place requires 30 minutes with a normal individual and 40 minutes with a diabetic. In normal subjects, Yarnitsky and Sprecher20 found that the method of levels and the staircase method take a bit longer than the method of limits. Claus and associates2 noted that the method of forced choice took six times longer than the method of limits. It is difficult to compare our average duration with the times obtained in other studies. Such a comparison might be unfair considering the fact that our algorithm is certainly not the most reliable one. However, it is necessary to recognize that the obtained average duration of measurement would be very welcome in clinical practice. Because of that, it seems reasonable to repeate the study of reliability of the proposed TQST algorithm in the future using advanced technical equipment.
Conclusions
In the past, there have been numerous studies oriented primarily to the detection of simple methods for testing and evaluation of sensory thresholds, which could be acquired using computerized sensory systems and would allow quick and precise sensory thresholds.6, 20 Recent research deals with identifying the most suitable (repeatable, reliable, accurate) one among the existing quantitative sensory testing algorithm and estimating the most credible normal sensory thresholds in healthy humans and humans with different pathologies. By designing the MALS algorithms, we looked back to the past in the sense that we wanted to develop an algorithm that would meet all the requirements of high-quality measurements. We found good repeatability of MALS within each series of measurements,
whereby better repeatability of the algorithm was observed for obtaininng the warm sensory threshold and interthreshold interval compared with the observed cool sensory thresholds. When comparing total series of measurements with each other and when comparing corresponding measurements between series, poor reliability of MALS was found. Despite relatively poor reliability, the proposed TQST algorithm showed interthreshold interval to be the most stable among the three variables of interest. The average duration of each measurement using MALS was only 7.1 minutes, which is a good prospect for the application of the proposed algorithm for clinical purposes. It would be useful to repeat the study of repeatability and reliability of MALS, provided that better technical support is assured in order to reduce the human impact on the measurement process.
References
1. Yarnitsky D, Ochoa JL. Warm and cold specific somatosensory systems. Brain 1991; 114: 1819-26.
2. Yarnitsky D. Quantitative sensory testing. Muscle Nerve 1997; 20: 198-204.
3. Shy ME, Frohman EM, So YT, Arezzo JC, Cornblath DR, Giuliani MJ, et al. Quantitative sensory testing: Report of the therapeutics and technology assesment. Neurology 2003; 60: 898-904.
4. Gibbons C, Freeman R. The evaluation of small fibre function - autonomic and quantitative sensory testing. Neurol Clin 2004; 22: 683-702.
5. Nicotra A, Ellaway PH. Thermal perception thresholds: assessing the level of human spinal cord injury. Spinal Cord 2006; 44: 617-24.
6. Chong PS, Cros DP. Technology literature review: Quantitative sensory testing. Muscle Nerve 2004; 29: 734-47.
7. Gruener G, Dyck PJ. Quantitative sensory testing: methodology, applications and future directions. J Clin Neurophysiol 1994; 11: 568-83.
8. Slagle J, Weinger MB, Dinh MT, Brumer VV, Williams K. Assessment of the intrarater and interrater reliability of an established clinical task analysis methodology. Anesthesiology 2002; 96: 1129-39.
9. McGraw KO, Wong SP. Forming inferences about some intraclass correlation coefficients. Psychol Methods 1996; 1: 30-46.
10. Mutlu A, Livanelioglu A, Gunel MK. Reliability of Ashworth and modified Ashworth scales in children with spastic cerebral palsy. BMC Musculoskeletal Disord 2008; 9: 44-51.
11. Atkinson G, Nevill AM. Statistical methods for assessing measurement error (reliability) in variables relevant to sports medicine. Sports Med 1998; 26: 217-38.
12. Defrin R, Shachal-Shiffer M, Hadgadg M, Peretz C. Quantitative somatosensory testing of warm and heat-pain thresholds: the effect of body region and testing method. Clin J Pain 2006; 22: 130-6.
13. Kropmans JB, Dijkstra PU, Stegenga B, Stewart R, de Bont LGM. Smallest detectable difference in outcome variables related to painful restriction of the temporomandibular joint. J Dent Res 1999; 78: 784-9.
14. Maeda S, Sakakibara H. Thermotactile perception thresholds measurement conditions. Ind Health 2002; 40: 353-61.
15. Guyatt GH, Kirschner B, Jaeschke R. Measuring health status: What are the necessary measurement properties? J Clin Epidemiol 1992; 45: 1341-5.
16. Pfennings LEMA, Ploeg HM, van der Cohen L, Polman CH. A comparison of responsiveness indices in multiple sclerosis patients. Qual Life Res 1999; 8: 481-9.
17. Beckerman H, Roebroeck ME, Lankhorst GJ, Becher JG, Bezemer PD, Verbeek ALM. Smallest real difference, a link between reproducibility and responsiveness. Qual Life Res 2006; 10: 571-8.
18. Stratford PW, Binkley J, Solomon P, Finch E, Gill C, Moreland J. Defining the minimum level of detectable change for the Roland-Morris questionnaire. Phys Ther 1996; 76: 359-65.
370
Zdrav Vestn 2009; 78
19. Leggin BG, Shaffer MA, Neuman RM, Williams GR, Iannotti JP. Relationship of the Penn shoulder score with measures of range of motion and strength in patients with shoulder disorders: a preliminary report. J Orthop Sports Phys Ther 2003; 16: 39-44.
20. Chong PS, Cros DP. Quantitative sensory testing equipment and reproducibility studies. AANEM 2004. Dosegljivo na: http://www. aanem.org/documents/qstReview.pdf
21. Ferligoj A, Leskošek K, Kogovšek T. Zanesljivost in veljavnost merjenja. Ljubljana: FDV; 1995.
22. American Association of Electordiagnostic Medicine. Tehnology review: the Neurometer current perception threshold (CPT). Muscle Nerve 1999; 22: 523-31.
23. Golja P, Tipton MJ, Mekjavic IB. Cutaneous thermal thresholds - the reproducibility of their measurements and the effect of gender. J Therm Biol 2003; 28: 341-6.
24. Dyck PJ, O'Brien PC, Kosanke JL, Gillen DA, Karnes JL. A 4, 2, and 1 stepping algorithm for quick and accurate estimation of cutaneous sensation threshold. Neurology 1993; 43: 1508-12.
25. Dyck PJ, Zimmerman I, Gillen DA, Johnson D, Karnes JL, O'Brien PC. Cool, warm, and heat-pain detection thresholds: testing methods and inferences about anatomic distribution of receptors. Neurology 1993; 43: 1500-8.
26. American Diabetes Association. Proceedings of a consensus development conference on standardised measures in diabetic neuropathy: quantitative sensory testing. Muscle Nerve 1992; 15: 1155-7.
27. Lilič A. Ponovljivost in zanesljivost merjenja porazdelitve pritiskov na podplatih z elektronskim pedobarografom med stojo [diplomsko delo]. Ljubljana: Visoka šola za zdravstvo; 2007.
Arrived 2009-03-02, accepted 2009-06-19