Kinesiologia Slovenica, 27, 3, 5 -20 (2021), ISSN 1318 -2269 Original article 5 ABSTRACT The ability to visually perceive the movement of athletes or objects they handle is an important skill in sport training practice. The aim of this study was to establish whether basketball coaches are capable of uniformly (congruently) evaluating the relea se time, release angle, ball rotation and overall correctness of a jump shot by observing it with the naked eye. The study included a sample of 13 coaches (six with more and seven with less experience) whose task was to evaluate basketball jump shots made by young basketball players aged 15.1 (± 0.75) years on average from a distance of 4.5 m. The values of Pearson’s correlation coefficients and multiple correlation coefficients (generally above 0.7) as well as Cronbach’s alpha coefficients (above 0.9) and Kendall’s W coefficients of concordance (mostly below 0.1) show a high level of uniformity and congruence in the evaluations of the quality (correctness) of a shots as a whole and also individual shot parameters. Quite a lower level of uniformity and congruence can be established in the evaluations of the consistency of individual parameters. This applies to both groups of more experienced and less experienced coaches (raters). The results allowed an assessment that the levels of reliability of the evaluati on of the group of at least six raters are appropriate for scientific research. Given the low values of the ICC (Inter Class Correlations), only equalling about 0.6, we can say that only one rater did not meet the scientific evaluation criteria. Keywords: basketball shot, judges, accuracy, kinematics 1 University of Ljubljana, Faculty of Sport, Ljubljana, Slovenia IZVLEČEK Vizualno zaznavanje gibanj športnikov ali predmetov s katerimi le -ti upravljajo je ena od pomembnih veščin v praksi športnega treniranja. V pričujoči raziskavi nas je zanimalo ali so košarkarski trenerji sposobni na osnovi opazovanja meta s prostim očesom enotno (skladno) oceniti izmetni čas, izmetni kot, rotacijo žoge in pravilnost meta v celoti. V raziskavi je sodelovalo 13 trenerjev (6 bolj izkušenih in 7 manj izkušenih), ki so ocenjevali mete iz skoka z razdalje 4.5 m, ki so jih izvajali mladi košarkarji stari v povprečju 15,1 (±0,75) let. Vrednosti Personovih korelacijskih koeficientov in koeficientov multiple korelacije (praviloma nad 0.7) ter vrednosti koeficientov Cronbach's alpha (nad 0.9) in Kendallovih koeficientov konkordance W (praviloma pod 0.1) kažejo na visoko stopnjo enotnosti in skladnosti v ocenjevanju kakovosti (pravilnosti) meta v celoti in tudi posameznih parametrov meta. Precej manjšo stopnjo enotnosti in skladnosti lahko ugotovimo pri ocenjevanju doslednosti (konsistenčnosti) posameznih parametrov. To velja tako za skupino bolj izkušenih, kot tudi za manj izkušene trenerje (ocenjevalce). Na osnovi rezultatov ocenjujemo, da so stopnje zanesljivosti ocenjevanja skupine najmanj šestih ocenjevelcev ustrezne za znanstveno proučevanje. Glede na nizke vrednosti ICC (Inter Class Correlations), ki znašajo samo okoli 0.6, lahko rečemo, da en sam ocenjevalec ne zadošča kriterijem znanstvenega ocenjevanja. Ključne besede : košarka, met, sodniki, natančnost, kinematika Corresponding author*: Frane Erčulj, University of Ljubljana, Faculty of Sport, Gortanova 22, 1000 Ljubljana, Slovenia E-mail: frane.erculj@fsp.uni -lj.si Frane Erčulj 1 * Marko Marković 1 Bojan Leskošek 1 Matej Supej 1 INTER - RATER RELIABILITY OF CERTAIN PARAMETERS OF BASKETBALL JUMP SHOT MEDOCENJEVALNA SKLADNOST NEKATERIH PARAMETROV PRI METU NA KOŠ IZ SKOKA Kinesiologia Slovenica, 27, 3, 5 -20 (2021), ISSN 1318 -2269 Evaluation of the basketball jump shot kinematics 6 I NTRODUCTION The ability to visually perceive the movement of athletes or the objects they handle is considered to be an important skill of sport judges/raters and coaches (Heinen, Vinken, & Velentzas, 2012; Williams, 2002). Studies on the evaluation of the technical components of athletes’ movement show that the ability to perceive the actions of other people mainly depends on the observation method, execution of the element and the level of motor and other experience of the raters (Raab, de Oliveira, & Heinen, 2009; Ward, Williams, & Bennett, 2002). In his study of the classification of psycho -motor abilities, Fleischman (1957) already defined nine factors indi rectly associated with the issue of perceiving athletes’ movement depending on the type of activity. One factor of visual perception is considered to be problematic, namely when movement is observed from the perspective of an external, passive observer or rater, without any direct active contact with the observed object. The following questions typically arise: Where is the object heading for, how fast does it move and in which direction does it move? In the case of an athlete rotating an item they are hand ling, the question of the speed of rotation also arises (Williams, 1968). Besides being skilful in imparting information about the technical execution of movement, good coaches should also have the ability to perceive changes in the performance of movement and to objectively evaluate the performance quality (Colby, & Witt, 2000) . The reliability and objectivity of the evaluation can improve if a motor task is assessed by several judges, as is the case in some ‘aesthetic’ sports that are evaluated by a panel of judges (sports gymnastics, rhythmic gymnastics, figure skating, ski ju mping). Yet it is difficult to use such an approach in the sport training practice and/or athlete training process and it is thus less frequent, if not unrealistic. In the mentioned aesthetic sports and those where the execution (technical) component of t he movement is highlighted and the result depends on details of its execution, it is vital that the athlete and their coach obtain accurate and objective information concerning the carrying out of the movement. Such small technical details of the movement and their changes can be crucial for an athlete’s performance or form, although even an experienced and skilful coach with expert knowledge can find it difficult to correctly perceive and evaluate with the naked eye and in real time (Crowley, 2011). Theref ore, different technologies or visual means are often used in such sports to obtain accurate and objective information on an athlete’s condition; moreover, Kinesiologia Slovenica, 27, 3, 5 -20 (2021), ISSN 1318 -2269 Evaluation of the basketball jump shot kinematics 7 they facilitate a prompt or subsequent video -analysis of the movement technique or an even more prof ound kinematic movement analysis. In complex sports and especially team sports where the movement technique is just one of many performance factors, the first (a higher number of competent judges) and second (the use of visual means/technologies) approache s are rarely employed. As a rule, the movement technique is evaluated by coaches through observation and visual perception. This takes place directly during training, i.e. at the exact time an athlete performs the movement. Due to the collective approach i n training and the large number of athletes in a group (team), individual treatment is unfeasible and the coach often cannot focus attention on only one athlete and their movement technique. In our opinion, such an evaluation can become generalised, approx imate or even incorrect, and it is difficult for a coach to evaluate the movement technique details and/or kinematic parameters of an athlete’s movement, which can affect importantly the efficiency of movement as a whole. This is particularly important in some sports (e.g. basketball) as the precision of movement is connected with the precision of launching a projectile (a ball) and the latter with the number of goals as the main performance criterion. A basketball jump shot is considered one of those moto r activities where the ‘ability to score with a launched (thrown) projectile’ is at the forefront. During a throw at the basket the afferent synthesis of visual and kinaesthetic information must offer all elements for the definition of the trajectory (curv e, path) and the force which are both indispensible for the projectile (ball) to reach its destination – the basket (Pistotnik, 2003). Like any complex movement, a basketball jump shot consists of individual segments or phases that make up the movement as a whole. In their studies, authors report different numbers of basketball jump shot phases. Hidrian (2010) divides a jump shot into four phases and Fontanella (2006) into three. Each phase includes specific kinematic parameters which, in the case of a qua ntitative analysis, can provide an objective evaluation of the execution of the phase. When we speak about physical or kinematic parameters, the release height, release angle and release speed are particularly important. Miller and Bartlett (1993) refer to some other factors, presented in Figure 1, which influence the abovementioned parameters. Kinesiologia Slovenica, 27, 3, 5 -20 (2021), ISSN 1318 -2269 Evaluation of the basketball jump shot kinematics 8 Figure 1. Basic factors determining the outcome of a basketball shot (Miller, & Bartlett, 1993) . A problem frequently encountered with young basketball players that influences the precision of a jump shot is a too small release angle (Okazaki, & Rodacki, 2005). This results in a low curve (parabola) of the ball’s flight and small entry angle of the ba ll into the basket which reduces the possibility of the ball penetrating the basket. The release speed is largely associated with the ball flight curve and the release time (Fontanella, 2006). In principle, it is desirable that the release time (the time f rom the moment the player receives/holds the ball to the moment the ball is released) is as short as possible. During play, a player shooting towards the basket is usually obstructed by a defence player and thus only has little time and space to throw the ball. It is generally accepted that the optimal release time for a player to perform a jump shot is less than 0.8 of a second (Rojas, Cepero, & Gutierrez, 2000) because the technically correct execution of a shot requires a minimum lowering of the ball and does not involve superfluous movements slowing down the release phase and facilitating the possibility of blocking the jump shot in the game (Marković, Supej, & Erčulj, 2013). Due to them having less strength in their arms and shoulder girdle, young baske tball players can lower the ball to a greater extent which prolongs the path of the ball and thus the release time. It often occurs that these players, due to their less developed strength, lower the release height or use the low -shot technique (release at chin or even chest level) which is generally less effective than the high technique (Podmenik, Supej, & Erčulj, 2011). Another common problem with young basketball players is inadequate (insufficient) ball rotation that is mainly a consequence of an inapp ropriate release technique or movement in the wrist and/or with the fingers (Palubinskas, 2004). In the training process a basketball player must obtain adequate and objective information about the abovementioned kinematic parameters which considerably aff ect the jump shot technique Kinesiologia Slovenica, 27, 3, 5 -20 (2021), ISSN 1318 -2269 Evaluation of the basketball jump shot kinematics 9 and the precision of a shot at the basket. As a rule, basketball coaches evaluate the jump shot technique during training (in real time) and rely on their own knowledge and visual perception ability. It is for this reason that t his study focused on ascertaining whether basketball coaches are truly capable of uniformly evaluating the release time, release angle and ball rotation solely by observing a jump shot with their naked eye. We assumed that this task is very demanding and e ven too difficult for one coach (evaluator) only. In our opinion he/she is not able to evaluate the jump shot kinematics in a correct way and to give appropriate feedback to the player/shooter. But how many coaches (evaluators) we need for correct evaluati on? That is way, the focus of the study was on whether the coaches are capable of making an accurate and reliable quantitative evaluation of a parameter which affects the efficiency of the jump shot, and whether they are uniform in their evaluation of whic h jump shot was correct or incorrect. Our assumption was also that the adequacy of the evaluation of the correct technical execution of a jump shot is influenced by coaches’ experience. That is way the differences between more and less experienced coaches in evaluation terms were also identified. M ETHODS Subjects The study included 22 young basketball players attending a National Basketball Camp organised by the Basketball Federation of Slovenia in Postojna, Slovenia. The players’ age was 15.1 (± 0.75) years on average and their years of playing 4.68 (± 1.04). All players were included in the regional selection programme run by the Basketball Federation of Slovenia, and were considered talented players of their generation in Slovenia. The study also encompassed 13 coaches who were divided into two groups, namely si x who were more experienced and seven who were less experienced. The coaches’ experience was determined according to the number of years they had spent coaching. The group of more experienced coaches included those who had been coaching for at least 10 yea rs while the less experienced group included those who had been coaching for 5 years or less. The average coaching period of the more experienced coaches was 11.51 (± 4.42) years and that of the less experienced ones 4.13 (± 0.69) years. All coaches/judges also had several years’ experience as basketball players. Kinesiologia Slovenica, 27, 3, 5 -20 (2021), ISSN 1318 -2269 Evaluation of the basketball jump shot kinematics 10 Procedures The experimental situation was a set basketball jump shot after receiving the ball at a distance of 4.5 m from the basket. The players first warmed up with group exercises and made thre e trial jump shots at the basket from the mentioned distance. The coaches/raters observed the trial jump shots but did not evaluate them. Then every player made 10 shots that were all observed and kinematic parameters were evaluated after last shot by each coach. Shots were taken every 5 seconds. The coaches received evaluation forms in advance that contained a detailed description and explanation of each evaluation element and a brief qualitative description (criterion) for an individual quantitative eva luation. They were concisely instructed about the procedure at a meeting held prior to the start of the evaluation. Both groups of coaches were allowed an equal view over the experimental situation. The evaluation parameters included the release time, flig ht curve and ball rotation as these are most frequently used in the literature as the key parameters associated with the success of shooting at the basket and/or are often an issue among young basketball players. The release time was defined as the period from the moment the player receives the ball (contacts the ball after a pass) until the moment the ball leaves his hands during the release phase. The coaches’ task was to evaluate the mentioned parameters of each shot in real time, with the naked eye and by using a five -point scale (from 1 to 5). They applied the following criteria: Table 1. Criteria for the five -point evaluation of the three jump shot parameters . Score Release time Ball rotation Ball flight curve 1 the player executes the shot very slowly the player executes the shot without rotation the player executes the shot with a very low curve 2 the player executes the shot slowly the player executes the shot with minimum rotation the player executes the shot with a low curve 3 the player executes the shot moderately fast the player executes the shot with a moderate rotation the player executes the shot with a medium curve 4 the player executes the shot fast the player executes the shot with a fast rotation the player executes the s hot with a moderately high curve 5 the player executes the shot very fast the player executes the shot with a very fast rotation the player executes the shot with a very high curve Kinesiologia Slovenica, 27, 3, 5 -20 (2021), ISSN 1318 -2269 Evaluation of the basketball jump shot kinematics 11 Besides the above jump shot parameters, using scores from 1 to 5 the coaches also evaluated the quality of the execution (correctness) of each jump shot as a whole (in general). The following criteria were taken into account: Score 1: The player executes every jump shot consistently and technically correctly, has very high scores for the parameters and is very precise. Score 2: The player executes most jump shots consistently and technically correctly, has high scores for the parameters and is fairly precise. Score 3: The player executes jump shots with low consistency and a perfunctory technique, has medium scores for the parameters and is moderately precise. Score 4: The player executes jump shots with very low consistency and a poor technique, has low scores for the parameters and is imprecise. Score 5: The player execut es jump shots inconsistently and technically incorrectly, has very low scores for the parameters and is very imprecise. Moreover, the coaches also evaluated the consistency of every individual’s jump shot execution (with scores from 1 to 4) based on the ev aluation criteria specified in the form (Table 2). Table 2. Criteria for the four -point evaluation of jump shot consistency. Score Release time Ball rotation Ball flight curve 1 the player executes practically every shot with the same or a very similar release time the player executes practically every shot with the same or a very similar rotation the player executes practically every shot with the same or a very similar curve 2 the player executes most shots with the same or a very similar release time the player executes most shots with the same or a very similar rotation the player executes most shots with the same or a very similar curve 3 the player executes most shots with a different release time the player executes most shots with a different rotation the player executes most shots with a different curve 4 the player executes practically every shot with a different release time the player executes practically every shot with a different rotation the player executes practically every shot with a different curve Statistical Analysis The data were processed using the IBM SPSS Statistics 21 program. The basic distribution statistics were calculated (median, interquartile range) for each coach. The efficacy of a coach’s Kinesiologia Slovenica, 27, 3, 5 -20 (2021), ISSN 1318 -2269 Evaluation of the basketball jump shot kinematics 12 evaluation was processed using r it and corrected by the item -total c orrelation (Pearson’s correlation coefficient between the scores of an individual coach and the sum total of the scores of other coaches) and R 2 , a squared multiple correlation between the individual coaches’ results and the sum total of scores of the othe r coaches. The reliability of the scale (the sum total of all coaches’ scores) was established for all coaches together and separately for the groups of more and less experienced coaches based on the two -way mixed model with intra-class correlation (ICC), for individual (ICCsng) and average (ICCavg) scores and Cronbach’s alpha. All ICC coefficients were calculated based on consistency and the absolute agreement model. Partiality was assessed using Kendall’s W coefficient of concordance. R ESULTS Table 1 shows the medians and inter -quartile ranges for the scores of individual coaches for all evaluated basketball jump shot parameters. In most parameters the median scores are mostly close to 3, i.e. the midpoint of the scale (1 – 5), while the medians of consistency are far below (i.e. better than) the midpoint of the scale (1 – 4) in all cases. As a consequence of the strong asymmetry (grouping of scores at a value of 1), the variability (IQR) of the consistency scores is in most cases much lower than th e variability of the scores. There are also some minor differences between the parameters. The scores for the medians of flight curve are generally higher than the ball rotation and general scores, while the release speed scores tend to be the lowest of al l. However, no such differences are found in the consistency score as most medians are around 1.5 irrespective of the parameter. The differences are generally small between the group of less (# 1 – 7) and group of more (# 8 – 13) experienced coaches. Probably the most noticeable difference is the flight curve score medians which tend to be a little higher in the more experienced group than in the less experienced group of coaches. Kinesiologia Slovenica, 27, 3, 5 -20 (2021), ISSN 1318 -2269 Evaluation of the basketball jump shot kinematics 13 Table 3. Medians (Me)and interquartile range (IQR) for an individual coach. Ge neral score Release time score Release time consistency Ball rotation score Ball rotation consistency Curve score Curve consistency Coach Me IQR Me IQR Me IQR Me IQR Me IQR Me IQR Me IQR #1 3.17 1.86 2.93 1.69 1.23 0.65 3.33 1.64 1.38 0.96 3.31 1.75 1.32 0.85 #2 2.87 1.56 2.64 1.70 1.59 0.97 2.85 1.17 1.50 1.00 3.06 1.37 1.67 0.97 #3 3.06 1.26 2.56 1.36 1.57 1.04 3.11 1.21 1.73 0.74 3.65 1.42 1.80 0.83 #4 3.53 1.45 2.88 1.41 1.71 0.89 2.89 1.21 1.76 0.78 3.45 1.17 1.85 0.73 #5 3.31 1.80 2.69 1.78 1.38 0.96 3.31 1.55 1.53 1.18 2.87 1.56 1.57 1.04 #6 3.33 1.69 2.53 1.66 1.36 0.92 3.47 0.67 1.55 0.99 3.19 1.45 1.45 0.99 #7 3.07 1.51 2.73 1.58 1.52 1.05 2.86 1.62 1.57 1.04 3.39 1.31 1.57 1.04 #8 3.25 1.52 2.71 1.49 1.38 0.96 3.33 1.30 2.18 0.61 3.60 1.63 1.76 0.78 #9 3.25 1.48 2.50 1.98 1.64 0.92 2.89 1.24 1.64 0.92 3.28 1.32 1.68 0.85 #10 3.29 1.49 2.92 1.91 1.33 0.88 3.27 1.64 1.95 0.65 3.40 1.62 1.57 1.04 #11 3.37 1.28 2.73 1.54 1.30 0.78 3.16 1.21 1.30 0.78 3.44 1.36 1.05 0.52 #12 2.83 1.31 2.71 1.74 1.41 0.97 3.00 1.58 1.48 1.04 3.60 1.63 1.32 0.85 #13 3.00 1.92 2.82 2.02 1.41 0.97 2.93 1.69 1.35 0.93 3.82 1.32 1.38 0.96 #1-13 3.18 1.55 2.72 1.68 1.45 .92 3.11 1.36 1.61 .90 3.39 1.45 1.54 .88 #1-7 3.19 1.59 2.71 1.60 1.48 .93 3.12 1.30 1.57 .96 3.27 1.43 1.60 .92 #8-13 3.17 1.50 2.73 1.78 1.41 .91 3.10 1.44 1.65 .82 3.52 1.48 1.46 .83 -.02 -.09 .02 .18 -.07 -.01 -.02 .15 .08 -.14 .25 .05 -.14 -.09 The performance of individual coaches expressed as a corrected item -total correlation, r it , and squared multiple correlation, R 2 , is shown in Table 2. r it is calculated as a Pearson correlation coefficient between a single coach’s scores and the sum of the scores of the remaining coaches, while R 2 is a multiple correlation between individual coaches’ scores and the scores of all other coaches. A low value o f R 2 and especially r it indicate the poor performance of a coach. In relation to the scores, most coefficients found in this study are between .7 and .9, with two notable exceptions: coach #6 in the release speed score with r it =.34 and coach #1 in the flig ht curve score with r it =.50. The ball rotation scores generally tend to be somewhat lower than in other parameters. Similarly, R 2 is generally high (above .7); again, coach #6 in release speed score with R 2 =.5 is the most extreme exception. For consistency , as expected due to the heavy grouping of values on the low end of the 1 – 4 scale, both r it and R 2 are much lower than for the scores. Kinesiologia Slovenica, 27, 3, 5 -20 (2021), ISSN 1318 -2269 Evaluation of the basketball jump shot kinematics 14 Table 4. Pearson’s correlation coefficient (r it ) and squared multiple correlation (R 2 ) of coefficients between a single coach’s scores and the scores given by other coaches . General score Release time score Release time consistency Ball rotation score Ball rotation consistency Curve score Curve consistency Coach r it R 2 r it R 2 r it R 2 r it R 2 r it R 2 r it R 2 r it R 2 #1 .72 .74 .73 .84 .10 .69 .62 .83 .51 .85 .50 .70 -.08 .61 #2 .81 .81 .89 .87 .31 .53 .68 .85 .53 .86 .74 .87 .62 .61 #3 .91 .91 .76 .83 .36 .65 .78 .84 .41 .70 .85 .94 .43 .47 #4 .80 .84 .83 .84 .24 .61 .74 .81 .30 .55 .84 .93 .37 .58 #5 .80 .78 .72 .78 .28 .71 .70 .76 .16 .53 .74 .66 .30 .31 #6 .80 .88 .34 .50 .49 .61 .81 .80 .08 .59 .81 .92 .36 .52 #7 .76 .85 .71 .69 .26 .68 .86 .90 .40 .85 .89 .92 .30 .60 #8 .77 .85 .82 .82 .16 .68 .70 .76 -.08 .60 .90 .95 .40 .58 #9 .81 .89 .89 .95 .62 .74 .64 .69 .58 .65 .79 .90 .70 .67 #10 .85 .93 .75 .88 .12 .54 .76 .79 .25 .73 .80 .81 .22 .60 #11 .73 .80 .90 .91 .21 .63 .65 .85 .21 .37 .76 .86 .19 .45 #12 .77 .89 .77 .78 .32 .52 .87 .92 .15 .58 .84 .88 .35 .62 #13 .74 .83 .85 .86 .00 .50 .64 .67 .30 .39 .73 .76 .55 .62 The reliability (and bias) of the coaches’ scores taken together as a scale was evaluated by Cronbach's alpha coefficient of internal consistency, Kendall's coefficient of concordance and intraclass correlation coefficients (ICC) (Table 5). Four types of ICC were used: two for individual measures (i.e. the evaluation of a single coach's ratings) and two for average measures (i.e. the evaluation of the reliability of the mean of the ratings of all coaches/raters). Within each of these ICC groups, one coefficient was calculated with the absolute agreement model (requiring equal ratings of all coaches for each players’ performance for ICC to take the value of 1) and one with the consistency model (requiring the coach es’ score only to be correlated, not equal). Again, parameter scores point to high reliability with Cronbach's alpha and ICC for average measures around .95, while ICC for single measures were much lower (around .6). Kendall's Kinesiologia Slovenica, 27, 3, 5 -20 (2021), ISSN 1318 -2269 Evaluation of the basketball jump shot kinematics 15 W are generally close to zero (i.e. showing no systematic differences between average scores of the judges), although some are statistically significant. The parameter consistency scores show low reliability, with Cronbach's alpha and ICC for average measures mostly in the .6 – .7 range and ICC for single measures in the .1 – .2 range. However, Kendall's W coefficients are similar to those of the parameter scores. Table 5. Reliability of the coaches’ evaluation. General score Release time score Release time consistency Ball rotation score Ball rotation consistency Flight curve Flight curve consistency Kendall's W .09* .05 .08 .09* .17* .17* .20* ICC single (consistency) .63 (.48– .78) .60 (.46– .77) .11 (.04– .25) .54 (.39– .71) .13 (.05– .28) .62 (.47– .78) .18 (.08– .35) ICC average (consistency) .96 (.92– .98) .95 (.92– .98) .62 (.33– .82) .94 (.89– .97) .65 (.40– .83) .96 (.92– .98) .74 (.54– .87) ICC single (agreement) .62 (.47– .78) .60 (.46– .76) .11 (.04– .25) .52 (.37– .70) .11 (.04– .25) .59 (.44– .75) .16 (.07– .31) ICC average (agreement) .96 (.92– .98) .95 (.92– .98) .61 (.33– .81) .93 (.89– .97) .62 (.36– .81) .95 (.91– .98) .70 (.50– .85) Cronbach's alpha .96 .95 .62 .94 .65 .95 .74 Cronbach's alpha #1 -7 .93 .88 .53 .90 .59 .91 .59 Cronbach's alpha #8-13 .92 .94 .34 .87 .37 .93 .67 Note: * – significant at p<.05; values in parentheses are at a 95% confidence interval for ICC D ISCUSSION AND CONCLUSION The results show that the coaches (raters ) were fairly uniform and congruent in their evaluation of the quality (correctness) of a jump shot as a whole and in the evaluation of individual jump Kinesiologia Slovenica, 27, 3, 5 -20 (2021), ISSN 1318 -2269 Evaluation of the basketball jump shot kinematics 16 shot parameters, whereas quite a smaller degree of uniformity and congruence was established in the eval uation of the consistency of individual parameters. Table 3 shows a relatively small range between the highest and lowest scores for the jump shots as a whole (0.54) as well as the scores for the release time (0.43) and ball rotation (0.62). A slightly bi gger range is seen in the scores for ball flight curve (0.95), which is slightly surprising in our estimate. We had expected the coaches to be the most uniform in their evaluation of the ball flight curve since, in our opinion, this parameter is the easies t to evaluate based on visual perception. As regards the size of the interquartile range (Table 3), it can also be established that there were no major differences between the coaches/raters in terms of score variability. The fairly high values of Pearson ’s correlation coefficients and multiple correlation coefficients (with only a few exceptions) confirm the uniformity of the coaches (Table 4). In terms of the value of the abovementioned coefficients, it is on average highest in the evaluation of a jump s hot as a whole (0.72 to 0.93). Similar values of correlation coefficients can also be seen for the scores of release time (with the exception of one coach whose score was quite below average), whereas for the scores of ball rotation (0.62 to 0.92) and espe cially flight curve (0.50 to 0.94) the values are slightly lower. The high values of Cronbach’s alpha coefficients (Table 5) also confirm a high degree of reliability in terms of the evaluation of jump shots as a whole and also all three parameters of the jump shot. For all four variables, the values are close to 1 (0.94 to 0.96). On the other hand, the good quality and reliability of the evaluations are also confirmed by the low values of Kendall’s W coefficients of concordance which exceed 0.1 only in the jump shot curve scores. Something similar can be said for the ICC (InterClassCorrelations) coefficients. On average, a high level of agreement and correlations (consistency) can be confirmed. A different picture emerges regarding the coaches’ uniformity a nd congruence in their evaluation of the consistency of the individual jump shot parameters. It is evident that the coaches were not capable of uniformly evaluating the basketball players’ consistency of shooting at the basket in terms of release time, bal l rotation and ball curve. This is confirmed both by the values of correlation coefficients (Table 4) and all coefficients of evaluation reliability (Table 5) as these were much lower in the consistency evaluation for all three variables. Nevertheless, one should be careful when interpreting the evaluation of consistency because lower coefficient values can also be due to the values and grouping of the consistency scores (mainly scores 1 and 2). Kinesiologia Slovenica, 27, 3, 5 -20 (2021), ISSN 1318 -2269 Evaluation of the basketball jump shot kinematics 17 The results show that it is practically impossible to speak ab out differences between the groups of more and less experienced coaches (raters). A look at Table 1 shows that the differences in the medians (Me) and interquartile range (IQR) between the group of less experienced (# 1 – 7) and more experienced (# 8 – 13) coa ches are minimal. Something similar can be established when speaking about the correlation (Table 4) and reliability (Table 5) of the scores of both groups of coaches. One cannot say that the correlation between the scores of the more experienced coaches i s stronger and the reliability of their scores higher. The differences in the Cronbach’s alpha coefficient are relatively small and inconsistent (sometimes ‘in favour’ of more experienced and sometimes ‘in favour’ of less experienced coaches). Slightly big ger differences are seen in the reliability of the scores of the consistency of individual parameters, but once again sometimes in favour of one and sometimes in favour of the other group of coaches. It is found in the literature that expert judges of gymn astics (with more than 10 years’ experience) are superior to novice judges (with up to 3 years’ experience) because they are more effective at interpreting the biomechanical information available from a gymnast’s body (Abernethy, 1997), have a greater brea dth and depth of knowledge (Ste -Marie, 1999) and can focus on different areas of the body better than novice judges (Bard, Fleury, Carriere, & Halle, 1980). In addition, expert judges are more accurate when recognising form errors (correct body positions) than novice judges (Ste-Marie, & Lee, 1991). This is because they are more able to predict which elements follow during the performance of one or more combinations of elements (Ste -Marie, & Lee, 1991) and can better monitor the speed of performing on vario us apparatus. Unlike gymnastics, the evaluation of a basketball jump shot involves a less complex movement that players and coaches encounter all the time, while also practising and perfecting it. In gymnastics, the gymnasts perform a higher number of tech nical elements, change their technical compositions and add new elements to them. This makes the evaluation of such elements more demanding. The results of our study are perhaps also influenced by the fact that the period of coaching in the group of less e xperienced coaches is also relatively long and that these coaches also have experience as basketball players. Perhaps this was the reason no differences were established between the groups of coaches. The results allows the conclusion that the selected co aches showed a high level of uniformity and congruence in their evaluation of the quality (correctness) of the jump shots as a whole as well as the individual jump shot parameters (release time, ball rotation, ball flight curve). This not only applies to the group of more experienced but also to the group of less experienced coaches (raters). It should be noted that the degree of reliability of the evaluation by a group of Kinesiologia Slovenica, 27, 3, 5 -20 (2021), ISSN 1318 -2269 Evaluation of the basketball jump shot kinematics 18 at least six raters is appropriate for scientific research, unlike the case with just one rater (ICC for single measures is only about .6 – Table 5). Even if the evaluation reliability criteria were milder for the purpose of training, it is still desirable to have a greater number of raters as the evaluation of one rater can also be non -ob jective even where an expert is experienced. This was confirmed by the individual cases in our study. A high level of congruence of scores and, in our opinion, also expertise of the coaches largely guarantees correctness in the sense of the objectivity of an evaluation of a jump shot at the basket and individual jump shot parameters. To confirm this with absolute certainty, the coaches’ scores would have to be compared with objective measurement methods for physical and kinematic parameters of the jump sho t using appropriate technology, which also represents our future aim. Although the coaches are capable of evaluating the jump shots (and throw parameters) of different players congruently and reliably, this is not absolutely true in the case of evaluating different jump shots by the same player. The lower level of congruence and reliability of scores of consistency for jump shots by the same player may reveal that coaches find it difficult to recognise differences in the parameters of different jump shots by the same player. The reason could be that these differences are smaller and coaches are not sensitive enough in their evaluation (visual observation) to detect them. In this case, there is a greater need to use technologies that could provide the coach with objective and reliable information about changes in the parameters of different jump shots by the same player. In training practice, this could ensure a higher degree of stability in implementation of the parameters and probably also greater efficienc y (precision) in shooting at the basket. Despite some specifics and limitations (age of the evaluated athletes, the criteria for selecting the more and the less experienced coaches, the parameters evaluated), the results of the study can, in our opinion, b e generalised to other ball sports (e.g. handball, volleyball, football, American football) where coaches must often evaluate the correctness and precision of the performance of movements associated with the launching of a projectile (a ball). The coaches in these sports evaluate such movements in similar circumstances and in a similar way as in basketball and therefore some of this study’s findings could help them in their work. A cknowledgment The study was conducted in the framework of the research programme “Kinesiology of Monostructural, Polystructural and Conventional Sports” led by Dr. Milan Čoh. It was Kinesiologia Slovenica, 27, 3, 5 -20 (2021), ISSN 1318 -2269 Evaluation of the basketball jump shot kinematics 19 supported by the Slovenian Research Agency and the Foundation for Financing Sport Organisations in Slovenia. Declaration of Conflicting Interests The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article. REFERENCES Abernethy, B. (1997). Movement expertise: A juncture between psychology, theory and practice. Paper presented at the meeting of the Association for the Advancement of Applied Sport Psychology . San Diego, California, June. Bard, C., Fleury, M., Carriere, L., & Halle, M. (1980). Analysis of gymnastic judges visual search. Research Quarterly for Exercise and Sport , 51, 267 – 273. Colby, J., & Witt M. (2000). Defining quality in education . Programme Division, Unicef, 00(2). New Y ork: Unicef. Crowley, M. (2011). Monitoring of physical training events. United States Patent Application Publication, US 2012/ 0029666 A1. Erčulj, F., & Supej, M. (2006). Vpliv utrujenosti na natančnost pri metu na koš iz velike razdalje. [Effect of fatig ue on precision of a throw at the basket from a large distance. In Slovenian.] Šport , 54(4), 22 – 26. Fleischman, E.A. (1957). A comparative study of aptitude patterns in unskilled and skilled psychomotor performances. Journal of Applied Psychology , 41, 263 – 272. Fontanella, J.J. (2006). The physics of basketball . Baltimore, Maryland: Johns Hopkins University Press. Heinen, T., Vinken, P.M., & Velentzas, K. (2012). Judging performance in gymnastics: a matter of motor or visual experience? Science of Gymnastics Journal, 4 (1), 63 – 72. Hidrian, A. (2010). The basketball set shot. Kinesiology II.4 – Human movement analysis . Retrieved June 6, 2012 from: http://www.scribd.com/doc/26028188/Human -Movement -Analysis -Basketball -Set -Shot Marković, M., Supej, M., & Erčulj, F . (2013). Uporaba programa Tracker za kinematično analizo gibanja športnikov. [Use of the Tracker software for kinematic analysis of athletes’ movement. In Slovenian.] Šport , 61(1/2), 19 – 36. Miller, S. & Bartlett, R. (1993). The effects of increased shooting distance in the basketball jump shot. Journal of Sport Sciences , 11 , 285 – 293. Okazaki, V.H.A., Rodacki, A.L.F. (2005). Changes in basketball shooting coordination in children performing wi th different balls. Fédération Internationale d'Education Physique, 75(2), 368 – 371. Palubinskas, E. (2004). The jump shot. Fiba Assist Magazine , 7, 6 – 11. Pistotnik, B. (2003). Osnove gibanja. [Basics of movement. In Slovenian.] Ljubljana: Fakulteta za špor t. Inštitut za šport. Podmenik, N., Supej, M., & Erčulj, F. (2011). Kako se tehnika meta na koš spreminja glede na oddaljenost od koša. [How the throwing technique changes with distance from the basket. In Slovenian.] Šport , 59(3), 179– 184. Raab, M., de Ol iveira, R.F., & Heinen, T. (2009). How do people perceive and generate options. In M. Raab, H. Hekeren, & J.G. Johnson (Eds.), Progress in Brain Research: Vol. 174. Mind and Motion: The Bidirectional Link between Thought and Action (pp. 49 – 59). Amsterdam: Elsevier. Kinesiologia Slovenica, 27, 3, 5 -20 (2021), ISSN 1318 -2269 Evaluation of the basketball jump shot kinematics 20 Rojas, F.J., Cepero, M.O.A., & Gutierrez, M. (2000). Kinematic adjustments in the basketball jump shot against an opponent. Ergonomics , 43(10), 1681– 1660. Ste -Marie, D.M. (1999). Expert novice differences in gymnastic judging: An information -processing perspective. Applied Cognitive Psychology , 13, 269 – 281. Ste -Marie, D.M., & Lee, T.D. (1991). Prior processing effects on gymnastic judging. Journal of Experimental Psychology , 17(1), 126 – 136. Ward, P., Williams, A.M., & Bennett, S.J. (2002). Visual search and biological motion perception in tennis. Research Quarterly for Exercise and Sport, 73 (1), 107– 112. Williams, G.H. (1968). The effects of systematic variation of speed and direction of object flight and of skill and age classifications upon visuo -perceptual judgments of moving objects in three -dimensional space . Toledo: University of Ohio. Williams, M. (2002). Perceptual and cognitive expertise in sport. The Psychologist , 15(8), 416-417.