članki
Professional Article
INVESTIGATING THE ABILITY OF RADIOGRAPHERS TO RECOGNISE TYPICAL NON ACCIDENTAL INJURY FRACTURES IN CHILDREN
Moore David, david.moore.6@ucdconnect.ie, dr. Davis Mihaela, michaela.davis@ucd.ie,
School of Medicine and Medical Science, University College Dublin, Belfield, Dublin 4; Starc Tina, tina.starc@zf.uni-lj.si,
Faculty of Health Sciences, Zdravstvena pot 5, 1000 Ljubljana, Slovenia
ABSTRACT
Introduction: Child abuse is often underreported by the general population and healthcare professionals for a variety of reasons and, therefore, all avenues to improve the discovery and prevention of child abuse should be explored. Radiographers are well positioned to identify child abuse and in particular Non Accidental Injuries (N.A.I.); having a wealth of experience in image acquisition and visualisation.^ the Republic of Ireland Health Care professionals are legally required to report suspicious cases as Designated Officers.
Aim: Our aim was to measure radiographers' ability to recognise non-accidental injury fractures in children comparing radiographers from two different European Countries.
Method: 22 radiographers working in hospitals in the Republic of Ireland and Slovenia viewed 26 plain radiographic images (13 NAI fractures/13 accidental fractures) and rated their confidence on a scale from 1 to 6 that a fracture was either accidental (1-3) or non-accidental (4-6).The images were viewed using ViewDex software on a laptop calibrated to DICOMgreyscale standard display function.Viewing conditions were standardised and optimised. An ROC curve was plotted and sensitivity, specificity and area under the curve (AUC) were calculated for each radiographer.
Results: The results demonstrated a difference in values which AUCwas statistically significant (P=0.0111). Conclusion: Further research would be beneficial with more images, full skeletal surveys, and paediatric radiologist(s) as a gold standard for comparison.
Key words: non-accidental injury, children, radiographer
INTRODUCTION
Radiographers are well positioned to recognise non-accidental injury in children as they may be the first person to spend time alone with a child who may use this opportunity to confide in them. Radiographers may discover hidden injuries when clothing is removed for imaging, and they are the first to view the diagnostic image of the child which might be the only indication of abuse (Davis and Reeves, 2009, Rigney and Davis, 2004, Hogg et al1999). The radiographic image is an important aspect of the
NAI investigative processas is its interpretation.Research conducted by McNulty et al. (2011) suggests collaboration between radiographers and junior doctors reduces errors in image interpretation.Furthermore, previous research conducted by Brealey et al. (2005) provides evidence that, with appropriate education and training, the accuracy of radiographers in interpreting plain x-ray images is comparable to that of radiologists. The researchers were interested in exploring radiographers' ability to recognisenon-accidental injury in children's radiographs. The aim of our research was to investigate the ability of radiographers in the Republic of Ireland and Slovenia to recognise typical non-accidental injury fractures in children, and compare both groups.
METHODS Pilot
Before the main study commenced a pilot was performed using academics in one ofthe diagnostic imaging departments as a sample.A number of images were excluded following the pilot due to poor image quality or an unclear fracture site.Some images were tagged with an age to indicate whether the patient was ambulatory, allowing better differentiation between toddlers' fractures and spiral fractures.
For the main study; a random sample of diagnostic radiographers working within paediatric Diagnostic Imaging departments was selected from each country. The sample of 22 radiographers working in hospitals in the Republic of Ireland and Slovenia viewed 26 digital x-ray images of fractures, and rated their confidence on a scale from 1 to 6 that the aetiology of each fracture was either accidental (1-3) or non-accidental (4-6).
Participants
The participants in this study were qualified radiographers who worked in large paediatric referral centres with Accident and Emergency departments in the Republic of Ireland or Slovenia. The inclusion criteria were that all imaged paediatric patients at least once every two days on average. A random selection was utilised based on the sample. The Slovene group had a mean experience of 14.95 years and the Irish group had a mean experience of 12.64 yearsworking as radiographers.
članki
Images
All the images used in the study came from the radiographic teaching file within the Universities and a large paediatric teaching hospital. The participantswere presentedwith 8 previously diagnosed accidental injury images including metaphyseal lesions, rib fractures, occipital compression fractures and spiral fractures of the femur.A further8 images with previously diagnosed non-accidental injury includingColles fractures, greenstick fracture of the radius, clavicle fractures and elbow fractures were also presented. An additional 5 images were selected at random from each group of 8 images and were given minor alterations (that did not alter the fracture site).Each radiographer viewed all of these images.
Image Display Conditions
The images were viewed in random order using ViewDex software(Borjesson et al. 2005) on the same laptop calibrated to DICOMgreyscale standard display function using VeriLum software (Image Smiths, Inc) and luminance pod.Quality checks performed throughout the work with the Society of Motion Picture and Television Engineers test pattern(Society of Motion Picture and Television Engineers 1986) American Association of Physicists in Medicine Task Group 18 findings; and geometry, luminance uniformity, temporal stability, resolution, and veiling glare remained within recommended levels.Ambient lighting was measured using a calibrated photometer to ensure it remained 25-40 luxas suggested by Brennan et al (2006).
Statistical Analysis
The data was inputted into JROCFIT softwareto calculate a range of values for each radiographer including sensitivity, specificity, an ROC curve and an AUCvalue an Az value the area under an ROC curve used as an indicator for performance (Eng, 2006).It was not possible to estimate smooth ROC curves for some of the radiographers due to the presence of asymptotes.Consequently it was difficult, therefore, to generate confidence intervals for these participants.The radiographers were compared on the basis of empirical AUC values using the Mann-Whitney U-Test. We also used ROCKIT software to confirm all the above values were correct and to generate confidence intervals where the raw data allowed (Metz, 2006).
The original intention was to generate a fitted ROC curve, fitted Az values and their confidence intervals; however, the data for a significant minority of the ROC curves was degenerate.This problem was particularly common in the top performers in the Irish sample; the strong confidence they had in their diagnoses coupled with a relatively small number of images and categories on the confidence scale resulted in asymptotes on their ROC curves.The results have shown the fitted Az values and asymmetric confidence intervals where possible, however, the reader should keep in mind that the "degree of non-accidental injury" depicted by the images
may not follow a binormal distribution and, consequently the results may be biased (Table 2, Table 3).
By comparing the observers using non-parametric methods, empirical ROC curves and empirical Az values (Table 2,3), we neither had to assume binormal distribution of the images nor was there any degenerate data.However when considering empirical AUC values it is important to remember they tend to underestimate the actual AUC values (Seong, 2004).
The mean Az value of each population sample of radiographers was compared using the non-parametric mean value Mann-Whitney U-Test (Table 1).The mean was for convenience yet is a low measure of central tendency due to the effect of a small number of outliers with low Az values. The Mann-Whitney U-test was used as it is not reasonable to assume the results of the samples followed a normal distribution.
Some images were rated by each radiographer more than once and by comparing the answers, using Kappa with linear weighting, we gained some idea of the consistency/ repeatability of the participants.Linear weighting rather than Cohen's un-weighted Kappa was used as it is appropriate that larger differences in answers would be treated as such (i.e. answering 1 and 5 for the same image is treated as more discordant then answer 1 and 2).A Kappa statistic of 1 is equal to perfect agreement while a score of 0 indicates chance agreement.Participants with high Az values generally had high Kappa statistics, indeed the Kappa statistics were high overall (Table 2, Table 3).
RESULTS
Table 1: The mean values for AUC, sensitivity and specificity.The difference in AUC values compared using a one way mean value Mann-Whitney U Test is statistically significant (U=20, Z=2.54, P=0.0111)
	AUC	Sensitivity	Specificity
Irish Group X =	0.825 (o 0.133)	70.4%	88.2%
Slovene Group X =	0.698 (o 0.135)	46.9%	81.2%
članki
Table 2: The empirical AUC values, the fitted AUC values with 95 % asymmetric confidence intervals, and linear weighted Kappa statistics with standard errors of the Irish sample
Irish Sample	Empirical AUC	Fitted AUC (Asymmetric 95% Confidence Intervals)	Linear Weighted Kappa (Standard Error)
Radiographer 1	0.849	N/A	0.9211 (0.0797)
Radiographer 2	0.935	0.944 (0.7763 - 0.9921)	0.7568 (0.1469)
Radiographer 3	0.97	N/A	0.7692 (0.1729)
Radiographer 4	0.84	N/A	0.8421 (0.0528)
Radiographer 5	0.825	N/A	0.4 (0.2991)
Radiographer 6	0.82	0.867 (0.657 - 0.966)	0.5333 (0.268)
Radiographer 7	0.787	0.807 (0.590 - 0.934)	1(0)
Radiographer 8	0.488	0.499 (0.272 - 0.726)	0.6154 (0.1938)
Radiographer 9	0.63	0.637 (0.395 - 0.833)	0(0.1863)
Radiographer 10	0.917	N/A	0.8378 (0.1164)
Radiographer 11	0.935	N/A	0.6667 (0.2309)
Radiographer 12	0.873	N/A	0.625 (0.1236)
Radiographer 13	0.861	0.88 (0.69 - 0.968)	0.4857 (0.2623)
Table 3:The empirical AUC values, the fitted AUC values with 95 % asymmetric confidence intervals, and linear weighted Kappa statistics with standard errors of the Slovene sample
Slovene Sample	Empirical AUC	Fitted AUC (Asymmetric 95% Confidence Intervals)	Linear Weighted Kappa
Radiographer 1	0.846	N/A	0.6129 (0.2507)
Radiographer 2	0.754	0.777 (0.547 - 0.92)	0.85 (0.1063)
Radiographer 3	0.814	0.805 (0.547 - 0.945)	0.9143 (0.086)
Radiographer 4	0.76	0.781 (0.556 - 0.921)	0.4783 (0.1324)
Radiographer 5	0.642	0.666 (0.431 - 0.849)	0.5 (0.1932)
Radiographer 6	0.725	0.761 (0.525 -0.913)	0.7857 (0.2047)
Radiographer 7	0.405	0.409 (0.207 - 0.639)	< 0
Radiographer 8	0.743	0.754 (0.515 - 0.91)	0.4706 (0.0928)
Radiographer 9	0.589	0.593	0.85 (0.1063)
DISCUSSION
There was a statistically significant difference between the Irish and Slovene groups.Both groups have a similar level of experience so that does not appear to be an important factor. It appears that with minimal training and involvement in radiography interpretation that radiographers may improve their ability to recognise typical non accidental injury fractures in children.
The 8 week time constraint of this project placed certain limitations of the research.It was not possible to obtain a sufficient number of images to generate a significant AUC value for each radiographer; indeed it remains statistically possible, though unlikely, that the radiographers performed no better than chance.Secondly, the researchers did not utilise a paediatric radiologist to use as a gold standard to compare the radiographers with and consequently cannot comment on the value of radiographer recognition of paediatric fractures in the context of non-accidental injury fractures.
Limitations
With more images it would be possible to produce narrower AUC values and, with additional images and categories on the confidence scale there would probably be much less degenerate data.
The results indicate radiographers can recognise typical non accidental injury fractures better then chance, but it is important to note that these results only apply to the images used in this study.It is debatable whether or not these results can be applied to the radiographer population as a whole because without a paediatric radiologist to use as a gold standard with which to compare our results we have no real notion of how "difficult" the test was.These limitations were an unfortunate consequence of the 8 week time constraint for the completion of the research.
Ability To Recognise Typical Non-Accidental Injury Fractures?
Keeping these limitations in mind the Irish sample seems to have performed rather well.Half the Az values were 0.84 or above, and only two were below 0.787.When these results are considered along with the high Kappa scores there is strong indication that the Irish group can recognise typical non-accidental injury fractures.It is also likely the Slovene sample can distinguish between accidental and non-accidental injury fractures at some level but not to the same extent.
The Effect of Radiographer Experience and Undergraduate /Training
Indeed, there is a statistically significant difference between the Irish and Slovene samples.Both samples have a similar level of experience so it does not appear to be an important factor.The fact that the Irish group works exclusively with children may be a confounding factor yet it seems unlikely to contribute to total the magnitude of the difference.In the opinion of the authors it is reasonable to conclude that
članki
possible undergraduate training and recognition of potential NAI markers in radiography has improved the Irish samples ability to recognise typical non-accidental injury fractures.
In conclusion, there is some indication that radiographers working in the Republic of Ireland can recognise typical non-accidental injury fractures in children; however, further research is needed to confirm or disprove this. The radiographers working in the Republic of Ireland were slightly better at recognising typical non accidental injury fractures then the radiographers working in Slovenia. However it would be useful to repeat this study using a lager sample size.
CONCLUSION
In conclusion the results of this study seem to indicate that radiographers with training and involvement in limited radiographer interpretation can recognise typical non-accidental injury fractures, while those who are not involved in this only performed slightly better than chance.Ashealth care professionals radiographershave a useful contribution to maketo the Multidisciplinaryteam approach regardingthe protection of children.
References
American Association of Physicists in Medicine (2005). Assessment of display performance for medical imaging systems. Med Phys, 32:1205 -1225. BôrjessonS,Hâkansson M, Bâthhttp://rpd.oxfordjournals. org/content/114/1-3/45.abstract - aff-1 M, Kheddache S, Svensson S,Tingberg A, Grahn A,Ruschin M, Hemdal B,Mattsson S and Gunnar Mânsson L( 2005) "A software tool for increased efficiency in observer performance studies in radiology'; Radiation Protection Dosimetry, Vol. 114 (1-3): 45-52.
Brealey S, Scally A, Hahn S, et al. Accuracy of radiographer plain radiograph reporting in clinical practice: a meta-analysis. ClinRadiol, 60: 234-241.
Brennan PC, McEntee M, Evanoff M, ,Phillips P, O'Connor OT, Manning DJ(2006). Ambient Lighting: Effect of Illumination on Soft-Copy Viewing of Radiographs of the Wrist. AJR, vol. 188 no. 2 W177-W18.
Davis M, Reeves P (2009). Diagnostic Radiographers and Their Role in Child Protection Situations -AnExploration of Bystander Intervention. Child Abuse review, Vol (3): 205-214. Eng J. ROC analysis: web-based calculator for ROC curves. Baltimore: Johns Hopkins University [updated 2006 May 17; cited 2012 January 17]. Available from: http://www.jrocfit.org. Hogg P, Hogg D, Eaton C,Sudberry J (1999). Guest Editorial-Child protection in radiographic practice. Radiography,Vol. 5 (1): 127-129.
McNulty JP, Kelly B, Rainford L, McEntee MF (2011). Can collaboration between radiographers and non-consultant hospital doctors during image interpretation improve the accuracy of diagnostic decisions? European Congress of Radiology 2011 Vienna, Austria, 3.3. - 7.3.2011. Rigney D and Davis M (2004) Radiographers and Non
Accidental Injury in Children-An Irish Perspective. Radiography,Vol.10 (1): 7-13.
Society of Motion Picture and Television Engineers (1986). SMPTE recommended practice, RP 133-1986, specifications for medical diagnostic imaging test pattern for television monitors and hard-copy recording cameras. SMPTE J 1986; 95:693 -69.
SeongHoPar,1 Jin Mo Goo,1 Chan-Hee Jo (2004). Receiver Operating Characteristic (ROC) Curve: Practical Review for Radiologists. Korean J Radiol, 5(1): 18. Sudbery J, Hancock V, Eaton C, Hogg P (1997). Child protection and radiography: clinical and technical issues. Child Abuse Rev, 6: 191-198.