79 Enhanced registration of point clouds for change detection on the Case of Strunjan Coastal Cliff Klemen Kregar 1 , Wolfgang Wiedemann 2 , Marko Vrabec 3 , Christoph Holst 2 Abstract Terrestrial laser scanning (TLS) is a suitable method for detecting changes in the natural and built environment. In the project "Erosion processes on coastal flysch cliffs with risk assessment" we use TLS as a reference method to determine the actual changes in the cliff surface. Despite adhering to the highest standards of planning and data collection: using the appropriate method, ensuring the best possible scanning geometry, good geometric distribution of the control points, high accuracy of determination and stability monitoring and updating of the positions of the control points, and the quality of registration within the accuracy of random errors of the measurement methods, the measurement results are not guaranteed to be of the expected quality. We propose a method for the detection of stable areas based on the segmentation of raster images interpolated from a point cloud. The stability criteria are the differences between the cloud positions of two epoch measurements and the differences in surface orientation. The identified stable areas are used for the ICP registration, which improves the quality of the registration and thus provides a more realistic representation of the surface changes at the cliff. Keywords: Terrestrial laser scanning, Deformation monitoring, Sveti Križ bay, Eocene flysch, Coastal cliff Introduction Terrestrial laser scanning (TLS) has proven to be an extremely useful method of detecting changes in the natural and built environment in recent decades. The Slovenian Research Agency is funding the three-year project "Erosion processes on coastal flysch cliffs with risk assessment". Surveyors, in collaboration with geologists, are looking at possible methods for monitoring changes on the surface of cliffs in larger areas. We use TLS as a reference method to determine the actual changes in the cliffs.TLS was chosen as the reference method because there has been sufficient literature on the use of this method for monitoring coastal cliffs over the last two decades (Hoffmeister et al., 2012; Kersten et al., 2020; Kuhn & Prüfer, 2014; Poulton et al., 2006; Rosser et al., 2005b). The literature suggests that the method for monitoring coastal cliffs is appropriate and scientifically sound. The coastal cliff has a difficult terrain configuration for TLS scans. The narrow coastal strip between the cliff and the escarpment limits the choice of suitable scanning sites. The presence of vegetation in the lower part of the cliff and the relative proximity of the stations to the cliff cause shadows in the collected data. The problem can be solved by setting up many stations, which leads to an accumulation of errors in the registration of the stations. Another, more important problem is to provide a stable reference base that allows comparison of data collected in different time epochs. 1 UL, Fakulteta za gradbeništvo in geodezijo, Jamova cesta 2, 1000 Ljubljana 2 TUM, Lehrstuhl für Ingenieurgeodäsie, Arcisstraße 21, 80333 München 3 UL, Naravoslovnotehniška fakulteta, Aškerčeva cesta 12, 1000 Ljubljana 80 For the above reasons, we chose the section of the cliff above the bay of sveti Križ near Strunjan, where the coast curves from east-west to north-south at a distance of 500 metres. Such a configuration makes it possible to place the scanner station at the top of the cliff to the west in such a way that we can scan the entire area of the cliff bordering the bay to the east from one station (Figure 1). Figure 1: A view of the area under consideration from the position of the scanner The coastal cliff under consideration consists of Eocene flysch in which hard sandstone layers alternate with soft intermediate layers of marl. Erosion mainly erodes marl, and the intervening hard sandstone layers act as barriers that partially retain the eroded marl material. The erosion eventually erodes the underlying sandstone layers, which then collapse along tectonic cracks. This makes cliff erosion, on a small scale, a transient process with slow, continuous erosion of soft marl and occasional sudden major surface changes. The measurements at the cliff are carried out twice a year, in spring and autumn. We assume that the entire area under consideration is not deformed between two measurement epochs, but that noticeable changes occur in locally limited areas. To ensure a stable reference base, reference points in stable areas are generally used (Rosser et al., 2005a). The centres of the targets at the reference points are evaluated from the point cloud and the scan is transformed into a reference coordinate system using the similarity transformation. In the present case, the control points were stabilised with metal anchors in layers of flysch sandstone in the area under consideration, which is subject to change. The solution is not optimal, as we cannot assume complete stability of the control points. However, the position of the scanner at such a great distance from the considered area prevents better solutions. The surroundings of the station are overgrown with vegetation, there is no stable ground in the surroundings, points close to the station would have too great an influence of the extrapolation on the transformation of the point cloud. The positions of the control points are measured with a precise geodetic instrument before each measurement using the polar method. Before we register the point cloud, we check the stability of the control points by comparing them with their positions in previous measurements. Despite everything, it turned out that the control points do not provide a sufficiently stable coordinate base. There are two possible explanations for this: the actual instability of the hard sandstone layers and/or errors in determining the positions of the centres of the targets from the point cloud due to the large scan distance. Proposed solution is to adjust the scanned point cloud to the previously measured cloud in areas that have not changed between epochs (Wujanz et al., 2018). 81 Methods Data collection and preparation So far, we have carried out the measurement seven times in a row in autumn and spring. Scanning is done with a Riegl VZ-400 scanner with a nominal spatial accuracy of 3 mm and a range of up to 600 m (Riegl LMS, 2014). From the selected position, we scan the cliff area at a distance of 300 to 550 m with a resolution of 1.6 cm at 400 m in "long range" mode. The control points are scanned using the fine target scanning application within the RiSCAN PRO software, which automatically sets the resolution and scanning mode. The control points are realised with round tiles with a diameter of 5 cm and a retroreflective surface. The centres of the targets from the scan are evaluated as the average of the points on the target weighted by the intensity of the intensities that exceed the selected threshold. Registration on is done with a similarity 6-parameter transformation, where we achieve deviations of the points after the transformation of up to 6 mm. The positions of the control points were determined during the first measurement in the national coordinate system D96/TM with ellipsoid heights. With each measurement we check the positions of the control points with a precise polar measurement and correct them if movements are detected. The result is a georeferenced point cloud with about 30 million points. Before further processing, the cloud is roughly clipped and filtered to remove as much of the interfering vegetation as possible. We use the method of geometric features filtering, i.e. we remove points with less than 2000 neighbours in a radius of 1 m and points with an planarity factor in radius 0.2 m of less than 0.3. The configuration of the terrain is extremely challenging for filtering, so we also remove some points on the cliff with filtering while some vegetation remains. Manually clipping such a large area in so many repeated terms would be too time consuming. The final step of data preparation is to calculate the normal of the planes through the points in radius 6 cm around each point. The equal alignment of the normals is ensured by the condition that the third component is always positive. Representation of surface changes - Coordinate system There are several methods for displaying surface changes from two point clouds of successive epoch measurements. The most basic is to display the distances between the nearest pairs of points in both clouds, i.e. "Cloud2Cloud" distance. With this method we cannot distinguish between erosion and sedimentation, positive and negative surface changes. Another method is e.g. M3C2 (James et al., 2017; Lague et al., 2013), which shows changes in the direction of the surface normal vectors. More recent methods show changes using vector fields (Gojcic et al., 2020; Holst et al., 2021). Given the nature of surface changes, the falling of whole hard blocks and the spreading of sand from soft layers, we estimate that the changes are most realistically represented as differences in digital elevation models (Wheaton et al., 2010). For such a method, it is necessary to transform the cliff cloud so that the height component is perpendicular to the 82 average plane of the cliff wall. The x-axis should be horizontal. The rounded value of the smallest coordinates is subtracted from the cloud before transformation. Figure 2: Transformation of the point cloud into the coordinate system of the cliff wall Identification of stable areas The approximate registered point cloud of the measurement can be fitted to the previous measurement cloud using the Iterative Closest Point (ICP) procedure (Besl & McKay, 1992). If the surface is expected to have changed between epochs, this procedure should not be used. If one were to register the entire deformed cloud of second measurement, the actual changes would affect the fitting, which would therefore be impossible to determine independently. Only sections that have not changed can be used for the fitting. For each point we calculate the direction and slope of the normal vector through the plane of its nearest neighbours. = ata n = asin ( ) where , and are the components of the normal vector. If we plot a point cloud coloured according to the point height, direction and inclination of the normals, we can see that the changes are well expressed, especially in the last two diagrams (Figure 3). We are looking for a method to automatically detect the differences observed in the plots and in this way impartially identify unstable areas. Slika 3: Visually observed changes in the point cloud, coloured by point height; direction and slope of the normals 83 Due to the size of the point cloud, we have limited the analysis to identify stable areas to three sub-areas where we expect to find stable segments. Figure 4: Sub-areas for the search for stable segments By linear interpolation between the points we create three raster images for both compared epoch measurements: the image of the height of the points (DEM), the image of the direction and the image of the slopes of the normal vectors. A resolution of 2 cm was chosen, which roughly corresponds to the resolution of scanning the surface. Figure 5: Raster images of DMV, directions and slopes of the normal vectors and their differences between successive epochs Assuming that the two clouds are sufficiently well approximately registered (with control points), the stability assessment can be based on the differences of all three images. If the approximate registration were worse, only the normal images could be used. We calculate the differences between the images of the first and second compared epoch measurements and enter them as channels R, G, B in the common image of differences. The three-channel difference image is then segmented into areas with similar values using the SLIC method (Achanta et al., 2012; Moore et al., 2008). For each segment, we calculate the average value of the differences of all three channels. The selected proportion of segments (we decided to take 5%) with the smallest average differences is marked as stable segments. 84 Figure 6: SLIC segmentation of the overall picture of differences and identification of the segments with the smallest average change On the images obtained by interpolation from the point cloud, we have successfully identified the segments where the differences between successive epoch measurements are the smallest. The next step is to find points in the cloud that correspond to stable areas of the image. The starting coordinate of the image, i.e. the position of the lower left corner, is subtracted from the x, y coordinates of the points from the cloud. The coordinates are then divided by the dimension of the raster cell and rounded down. In this way we obtain the indices of the image cells to which each cloud point belongs. Then, for each point, we check whether it lies in one of the stable areas identified by the SLIC segmentation. Points that match stable areas are then used for ICP registration. Commercial programmes such as Cyclone Core, Cyclon 3DR or even Matlab can be used for ICP registration, but the open source programme CloudCompare is most commonly used. We expect a small transformation of the second point cloud to allow a correct comparison of the two clouds and a realistic representation of the actual changes between the two clouds. Figures 5 and 6 show only one of the three selected sub-areas for clarity. In Figure 7 we show the points that are within 5% of the most stable segments in all three sub-regions. In Figure 7 we can see how in the upper right region the clouds already fit together well enough, while on the left one cloud clearly overlaps the other. The ICP registration will ensure that the clouds fit together as well as possible everywhere. Figure 7: Points that lie within the most stable segments: first epoch in blue and second epoch in red 85 Figure 8: ICP transformation, rotation around the yellow axis and translation for the violet vector are shown In Figure 8 we show the result of the transformation with the parameters calculated by ICP registration at the points of the stable areas. The rotation around the yellow axis is relatively small (46.5'') and causes a movement of less than 40 mm on the cliff dimension. The translation is 15, 2 and 43 mm along the violet axes. In the next chapter we will show how the implemented transformation improves the detection of changes in the whole considered area of the cliff. Results In Figure 9 we show the final changes between the two point clouds with and without the use of enhanced registration based on the identification of stable areas. The surface changes are shown as DEM differences (Wheaton et al., 2010). The red colour represents negative changes or erosion, while the blue colour represents positive changes or sedimentation. As the changes are small to be visible in the existing format of the article, we have artificially highlighted them. Large changes at the edges of the area are the result of incomplete overlap of the cloud areas of the two epoch measurements and incomplete filtering of the vegetation, so they should be ignored. 86 Figure 9: Changes in cliff surface, above without, below with improved registration. The units on the colour scale are centimetres The red colour is expected to appear on the steep sections as the material from these areas collapses. The blue colour is expected on the scree below the walls where the material is piled up. The blue streaks in the wall are also realistic where sediment accumulates on the harder layers. In the image above we see larger bluish areas on the steep parts of the cliff which are the result of registration errors with control points. Registration enhancement eliminates such areas, as can be seen in the image below. Discussion The motivation for the study described was dissatisfaction with the results of the measurements. The following criteria were considered in the planning and implementation of data collection:  using a method that, according to the literature review, is suitable for the purpose of monitoring cliff changes,  ensuring the best scanning geometry in a given terrain,  good geometry of the distribution of the control points,  high accuracy in the determination and stability monitoring and updating of the control point positions,  a registration quality that is within the accuracy of the random errors of the measurement methods. Despite all the above criteria, which are much higher than those used in the literature (Hoffmeister et al., 2012; Kersten et al., 2020; Kuhn & Pruefer, 2014; Poulton et al., 2006; Rosser et al., 2005b), the results obtained did not reach the expected quality. The main 87 problem was the areas with slight positive changes in areas for which no meaningful explanation could be found. We believe that the proposed method is general enough and can work in other surface change detection applications where we cannot guarantee the sufficient quality of a stable reference system. Using the normal orientation of the point cloud is robust enough to work even with poorly pre-registered point clouds. The height differences, which appear as one of the three stability criteria, can be replaced by a deviation from the average difference or omitted for poorly pre-registered clouds. In calculating the average value of each segment's differences from all three criteria, we used centimetres for the DMV differences and degrees for the normal difference in direction and slope. It would also be possible to use standardised units and examine the influence of larger or smaller weights on a single channel. The method requires the projection of a point cloud into the image plane, which can be an obstacle in spatially diverse terrain. Nevertheless, the cloud can be divided into several approximately planar segments. The ICP method works on the principle of minimising the distances between points. In the case of a cliff, all stable sectors lie on more or less similarly oriented planes. ICP can only define displacements in the direction of the plane normal well, displacements along the plane are worse defined. Therefore, it would be optimal to select the areas also according to the criterion that the local plots of the segments are oriented as differently as possible. The method used provides for the manual selection of sub-regions, which we can use to enforce a better geometric arrangement of the segments for ICP registration. Further research is being conducted towards full automation of the process. Instead of image interpolation and segmentation, it would be useful to move to a fully 3D-based process by segmenting the point cloud into super-voxels (Lin et al., 2018; Papon et al., 2013; Verdoja et al., 2017). For a large number of potentially stable segments, the RANSAC random selection method could also be used. Literature Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., & Susstrunk, S. (2012). SLIC Superpixels Compared to State-of-the-Art Superpixel Methods. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 34(11), 2274–2281. Besl, P. J., & McKay, N. D. (1992). A Method for Registration of 3-D Shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(2), 239–256. https://doi.org/10.1109/34.121791 Gojcic, Z., Zhou, C., & Wieser, A. (2020). F2S3: Robustified determination of 3D displacement vector fields using deep learning. Journal of Applied Geodesy, 14(2), 177–189. https://doi.org/10.1515/jag-2019-0044 Hoffmeister, D., Tilly, N., Curdt, C., Aasen, H., Ntageretzis, K., Hadler, H., Willershäuser, T., Vött, A., & Bareth, G. (2012). Terrestrial Laser Scanning for Coastal Geomorphologic Research in Western Greece. ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XXXIX-B5(June 2018), 511–516. https://doi.org/10.5194/isprsarchives-xxxix-b5-511-2012 Holst, C., Janßen, J., Schmitz, B., Blome, M., Dercks, M., Schoch-Baumann, A., Blöthe, J., Schrott, L., Kuhlmann, H., & Medic, T. (2021). Increasing spatio-temporal resolution for monitoring alpine solifluction using terrestrial laser scanners and 3d vector fields. Remote Sensing, 13(6). https://doi.org/10.3390/rs13061192 James, M. R., Robson, S., & Smith, M. W. (2017). 3-D uncertainty-based topographic change detection with structure-from-motion photogrammetry: precision maps for ground control and directly georeferenced surveys. Earth Surface Processes and Landforms, 42(12), 1769–1788. https://doi.org/10.1002/esp.4125 88 Kersten, T. P., Lindstaedt, M., & Mechelke, K. (2020). Coastal cliff monitoring using UAS photogrammetry and TLS. June. https://doi.org/10.23784/HN116-02 Kuhn, D., & Prüfer, S. (2014). Coastal cliff monitoring and analysis of mass wasting processes with the application of terrestrial laser scanning: A case study of Rügen, Germany. Geomorphology, 213, 153–165. https://doi.org/10.1016/j.geomorph.2014.01.005 Lague, D., Brodu, N., & Leroux, J. (2013). Accurate 3D comparison of complex topography with terrestrial laser scanner: Application to the Rangitikei canyon (N-Z). ISPRS Journal of Photogrammetry and Remote Sensing, 82(February 2013), 10–26. https://doi.org/10.1016/j.isprsjprs.2013.04.009 Lin, Y., Wang, C., Zhai, D., Li, W., & Li, J. (2018). Toward better boundary preserved supervoxel segmentation for 3D point clouds. ISPRS Journal of Photogrammetry and Remote Sensing, 143(May), 39–47. https://doi.org/10.1016/j.isprsjprs.2018.05.004 Moore, A. P., Prince, S. J. D., Warrell, J., Mohammed, U., & Jones, G. (2008). Superpixel lattices. 26th IEEE Conference on Computer Vision and Pattern Recognition, CVPR. https://doi.org/10.1109/CVPR.2008.4587471 Papon, J., Abramov, A., Schoeler, M., & Worgotter, F. (2013). Voxel cloud connectivity segmentation - Supervoxels for point clouds. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2027–2034. https://doi.org/10.1109/CVPR.2013.264 Poulton, C. V. L., Lee, J. R., Hobbs, P. R. N., Jones, L., & Hall, M. (2006). Preliminary investigation into monitoring coastal erosion using terrestrial laser scanning : case study at Happisburgh, Norfolk. Bulletin of the Geological Society of Norfolk, 56(January), 45–64. http://www.norfolkgeology.co.uk/index.htm Riegl LMS. (2014). Riegl Vz-400 - datasheet (p. 4). Rosser, N. J., Petley, D. N., Lim, M., Dunning, S. A., & Allison, R. J. (2005a). Terrestrial laser scanning for monitoring the process of hard rock coastal cliff erosion. Quarterly Journal of Engineering Geology and Hydrogeology, 38(4), 363–375. https://doi.org/10.1144/1470-9236/05- 008 Rosser, N. J., Petley, D. N., Lim, M., Dunning, S. A., & Allison, R. J. (2005b). Terrestrial laser scanning for monitoring the process of hard rock coastal cliff erosion. Quarterly Journal of Engineering Geology and Hydrogeology., 38(4), 363–375. http://dro.dur.ac.uk/2823/ Verdoja, F., Thomas, D., & Sugimoto, A. (2017). Fast 3D point cloud segmentation using supervoxels with geometry and color for 3D scene understationing. Proceedings - IEEE International Conference on Multimedia and Expo, July, 1285–1290. https://doi.org/10.1109/ICME.2017.8019382 Wheaton, J. M., Brasington, J., Darby, S. E., & Sear, D. A. (2010). Accounting for uncertainty in DEMs from repeat topographic surveys: Improved sediment budgets. Earth Surface Processes and Landforms, 35(2), 136–156. https://doi.org/10.1002/esp.1886 Wujanz, D., Avian, M., Krueger, D., & Neitzel, F. (2018). Identification of stable areas in unreferenced laser scans for automated geomorphometric monitoring. Earth Surface Dynamics, 6(2), 303–317. https://doi.org/10.5194/esurf-6-303-2018