Metoda prilagodljivega uglaševanja slojev konvolucijskih nevronskih mrež pri strojnem učenju s prenosom znanja : doktorska disertacija

slovenian

doktorska dela

2021

classification   deep learning   fine-tuning   machine learning   optimization   transfer learning

Univerza v Mariboru, Fakulteta za elektrotehniko računalništvo in informatiko

In this Doctoral Dissertation, we present the problem of selecting fine-tunable layers when utilizing transfer learning with the fine-tuning approach for training deep convolutional neural networks. With the conducted empirical analysis of layer selection impact on the training performance, we confirmed the assumption that the most suitable selection of fine-tuned layers depends on the chosen convolutional neural network architecture, as well as on the target problem. In order to address the problem of selecting the most suitable combination of fine-tunable layers, we developed and proposed an adaptive method, DEFT, based on a differential evolution algorithm, which works in a straightforward automatic manner using different convolutional neural network architectures. Due to the high time complexity of the proposed method, we developed and proposed a metric derived from the loss value, which is capable of detecting less suitable selections of fine-tunable layers at an early stage of training, which allows us to terminate training early, and, thus, reduce the time complexity of the proposed method. The performance of the proposed method was evaluated by utilizing three different convolutional neural network architectures against three different image datasets. Classification performance of the proposed DEFT method, with or without the proposed metric LDM, was compared against conventional approaches for training convolutional neural networks. The performance comparison was conducted using the most common classification metrics, consumed time for training, and consumed number of epochs. The statistical analysis of the obtained results was conducted using conventional statistical methods, as well as modern Bayesian analysis based approaches. The results confirmed the initial thesis that the problem of layer selection when utilizing transfer learning with fine-tuning, can be addressed successfully using the proposed adaptive DEFT method, and that utilization of the proposed LDM metric reduces the number of epochs needed for training effectively, while achieving comparable results.

07.08.2023