Learning rate optimisation of an image processing deep convolutional neural network

Abstract

The major contribution of this dissertation is the proposal of the use of mathematical models to identify an optimal learning rate for an image processing deep convolutional neural network (DCNN). This model is derived from a nonlinear regression relationship between the learning rate and the accuracy of a test DCNN model. This relationship is meant to (A) resolve the problem of arbitrarily selecting the initial learning rate (B) reduce computational resource requirement and (C) reduce training instabilities. An algorithm is developed to analyse an inputted DCNN model and subsequently render output parameters that may be used to aid in the selection of an OLR. The benefit of an OLR includes improved training stability and reduced computational resources. The results rendered by the OLR algorithm proposes that an optimal learning rate improves model performance; this is described by the test model average accuracy of 91%. Furthermore, a model validation graph is also extrapolated. which will illustrate the mathematical model accuracy and the region of interest (ROI). The ROI defines the region in the learning rate spectrum with a positive effect on model performance.