Reservoir Computing Networks incorporating Mathematical Analysis

Abstract

Recently, there are many successful studies on machine learning, deep learning, and AI. The wave of the Fourth Industrial Revolution using such technologies is flowing rapidly throughout the whole industry. However, there are still many problems yet to be investigated in machine learning. In addition to applying machine learning to increase the performance for the tasks, there are crucial problems such as AI security, lack of data, explainabilty all of which require expanding or making the new theories. To tackle such problems and establish new theories, researchers from various fields, including mathematics, physics and etc. as well as computer science, are studying the fusion of machine learning and other fields. As part of those studies, we have investigated mutual applications of machine learning techniques and mathematical analysis. Conversely, we conducted a study that applied mathematical analysis to machine learning. We intend to use machine learning techniques in this work to enhance the following two conventional mathematical methods and analysis: i) variational method in mathematical image processing and ii) distance measures used in data clustering. In principle, it doesn’t matter to use any machine learning with a certain computing ability, but we used RCN that has flexibility and scalability as a machine learning technique used in the methods proposed by this work. In the first part of the dissertation, we would like to present how reservoir computing can be strengthened in combination with PDEs for image denoising. In fact, the way we combine reservoir computing with PDEs is that we choose for each task a proper PDE model that turns an image into a sequential dataset, which is to be fed into a reservoir. At the learning stage, we also notify the network in a way that the sequential dataset has image structures. This whole procedure is called RCPDE. What makes RCPDE distinctive is that even though it is still not mathematically clear of the working principles of reservoir computing networks, we can tell the networks how they should produce solutions through proper guides from the solutions of mathematically well understood PDE models. It turns out that RCPDE outperforms both the usual reservoir computing and existing PDE approaches for image denoising. In addition, RCPDE outperforms deep neural networks, which is popular in image processing tasks, both in quality and in time with a small training set available. Next, in the second part of the dissertation, we present the new data measure which is based on a general machine learning technique. It was found that the newly defined distance tracks the appropriate parameter values of certain processes (image denoising, image segmentation,time series classification) even without the ground truth or comparative target data. We used the inconstructability of reservoir computing in the work. The inconstructability I(Fc; Fr) is an indicator of how well the characteristics of the processed data Fr are constructed from the characteristics of a given data Fc. The inconstructability may be considered a kind of trainingerror in the learning stage of machine learning. We have found that the inconstructability are relatively robust for different types of error measure(RMSE, SSIM, PSNR) and give adaptive optimal parameter values for different images in image denoising and segmentation.