Reservoir concatenation and the spectrum distribution of concatenated reservoir state matrices

Abstract

Reservoir computing, one of the state-of-the-art machine learning architectures, processes time-series data generated by dynamical systems. Nevertheless, we have realized that reservoir computing with the conventional single-reservoir structure suffers from capacity saturation. This leads to performance stagnation in practice. Therefore, we propose an extended reservoir computing architecture called reservoir concatenation to further delay such stagnation. Not only do we provide training error analysis and test error comparison of reservoir concatenation, but we also propose a crucial measure, which is the trace associated with a reservoir state matrix, that explains the level of responsiveness to reservoir concatenation. Two reservoir dynamics are compared in detail, one by using the echo state network and the other by using a synchronization model called an explosive Kuramoto model. The distinct eigenvalue distributions of the reservoir state matrices from the two models are well reflected in the trace values that are shown to account for the different reservoir capacity behaviors, determining the different levels of responsiveness.