TY - JOUR
T1 - Training recurrent neural networks using optimization layer-by-layer recursive least squares algorithm for vibration signals system identification and fault diagnostic analysis
AU - Cho, S. Y.
AU - Chow, T. W.S.
AU - Fang, Y.
N1 - Funding Information:
This Research is supported by the City University Strategic Research Grant of project No.7001081. The authors would like to thank Prof. Andrew. Chan, Department of Electrical Engineering, Texas A. & M. University, USA, for providing us with some of the data.
PY - 2001
Y1 - 2001
N2 - Depending on the representation ability and the dynamic capability of recurrent neural networks for nonlinear models, a neural-based approach for general practical nonlinear systems identification is demonstrated in this paper. The paper presents a new method for the training of recurrent neural networks, namely the Optimization Layer-by-Layer Recursive Least Squares (OLL-RLS) algorithm. The method is based upon the weights to be optimized in layer-by-layer fashion, using the standard RLS method to speed up the convergence rate. The derivations of the OLL-RLS algorithm for the recurrent networks and the difficulties by the numerical instability and convergence stalling are also included and addressed in this paper. These difficulties can be tackled by introducing noise to the OLL-RLS algorithm, which enhances the convergence property significantly. Further, the convergence analysis and the computational complexity of the OLL-RLS algorithm are discussed. The validation results show that when applying to the systems identification, the novel OLL-RLS algorithm significantly outperforms other training algorithms, in both convergence speed and generalization capability. The proposed neural-based approach is also applied to the railway carriage system identification by identifying the train vibration signal. The identification results are very promising and show that the OLL-RLS algorithm can provide a significant contribution to fault diagnostic analysis by monitoring the identification error characteristics.
AB - Depending on the representation ability and the dynamic capability of recurrent neural networks for nonlinear models, a neural-based approach for general practical nonlinear systems identification is demonstrated in this paper. The paper presents a new method for the training of recurrent neural networks, namely the Optimization Layer-by-Layer Recursive Least Squares (OLL-RLS) algorithm. The method is based upon the weights to be optimized in layer-by-layer fashion, using the standard RLS method to speed up the convergence rate. The derivations of the OLL-RLS algorithm for the recurrent networks and the difficulties by the numerical instability and convergence stalling are also included and addressed in this paper. These difficulties can be tackled by introducing noise to the OLL-RLS algorithm, which enhances the convergence property significantly. Further, the convergence analysis and the computational complexity of the OLL-RLS algorithm are discussed. The validation results show that when applying to the systems identification, the novel OLL-RLS algorithm significantly outperforms other training algorithms, in both convergence speed and generalization capability. The proposed neural-based approach is also applied to the railway carriage system identification by identifying the train vibration signal. The identification results are very promising and show that the OLL-RLS algorithm can provide a significant contribution to fault diagnostic analysis by monitoring the identification error characteristics.
UR - http://www.scopus.com/inward/record.url?scp=0034882911&partnerID=8YFLogxK
U2 - 10.1515/JISYS.2001.11.2.125
DO - 10.1515/JISYS.2001.11.2.125
M3 - Article
AN - SCOPUS:0034882911
SN - 0334-1860
VL - 11
SP - 125
EP - 154
JO - Journal of Intelligent Systems
JF - Journal of Intelligent Systems
IS - 2
ER -