Automatic Learning Rate Adaption for Memristive Deep Learning Systems

Yang Zhang, Linlin Shen

Research output: Journal PublicationArticlepeer-review


As a possible device to further enhance the performance of the hybrid complementary metal oxide semiconductor (CMOS) technology in the hardware, the memristor has attracted widespread attention in implementing efficient and compact deep learning (DL) systems. In this study, an automatic learning rate tuning method for memristive DL systems is presented. Memristive devices are utilized to adjust the adaptive learning rate in deep neural networks (DNNs). The speed of the learning rate adaptation process is fast at first and then becomes slow, which consist of the memristance or conductance adjustment process of the memristors. As a result, no manual tuning of learning rates is required in the adaptive back propagation (BP) algorithm. While cycle-to-cycle and device-to-device variations could be a significant issue in memristive DL systems, the proposed method appears robust to noisy gradients, various architectures, and different datasets. Moreover, fuzzy control methods for adaptive learning are presented for pattern recognition, such that the over-fitting issue can be well addressed. To our best knowledge, this is the first memristive DL system using an adaptive learning rate for image recognition. Another highlight of the presented memristive adaptive DL system is that quantized neural network architecture is utilized, and there is therefore a significant increase in the training efficiency, without the loss of testing accuracy.

Original languageEnglish
Pages (from-to)1-12
Number of pages12
JournalIEEE Transactions on Neural Networks and Learning Systems
Publication statusAccepted/In press - 2023
Externally publishedYes


  • Adaptation models
  • Adaptive learning
  • Adaptive learning rate
  • Computer architecture
  • deep learning (DL)
  • fuzzy logic
  • image recognition
  • Image recognition
  • memristor
  • Memristors
  • Neural networks
  • quantized neural network
  • Tuning

ASJC Scopus subject areas

  • Software
  • Computer Science Applications
  • Computer Networks and Communications
  • Artificial Intelligence


Dive into the research topics of 'Automatic Learning Rate Adaption for Memristive Deep Learning Systems'. Together they form a unique fingerprint.

Cite this