Dynamic Transfer Learning Switching Approach Using Resource Benchmark in Edge Intelligence

Machine learning (ML) techniques are applied for profiling computing and processing resources data collected while running deep neural network models on edge devices. Adaptive deep neural network (DNN) model switching requires proper benchmarking for categorizing AI models based on their applications and computational resources enabled by their processing accelerators. Based on benchmark metrics, DNN models can be classified into tiny, low, small, medium, and large resources, then identify DNN models that perform well within resource constraints. Ensure efficient resource allocation, latency management, and trade-off between accuracy and resource. In this work, we propose a benchmark for edge transfer artificial intelligence learning service (TALS) that uses ML techniques. They aim at classifying DNN models by their target edge applications while running edge inferences. We used both unsupervised learning (UL) and supervised learning (SL) techniques to identify the most effective features for the TALS models and to benchmark the performance of edge devices. To achieve this, two approaches were investigated: first, determining features based on edge inference’s computing resources profiling using principal component analysis; and second, classifying the DNN models at the target application level using a regression approach based on historical resource utilization data. In addition, we propose a dynamic model transfer learning that switches between a set of pre-trained and optimized and quantized DNN models based on the cost function. ML techniques learn resource-aware prediction from new resource allocation data and ensure that the multicriteria switch cost selects the inference task models that meet the edge resource constraint requirements. The experimental results highlight a strong relationship between the supervised learning model and the clustering execution method. The dynamic switching approach on real edge devices demonstrates dynamic switching between models according to inference task complexity. We conclude that dynamic switching models allows to ensure smooth operation without overloading resources in edge intelligence.