A Systematic Review of Model Optimization, Resource Allocation, and Online Learning in TinyDL

With the expansion of the Internet of Things (IoT), latency and bandwidth challenges in cloud architectures have led to the emergence of TinyML and its evolution into TinyDL. This technology enables the execution of deep learning models on resource-constrained microcontrollers (MCUs). This study aims to provide a systematic review of ۲۴ leading research papers, focusing on the transition from static inference to On-device Online Learning. The findings indicate that compression techniques such as "Deep Compression" (a combination of pruning and quantization) can reduce model size by up to ۴۹ times. Furthermore, hardware-software co-design approaches, such as MCUNet, have achieved over ۷۰% accuracy in image classification while reducing SRAM memory consumption by ۳.۴ times. Analytical results suggest that utilizing optimized architectures like AdderNet and Sparse Transformer has significantly enhanced energy efficiency in edge devices. Additionally, Tiny Distributed Machine Learning (TDML) systems have improved model stability in the face of environmental changes. This study confirms that TinyDL is moving toward dynamic learning, although challenges such as managing unlabeled data and hardware heterogeneity persist. Finally, the development of Federated Learning and MLOps tools are proposed as future research directions.

کلیدواژه ها:

Tiny Machine Learning (TinyML) ، Microcontroller (MCU) ، Model Compression ، Online Learning ، Tiny Distributed Machine Learning (TDML) ، Neural Architecture Search (NAS)