EdgeVLM: A Compression-Aware Vision–Language Framework with Hybrid Temporal Modeling for Real-Time Mobile Inference

Vision–language models have achieved strong performance across many multimodal tasks, yet deploying them directly on mobile and edge devices remains difficult because of their large model size and high computational requirements. In this work, we present EdgeVLM, a compact vision–language model designed for edge-oriented multimodal inference. The architecture integrates MobileViTv۲ as the visual encoder and TinyLLaMA as the language backbone. The two components are connected through a lightweight ۱.۲ MB adapter that reduces cross-modal fusion parameters by ۹۲% compared with standard cross-attention approaches. The model is compressed through a three-stage pipeline that includes QLoRA fine-tuning, knowledge distillation from a larger teacher model, and ۴-bit quantization-aware training. Through this process, the deployment footprint is reduced from approximately ۱۴.۵ GB for the LLaVA-۱.۵ ۷B teacher model to about ۷۳۰ MB, representing nearly a twenty-fold reduction in size. In addition, a lightweight hybrid TCN–LSTM module with sample-dependent dynamic gating enables temporal reasoning over short video clips by capturing both short-term dynamics and longer-range dependencies. Evaluation on NExT-QA shows that EdgeVLM preserves descriptive visual grounding more effectively than causal and temporal reasoning under aggressive compression. Component-level CPU benchmarks further show that the temporal fusion workload can run under CPU-only conditions. Overall, these findings suggest that EdgeVLM offers a practical path toward compact, edge-oriented multimodal inference.

کلیدواژه ها:

Vision-Language Models ، Edge AI ، Model Compression ، Quantization-Aware Training ، Mobile Inference ، Temporal Modeling