Privacy-Preserving Federated Collaborative Framework for Multimodal Content Understanding in Mobile Livestreaming with Energy Optimization

Objective: The rapid growth of mobile livestreaming services has created an urgent need for real-time, automated understanding of multimodal content, including both visual and speech data. Existing centralized cloud-based frameworks face critical challenges such as high latency, substantial communication costs, and growing concerns regarding user privacy. Recent device-cloud collaborative frameworks have addressed some of these issues by offloading preliminary processing to mobile devices; however, they exhibit fundamental limitations in three key areas: preserving the privacy of sensitive streamer data, optimizing energy consumption on resource-constrained mobile devices, and enabling knowledge sharing among different streamers. Method: This paper proposes a comprehensive solution that integrates three key technical contributions. First, a federated learning protocol incorporating differential privacy and secure aggregation is designed to protect local streamer data while enabling collaborative knowledge sharing. Second, a multimodal personalization module based on lightweight adapters is developed to replace full model fine-tuning, thereby substantially reducing energy and memory consumption. Third, an energy optimization mechanism featuring dynamic sampling frequency scheduling is introduced to adaptively adjust processing rates according to device battery levels and processor utilization. Results: Simulation-based evaluations conducted on the Fashion-Gen dataset and a custom-built simulated livestreaming scenario demonstrate that the proposed framework achieves the following: mathematical privacy guarantees with strong differential privacy parameters, a ۳۴% reduction in device energy consumption, a ۹۷% reduction in communication overhead, and a ۲۱.۶% improvement in accuracy for new streamers experiencing cold-start conditions-all while maintaining recognition accuracy with a marginal degradation of only ۱.۲% compared to the baseline method. Statistical significance of the accuracy comparisons is confirmed through paired t-tests at the ۹۵% confidence level.

کلیدواژه ها:

Federated learning ، privacy preservation ، differential privacy ، multimodal content understanding ، mobile livestreaming ، energy optimization ، device-cloud collaborative learning