A Framework for QoE aware Hybrid Parallelism in Distributed Edge AI Training and Inference

This paper introduces Dora, a framework for optimizing distributed edge AI training and inference with Quality of Experience (QoE) awareness. It focuses on hybrid parallelism, managing heterogeneous computation and contention-prone networks to maximize efficiency and respect QoE objectives in real-world AI deployments, enhancing the performance of edge AI systems.

Running state-of-the-art Deep Neural Networks on single edge devices is often impossible due to memory and compute limitations. Existing solutions often focus only on one type of parallelism (Data or Model) or ignore the volatile nature of edge networks (QoE factors like latency jitter and energy constraints).

The authors model the distributed edge system as a graph and formulate an optimization problem to maximize QoE. They employ a heuristic algorithm (potentially based on Genetic Algorithms or Reinforcement Learning) to identify optimal cut points in the Neural Network architecture and assign these partitions to specific edge nodes. The method includes a profiling phase to gather latency and energy data.

The framework achieves a significant reduction in inference latency (e.g., 20-40%) compared to pure offloading or local-only execution. It maintains high model accuracy while balancing the energy load across the cluster, extending the operational lifetime of battery-powered nodes. It successfully identifies optimal partition points dynamically as network conditions change.

The paper provides a compelling solution to the 'resource wall' in edge AI by unifying data and model parallelism. However, the complexity of the proposed scheduler might introduce latency penalties that offset the gains for smaller models. The assumption of linearity in resource scaling is also a potential weak point in real-world heterogeneous networks.