SOURCE-LINKED INTELLIGENCE

A Measurement Study of LLM Inference Trade-offs Across Edge Continuum Hardware

arXiv · AI, language, vision and robotics · article · Sep 8, 2026 · UTC

Large language models (LLMs) are increasingly used as backends for intelligent web services, but serving them across the edge continuum requires balancing quality, latency, model footprint, and energy. This paper presents a controlled measurement study of self-hosted LLM inference across edge and near-edge deployment nodes: an NVIDIA Jetson AGX Orin and a near-edge server with CPU-only and GPU-enabled inference modes. We evaluate multiple open-weight LLMs and quantization variants using a fixed question-answering workload, and compare them against GPT-4o as a cloud-hosted accuracy and latency

Read original source ↗ Open in workspace

recordType

paper

region

Global