Neural pre-processing can substantially improve the rate-distortion efficiency of Adaptive Bitrate (ABR) video streaming, but existing solutions either require non-standard client-side filters or are too complex for real-time deployment on CPU. In this context, we propose LoResNet, a lightweight learned downscaler architecture paired with a standard, fixed bicubic upsampler. Codec-awareness is learned once by a high-capacity teacher trained through the codec via a state-of-the-art surrogate gradient. We find that this knowledge transfers to lightweight architectures by distilling in the latent (pre-codec) domain, removing the codec from student training and enabling fast, exact-gradient learning that preserves the teacher's gains. This makes a broad architecture sweep affordable, yielding a (FLOPs, BD-Rate) Pareto frontier and three deployment-ready points: LoResNet-fast, -medium, and -slow, spanning an order of magnitude in complexity. The fast point requires only 1.48 GFLOPs, up to 403× fewer operations than the ProgDown-Lite teacher, and processes 1080p at 165 fps on a CPU (up to ∼10× faster than the teacher), while still delivering consistent BD-Rate gains over Lanczos across x264, x265 and VVenC, up to -8.77% in PSNR and -18.45% in VMAF.
Key Idea: Distilling Codec-Awareness in the Latent Domain
Training a codec-aware downscaler end-to-end (e.g. with a surrogate gradient such as SCALED) requires running the actual codec in the training loop, which is affordable for a single high-capacity teacher but prohibitive for sweeping many lightweight student architectures. LoResNet decouples the two: a teacher is trained once, through the codec, to produce codec-aware low-resolution outputs. Lightweight students are then trained to match the teacher's pre-codec latent representation directly, via a standard exact-gradient regression loss - no codec, no surrogate, no per-architecture codec calls. This turns an expensive per-model codec-in-the-loop search into a cheap architecture sweep, while preserving most of the teacher's rate-distortion gains.
Key Results
-8.77%
BD-Rate PSNR vs Lanczos
-18.45%
BD-Rate VMAF vs Lanczos
403×
fewer FLOPs than the teacher
165 fps
1080p on CPU (LoResNet-fast)
Contributions
🧪 Latent distillation Transfers codec-awareness from a high-capacity teacher to lightweight students by matching pre-codec latents, removing the codec from student training entirely.
⚡ Real-time CPU deployment LoResNet-fast runs at 165 fps on 1080p CPU with only 1.48 GFLOPs, up to 403× cheaper than the teacher.
📈 Deployment-ready Pareto frontier Three points - fast, medium, slow - spanning an order of magnitude in complexity, validated across x264, x265 and VVenC.
Citation
@INPROCEEDINGS{pesnel:hal-05749683,
author = {Pesnel, Esteban and Le Tanou, Julien and Ropert, Michael
and Roumy, Aline and Maugey, Thomas},
title = {LoResNet: Light Codec-Aware Downscaling via Latent
Distillation for Real-Time ABR Streaming},
booktitle = {IEEE International Conference on Visual
Communications and Image Processing (VCIP)},
address = {Singapore, Singapore},
year = {2026},
month = {dec},
hal_id = {hal-05749683}
}
