MPEG Column: 156th MPEG Meeting

By | | , , ,

Author: Christian Timmerer, christian.timmerer@aau.at
Affiliation: Alpen-Adria-Universität (AAU) Klagenfurt, Austria & Bitmovin Inc.
Web site: http://timmerer.com

At the 156th MPEG meeting in Hangzhou, China, MPEG Video Coding (WG 4) advanced Video Coding for Machines (VCM, ISO/IEC 23888-2) to Final Draft International Standard (FDIS) and Feature Coding for Machines (FCM, ISO/IEC 23888-4) to Draft International Standard (DIS). VCM compresses video for analysis after decoding, while FCM compresses the intermediate features exchanged between parts of a model. In parallel, MPEG Systems (WG 3) is exploring how to package both types of bitstream for storage and delivery. The full press release is available here.

Video Coding for Machines

VCM combines a conventional video codec with tools that adapt the video before encoding and process the decoded signal before analysis. An encoder can reduce spatial resolution, transmit fewer frames, or allocate more bits to regions that matter for the downstream task. The receiver can then apply operations such as upsampling or frame synthesis before passing the reconstructed video to the vision model [Eimon 2026].

In resolving comments on the VCM draft, WG 4 addressed the delivery of neural network weights and set an upper limit on the number of regions of interest. The group also updated the reference software so that the VCM multiplexer and demultiplexer can operate independently of the Versatile Video Coding (VVC) Test Model. This allows the reference pipeline to use an alternative inner codec such as High Efficiency Video Coding (HEVC). Other updates configure bit-depth restoration automatically based on day or night conditions and add supplemental signaling related to decoder timing and buffering. WG 4 also established a study on verification testing.

Feature Coding for Machines

The input to an FCM encoder is a feature tensor, an array of activations produced by the first part of a model. The encoder can transform the tensor into a more compact representation and quantize its values for transmission. The receiver reconstructs the tensor and passes it to the remaining layers. The split point affects how much computation each device performs, how much data the sender transmits, and which features are available to downstream tasks [Eimon 2026, Yang 2024].

The FCM DIS specifies the bilinear interpolation used in reconstruction and the bit widths of floating-point syntax elements. Its levels account for the neural network used to invert the feature transform and the size of the transformed tensor. WG 4 also simplified parts of the reconstruction network, introduced lower-complexity integer convolution, and adapted feature refinement to the distribution of the reconstructed values. The draft supports temporal scalability and aligns the ordering of picture-level metadata with that of the inner coded data to support random access.

Separately, WG 4 updated channel rearrangement and nonlinear quantization in the FCM Test Model and introduced a method based on singular value decomposition to estimate channel importance. The group updated the Common Test and Training Conditions with a single-layer feature-transform configuration, 4:2:0 input for the HEVC Main 10 inner codec, and a revised configuration for the Neural Network Coding (NNC) inner codec. Further studies will examine FCM levels and deterministic operation for conformance testing.

Systems Carriage for VCM and FCM

Through the Exploration on Systems technology for AI-based media standards (SyfAI), MPEG Systems (WG 3) is studying how to carry VCM and FCM bitstreams. The Hangzhou outputs include separate preliminary carriage drafts for VCM and FCM [MPEG156]. The VCM proposal describes how to package single-layer VCM streams in the ISO Base Media File Format (ISOBMFF, ISO/IEC 14496-12), either as a file for storage or as an initialization segment followed by media segments for streaming.

The VCM proposal also describes how the container conveys timing and decoder configuration. An inner-coded picture and its restoration data share the same timed sample. A picture synthesized at the decoder has its own timed sample carrying the synthesis instructions. The proposed configuration box carries initialization information for both VCM and its inner codec.

Offshore Wind Turbine Inspection

When several inspection drones share a limited wireless link, the choice of what to send depends on how much analysis each drone can perform onboard and what the ground station needs to combine observations from different cameras.

DORBINE [DORBINE], a collaboration between the University of Klagenfurt and AIR6 Systems funded by the Austrian Research Promotion Agency (FFG), aims to inspect wind turbine blades while the turbines continue generating power. In their IEEE Access paper, Leopold et al. describe how several drones divide the work of observing the turbine, capturing close-up blade imagery, and measuring blade deflection. A ground station coordinates the drones and combines data from their cameras. Experiments on a scaled turbine model demonstrated persistent blade identification and deflection estimation during rotation [Leopold 2026].

Figure 1 shows how VCM and FCM could fit into a DORBINE-style inspection. In the VCM path, the drone encodes video, and the ground station reconstructs the frames and runs the full inspection model. In the FCM path, the drone runs the first layers of the model and encodes their output. The ground station reconstructs those features and runs the remaining layers. In both paths, the sender packages the bitstream for storage or transmission, and the receiver extracts it and passes it to the appropriate decoder.

Figure 1: Proposed VCM and FCM workflow for drone-based wind turbine inspection. Both paths support blade identification and deflection estimation at the ground station.

Research Directions for the ACM SIGMM Community

Compression for machine accuracy and human review. How much can VCM reduce bitrate while preserving small-defect recall and keeping deflection estimates within a target error? Compare how encoders allocate bits across blade regions and frames, and assess whether engineers can verify suspected defects from selected clips [Eimon 2026].

Adaptive split inference. When should a drone send VCM video, transmit FCM features, or run the full model onboard? Replay the same inspection under different network conditions and onboard compute budgets. Compare bitrate, time to result, and onboard energy per inference at comparable task performance [Eimon 2026].

Feature reuse across tasks and model versions. Can one compressed feature stream support blade identification, surface defect detection, and deflection estimation? Test which tasks can share a representation and whether a ground-station model can be upgraded while remaining compatible with feature extractors already deployed on drones [Yang 2024].

Coordinated coding across drones. When several drones observe the same rotating blade over a shared uplink, which views should receive more of the available bitrate? Compare independent encoding with a scheduler that prioritizes complementary views. Measure how synchronization errors, missing views, and compression affect the combined estimate [Leopold 2026].

Carriage and recovery under variable network conditions. What information does a receiver need to join a live stream, recover after segment loss, or resume inference after a configuration change? Measure the time to the first usable result and the effect of stale or missing inputs on the inspection outcome [MPEG156].

Reproducible end-to-end benchmarks. Pair synchronized footage and task annotations with recorded network traces. Compare conventional video coding with remote inference, VCM, FCM, and fully onboard inference on the same source material. Hold out entire turbines or flights to test whether the results generalize beyond the sequences used for tuning.

Concluding Remarks

The Hangzhou milestones give the multimedia research community a basis for evaluating coding, inference, and delivery together. Offshore wind turbine inspection offers a concrete test of whether these technologies can deliver reliable blade measurements within the bandwidth and energy available during a flight.

The 157th MPEG meeting will be held in Brisbane, Australia, from January 18 to 22, 2027. Click here for more information about MPEG meetings and ongoing developments.

References

Bookmark the permalink.