Category Archives: Feature

At the 156th MPEG meeting in Hangzhou, China, MPEG Video Coding (WG 4) advanced Video Coding for Machines (VCM, ISO/IEC 23888-2) to Final Draft International Standard (FDIS) and Feature Coding for Machines (FCM, ISO/IEC 23888-4) to Draft International Standard (DIS). VCM compresses video for analysis after decoding, while FCM compresses the intermediate features exchanged between parts of a model. In parallel, MPEG Systems (WG 3) is exploring how to package both types of bitstream for storage and delivery. The full press release is available here.

Video Coding for Machines

VCM combines a conventional video codec with tools that adapt the video before encoding and process the decoded signal before analysis. An encoder can reduce spatial resolution, transmit fewer frames, or allocate more bits to regions that matter for the downstream task. The receiver can then apply operations such as upsampling or frame synthesis before passing the reconstructed video to the vision model [Eimon 2026].

In resolving comments on the VCM draft, WG 4 addressed the delivery of neural network weights and set an upper limit on the number of regions of interest. The group also updated the reference software so that the VCM multiplexer and demultiplexer can operate independently of the Versatile Video Coding (VVC) Test Model. This allows the reference pipeline to use an alternative inner codec such as High Efficiency Video Coding (HEVC). Other updates configure bit-depth restoration automatically based on day or night conditions and add supplemental signaling related to decoder timing and buffering. WG 4 also established a study on verification testing.

Feature Coding for Machines

The input to an FCM encoder is a feature tensor, an array of activations produced by the first part of a model. The encoder can transform the tensor into a more compact representation and quantize its values for transmission. The receiver reconstructs the tensor and passes it to the remaining layers. The split point affects how much computation each device performs, how much data the sender transmits, and which features are available to downstream tasks [Eimon 2026, Yang 2024].

The FCM DIS specifies the bilinear interpolation used in reconstruction and the bit widths of floating-point syntax elements. Its levels account for the neural network used to invert the feature transform and the size of the transformed tensor. WG 4 also simplified parts of the reconstruction network, introduced lower-complexity integer convolution, and adapted feature refinement to the distribution of the reconstructed values. The draft supports temporal scalability and aligns the ordering of picture-level metadata with that of the inner coded data to support random access.

Separately, WG 4 updated channel rearrangement and nonlinear quantization in the FCM Test Model and introduced a method based on singular value decomposition to estimate channel importance. The group updated the Common Test and Training Conditions with a single-layer feature-transform configuration, 4:2:0 input for the HEVC Main 10 inner codec, and a revised configuration for the Neural Network Coding (NNC) inner codec. Further studies will examine FCM levels and deterministic operation for conformance testing.

Systems Carriage for VCM and FCM

Through the Exploration on Systems technology for AI-based media standards (SyfAI), MPEG Systems (WG 3) is studying how to carry VCM and FCM bitstreams. The Hangzhou outputs include separate preliminary carriage drafts for VCM and FCM [MPEG156]. The VCM proposal describes how to package single-layer VCM streams in the ISO Base Media File Format (ISOBMFF, ISO/IEC 14496-12), either as a file for storage or as an initialization segment followed by media segments for streaming.

The VCM proposal also describes how the container conveys timing and decoder configuration. An inner-coded picture and its restoration data share the same timed sample. A picture synthesized at the decoder has its own timed sample carrying the synthesis instructions. The proposed configuration box carries initialization information for both VCM and its inner codec.

Offshore Wind Turbine Inspection

When several inspection drones share a limited wireless link, the choice of what to send depends on how much analysis each drone can perform onboard and what the ground station needs to combine observations from different cameras.

DORBINE [DORBINE], a collaboration between the University of Klagenfurt and AIR6 Systems funded by the Austrian Research Promotion Agency (FFG), aims to inspect wind turbine blades while the turbines continue generating power. In their IEEE Access paper, Leopold et al. describe how several drones divide the work of observing the turbine, capturing close-up blade imagery, and measuring blade deflection. A ground station coordinates the drones and combines data from their cameras. Experiments on a scaled turbine model demonstrated persistent blade identification and deflection estimation during rotation [Leopold 2026].

Figure 1 shows how VCM and FCM could fit into a DORBINE-style inspection. In the VCM path, the drone encodes video, and the ground station reconstructs the frames and runs the full inspection model. In the FCM path, the drone runs the first layers of the model and encodes their output. The ground station reconstructs those features and runs the remaining layers. In both paths, the sender packages the bitstream for storage or transmission, and the receiver extracts it and passes it to the appropriate decoder.

Research Directions for the ACM SIGMM Community

Compression for machine accuracy and human review. How much can VCM reduce bitrate while preserving small-defect recall and keeping deflection estimates within a target error? Compare how encoders allocate bits across blade regions and frames, and assess whether engineers can verify suspected defects from selected clips [Eimon 2026].

Adaptive split inference. When should a drone send VCM video, transmit FCM features, or run the full model onboard? Replay the same inspection under different network conditions and onboard compute budgets. Compare bitrate, time to result, and onboard energy per inference at comparable task performance [Eimon 2026].

Feature reuse across tasks and model versions. Can one compressed feature stream support blade identification, surface defect detection, and deflection estimation? Test which tasks can share a representation and whether a ground-station model can be upgraded while remaining compatible with feature extractors already deployed on drones [Yang 2024].

Coordinated coding across drones. When several drones observe the same rotating blade over a shared uplink, which views should receive more of the available bitrate? Compare independent encoding with a scheduler that prioritizes complementary views. Measure how synchronization errors, missing views, and compression affect the combined estimate [Leopold 2026].

Carriage and recovery under variable network conditions. What information does a receiver need to join a live stream, recover after segment loss, or resume inference after a configuration change? Measure the time to the first usable result and the effect of stale or missing inputs on the inspection outcome [MPEG156].

Reproducible end-to-end benchmarks. Pair synchronized footage and task annotations with recorded network traces. Compare conventional video coding with remote inference, VCM, FCM, and fully onboard inference on the same source material. Hold out entire turbines or flights to test whether the results generalize beyond the sequences used for tuning.

Concluding Remarks

The Hangzhou milestones give the multimedia research community a basis for evaluating coding, inference, and delivery together. Offshore wind turbine inspection offers a concrete test of whether these technologies can deliver reliable blade measurements within the bandwidth and energy available during a flight.

The 157th MPEG meeting will be held in Brisbane, Australia, from January 18 to 22, 2027. Click here for more information about MPEG meetings and ongoing developments.

References

• [DORBINE] DORBINE, Wind-turbine inspection with autonomous drone swarms. https://dorbine.itec.aau.at/.
• [Leopold 2026] Mario Leopold, Klaus Schoeffmann, and Farzad Tashtarian. 2026. In-Service Wind Turbine Blade Inspection: A Multi-UAV Architecture Design and Preliminary Validation. IEEE Access 14 (2026), 142383–142399. https://doi.org/10.1109/ACCESS.2026.3730692.
• [Eimon 2026] Md Eimran Hossain Eimon, Velibor Adzic, Hari Kalva, and Borko Furht. 2026. Emerging Standards for Machine-to-Machine Video Coding. In Proceedings of the 5th Mile-High Video Conference (MHV ’26). Association for Computing Machinery, New York, NY, USA, 128–134. https://doi.org/10.1145/3789239.3793282.
• [Yang 2024] Wenhan Yang, Haofeng Huang, Yueyu Hu, Ling-Yu Duan, and Jiaying Liu. 2024. Video Coding for Machines: Compact Visual Representation Compression for Intelligent Collaborative Analytics. IEEE Transactions on Pattern Analysis and Machine Intelligence 46, 7 (July 2024), 5174–5191. https://doi.org/10.1109/TPAMI.2024.3367293.
• [MPEG156] MPEG. 2026. Outputs from the 156th MPEG Meeting. https://www.mpeg.org/meetings/mpeg-156/.

JPEG XE becomes an International Standard

The 112th JPEG meeting was held in Leiria, Portugal, from 6 to 10 July 2026.

This meeting was marked by several major achievements. JPEG XE final text of the International Standard was issued, and it is expected to be published in the near future by both ISO/IEC and ITU-T as twin texts. This is a major milestone, as JPEG XE defines the first standard for event-based vision, with the support of major industrial and academic players and under the auspices of three standardization organizations.

The following sections summarise the main highlights of the 112th JPEG meeting.

• JPEG XE becomes an international standard.
• JPEG Trust has major advances in its different parts.
• JPEG DNA proceeds as Draft International Standard.
• JPEG Pleno continues work on quality evaluation of coded light fields.
• JPEG XS enables slice-level embedded metadata.
• JPEG AI proceeds with the implementation’s performance evaluation.
• JPEG RF exploration launches a call for datasets.

JPEG XE

The joint effort between ITU-T SG21 and ISO/IEC JTC1/SC29/WG1 on JPEG XE continues to make significant progress toward establishing a comprehensive international framework for event-based vision coding. At its 112th meeting, the JPEG Committee completed the final International Standard text of ISO/IEC 26112-1 after successful approval of the DIS ballot and requested its publication. This milestone marks JPEG XE as the first international standard for lossless coding of visual events produced by event vision sensors, providing a standardized and interoperable foundation for the rapidly growing event-based vision ecosystem. Developed jointly between ISO, IEC and ITU-T, the standard ISO/IEC 26112 will also be published as ITU-T Recommendation T.841.1 in the near future.

The JPEG Committee also completed the DIS texts of ISO/IEC 26112-2, Profiles and Levels, and ISO/IEC 26112-3, Reference Software, both of which were approved for progression to the DIS stage. Under the joint ISO, IEC and ITU-T collaboration, these standards will also become ITU-T Recommendations T.841.2 and T.841.3, respectively. ISO/IEC 26112-2 includes the Embedded Lossless Constrained Profile, specifically designed to enable interoperability with the MIPI CSI-2 v4.2 Event Stream Protocol (ESP) specification. The profile represents the successful outcome of a close liaison between the MIPI Alliance and the JPEG Committee to promote industry-wide interoperability for event-based imaging systems.

Work will now continue on ISO/IEC 26112-4, which will define formal compliance testing procedures, and on ISO/IEC 26112-5, which will specify file formats and metadata structures for the storage and exchange of event data. In particular, Part 5 will introduce mechanisms enabling synchronization of event timestamps with real-world atomic time references, supporting demanding industrial, scientific, automotive, and machine vision applications. In parallel, the JPEG Committee will explore advanced coding technologies aimed at further improving both lossless and lossy compression efficiency for event data.

The JPEG XE Joint Ad Hoc Group, established under the collaboration between ITU-T SG21/Q7 and ISO/IEC JTC1/SC29/WG1, was reestablished to continue the development of the JPEG XE family of standards. The JPEG Committee remains committed to delivering a unified, open, and internationally recognized framework for event-based vision technologies, accelerating interoperability, innovation, and adoption across this emerging field. If you are interested in contributing to this effort, please consider joining this joint AHG.

JPEG Trust

At the 112th JPEG meeting in Leiria, major advances were made across all parts of JPEG Trust. The 2nd edition of JPEG Trust Part 1 – Core Foundation has been completed and is progressing towards publication, incorporating assertions for attributions and declarations of rights. JPEG Trust Part 2 – Trust Profiles and Reports, which extends the framework specified in Part 1 with functionalities for Trust Profiles and Trust Reports, was promoted to FDIS stage. JPEG Trust Part 3 – Media Asset Watermarking, incorporating specific tools and associated assessment methodologies for usage scenarios that rely on invisible watermarking of media assets, is currently at the DIS stage. JPEG Trust Part 4 – Reference Software is also at the DIS stage, with further extensions planned to include additional software implementations and test datasets.

Looking ahead, JPEG Trust is expanding its activities to address emerging requirements for media authenticity and transparency, while continuing to evolve the JPEG Trust framework. A revised structure for the JPEG Trust family of standards is being considered to provide a clearer separation between the core trust framework, provenance, attribution, and rights declarations, watermarking, and emerging detection technologies.

Preparations for the first JPEG Trust Summit, to be held on 12–13 November 2026 in London, are well underway. More information on upcoming events related to JPEG Trust can be found on the jpeg.org website.

JPEG DNA

At its 112th meeting, the JPEG Committee continued the standardization and validation activities for JPEG DNA Part 1 (ISO/IEC 25508-1), which is currently undergoing DIS ballot. The DIS stage follows a multi-year standardization effort that began with the Final Call for Proposals at the 99th JPEG meeting and progressed through the creation of the Verification Model at the 102nd meeting, the first Working Draft at the 103rd meeting, the Committee Draft, and the study DIS text produced at the 108th meeting. A significant part of the meeting was dedicated to analyzing and continuing the JPEG DNA Core Experiments. In particular, the results of Core Experiment 11 (CE11) were discussed, providing further validation of the end-to-end JPEG DNA workflow on synthetic DNA. The experiment demonstrated the feasibility of recovering and decoding the information stored in DNA while also highlighting areas where further improvements to the decoding process are desirable, especially for the header encoding/decoding. Based on these results, the JPEG Committee identified the need for a further Core Experiment, CE12, aimed in particular at improving and further validating the decoding process under realistic synthesis and sequencing conditions.

JPEG Pleno

JPEG Pleno Light Field Quality Assessment advanced its work on subjective and objective quality assessment activities. The group reviewed the user documentation for the subjective assessment software, further investigated the performance of the deep-learning-based metric currently under consideration for inclusion in the standard, and discussed the next steps for objective quality assessment standardization activities.

At its 112th meeting, the JPEG Committee approved the release of the Common Test Conditions (CTC) for JPEG Pleno Light Field Coding v4.0 document, establishing a unified framework to ensure consistency and comparability across core experiments. The document defines testing procedures, software reference configurations, and environmental settings required to evaluate and compare light field coding solutions accurately. The CTC document is supported by the JPEG Pleno Light Field Coding (LFC) CTC Toolkit, a modular software toolkit with automated scripts for file conversion, coding, and quality metrics to assist in executing the CTC guidelines. AHG.

JPEG XS

JPEG XS, the lightweight low-latency image and video compression standard, continues to expand its capabilities while preserving its core strengths of visually lossless quality, ultra-low latency, and low implementation complexity. Following the 112th JPEG meeting, the JPEG Committee is happy to announce the publication of ISO/IEC 21122-1:2024 AMD1 that introduces the SYM marker to allow user-defined metadata to be embedded at the slice level. This feature can be leveraged to facilitate the needs of video transmission in AR/VR applications. Furthermore, ISO/IEC 21122-2:2024 AMD1, which adds new levels, sublevels, and a frame buffer level to better address proxy-stream extraction use cases, completed the DIS approval process and was issued for publication as a new amendment. The JPEG Committee also progressed ISO/IEC 21122-2:2024 AMD2, which introduces the new CMainBayer profile for raw Bayer content. The profile is designed to simplify implementations while maintaining backward decoding compatibility with existing MainBayer and HighBayer decoders. Following a successful technical evaluation, the amendment was approved to advance to the DIS stage, marking the next step toward standardization.

JPEG AI

At its 112th meeting, the JPEG Committee continued advancing JPEG AI, reviewing progress in its Core Experiments on hardware deployment performance, bit-exact reconstruction, and objective and subjective evaluation. New results were reported for a heterogeneous CPU–GPU–FPGA deployment in which the codec is split at the quantized latent representation, mapping the main analysis and synthesis transforms to GPU and the hyperprior transforms to FPGA, while entropy coding and bitstream handling remain on CPU, and the rate–distortion behaviour is preserved. Evaluated on 100 images at native 2K resolution across five bitrates, this deployment achieved a runtime close to GPU-only execution while reducing energy consumption, yielding an improved end-to-end energy-delay product with respect to both GPU-only and FPGA-only execution. Progress was also reported towards cross-platform reproducible reconstruction, where floating-point convolutions are replaced by per-channel integer arithmetic derived from a two-step quantization procedure; all convolutional layers of the base operating point encoder have now been quantized, with the remaining components to follow. In parallel, a systematic evaluation of JPEG AI was set in motion, encompassing a new dataset and updated common training and test conditions, ablation studies, objective and subjective comparisons with state-of-the-art conventional and learning-based codecs, and a computational complexity assessment. Further contributions addressed low-latency FPGA deployment and compressed-domain processing for geospatial segmentation. AHG.

JPEG RF

At the 112th JPEG meeting, the JPEG Committee made significant progress in its JPEG Radiance Fields activity, which explores future standardization needs for compact and high-quality coding of radiance field representations. A Call for Datasets was finalized to broaden the content available for future benchmarking, advanced discussions on Common Test Conditions and evaluation protocols, and the roadmap toward a future Call for Proposals was reviewed. The meeting also progressed work on subjective and objective quality assessment pipelines, including an extensive evaluation of objective metrics and the identification of promising directions for future anchor coding technologies. These activities strengthen the technical foundation for a transparent and reproducible JPEG RF standardization process.

“At the 112th meeting, the JPEG Committee released JPEG XE as the first ever International Standard for visual event lossless coding, providing a crucial tool for interoperability between event-based imaging systems. This will be instrumental for the success of imaging and machine vision applications that rely on event sensors.” said Prof. Touradj Ebrahimi, the Convenor of the JPEG Committee.

The 155th MPEG meeting took place in Geneva, Switzerland, from July 13 to 17, 2026. The official MPEG press release can be found here. This report highlights key outcomes from the meeting, with a focus on research directions relevant to the ACM SIGMM community:

• Gaussian Splat Coding Use Cases and Current Status
• Call for Proposals on Media Authenticity and Provenance Indication
• Joint Call for Proposals on video compression with capability beyond VVC
• Joint ITU-T SG 21 and ISO/IEC JTC 1/SC 29 Workshop on Media Streaming Services
• Exploration on Systems technologies for AI-based media standards (SyfAI)

Gaussian Splat Coding Use Cases and Current Status

The previous MPEG column [MPEG154] introduced MPEG’s Gaussian Splat Coding (GSC) exploration, its two tracks (I-3DGS and A-3DGS), and a first set of 27 draft use cases. At the 155th meeting, WG 2 (Technical Requirements) approved an updated document (WG 2 N531) that drops the draft qualifier and adds two use cases.

The additions matter less for their number than their nature: both concern storage and delivery rather than compression. Use case 28 stores a static splat asset in a single file together with a cover image, thumbnail, and audio annotation, so that a capable receiver renders it interactively while a legacy receiver displays the cover image, with no modification of the file. Use case 29 is the temporal counterpart, delivering a dynamic splat track by HTTP adaptive streaming alongside synchronized audio, a pre-rendered video track serving as preview and fallback, and timed text, with representations offered per bitrate, level of detail, spherical harmonics subset, or attribute subset. The application-oriented use cases are unchanged, although the derived requirements are being refactored, notably by separating attribute subset scalability from random access.

GSC remains an exploration activity, but a busy one: 83 input contributions and nine Joint Exploration Experiments re-conducted across WG 2, WG 4 (Video Coding), WG 5 (Joint Video Experts Team, JVET), and WG 7 (3D Graphics and Haptics), with WG 1 (JPEG) now engaged on quality metrics and WG 4 examining Neural Network Coding (NNC) as a compression tool. On the fast track, GS4 (Video-based Point Cloud Compression, V-PCC, Amd. 1) and GS5 (Geometry-based Point Cloud Compression, G-PCC, Amd. 1) progress on reference software and conformance, while the JVET codec track carries five competing proposals. The Call for Proposals (CfP) still has no date, and I-3DGS planning is more mature than A-3DGS. The clearest message from the meeting is that single-frame compression is essentially solved and that the difficulty and the expected gain now lie in dynamic content.

Research aspects: Temporal coding is no longer one open question among many, but the central one, covering inter-prediction for anisotropic primitives, deformation models, and primitive correspondence across frames. Quality assessment remains unresolved: current test conditions score rendered views with peak signal-to-noise ratio (PSNR), structural similarity (SSIM), and learned perceptual image patch similarity (LPIPS) along a pose trace, a fragile proxy for artifacts such as floaters and popping; the involvement of JPEG makes this a good moment to contribute. Use case 29 gives the streaming question a concrete form: when quality varies along several orthogonal axes at once, what is the right abstraction for a representation, and what should an adaptation algorithm optimize?

Call for Proposals on Media Authenticity and Provenance Indication

At the 155th meeting, WG 2 approved a CfP on media authenticity and provenance indication with the MPEG Systems technologies (WG 2 N535), following an exploration phase in WG 3 (Systems) that produced requirements, a gap analysis, and several technical proposals (WG 3 N1842, attached to the call). The scope is the system layer: carriage and signaling of metadata that allows a receiver to verify that a media asset comes from a trusted producer and to convey provenance information, rather than the definition of a provenance language itself.

Three interfaces are addressed, namely (i) elementary streams and non-timed items, (ii) the file format level (ISO Base Media File Format (ISOBMFF) and Common Media Application Format (CMAF)), and (iii) packaged delivery (Dynamic Adaptive Streaming over HTTP (DASH), CMAF, MPEG Media Transport (MMT), and MPEG-2 Transport Stream (MPEG-2 TS)). The use cases span deepfakes and manipulated news media, forgery in insurance claims, surveillance and investigations, labeling of AI-generated content, and legitimate modifications such as editing, transcoding, ad insertion, and archival preservation. Detecting whether an asset is fake without embedded data is explicitly out of scope.

A response may be a complete solution or a single tool addressing one or more requirements, and is evaluated on requirement coverage, computational complexity, the bandwidth needed to convey the authenticity information, compatibility and backward compatibility with existing MPEG standards, and extensibility, with self-evaluation tables provided in the annexes. Proposals are submitted as input contributions to the 157th meeting in Brisbane, Australia, by January 13, 2027. Review at that meeting will produce one or more working drafts or new work item proposals, and the preliminary development plan targets Committee Draft (CD) or Committee Draft Amendment (CDAM) at the 158th meeting, Draft International Standard (DIS) or Draft Amendment (DAM) at the 159th, and Final Draft International Standard (FDIS) or Final Draft Amendment (FDAM) in early 2028.

Research aspects: The requirements make this more interesting than a signing exercise. A conventional signature breaks on any bit change, yet the call demands that verification survive transcoding, dropped scalability layers, representation switching, splicing, late binding, and loss of frames or audio, and that the unmodified remainder of a presentation stay verifiable after an authorized edit. Designing verification structures with that granularity and quantifying their overhead per segment against the added latency in live scenarios is an open problem at the intersection of cryptography and streaming systems. A second question is joint verification: establishing that a given audio track was the one the producer intended to accompany a given video, including their synchronization, cannot be achieved by signing each file separately. Finally, interoperability with provenance schemes defined elsewhere, and the question of what a receiver should do with a verification result, leave room for work on usable trust signaling.

Joint Call for Proposals on Video Compression with Capability Beyond VVC

The draft of this call was described in the previous MPEG column [MPEG154], and JVET has now issued the final version (JVET-AQ2021 [JVET-AQ2021]), approved at its 43rd meeting in Geneva in July 2026. The target is video coding technology that significantly exceeds Versatile Video Coding (VVC) in compression, implementability, applicability across content types, and features such as latency, robustness, and scalability, benchmarked against the VVC Main 10 profile. Four test cases are defined: one for improved compression without runtime limits, and three in which the aggregate encoder run time is constrained to 5x, 1x, and 0.2x that of the VVC Test Model (VTM) anchor. All are evaluated over seven categories covering (i) standard dynamic range (SDR) random access at ultra-high definition (UHD) and (ii) at high definition (HD), (iii) SDR low delay HD, (iv) high dynamic range (HDR) with perceptual quantizer (PQ) and (v) with hybrid log-gamma (HLG) transfer functions, (vi) gaming, and (vii) user-generated content. A separate track invites technology offering additional functionality, together with proposals on how its benefit should be assessed.

Formal subjective testing uses degradation category rating, with objective results reported as PSNR, multiscale SSIM (MS-SSIM), and, for PQ content, weighted PSNR. The schedule is tight: anchors have been available since May 2026, registration runs from August 1 to September 1, 2026, and the main package of bitstreams, reconstructed sequences, and binaries must reach the test coordinator on physical media by October 26, 2026. Subjective assessment then runs until late December, blind cross-checking by other proponents is mandatory, and proposals are evaluated at the 45th JVET meeting in January 2027, with an initial test model selected during 2027 and the standard targeted for October 2029. Participation is not free: up to EUR 20k per test case is charged to cover the hiring of test subjects.

Research aspects: Two design choices in the call are worth attention. First, a supplemental set of sequences is disclosed to proponents only after the decoder binaries have been submitted, and results on it are due six weeks later, which turns the call into a held-out generalization test. Read together with the ban on training on test sequences and the obligation to disclose training material, this makes out-of-domain behavior of learned coding tools measurable at scale, and reporting it well is a contribution in itself. Second, run time is aggregated as the sum over threads, which measures total compute rather than latency and therefore reads very differently for a massively parallel or GPU-resident design than for a sequential one. How to characterize the rate, distortion, and complexity trade-off fairly across such architectures, and how to value functionality such as scalability or error resilience against a plain bitrate gain, are open questions the call poses rather than answers.

Joint ITU-T SG 21 and ISO/IEC JTC 1/SC 29 Workshop on Media Streaming Services

On July 14, 2026, ITU-T SG 21 and MPEG Systems held a joint half-day workshop in Geneva, collocated with their meetings, on “Media Streaming Service, What’s next” (call for presentations, program). The premise was that after the transitions from analog to digital, enabled by MPEG-2 Systems, and from broadcast to over the top (OTT), enabled by MPEG-DASH, integrated networks, edge computing, and AI are driving another shift, and that both bodies wanted industry views before committing to new work. On challenges, Netflix spoke on the shortcomings and evolution of ISOBMFF, Bitmovin on where streaming trends meet the container, ETRI on future streaming services, and Huawei on ultra-low latency communication and streaming. On opportunities, 5G-MAG addressed standards and open source, DVB heterogeneous networks, and 3GPP SA4 delivery beyond OTT. The closing session was an open mic with questions and answers from both speakers and the audience.

The wrap-up set the existing systems standards, namely MPEG-2 TS, ISOBMFF, DASH, and CMAF as well as the volumetric, scene, and augmented reality (AR) formats, plus ongoing work on authenticity, SyfAI (see below), common metadata, and Gaussian splats, against what industry actually asked for: low latency, low overhead, AI-driven media, and open source software. The resulting agenda is an exploration of a better or new container format, an analysis of overhead, processing-friendly metadata, including JSON and ontology issues, WebCodecs integration, and ingest. Just as notable are the conclusions about process: more open access and industry involvement, faster turnaround, possibly a new home or outlet for this work, and software with interoperability testing from day one rather than reference software at the end. Three ad hoc groups were set up: (i) on MP4, DASH, and file formats, reviewing ISOBMFF against emerging transports and the delivery of AI input and output data; (ii) on vision, analyzing bottlenecks and requirements beyond ISOBMFF; and (iii) on working methods.

Research aspects: Two of these items are directly researchable. Container overhead is asserted more often than measured, and as segments shrink toward frame level for low latency, the ratio of container to media bytes grows; a careful comparison across ISOBMFF, CMAF, and object-based transports at equal latency would inform the exploration rather than follow it. Media over QUIC (MOQ) is the bigger change, since it replaces the segment with the object as the unit of delivery and pulls streaming and real-time communication into a single design space, which reopens rate adaptation, interaction with congestion control, caching, and relay behavior, and the question of what a container still contributes when the transport itself frames media. Carrying AI input and output data alongside media, finally, links this work to coding for machines and to semantic streaming.

Exploration on Systems Technologies for AI-based Media Standards (SyfAI)

Several MPEG coding standards now put a neural network inside the decoder, among them video and feature coding for machines, AI-based point cloud coding, and neural network coding. The previous MPEG column [MPEG154] described that coding side through the MPEG-AI vision document. SyfAI, an MPEG Systems exploration started at the 153rd MPEG meeting, asks the complementary and much less glamorous question: what does the surrounding infrastructure have to do so that such content can actually be stored, delivered, and played back interoperably? The underlying shift is that a media file has traditionally been self-contained, meaning that a conforming decoder and the bitstream are sufficient. Once the decoder depends on a trained model that may be selected, delivered, or updated separately, that assumption no longer holds, and the system layer has to say how a player learns which model it needs, how that model reaches it, and how both sides can be sure they are using the same one. Work so far has advanced on the most concrete piece, namely storing video coding for machines content in the MPEG file format, while proposals for carrying compressed neural networks were sent back for clearer use cases. The more interesting development is a new thread on an AI update framework, opened jointly with WG 7, which collects five topics: (i) a format for AI parameter data, (ii) a manifest for updating parameters, (iii) a repository to serve them, (iv) integrity checking, and (v) bit exactness together with conformance.

Research aspects: Each of those topics is a research problem in its own right. Distributing and updating models alongside media turns into a delivery question that looks familiar but has not been studied: when to fetch an update, how to cache and version models, what a manifest must express about capability and compatibility, and how to treat weights as a second class of asset next to the media. Conformance is harder, because a standard normally guarantees that every decoder produces identical output, whereas neural inference varies with library and hardware, so deciding what conformance means and how to test it remains open and extends the reproducibility concerns already visible in MPEG-AI. A repository of updatable weights is also an attack surface, which is why this work sits naturally beside the media authenticity call discussed above: signing and verifying a model raises the same questions one level down from signing content. And because the choice of where a model lives affects how quickly playback can start and how smoothly a player can switch, the apparently dry container questions have measurable consequences for the quality of experience (QoE).

Concluding Remarks

Taken together, the five items show MPEG engaging early with technologies whose momentum comes from outside the committee. With Gaussian splatting, it is compressing a representation that already has a renderer and a user base, which is a better starting point than earlier attempts at three-dimensional video had, even if wider adoption will depend on capture and display ecosystems as much as on coding efficiency. The authenticity call is scoped pragmatically at establishing origin, which is achievable in the near term and provides the system-level plumbing that regulation and industry are asking for. The call for video coding beyond VVC visibly reflects deployment experience, with runtime-constrained test cases treating practicality as a first-class criterion, while questions of licensing and market uptake sit largely outside MPEG itself. In systems, the joint workshop and the new ad hoc groups show a willingness to re-examine long-standing assumptions as transports such as MOQ emerge, and SyfAI stakes out the system-level questions of the AI era before they become urgent. For our community, the result is an unusually rich set of open problems in evaluation and methodology.

The 156th MPEG meeting will be held in Hangzhou, China, from October 19 to 23, 2026. Click here for more information about MPEG meetings and ongoing developments.

References

• [MPEG154] C. Timmerer. 2026. MPEG Column: 154th MPEG Meeting. SIGMultimedia Rec. 18, 2, Article 8 (April 2026).
• [JVET-AQ2021] J.-R. Ohm, M. Wien, F. Bossen. 2026. Joint Call for Proposals on video compression with capability beyond VVC, JVET-AQ2021 (July 2026).

JPEG DNA reaches Draft International Standard stage at the 111th JPEG meeting

The 111th JPEG meeting was held in virtual mode from 13 to 17 April 2026.

This meeting was marked by several major achievements. JPEG DNA, the first image format that uses a quaternary representation suitable for synthesis into nucleotide sequences to be used for storage on DNA support, reached the Draft International Standard (DIS) stage. This was ensured after a successful wet-lab experiment, including DNA synthesis/sequencing that allowed a test of reliability in real conditions.

Furthermore, the DIS stage was also reached for JPEG XE, the first International standard for coding of visual events; JPEG Pleno Light Field Quality Assessment, which establishes a model for quality evaluation of encoded light fields; JPEG Trust Media Asset Watermarking, which provides watermarking support to media asset authenticity; and JPEG Trust reference software, which will provide a valuable tool for the development of applications using this family of standards. Furthermore, JPEG Pleno Light Field coding, 2nd edition, reached the Final Draft International Stage.

The following sections summarize the main highlights of the 111th JPEG meeting.

• JPEG DNA reaches DIS stage.
• JPEG XE reaches DIS stage.
• JPEG Trust part 4 – Reference Software – reaches DIS stage.
• JPEG Pleno Light Field Quality Assessment reaches DIS.
• JPEG XS Part 2 prepares a new amendment to define a new raw Bayer profile.
• JPEG RF explores methodologies for quality evaluation of Gaussian Splatting coding distortions.
• JPEG AI explores machine-oriented applications in the compressed domain.

JPEG DNA

At its 111th meeting, the JPEG Committee reached a major milestone with JPEG DNA Part 1 (ISO/IEC 25508-1) advancing to the Draft International Standard (DIS) stage. This achievement marks the culmination of a multi-year standardization effort, beginning with the Final Call for Proposals at the 99th JPEG meeting and proceeding through the creation of the Verification Model at the 102nd meeting, the first Working Draft at the 103rd meeting, the Committee Draft, and the study DIS text produced at the 108th meeting. With DIS now reached, the core technical specification for the efficient coding of images into quaternary representations suitable for archival on DNA synthetic polymers is frozen for the first edition of the standard. The JPEG Committee expects publication of Part 1 an International Standard before the end of 2026.

The DIS milestone was accompanied by the successful completion of wet-lab experiments initiated at the 109th JPEG meeting and whose synthesis and sequencing phases were carried out between the 110th and 111th meetings. The sequenced results from the independent parties have now been delivered to the JPEG Committee and analyzed. The experiments confirmed that the information recorded in the DNA solution was successfully extracted and that the corresponding compressed images were correctly decoded, in accordance with the end-to-end workflow defined in the specification, including biochemical constraints, the recovery mechanism used to cope with the noise introduced by the synthesis and sequencing processes, and the procedures for reading the encoded image information back from the synthesized oligonucleotides. These results provide concrete validation that the current specification of JPEG DNA satisfies the conditions required for applications relying on the current state of the art in DNA synthesis and sequencing and represents the first end-to-end demonstration of the standard on real biological media.

JPEG XE

The joint effort between ITU-T SG21 and ISO/IEC JTC1/SC29/WG1 on JPEG XE is ongoing, with JPEG XE Part 1 ready to become the first of a series of standards for coding of visual events. During its 111th meeting, the JPEG Committee made substantial progress on the ongoing development of the additional JPEG XE Parts 2, 3, 4, and 5. Technical discussions advanced the work on Part 2, which will define Profiles and Levels, as well as a normative buffer model to ensure safe and interoperable decoder operation across implementations. Further enhancements were made to the code of the open source reference software, which will be published under Part 3 as a proof‑of‑concept and conformance testing implementation. Preparatory work commenced for Part 4, which will establish the structure and scope of formal conformance testing and will become the focus during the coming months. Finally, consensus was reached about Part 5, where the JPEG Committee converged on an ISOBMFF‑based file format design supporting efficient storage, streaming, and precise event timestamping. The JPEG Committee, mandated for the joint effort between ISO/IEC/JTC1/SG29 and ITU-T SG21, remains committed to the development of a comprehensive and industry-aligned standard that meets the growing demand for event-based vision technologies. This collaborative approach underscores a shared vision for a unified, international standard to accelerate innovation and interoperability in this emerging field. The JPEG XE joint AHG (ITU-T SG21 and ISO/IEC JTC1 SC29 WG1) was reestablished to continue the JPEG XE standards development. If you are interested, please consider joining this public joint AHG.

JPEG Trust

There have been major advances in all parts of JPEG Trust. Recently, a 2nd edition of JPEG Trust Part 1 – Core Foundation, was published to accommodate assertions for attributions (based on The Dublin Core metadata element set, ISO 15836-1:2017) and declarations of rights (based on the W3C ODRL Information Model 2.2). JPEG Trust Part 2 – Trust profiles and reports extends the framework specified in Part 1 with extended functionalities for Trust Profiles and Trust Reports; it has now entered the DIS stage. JPEG Trust Part 3 – Media asset watermarking, incorporating specific tools and associated assessment methodologies for usage scenarios that rely on invisible watermarking of media assets, has now been released for DIS balloting. The DIS for JPEG Trust Part 4 – Reference Software was also released for balloting; this Part will be extended in the future with additional implementations and test datasets.

A first JPEG Trust Summit is planned on 12-13 November 2026 in London. More information on upcoming events related to JPEG Trust can be found here.

JPEG Pleno

The JPEG Pleno Light Field activity analyzed the DoCR for the Final Draft International Standard (FDIS) of the 2nd edition of ISO/IEC 21794-2 (“Plenoptic image coding system (JPEG Pleno) Part 2: Light field coding”). This 2nd edition integrates AMD1 of ISO/IEC 21794-2 (“Profiles and levels for JPEG Pleno Light Field Coding”) and includes the specification of a third coding mode entitled “Slanted 4D Transform Mode” and its associated profile. It is expected that at the 112th JPEG meeting this new edition will become an International Standard (IS).

The JPEG committee made significant progress in JPEG Pleno Light Field Quality Assessment (Part 7) activities. Key developments were achieved, including the review and finalization of the DIS text for ISO/IEC 21794-7, which was approved for registration at this meeting. Core experiments on light field quality assessment were presented and discussed, contributing to the validation of subjective testing frameworks and the further development of objective metrics, including improvements to learning-based approaches.

JPEG XS

JPEG XS, the image and video compression format for transmitting visually lossless, high-quality pictures with minimal latency and low resource consumption, is currently in a mature state. Two amendments, JPEG XS Part 1 and Part 2, are expected to be published in July of 2026. JPEG XS AMD1 provides a new syntax to allow embedding of user-defined metadata at the slice-level in addition to the frame-level that was already possible. JPEG XS Part 2 amendment AMD1 provides additional levels and sublevels, as well as a new frame buffer level, to better support proxy-stream extraction use cases. At its 111th meeting, the JPEG Committee prepared another amendment (AMD2) for JPEG XS Part 2 to define a new raw Bayer profile, called CMainBayer, that supports reducing the implementation complexity compared to the MainBayer profile. It will be an inclusive profile, meaning that existing HighBayer and MainBayer implementations will be capable of decoding the newly proposed CMainBayer profile. A core experiment was issued to verify the proposed parameters and the impact on the coding performance and the potential resource savings. The plan is to issue the DAMD2 text at the next JPEG meeting in July 2026.

JPEG RF

At its 111th meeting, the JPEG committee continued its active exploration of technologies for radiance field representations. Significant progress was achieved, particularly in the areas of quality assessment and visualization methods for radiance fields, where an exploration study was presented and discussed, contributing to a deeper understanding of how radiance field content should be rendered and evaluated in subjective experiments. These discussions support the development of reliable and practical assessment frameworks in preparation for a future Call for Proposals. JPEG also initiated a new exploration focusing on the identification of suitable objective quality metrics and subjectively annotated datasets, as well as the evaluation of metrics’ performance for radiance field content. This work builds on prior efforts covering coding methods, subjective assessment protocols, and trajectory design, and aims to support the development of standardized evaluation methodologies.

In parallel, outreach activities continued to expand community engagement and support the preparation of dissemination materials, while work progresses toward revising the JPEG RF Common Test Conditions. The outcome of these efforts will guide future standardization in radiance field coding.

JPEG AI

At its 111th meeting, the JPEG Committee continued advancing JPEG AI. The Committee reviewed progress in key JPEG AI Core Experiments, including energy-consumption analysis, and bit-exact reconstruction, with results showing promising implementation trade-offs across CPU, GPU, and FPGA platforms. The FPGA implementation was identified as the most energy-efficient, while GPU execution provided the lowest latency. JPEG also reported progress on compressed-domain machine analysis, where JPEG AI latent representations enabled earth-observation segmentation with nearly ten times fewer model parameters. Further work will continue through new Core Experiments on energy consumption, bit-exact reconstruction, and objective and subjective evaluation. This activity reinforces JPEG AI’s role as the first International Standard for end-to-end learning-based image coding and as a technology relevant to both human visualization and machine-oriented applications.

Final Quote

“At the 111th meeting, five JPEG standardization projects reached DIS stage, a testimony to the commitment of its experts who work tirelessly towards the development of efficient standardized solutions that guarantee interoperability between emerging imaging products and services,” said Prof. Touradj Ebrahimi, the Convenor of the JPEG Committee.