Standardization, open source software and datasets
Jump to Introduction, Reports from Standardization, Open Source, Datasets, Quality-of-Experience
Introduction
Standards Column: JPEG and MPEG
Introduction ISO/IEC JTC 1/SC 29 area of work comprises the standardization of coded representation of audio, picture, multimedia and hypermedia information and sets of compression and control functions for use with such information. SC29 basically hosts two working groups responsible for the development of international standards for the compression, decompression, processing, and coded representation of media content, ...
Reports from Standardization
JPEG Column: 104th JPEG Meeting in Sapporo, Japan
JPEG XE issues Call for Proposals on event-based vision representation The 104th JPEG meeting was held in Sapporo, Japan from July 15 to 19, 2024. During this JPEG meeting, a Call for Proposals on event-based vision representation was launched for the creation of the first standardised representation of this type of data. This CfP addresses lossless ...
VQEG Column: VQEG Meeting December 2023
Introduction The last plenary meeting of the Video Quality Experts Group (VQEG) was held online by the University of Konstantz (Germany) in December 18th to 21st, 2023. It offered the possibility to more than 100 registered participants from 19 different countries worldwide to attend the numerous presentations and discussions about topics related to the ongoing ...
MPEG Column: 146th MPEG Meeting in Rennes, France
The 146th MPEG meeting was held in Rennes, France from 22-26 April 2024, and the official press release can be found here. It comprises the following highlights: AI-based Point Cloud Coding*: Call for proposals focusing on AI-driven point cloud encoding for applications such as immersive experiences and autonomous driving. Object Wave Compression*: Call for interest in object ...
JPEG Column: 102nd JPEG Meeting in San Francisco, U.S.A.
JPEG Trust reaches Draft International Standard stage The 102nd JPEG meeting was held in San Francisco, California, USA, from 22 to 26 January 2024. At this meeting, JPEG Trust became a Draft International Standard. Moreover, the responses to the Call for Proposals of JPEG NFT were received and analysed. As a consequence, relevant steps were ...
JPEG Column: 101st JPEG Meeting
JPEG Trust reaches Committee Draft stage at the 101st JPEG meeting The 101st JPEG meeting was held online, from the 30th of October to the 3rd of November 2023. At this meeting, JPEG Trust became a Committee Draft. In addition, JPEG analyzed the responses to its Calls for Proposals for JPEG DNA. The 101st JPEG meeting had the ...
MPEG Column: 145th MPEG Meeting (Virtual/Online)
The 145th MPEG meeting was held online from 22-26 January 2024, and the official press release can be found here. It comprises the following highlights: Latest Edition of the High Efficiency Image Format Standard Unveils Cutting-Edge Features for Enhanced Image Decoding and Annotation MPEG Systems finalizes Standards supporting Interoperability Testing MPEG finalizes the Third Edition of MPEG-D Dynamic ...
MPEG Column: 144th MPEG Meeting in Hannover, Germany
The 144th MPEG meeting was held in Hannover, Germany! For those interested, the press release is available with all the details. It’s great to see progress being made in person (cf. also the group pictures below). The main outcome of this meeting is as follows: MPEG issues Call for Learning-Based Video Codecs for Study of Quality ...
JPEG Column: 100th meeting in Covilha, Portugal
JPEG AI reaches Committee Draft stage at the 100th JPEG meeting The 100th JPEG meeting was held in Covilhã, Portugal, from July 17th to 21st, 2023. At this meeting, in addition to its usual standardization activities, the JPEG Committee organized a celebration on the occasion of its 100th meeting. This face-to-face meeting, the second after the ...
VQEG Column: VQEG Meeting June 2023
Introduction This column provides a report on the last Video Quality Experts Group (VQEG) plenary meeting, which took place from 26 to 30 June 2023 in San Mateo (USA), hosted by Sony Interactive Entertainment. More than 90 participants worldwide registered for the hybrid meeting, counting with the physical attendance of more than 40 people. This ...
MPEG Column: 143rd MPEG Meeting in Geneva, Switzerland
The 143rd MPEG meeting took place in person in Geneva, Switzerland. The official press release can be accessed here and includes the following details: MPEG finalizes the Carriage of Uncompressed Video and Images in ISOBMFF MPEG reaches the First Milestone for two ISOBMFF Enhancements MPEG ratifies Third Editions of VVC and VSEI MPEG reaches the First Milestone of AVC ...
Open Source
Procedural Content Generation – The Open Source Success Story of Wave Function Collapse
With OpenAI’s Dall-E, Midjourney, and Adobe Firefly, computer-generated visual content has hit the mass market. Machine learning-based algorithms can now create, and re-mix multimedia content based on huge corpora of images and videos and relieve creative professionals of tedious work. While this has gained much momentum lately, procedurally generated content (PCG) has been around for ...
Autonomous driving research with CARLA simulator
Introduction The autonomous driving industry, in order to advance through its six levels of automation (as defined by SAE, Society of Automotive Engineers ), is going to be increasingly more data-driven. While the number of sensors and their technology has been increasing it is still both cost-effective and, in some cases, necessary to use a simulator, ...
COCO Annotator – Web-based Image Segmentation Tool for Object Detection, Localization, and Keypoints
Introduction COCO Annotator is an image annotation tool that allows the labelling of images to create training data for object detection and localization. It provides many features, including the ability to label an image segment by drawing, label objects with disconnected visible parts, efficiently store and export annotations in the well-known COCO format as well as ...
Multimedia Retrieval and Analysis with Cottontail DB
Introduction Analysis and retrieval of media collections get more and more challenging the larger the collections become. Keeping everything in the main memory becomes less feasible, and more and more time and effort have to be spent to deal with the data management. However, traditional relational databases do not support primitives often used in multimedia workloads, ...
Open Source for Video Games – A Shortlist of Game Engines
Introduction Open-source software is a relevant topic in video game development. Taking a look at the most frequently employed game engines for developing Android games we can see that seven out of ten ranked engines are OSS. Over the last decade, more and more game studios and individual developers switched to open-source software. Oliver Franzke ...
nteract – A Desktop-Based Tool for Notebooks
Introduction Writing source code for programs with lightweight text editors or fully featured integrated development environments is considered the main method of programming. Notebooks, however, are an extremely practical tool. In contrast to IDEs, projects are set up more easily and they allow for running programs in a read-eval-print loop (REPL) environment. The Jupyter Notebooks Quick ...
Open Source Video Streaming & Recording
By the time of writing, the CoViD-19 disease has spread, impacting the world. Education and research are affected as well, but fortunately the community can continue working due to all the developments over the last few years. Even more so, streaming tech and video conferences allow us to keep in touch with family and friends. ...
OpenVSLAM: A Versatile Visual SLAM Framework
Visual simultaneous localization and mapping (VSLAM) systems are essential for augmented reality devices, autonomous control of robots and drones, etc. However, conventional open-source visual SLAM frameworks are not designed to be called as libraries from third-party programs. To overcome this situation, we have developed OpenVSLAM , a novel visual SLAM framework, and released it as ...
Configurable image annotation on the Web
Image annotations are required in a wide range of applications including image classification (which requires textual labels), object detection (bounding boxes), or image segmentation (pixel-wise classification). The application we show here was presented in the ACM Multimedia 2018 Open Source Software Competition track, and provides a number of configurable manual image annotation tools for detection and segmentation. It is available online, but in ...
Deep Learning in the Browser: TensorFlow JS
Having already discussed MatConvNet and Keras, let us continue with an open source framework for deep learning, which takes a new and interesting approach. TensorFlow.js is not only providing deep learning for JavaScript developers, but it’s also making applications of deep learning available in the WebGL enabled web browsers, or more specifically, Chrome, Chromium-based browsers, ...
Datasets
Overview of Open Dataset Sessions and Benchmarking Competitions in 2023-2024 – Part 2 (MDRE at MMM 2023 and MMM 2024)
As already started in the previous Datasets column, we are reviewing some of the most notable events related to open datasets and benchmarking competitions in the field of multimedia in the years 2023 and 2024. This selection highlights the wide range of topics and datasets currently of interest to the community. Some of the events ...
Overview of Open Dataset Sessions and Benchmarking Competitions in 2023-2024 – Part 1 (QoMEX 2023 and QoMEX 2024)
In this and the following Dataset Columns, we present a review of some of the notable events related to open datasets and benchmarking competitions in the field of multimedia in the years 2023 and 2024. This selection highlights the wide range of topics and datasets currently of interest to the community. Some of the events ...
Overview of Benchmarking Platforms and Software for Multimedia Applications
In a time where Artificial Intelligence (AI) continues to push the boundaries of what was previously thought possible, the demand for benchmarking platforms that allow to fairly assess and evaluate AI models has become paramount. These platforms serve as connecting hubs between data scientists, machine learning specialists, industry partners, and other interested parties. They mostly ...
Overview of Open Dataset Sessions and Benchmarking Competitions in 2022 – Part 2 (MDRE at MMM 2022, ACM MM 2022)
In this Dataset Column, we present a review of some of the notable events related to open datasets and benchmarking competitions in the field of multimedia. This year’s selection highlights the wide range of topics and datasets currently of interest to the community. Some of the events covered in this review include special sessions on ...
Overview of Open Dataset Sessions and Benchmarking Competitions in 2022 – Part 3 (ImageCLEF 2022, MediaEval 2022)
In this Dataset Column, we present a review of some of the notable events related to open datasets and benchmarking competitions in the field of multimedia. This year’s selection highlights the wide range of topics and datasets currently of interest to the community. Some of the events covered in this review include special sessions on ...
Overview of Open Dataset Sessions and Benchmarking Competitions in 2022 – Part 1 (QoMEX 2022, ODS at MMSys ’22)
In this Dataset Column, we present a review of some of the notable events related to open datasets and benchmarking competitions in the field of multimedia. This year’s selection highlights the wide range of topics and datasets currently of interest to the community. Some of the events covered in this review include special sessions on ...
Two Interviews with renown Datasets Researchers
This issue of the Dataset Column provides two interviews with the researchers responsible for novel datasets of recent years. In particular, we first interview Nacho Reimat (https://www.cwi.nl/people/nacho-reimat), the scientific programmer responsible for the CWIPC-SXR, one of the first datasets on dynamic, interactive volumetric media. Second, we interview Pierre-Etienne Martin (https://www.eva.mpg.de/comparative-cultural-psychology/staff/pierre-etienne-martin/), responsible for contributions to datasets ...
Overview of Open Dataset Sessions and Benchmarking Competitions in 2021.
This issue of the Dataset Column proposes a review of some of the most important events in 2021 related to special sessions on open datasets or benchmarking competitions associated with multimedia data. While this is not meant to represent an exhaustive list of events, we wish to underline the great diversity of subjects and dataset ...
Dataset Column: Overview, Scope and Call for Contributions
Overview and Scope The Dataset Column (https://records.sigmm.org/open-science/datasets/) of ACM SIGMM Records provides timely updates on the developments in the domain of publicly available multimedia datasets as enabling tools for reproducible research in numerous related areas. It is intended as a platform for further dissemination of useful information on multimedia datasets and studies of datasets covering various ...
Report from the MMM 2020 Special Session on Multimedia Datasets for Repeatable Experimentation (MDRE 2020)
Introduction Information retrieval and multimedia content access have a long history of comparative evaluation, and many of the advances in the area over the past decade can be attributed to the availability of open datasets that support comparative and repeatable experimentation. Hence, sharing data and code to allow other researchers to replicate research results is needed ...
Quality-of-Experience
From Theory to Practice: System QoE Assessment by Providers
Service and network providers actively evaluate and derive Quality of Experience (QoE) metrics within their systems, which necessitates suitable monitoring strategies. Objective QoE monitoring involves mapping Quality of Service (QoS) parameters into QoE scores, such as calculating Mean Opinion Scores (MOS) or Good-or-Better (GoB) ratios, by using appropriate mapping functions. Alternatively, individual QoE monitoring directly ...
Energy-Efficient Video Streaming: Open-Source Tools, Datasets, and Solutions
Abstract: Energy efficiency has become a crucial aspect of today’s IT infrastructures, and video (streaming) accounts for over half of today’s Internet traffic. This column highlights open-source tools, datasets, and solutions addressing energy efficiency in video streaming presented at ACM Multimedia Systems 2024 and its co-located workshop ACM Green Multimedia Systems. Introduction Across various platforms, users seek ...
Towards Immersive Digiphysical Experiences
Immersive experiences have the potential of redefining traditional forms of media engagement by intricately combining reality with imagination. Motivated by necessities, current developments and emerging technologies, this column sets out to bridge immersive experiences in both digital and physical realities. Fitting under the umbrella term of eXtended Reality (XR), the first section describes various realizations ...
Explainable Artificial Intelligence for Quality of Experience Modelling
Data-driven Quality of Experience (QoE) modelling using Machine Learning (ML) arose as a promising alternative to the cumbersome and potentially biased manual QoE modelling. However, the reasoning of a majority of ML models is not explainable due to their black-box characteristics, which prevents us from gaining insights about how the model actually related QoE influence ...
Sustainability vs. Quality of Experience: Striking the Right Balance for Video Streaming
The exponential growth in internet data traffic, driven by the widespread use of video streaming applications, has resulted in increased energy consumption and carbon emissions. This outcome is primarily due to higher resolution or higher framerates content and the ability to watch videos on various end-devices. However, efforts to reduce energy consumption in video streaming ...
Green Video Streaming: Challenges and Opportunities
Introduction Regarding the Intergovernmental Panel on Climate Change (IPCC) report in 2021 and Sustainable Development Goal (SDG) 13 “climate action”, urgent action is needed against climate change and global greenhouse gas (GHG) emissions in the next few years . This urgency also applies to the energy consumption of digital technologies. Internet data traffic is responsible for ...
Towards the design and evaluation of more sustainable multimedia experiences: which role can QoE research play?
In this column, we reflect on the environmental impact and broader sustainability implications of resource-demanding digital applications and services such as video streaming, VR/AR/XR and videoconferencing. We put emphasis not only on the experiences and use cases they enable but also on the “cost” of always striving for high Quality of Experience (QoE) and better ...
What is the trade-off between CO2 emission and video-conferencing QoE?
It is a natural thing that users of multimedia services want to have the highest possible Quality of Experience (QoE), when using said services. This is especially so in contexts such as video-conferencing and video streaming services, which are nowadays a large part of many users’ daily life, be it work-related Zoom calls, or relaxing ...
Towards an updated understanding of immersive multimedia experiences
Bringing theories and measurement techniques up to date Development of technology for immersive multimedia experiences Immersive multimedia experiences, as its name is suggesting are those experiences focusing on media that is able to immerse users with different interactions into an experience of an environment. Through different technologies and approaches, immersive media is emulating a physical world through ...
MPEG Visual Quality Assessment Advisory Group: Overview and Perspectives
Introduction The perceived visual quality is of utmost importance in the context of visual media compression, such as 2D, 3D, immersive video, and point clouds. The trade-off between compression efficiency and computational/implementation complexity has a crucial impact on the success of a compression scheme. This specifically holds for the development of visual media compression standards which ...