Standardization, open source software and datasets
Jump to Introduction, Reports from Standardization, Open Source, Datasets, Quality-of-Experience
Introduction
Standards Column: JPEG and MPEG
Introduction ISO/IEC JTC 1/SC 29 area of work comprises the standardization of coded representation of audio, picture, multimedia and hypermedia information and sets of compression and control functions for use with such information. SC29 basically hosts two working groups responsible for the development of international standards for the compression, decompression, processing, and coded representation of media content, ...
Reports from Standardization
MPEG Column: 142nd MPEG Meeting in Antalya, Türkiye
The 142nd MPEG meeting was held as a face-to-face meeting in Antalya, Türkiye, and the official press release can be found here and comprises the following items:The 142nd MPEG meeting was held as a face-to-face meeting in Antalya, Türkiye, and the official press release can be found here and comprises the following items: MPEG issues Call ...
VQEG Column: Emerging Technologies Group (ETG)
Introduction This column provides an overview of the new Video Quality Experts Group (VQEG) group called the Emerging Technologies Group (ETG), which was created during the last VQEG plenary meeting in December 2022. For an introduction to VQEG, please check the VQEG homepage or this presentation. The works addressed by this new group can be of ...
JPEG Column: 98th JPEG meeting in Sydney, Australia
JPEG explores standardization in event-based imaging The 98th JPEG meeting was held in Sydney, Australia, from the 16th to 20th January 2023. This was a welcome return to face-to-face meetings after a long period of online meetings due to Covid-19 pandemics. Interestingly, the previous face-to-face meeting of the JPEG Committee was also held in Sydney, in ...
VQEG Column: VQEG Meeting December 2022
Introduction This column provides an overview of the last Video Quality Experts Group (VQEG) plenary meeting, which took place from 12 to 16 December 2022. Around 100 participants from 21 different countries around the world registered for the meeting that was organized online by Brightcove (United Kingdom). During the five days, there were more than ...
MPEG Column: 140th MPEG Meeting in Mainz, Germany
After several years of online meetings, the 140th MPEG meeting was held as a face-to-face meeting in Mainz, Germany, and the official press release can be found here and comprises the following items: MPEG evaluates the Call for Proposals on Video Coding for Machines MPEG evaluates Call for Evidence on Video Coding for Machines Feature Coding MPEG reaches ...
JPEG Column: 97th JPEG Meeting
JPEG initiates specification on fake media based on responses to its call for proposals The 97th JPEG meeting was held online from 24 to 28 October 2022. JPEG received responses to the Call for Proposals (CfP) on JPEG Fake Media, the first multimedia international standard designed to facilitate the secure and reliable annotation of media assets ...
VQEG Column: VQEG Meeting May 2022
Introduction Welcome to this new column on the ACM SIGMM Records from the Video Quality Experts Group (VQEG), which will provide an overview of the last VQEG plenary meeting that took place from 9 to 13 May 2022. It was organized by INSA Rennes (France), and it was the first face-to-face meeting after the series of online meetings due to the Covid-19 pandemic. Remote ...
JPEG Column: 96th JPEG Meeting
JPEG analyses the responses of the Calls for Proposals for the standardisation of the first codecs based on machine learning The 96th JPEG meeting was held online from 25 to 29 July 2022. The meeting was one of the most productive in the recent history of JPEG with the analysis of the responses of two Calls for ...
MPEG Column: 139th MPEG Meeting (virtual/online)
The original blog post can be found at the Bitmovin Techblog and has been modified/updated here to focus on and highlight research aspects. The 139th MPEG meeting was once again held as an online meeting, and the official press release can be found here and comprises the following items: MPEG Issues Call for Evidence for Video Coding for Machines (VCM) MPEG ...
JPEG Column: 95th JPEG Meeting
JPEG issues a call for proposals for JPEG Fake Media The 95th JPEG meeting was held online from 25 to 29 April 2022. A Call for Proposals (CfP) was issued for JPEG Fake Media that aims at a standardisation framework for secure annotation of modifications in media assets. With this new initiative, JPEG endeavours to provide ...
Open Source
Procedural Content Generation – The Open Source Success Story of Wave Function Collapse
With OpenAI’s Dall-E, Midjourney, and Adobe Firefly, computer-generated visual content has hit the mass market. Machine learning-based algorithms can now create, and re-mix multimedia content based on huge corpora of images and videos and relieve creative professionals of tedious work. While this has gained much momentum lately, procedurally generated content (PCG) has been around for ...
Autonomous driving research with CARLA simulator
Introduction The autonomous driving industry, in order to advance through its six levels of automation (as defined by SAE, Society of Automotive Engineers ), is going to be increasingly more data-driven. While the number of sensors and their technology has been increasing it is still both cost-effective and, in some cases, necessary to use a simulator, ...
COCO Annotator – Web-based Image Segmentation Tool for Object Detection, Localization, and Keypoints
Introduction COCO Annotator is an image annotation tool that allows the labelling of images to create training data for object detection and localization. It provides many features, including the ability to label an image segment by drawing, label objects with disconnected visible parts, efficiently store and export annotations in the well-known COCO format as well as ...
Multimedia Retrieval and Analysis with Cottontail DB
Introduction Analysis and retrieval of media collections get more and more challenging the larger the collections become. Keeping everything in the main memory becomes less feasible, and more and more time and effort have to be spent to deal with the data management. However, traditional relational databases do not support primitives often used in multimedia workloads, ...
Open Source for Video Games – A Shortlist of Game Engines
Introduction Open-source software is a relevant topic in video game development. Taking a look at the most frequently employed game engines for developing Android games we can see that seven out of ten ranked engines are OSS. Over the last decade, more and more game studios and individual developers switched to open-source software. Oliver Franzke ...
nteract – A Desktop-Based Tool for Notebooks
Introduction Writing source code for programs with lightweight text editors or fully featured integrated development environments is considered the main method of programming. Notebooks, however, are an extremely practical tool. In contrast to IDEs, projects are set up more easily and they allow for running programs in a read-eval-print loop (REPL) environment. The Jupyter Notebooks Quick ...
Open Source Video Streaming & Recording
By the time of writing, the CoViD-19 disease has spread, impacting the world. Education and research are affected as well, but fortunately the community can continue working due to all the developments over the last few years. Even more so, streaming tech and video conferences allow us to keep in touch with family and friends. ...
OpenVSLAM: A Versatile Visual SLAM Framework
Visual simultaneous localization and mapping (VSLAM) systems are essential for augmented reality devices, autonomous control of robots and drones, etc. However, conventional open-source visual SLAM frameworks are not designed to be called as libraries from third-party programs. To overcome this situation, we have developed OpenVSLAM , a novel visual SLAM framework, and released it as ...
Configurable image annotation on the Web
Image annotations are required in a wide range of applications including image classification (which requires textual labels), object detection (bounding boxes), or image segmentation (pixel-wise classification). The application we show here was presented in the ACM Multimedia 2018 Open Source Software Competition track, and provides a number of configurable manual image annotation tools for detection and segmentation. It is available online, but in ...
Deep Learning in the Browser: TensorFlow JS
Having already discussed MatConvNet and Keras, let us continue with an open source framework for deep learning, which takes a new and interesting approach. TensorFlow.js is not only providing deep learning for JavaScript developers, but it’s also making applications of deep learning available in the WebGL enabled web browsers, or more specifically, Chrome, Chromium-based browsers, ...
Datasets
Overview of Open Dataset Sessions and Benchmarking Competitions in 2022 – Part 2 (MDRE at MMM 2022, ACM MM 2022)
In this Dataset Column, we present a review of some of the notable events related to open datasets and benchmarking competitions in the field of multimedia. This year’s selection highlights the wide range of topics and datasets currently of interest to the community. Some of the events covered in this review include special sessions on ...
Overview of Open Dataset Sessions and Benchmarking Competitions in 2022 – Part 3 (ImageCLEF 2022, MediaEval 2022)
In this Dataset Column, we present a review of some of the notable events related to open datasets and benchmarking competitions in the field of multimedia. This year’s selection highlights the wide range of topics and datasets currently of interest to the community. Some of the events covered in this review include special sessions on ...
Overview of Open Dataset Sessions and Benchmarking Competitions in 2022 – Part 1 (QoMEX 2022, ODS at MMSys ’22)
In this Dataset Column, we present a review of some of the notable events related to open datasets and benchmarking competitions in the field of multimedia. This year’s selection highlights the wide range of topics and datasets currently of interest to the community. Some of the events covered in this review include special sessions on ...
Two Interviews with renown Datasets Researchers
This issue of the Dataset Column provides two interviews with the researchers responsible for novel datasets of recent years. In particular, we first interview Nacho Reimat (https://www.cwi.nl/people/nacho-reimat), the scientific programmer responsible for the CWIPC-SXR, one of the first datasets on dynamic, interactive volumetric media. Second, we interview Pierre-Etienne Martin (https://www.eva.mpg.de/comparative-cultural-psychology/staff/pierre-etienne-martin/), responsible for contributions to datasets ...
Overview of Open Dataset Sessions and Benchmarking Competitions in 2021.
This issue of the Dataset Column proposes a review of some of the most important events in 2021 related to special sessions on open datasets or benchmarking competitions associated with multimedia data. While this is not meant to represent an exhaustive list of events, we wish to underline the great diversity of subjects and dataset ...
Dataset Column: Overview, Scope and Call for Contributions
Overview and Scope The Dataset Column (https://records.sigmm.org/open-science/datasets/) of ACM SIGMM Records provides timely updates on the developments in the domain of publicly available multimedia datasets as enabling tools for reproducible research in numerous related areas. It is intended as a platform for further dissemination of useful information on multimedia datasets and studies of datasets covering various ...
Report from the MMM 2020 Special Session on Multimedia Datasets for Repeatable Experimentation (MDRE 2020)
Introduction Information retrieval and multimedia content access have a long history of comparative evaluation, and many of the advances in the area over the past decade can be attributed to the availability of open datasets that support comparative and repeatable experimentation. Hence, sharing data and code to allow other researchers to replicate research results is needed ...
MediaEval Multimedia Evaluation Benchmark: Tenth Anniversary and Counting
MediaEval Multimedia Challenges MediaEval is a benchmarking initiative that offers challenges in multimedia retrieval, analysis and exploration. The tasks offered by MediaEval concentrate specifically on the human and social aspects of multimedia. They encourage researchers to bring together multiple modalities (visual, text, audio) and to think in terms of systems that serve users. Our larger ...
Dataset Column: ToCaDa Dataset with Multi-Viewpoint Synchronized Videos
This column describes the release of the Toulouse Campus Surveillance Dataset (ToCaDa). It consists of 25 synchronized videos (with audio) of two scenes recorded from different viewpoints of the campus. An extensive manual annotation comprises all moving objects and their corresponding bounding boxes, as well as audio events. The annotation was performed in order to ...
Dataset Column: Report from the MMM 2019 Special Session on Multimedia Datasets for Repeatable Experimentation (MDRE 2019)
Special Session Information retrieval and multimedia content access have a long history of comparative evaluation, and many of the advances in the area over the past decade can be attributed to the availability of open datasets that support comparative and repeatable experimentation. Sharing data and code to allow other researchers to replicate research results is needed ...
Quality-of-Experience
Sustainability vs. Quality of Experience: Striking the Right Balance for Video Streaming
The exponential growth in internet data traffic, driven by the widespread use of video streaming applications, has resulted in increased energy consumption and carbon emissions. This outcome is primarily due to higher resolution or higher framerates content and the ability to watch videos on various end-devices. However, efforts to reduce energy consumption in video streaming ...
Green Video Streaming: Challenges and Opportunities
Introduction Regarding the Intergovernmental Panel on Climate Change (IPCC) report in 2021 and Sustainable Development Goal (SDG) 13 “climate action”, urgent action is needed against climate change and global greenhouse gas (GHG) emissions in the next few years . This urgency also applies to the energy consumption of digital technologies. Internet data traffic is responsible for ...
Towards the design and evaluation of more sustainable multimedia experiences: which role can QoE research play?
In this column, we reflect on the environmental impact and broader sustainability implications of resource-demanding digital applications and services such as video streaming, VR/AR/XR and videoconferencing. We put emphasis not only on the experiences and use cases they enable but also on the “cost” of always striving for high Quality of Experience (QoE) and better ...
What is the trade-off between CO2 emission and video-conferencing QoE?
It is a natural thing that users of multimedia services want to have the highest possible Quality of Experience (QoE), when using said services. This is especially so in contexts such as video-conferencing and video streaming services, which are nowadays a large part of many users’ daily life, be it work-related Zoom calls, or relaxing ...
Towards an updated understanding of immersive multimedia experiences
Bringing theories and measurement techniques up to date Development of technology for immersive multimedia experiences Immersive multimedia experiences, as its name is suggesting are those experiences focusing on media that is able to immerse users with different interactions into an experience of an environment. Through different technologies and approaches, immersive media is emulating a physical world through ...
MPEG Visual Quality Assessment Advisory Group: Overview and Perspectives
Introduction The perceived visual quality is of utmost importance in the context of visual media compression, such as 2D, 3D, immersive video, and point clouds. The trade-off between compression efficiency and computational/implementation complexity has a crucial impact on the success of a compression scheme. This specifically holds for the development of visual media compression standards which ...
ITU-T Standardization Activities Targeting Gaming Quality of Experience
Motivation for Research in the Gaming Domain The gaming industry has eminently managed to intrinsically motivate users to interact with their services. According to the latest report of Newzoo, there will be an estimated total of 2.7 billion players across the globe by the end of 2020. The global games market will generate revenues of $159.3 billion in 2020 . This ...
Immersive Media Experiences – Why finding Consensus is Important
An introduction to the QUALINET White Paper on Definitions of Immersive Media Experience (IMEx) . Introduction Immersive media are reshaping the way users experience reality. They are increasingly incorporated across enterprise and consumer sectors to offer experiential solutions to a diverse range of industries. Current technologies that afford an immersive media experience (IMEx) include Augmented Reality (AR), ...
Towards Interactive QoE Assessment of Robotic Telepresence
Telepresence robots (TPRs) are remote-controlled, wheeled devices with an internet connection. A TPR can “teleport” you to a remote location, let you drive around and interact with people. A TPR user can feel present in the remote location by being able to control the robot position, movements, actions, voice and video. A TPR facilitates human-to-human ...
Definitions of Crowdsourced Network and QoE Measurements
1 Introduction and Definitions Crowdsourcing is a well-established concept in the scientific community, used for instance by Jeff Howe and Mark Robinson in 2005 to describe how businesses were using the Internet to outsource work to the crowd , but can be dated back up to 1849 (weather prediction in the US). Crowdsourcing has enabled a ...