Open Science

Standardization, open source software and datasets

Jump to Introduction, Reports from Standardization, Open Source, Datasets, Quality-of-Experience

Introduction

Standards Column: JPEG and MPEG

Standards Column: JPEG and MPEG

Introduction ISO/IEC JTC 1/SC 29 area of work comprises the standardization of coded representation of audio, picture, multimedia and hypermedia information and sets of compression and control functions for use with such information. SC29 basically hosts two working groups responsible for the development of international standards for the compression, decompression, processing, and coded representation of media content, ...


Reports from Standardization

  • MPEG Column: 146th MPEG Meeting in Rennes, France

    The 146th MPEG meeting was held in Rennes, France from 22-26 April 2024, and the official press release can be found here. It comprises the following highlights: AI-based Point Cloud Coding*: Call for proposals focusing on AI-driven point cloud encoding for applications such as immersive experiences and autonomous driving. Object Wave Compression*: Call for interest in object ...

  • JPEG Column: 102nd JPEG Meeting in San Francisco, U.S.A.

    JPEG Column: 102nd JPEG Meeting in San Francisco, U.S.A.

    JPEG Trust reaches Draft International Standard stage The 102nd JPEG meeting was held in San Francisco, California, USA, from 22 to 26 January 2024. At this meeting, JPEG Trust became a Draft International Standard. Moreover, the responses to the Call for Proposals of JPEG NFT were received and analysed. As a consequence, relevant steps were ...

  • JPEG Column: 101st JPEG Meeting

    JPEG Column: 101st JPEG Meeting

    JPEG Trust reaches Committee Draft stage at the 101st JPEG meeting The 101st JPEG meeting was held online, from the 30th of October to the 3rd of November 2023. At this meeting, JPEG Trust became a Committee Draft. In addition, JPEG analyzed the responses to its Calls for Proposals for JPEG DNA. The 101st JPEG meeting had the ...

  • MPEG Column: 145th MPEG Meeting (Virtual/Online)

    MPEG Column: 145th MPEG Meeting (Virtual/Online)

    The 145th MPEG meeting was held online from 22-26 January 2024, and the official press release can be found here. It comprises the following highlights: Latest Edition of the High Efficiency Image Format Standard Unveils Cutting-Edge Features for Enhanced Image Decoding and Annotation MPEG Systems finalizes Standards supporting Interoperability Testing MPEG finalizes the Third Edition of MPEG-D Dynamic ...

  • MPEG Column: 144th MPEG Meeting in Hannover, Germany

    MPEG Column: 144th MPEG Meeting in Hannover, Germany

    The 144th MPEG meeting was held in Hannover, Germany! For those interested, the press release is available with all the details. It’s great to see progress being made in person (cf. also the group pictures below). The main outcome of this meeting is as follows: MPEG issues Call for Learning-Based Video Codecs for Study of Quality ...

  • JPEG Column: 100th meeting in Covilha, Portugal

    JPEG Column: 100th meeting in Covilha, Portugal

    JPEG AI reaches Committee Draft stage at the 100th JPEG meeting The 100th JPEG meeting was held in Covilhã, Portugal, from July 17th to 21st, 2023. At this meeting, in addition to its usual standardization activities, the JPEG Committee organized a celebration on the occasion of its 100th meeting. This face-to-face meeting, the second after the ...

  • VQEG Column: VQEG Meeting June 2023

    VQEG Column: VQEG Meeting June 2023

    Introduction This column provides a report on the last Video Quality Experts Group (VQEG) plenary meeting, which took place from 26 to 30 June 2023 in San Mateo (USA), hosted by Sony Interactive Entertainment. More than 90 participants worldwide registered for the hybrid meeting, counting with the physical attendance of more than 40 people. This ...

  • MPEG Column: 143rd MPEG Meeting in Geneva, Switzerland

    MPEG Column: 143rd MPEG Meeting in Geneva, Switzerland

    The 143rd MPEG meeting took place in person in Geneva, Switzerland. The official press release can be accessed here and includes the following details: MPEG finalizes the Carriage of Uncompressed Video and Images in ISOBMFF MPEG reaches the First Milestone for two ISOBMFF Enhancements MPEG ratifies Third Editions of VVC and VSEI MPEG reaches the First Milestone of AVC ...

  • JPEG Column: 99th JPEG Meeting

    JPEG Column: 99th JPEG Meeting

    JPEG Trust on a mission to re-establish trust in digital media The 99th JPEG meeting was held online, from 24th to 28th April 2023. Providing tools suitable for establishing provenance, authenticity and ownership of multimedia content is one of the most difficult challenges faced nowadays, considering the technological models that allow effective multimedia data manipulation and generation. ...

  • MPEG Column: 142nd MPEG Meeting in Antalya, Türkiye

    MPEG Column: 142nd MPEG Meeting in Antalya, Türkiye

    The 142nd MPEG meeting was held as a face-to-face meeting in Antalya, Türkiye, and the official press release can be found here and comprises the following items: MPEG issues Call for Proposals for Feature Coding for Machines MPEG finalizes the 9th Edition of MPEG-2 Systems MPEG reaches the First Milestone for Storage and Delivery of Haptics Data MPEG completes ...


Open Source

  • Procedural Content Generation – The Open Source Success Story of Wave Function Collapse

    Procedural Content Generation - The Open Source Success Story of Wave Function Collapse

      With OpenAI’s Dall-E, Midjourney, and Adobe Firefly, computer-generated visual content has hit the mass market. Machine learning-based algorithms can now create, and re-mix multimedia content based on huge corpora of images and videos and relieve creative professionals of tedious work. While this has gained much momentum lately, procedurally generated content (PCG) has been around for ...

  • Autonomous driving research with CARLA simulator

    Autonomous driving research with CARLA simulator

    Introduction The autonomous driving industry, in order to advance through its six levels of automation (as defined by SAE, Society of Automotive Engineers ), is going to be increasingly more data-driven. While the number of sensors and their technology has been increasing it is still both cost-effective and, in some cases, necessary to use a simulator, ...

  • COCO Annotator – Web-based Image Segmentation Tool for Object Detection, Localization, and Keypoints

    COCO Annotator - Web-based Image Segmentation Tool for Object Detection, Localization, and Keypoints

    Introduction COCO Annotator is an image annotation tool that allows the labelling of images to create training data for object detection and localization. It provides many features, including the ability to label an image segment by drawing, label objects with disconnected visible parts, efficiently store and export annotations in the well-known COCO format as well as ...

  • Multimedia Retrieval and Analysis with Cottontail DB

    Multimedia Retrieval and Analysis with Cottontail DB

    Introduction Analysis and retrieval of media collections get more and more challenging the larger the collections become. Keeping everything in the main memory becomes less feasible, and more and more time and effort have to be spent to deal with the data management. However, traditional relational databases do not support primitives often used in multimedia workloads, ...

  • Open Source for Video Games – A Shortlist of Game Engines

    Open Source for Video Games - A Shortlist of Game Engines

    Introduction Open-source software is a relevant topic in video game development. Taking a look at the most frequently employed game engines for developing Android games we can see that seven out of ten ranked engines are OSS. Over the last decade, more and more game studios and individual developers switched to open-source software. Oliver Franzke ...

  • nteract – A Desktop-Based Tool for Notebooks

    nteract - A Desktop-Based Tool for Notebooks

    Introduction Writing source code for programs with lightweight text editors or fully featured integrated development environments is considered the main method of programming. Notebooks, however, are an extremely practical tool. In contrast to IDEs, projects are set up more easily and they allow for running programs in a read-eval-print loop (REPL) environment. The Jupyter Notebooks Quick ...

  • Open Source Video Streaming & Recording

    Open Source Video Streaming & Recording

    By the time of writing, the CoViD-19 disease has spread, impacting the world. Education and research are affected as well, but fortunately the community can continue working due to all the developments over the last few years. Even more so, streaming tech and video conferences allow us to keep in touch with family and friends. ...

  • OpenVSLAM: A Versatile Visual SLAM Framework

    OpenVSLAM: A Versatile Visual SLAM Framework

    Visual simultaneous localization and mapping (VSLAM) systems are essential for augmented reality devices, autonomous control of robots and drones, etc. However, conventional open-source visual SLAM frameworks are not designed to be called as libraries from third-party programs. To overcome this situation, we have developed OpenVSLAM , a novel visual SLAM framework, and released it as ...

  • Configurable image annotation on the Web

    Configurable image annotation on the Web

    Image annotations are required in a wide range of applications including image classification (which requires textual labels), object detection (bounding boxes), or image segmentation (pixel-wise classification). The application we show here was presented in the ACM Multimedia 2018 Open Source Software Competition track, and provides a number of configurable manual image annotation tools for detection and segmentation. It is available online, but in ...

  • Deep Learning in the Browser: TensorFlow JS

    Having already discussed MatConvNet and Keras, let us continue with an open source framework for deep learning, which takes a new and interesting approach. TensorFlow.js is not only providing deep learning for JavaScript developers, but it’s also making applications of deep learning available in the WebGL enabled web browsers, or more specifically, Chrome, Chromium-based browsers, ...


Datasets

  • Overview of Benchmarking Platforms and Software for Multimedia Applications

    Overview of Benchmarking Platforms and Software for Multimedia Applications

    In a time where Artificial Intelligence (AI) continues to push the boundaries of what was previously thought possible, the demand for benchmarking platforms that allow to fairly assess and evaluate AI models has become paramount. These platforms serve as connecting hubs between data scientists, machine learning specialists, industry partners, and other interested parties. They mostly ...

  • Overview of Open Dataset Sessions and Benchmarking Competitions in 2022 – Part 2 (MDRE at MMM 2022, ACM MM 2022)

    Overview of Open Dataset Sessions and Benchmarking Competitions in 2022 - Part 2 (MDRE at MMM 2022, ACM MM 2022)

    In this Dataset Column, we present a review of some of the notable events related to open datasets and benchmarking competitions in the field of multimedia. This year’s selection highlights the wide range of topics and datasets currently of interest to the community. Some of the events covered in this review include special sessions on ...

  • Overview of Open Dataset Sessions and Benchmarking Competitions in 2022 – Part 3 (ImageCLEF 2022, MediaEval 2022)

    Overview of Open Dataset Sessions and Benchmarking Competitions in 2022 - Part 3 (ImageCLEF 2022, MediaEval 2022)

    In this Dataset Column, we present a review of some of the notable events related to open datasets and benchmarking competitions in the field of multimedia. This year’s selection highlights the wide range of topics and datasets currently of interest to the community. Some of the events covered in this review include special sessions on ...

  • Overview of Open Dataset Sessions and Benchmarking Competitions in 2022 – Part 1 (QoMEX 2022, ODS at MMSys ’22)

    Overview of Open Dataset Sessions and Benchmarking Competitions in 2022 - Part 1 (QoMEX 2022, ODS at MMSys '22)

    In this Dataset Column, we present a review of some of the notable events related to open datasets and benchmarking competitions in the field of multimedia. This year’s selection highlights the wide range of topics and datasets currently of interest to the community. Some of the events covered in this review include special sessions on ...

  • Two Interviews with renown Datasets Researchers

    Two Interviews with renown Datasets Researchers

    This issue of the Dataset Column provides two interviews with the researchers responsible for novel datasets of recent years. In particular, we first interview Nacho Reimat (https://www.cwi.nl/people/nacho-reimat), the scientific programmer responsible for the CWIPC-SXR, one of the first datasets on dynamic, interactive volumetric media. Second, we interview Pierre-Etienne Martin (https://www.eva.mpg.de/comparative-cultural-psychology/staff/pierre-etienne-martin/), responsible for contributions to datasets ...

  • Overview of Open Dataset Sessions and Benchmarking Competitions in 2021.

    Overview of Open Dataset Sessions and Benchmarking Competitions in 2021.

    This issue of the Dataset Column proposes a review of some of the most important events in 2021 related to special sessions on open datasets or benchmarking competitions associated with multimedia data. While this is not meant to represent an exhaustive list of events, we wish to underline the great diversity of subjects and dataset ...

  • Dataset Column: Overview, Scope and Call for Contributions

    Overview and Scope The Dataset Column (https://records.sigmm.org/open-science/datasets/) of ACM SIGMM Records provides timely updates on the developments in the domain of publicly available multimedia datasets as enabling tools for reproducible research in numerous related areas. It is intended as a platform for further dissemination of useful information on multimedia datasets and studies of datasets covering various ...

  • Report from the MMM 2020 Special Session on Multimedia Datasets for Repeatable Experimentation (MDRE 2020)

    Introduction Information retrieval and multimedia content access have a long history of comparative evaluation, and many of the advances in the area over the past decade can be attributed to the availability of open datasets that support comparative and repeatable experimentation. Hence, sharing data and code to allow other researchers to replicate research results is needed ...

  • MediaEval Multimedia Evaluation Benchmark: Tenth Anniversary and Counting

    MediaEval Multimedia Evaluation Benchmark: Tenth Anniversary and Counting

    MediaEval Multimedia Challenges MediaEval is a benchmarking initiative that offers challenges in multimedia retrieval, analysis and exploration. The tasks offered by MediaEval concentrate specifically on the human and social aspects of multimedia. They encourage researchers to bring together multiple modalities (visual, text, audio) and to think in terms of systems that serve users. Our larger ...

  • Dataset Column: ToCaDa Dataset with Multi-Viewpoint Synchronized Videos

    Dataset Column: ToCaDa Dataset with Multi-Viewpoint Synchronized Videos

    This column describes the release of the Toulouse Campus Surveillance Dataset (ToCaDa). It consists of 25 synchronized videos (with audio) of two scenes recorded from different viewpoints of the campus. An extensive manual annotation comprises all moving objects and their corresponding bounding boxes, as well as audio events. The annotation was performed in order to ...


Quality-of-Experience

  • Towards Immersive Digiphysical Experiences

    Towards Immersive Digiphysical Experiences

    Immersive experiences have the potential of redefining traditional forms of media engagement by intricately combining reality with imagination. Motivated by necessities, current developments and emerging technologies, this column sets out to bridge immersive experiences in both digital and physical realities. Fitting under the umbrella term of eXtended Reality (XR), the first section describes various realizations ...

  • Explainable Artificial Intelligence for Quality of Experience Modelling

    Explainable Artificial Intelligence for Quality of Experience Modelling

    Data-driven Quality of Experience (QoE) modelling using Machine Learning (ML) arose as a promising alternative to the cumbersome and potentially biased manual QoE modelling. However, the reasoning of a majority of ML models is not explainable due to their black-box characteristics, which prevents us from gaining insights about how the model actually related QoE influence ...

  • Sustainability vs. Quality of Experience: Striking the Right Balance for Video Streaming

    Sustainability vs. Quality of Experience: Striking the Right Balance for Video Streaming

    The exponential growth in internet data traffic, driven by the widespread use of video streaming applications, has resulted in increased energy consumption and carbon emissions. This outcome is primarily due to higher resolution or higher framerates content and the ability to watch videos on various end-devices. However, efforts to reduce energy consumption in video streaming ...

  • Green Video Streaming: Challenges and Opportunities

    Green Video Streaming: Challenges and Opportunities

    Introduction Regarding the Intergovernmental Panel on Climate Change (IPCC) report in 2021 and Sustainable Development Goal (SDG) 13 “climate action”, urgent action is needed against climate change and global greenhouse gas (GHG) emissions in the next few years . This urgency also applies to the energy consumption of digital technologies. Internet data traffic is responsible for ...

  • Towards the design and evaluation of more sustainable multimedia experiences: which role can QoE research play?

    Towards the design and evaluation of more sustainable multimedia experiences: which role can QoE research play?

    In this column, we reflect on the environmental impact and broader sustainability implications of resource-demanding digital applications and services such as video streaming, VR/AR/XR and videoconferencing. We put emphasis not only on the experiences and use cases they enable but also on the “cost” of always striving for high Quality of Experience (QoE) and better ...

  • What is the trade-off between CO2 emission and video-conferencing QoE?

    What is the trade-off between CO2 emission  and video-conferencing QoE?

    It is a natural thing that users of multimedia services want to have the highest possible Quality of Experience (QoE), when using said services. This is especially so in contexts such as video-conferencing and video streaming services, which are nowadays a large part of many users’ daily life, be it work-related Zoom calls, or relaxing ...

  • Towards an updated understanding of immersive multimedia experiences

    Towards an updated understanding of immersive multimedia experiences

    Bringing theories and measurement techniques up to date Development of technology for immersive multimedia experiences Immersive multimedia experiences, as its name is suggesting are those experiences focusing on media that is able to immerse users with different interactions into an experience of an environment. Through different technologies and approaches, immersive media is emulating a physical world through ...

  • MPEG Visual Quality Assessment Advisory Group: Overview and Perspectives

    MPEG Visual Quality Assessment Advisory Group: Overview and Perspectives

    Introduction The perceived visual quality is of utmost importance in the context of visual media compression, such as 2D, 3D, immersive video, and point clouds. The trade-off between compression efficiency and computational/implementation complexity has a crucial impact on the success of a compression scheme. This specifically holds for the development of visual media compression standards which ...

  • ITU-T Standardization Activities Targeting Gaming Quality of Experience

    ITU-T Standardization Activities Targeting Gaming Quality of Experience

    Motivation for Research in the Gaming Domain The gaming industry has eminently managed to intrinsically motivate users to interact with their services. According to the latest report of Newzoo, there will be an estimated total of 2.7 billion players across the globe by the end of 2020. The global games market will generate revenues of $159.3 billion in 2020 . This ...

  • Immersive Media Experiences – Why finding Consensus is Important

    Immersive Media Experiences – Why finding Consensus is Important

    An introduction to the QUALINET White Paper on Definitions of Immersive Media Experience (IMEx) . Introduction Immersive media are reshaping the way users experience reality. They are increasingly incorporated across enterprise and consumer sectors to offer experiential solutions to a diverse range of industries. Current technologies that afford an immersive media experience (IMEx) include Augmented Reality (AR), ...

Comments are closed.