Codec-Agnostic Dynamic and Distributed Adaptation of Scalable Multimedia Content
Supervisor(s) and Committee member(s): Supervisors: Hermann Hellwagner, Rik Van de Walle
Today’s Internet is accessible to diverse end devices through a wide variety of network types. Independent from this huge amount of usage contexts, content consumers desire to retrieve content with the best possible supported quality. The designers of new media codecs react to this diversity of usage contexts by including adaptation support into the codec design. Scalable media codecs, such as the new MPEG-4 Scalable Video Codec, enable to easily retrieve different qualities of the media content by simply disregarding certain media segments. All these variables (different end devices, network types, user preferences, media codec types, scalability options) lead to a manifold of needed and possible adaptation operations.
In order to counter this complexity, the MPEG-21 Digital Item Adaptation (DIA) standard specifies a set of descriptions (and related processes) in order to describe the media content, the adaptation possibilities and the usage context in the XML domain. The relevant descriptions are: 1) The generic Bitstream Syntax Description (gBSD), which uses a generic language to describe, for instance, the parts of a media content which may be removed for scalability purposes. 2) The Adaptation Quality of Service Description (AQoS), which describes how (segments of) a media content need(s) to be adapted in order to correspond to the various usage contexts, e.g., how many quality layers need to be dropped to correspond to the currently available network bandwidth. 3) The Usage Environment Descriptions (UEDs) which describe the usage context, e.g., the available network bandwidth. Since all of these descriptions, i.e., all codec-specific information, are provided together with the media content, this helps to enable codec-agnostic adaptation nodes, which support any type of scalable media which is properly described by those DIA descriptions.
This thesis extends the static, server-based, gBSD-driven adaptation mechanism towards dynamic and distributed environments. To achieve this, novel mechanisms for fragmentation, storage and transport of content-related XML metadata are introduced. One particular contribution is the introduction of the concept of samples for metadata by employing Streaming Instructions which steer the fragmentation of and provide timing for XML-based metadata. This enables the synchronized processing of such a metadata stream with the described media samples. Furthermore, investigations of the ISO Base Media File Format show how such metadata streams can be stored for later processing. Finally, the applicability of the Real-Time Transport Protocol (RTP) is analyzed for the transport of such metadata streams. A codec-agnostic adaptation node based on these novel mechanisms is implemented and evaluated with regards to its adaptation performance for different types of scalable media. Extensive measurements with these scalable media contents show which parts of the gBSD-based adaptation process (could) benefit most from optimization.
Additionally, a mechanism based on a novel binary header to enable codec-agnostic adaptation of media content is specified. This Generic Scalability Header (GSH) prefixes each media packet payload and is based on the concepts of the gBSD-based adaptation mechanism. It provides information on both the bitstream syntax and the adaptation options and therefore combines (some of) the information provided by the MPEG-21 DIA gBSD and AQoS descriptions. However it enables codec-agnostic adaptation at a considerably lower performance cost. As above, the adaptation performance of this mechanism is evaluated for several types of scalable media. Finally, both mechanisms are implemented in the same adaptation architecture and compared to each other and additionally to a codec-specific adaptation approach using several types of scalable media.
A concluding discussion analyzes the results of the quantitative and qualitative evaluation of both mechanisms. Most notably the measurements show that for MPEG-4 Scalable Video Codec and MPEG-4 Visual Elementary Streams the GSH-based mechanism’s throughput is only about 1.25 times lower than for the codec-specific mechanism and the metadata overhead is less than 1 percent. The gBSD-based mechanism comes at a higher cost for these codecs (about 10 times lower throughput and a maximum of 10 percent metadata overhead with compression). We conclude that, depending on the application scenario, both mechanisms can be viable alternatives to existing codec-specific adaptation approaches. In particular in scenarios where contents encoded with diverse (and potentially changing) scalable media codecs need to be adapted, the flexibility of codec-agnostic approaches can outweigh their reduced performance.
Multimedia Communication (MMC)
The research group "Multimedia Communication (MMC)" was founded and is being led by Prof. Hermann Hellwagner. In addition, the group currently has three research assistants, seven project staff members, and three administrative and technical staff members.
The research activities of the group are in the areas of: Multimedia communication and quality of service (QoS) provisioning; Adaptation of multimedia content with respect to.t. network, device and usage contexts; Standardization within the ISO/IEC MPEG group (MPEG-21 - Multimedia Framework); Mobile, adaptive multimedia applications; Multimedia in disaster management.
The focus of the MMC group is clearly on adaptive delivery of audio-visual contents, taking into account, for instance, fluctuating network and environmental conditions that can occure when users are on the move. In particular, we are currently investigating the use of Scalable Video Coding (SVC) technology in such networks.
The group actively participates in several international and national research projects on all levels, ranging from basic research to application-oriented projects and direct cooperation with industry.
In teaching, the MMC group covers the technical courses of the Informatics study programme such as Computer Organization, Operating Systems, Computer Networks, Servers and Clusters, Internet QoS, and Multimedia Coding.