Category Archives: Opinion: Interview Column

An interview with Irene Viola

By | | , ,
Leave a comment
Irene at the beginning of her research career.

Describe your journey into research from your youth up to the present. What foundational lessons did you learn from this journey? Why were you initially attracted to multimedia?

My passion for multimedia stems from graphic design, actually. As a teenager, I taught myself Photoshop and I was playing around with coding websites. I chose Cinema and Media Engineering as my bachelor to combine the programming aspects with a media-based sensibility, and there I discovered that all the filters I had used in Photoshop had clear mathematical bases. I was hooked! I think the fact that I was coming from a more graphics background led me to always keep in mind the users who would see the end product. Applying filters and changing the appearance of an image or video needs to consider how the final user will engage with the content, how they will experience it. I think it has been very helpful in my research in the quality of experience for multimedia content.

Tell us more about your vision and objectives behind your current roles? What do you hope to accomplish and how will you bring this about?

I am currently working on immersive multimedia systems, and in particular on real-time communication systems. The vision is to make remote communication more lifelike, and interaction more natural. I think we’re all aware of how different a video call feels from a face-to-face meeting. Immersive multimedia can help users feel more present and connected, even when displaced in different corners of the globe. What I aim to accomplish is to bring this technology to everyday users, overcoming the current limitations.

Can you profile your current research, its challenges, opportunities, and implications?

My research is currently focused on the quality of experience for immersive media systems. There are several aspects to it: one aspect is to improve media delivery systems, be it by creating new compression solutions, or by improving the transmission efficiency through user-adaptive solutions, for example. The core idea is that we need to optimize transmission by keeping in mind how the users will interact with the content. Then there’s the aspect of quantifying the reaction of the users to the contents they’re visualizing, identifying the influencing factors and building models that can predict them. It’s quite challenging because we don’t fully understand yet how, and why, humans react the way they do to certain stimuli. But that’s also what makes it fascinating.

How would you describe your top innovative achievements in terms of the problems you were trying to solve, your solutions, and the impact it has today and into the future?

In terms of impact, I would say my top achievements would be the contributions to standardization bodies. My subjective methodologies were adopted to conduct the evaluation of the JPEG Pleno Call for Proposals for Light Field Compression, and along with my colleagues in VQEG, I have contributed to ITU recommendations. It’s quite gratifying to know that your research can serve the scientific community this way.

Over your distinguished career, what are the top lessons you want to share with the audience?

I think my message would be: don’t be afraid to switch up. Throughout my studies, I changed focus many times: in my bachelor, the focus was on sociological aspects of media, as well as technological ones; in my master, I dived deeper into the engineering side of it; in my PhD, I tried to understand the user reaction to media. Switching up allows you to see the same problem from different sides, which can be extremely useful in order to do successful research.

What is the best joke you know?

Since an image is worth a thousand words, I will leave you with my favourite comic strip, by artist Lee Gatlin:

A comic strip by Lee Gatlin (Original post)

If you were conducting this interview, what questions would you ask, and then what would be your answers?

My question would be: how do you best balance work and life? Which is a question I don’t have an answer for, and I’d like to read what other people do about it. I think research is pretty tough in this sense because you always have the feeling that there’s more that you could do, and if you just spend half an hour more, you can reach greater results. So, it’s hard to step back, and your work becomes your life. I try to be mindful of it and remind myself to disconnect, which also helps to get a fresh perspective.

A recent photo of Irene.

Bio: Irene Viola is a tenure-track researcher at the Centrum Wiskunde & Informatica. Her research interests include multimedia compression, transmission, and quality evaluation (https://www.ireneviola.com).

An interview with Benoit Huet

By | | , ,
Leave a comment
Benoit at the beginning of his research career.

Describe your journey into research from your youth up to the present. What foundational lessons did you learn from this journey? Why were you initially attracted to multimedia?

This is an excellent question. Indeed, life is a journey, and every step is a lesson. I was originally attracted by electronics but as I was studying it, I discovered computers. Remember, for those who were old enough in the 1980’s, this was the start of personal computers. So I decided to learn more about them, and as I was studying computer science I found out about AI, yes AI 1990’s style. I was interested, but this coincided with one of the AI winters and I was advised, or rather decided, to go in a different direction. The area that attracted me most was computer vision. The reason was that it seemed like a very hard problem which would clearly have a very broad impact. It turns out that vision alone is indeed very hard and using additional information or signals could help obtain better results, hence reducing time to impact for such a scientific approach/method. This was what attracted me to multimedia and kept me busy for many years at EURECOM. What did I learn along the way? Follow your instinct and your heart as you go along as it is rare to know where to go from the very start. Your destination might not even exist at the time you started your journey!

Tell us more about your vision and objectives behind your current roles? What do you hope to accomplish and how will you bring this about?

Since July 2019 I have headed the Data Science team of MEDIAN Technologies. The objective is to bring recent advances from the field of computer vision, neural networks, and also multimedia to part from the way medical imaging is currently performed while providing solutions to detect cancer at the earliest possible stage of the disease and help identify the best treatment for each patient. Concretely, we are currently working on the identification of biomarkers for Hepatocellular Carcinoma (HCC) which is the most common type of primary liver cancer and which is known to be a difficult organ for medical imaging solutions.

Can you profile your current research, its challenges, opportunities, and implications?

To answer this very broad question concisely, I will limit myself to one challenge, one opportunity, and one implication. For the challenge, I will mention one key challenge which I have encountered many times in many projects: Interdisciplinary communication. In most projects, whether small or large and involving multiple domains of expertise, the communication between people of different backgrounds is not as straightforward as one would assume. It is important to address this challenge proactively. For the opportunity, medical imaging is nowadays still mostly employing “traditional” machine learning on top of man-made features (Grey-Level Co-occurrence Matrices, Gabor, etc). The “end to end” paradigm shift brought in by the recent developments in the field of deep neural nets is still to take place in the medical domain at large. This is what we aim to achieve for medical imaging for Oncology. The implication, a significant improvement in the detection of cancer, such as the early detection of tumors. Such early detection allows for therapy to take place at the earliest possible stage hence drastically increasing the patient chance of survival. Saving lives in short.

How would you describe your top innovative achievements in terms of the problems you were trying to solve, your solutions, and the impact it has today and into the future?

There is a number of research work originating from my team that would be worth mentioning here; EventEnricher for collecting media illustrating event, the Hyper Video Browser, an interactive tool for searching and hyperlinking in broadcast media, or the work resulting from the collaborative project NexGen-TV: Providing real-time insight during political debates in a second screen application… to name just a few.

But the one with the highest impact is the work performed while on sabbatical at IBM T.J. Watson research center. As I onboarded, the research group received a request from the 20th Century Fox, regarding the possibility for some AI to help generate the trailer of a sci-fi horror movie that was about to be released. The project was both challenging and interesting as I had previously addressed video summarization and multimedia emotion recognition as part of previous research projects. The challenge was the limited amount of time available to deliver the shots which using state of the art machine learning were identified as the best suited to be part of the trailer. The team worked hard and hard work was rewarded multiple times. First because the hard deadline was met, having the “AI Trailer” on time for the screening of the movie in the US. Second because Fox sent a whole video crew to shoot the making of the trailer behind the scene. The video was posted on YouTube and got about 2 million views in about a week. This was the level of impact this scientific research work had. And if that was not enough, the work got another reward at the ACM Multimedia 2017 conference for being the Best Brave New Ideas paper that year.

Over your distinguished career, what are your top lessons you want to share with the audience?

Over the years I have observed that as a researcher, one needs to be curious while being able to find a good compromise between being focused and exploring new or alternative options/approaches. I feel that it is easy for today’s young researchers to be overwhelmed with the pace at which high-quality publications are becoming available. Social media (i.e. Twitter) and online repositories (i.e. Arxiv) are no stranger to this situation. There will always be a new paper reporting something potentially interesting with respect to your research, yet it doesn’t mean you should keep reading and reading at the cost of making slow or no progress on your own work! Reading and being aware of the state of the art is one thing, contributing and being innovative is another and the latter is the key to a successful PhD. Life as a researcher whether in academia or in the industry is made of choices, directions to follow, etc. While more senior people may sometimes rely on their experience, I believe it is important to listen to your inner self and follow what motivates you whenever possible. I have always believed and often witnessed that it is easier to work toward something of interest (to yourself) and in most cases the outcome exceeds expectations.

What is the best joke you know?

I have a very bad memory for jokes. Tell me one and I will laugh because I have a good sense of humor. But ask me to tell the story the next day and I will not be able to. So I looked jokes on the internet and here is the first one that made me laugh (I did read quite a few before!!!):

Two men meet on opposite sides of a river.  One shouts to the other, “I need you to help me get to the other side!” The other guy replies, “You’re on the other side!”

Not the best, but it will do for now!

If you were conducting this interview, what questions would you ask, and then what would be your answers?

The COVID-19 pandemic is affecting people’s lives on an international scale, do you think this will have an influence on research and in particular multimedia research?

Indeed, the situation is forcing us to change the way we collaborate and interact. As a researcher, one regularly travels for project meetings, conferences, PhD presentations, etc. in addition to local activities and events such as teaching, labs, group meetings, etc. With current travel restrictions and social distancing recommendations, remote work relying heavily on high bandwidth internet has developed to an unprecedented level, exposing both its limitations and advantages. Similarly, scientific conferences where a lot of interaction takes place have been forced to adapt. At first, organizers postponed the events, hoping the situation will quickly return to normal. However, with the extended duration of the pandemic, the shift from physical to remote or virtual conferencing, using online tools and systems, had to be performed. This clearly demonstrated not only the possibility of organizing such events online but also showed some limitations regarding interaction. On this topic, this could be a great opportunity for the multimedia community to have an impact at large. Indeed, who would be better suited to contribute to the next generation of tools for effective interactive remote work and conferencing than the multimedia community. I believe we have a role to play and look forward to seeing and using such tools. I didn’t touch on the health aspect of this question but that is also something multimedia researchers, usually well acquainted with the state of the art machine learning, can contribute to. On that note, if Medical Imaging is a topic that attracts you and that you are motivated by, do not hesitate to reach out.

Disclaimer: All views expressed in this interview are my own and do not represent the opinions of any entity with which I have been or am now affiliated.

A recent photo of Benoit.

Bio: Benoit Huet heads the data science team at MEDIAN Technologies. His research interests include computer vision, machine learning, and large-scale multimedia data mining and indexing.

An interview with Associate Professor Duc-Tien Dang-Nguyen

By | | , ,
Leave a comment
Tien at the beginning of his research career.

Describe your journey into research from your youth up to the present. What foundational lessons did you learn from this journey? Why were you initially attracted to multimedia?

Looking back at the early days of my life, my love for science started quite young. I loved solving puzzles and small recreational mathematical problems. Actually, I still do this. It may also be because my mother “seeded” many stories with great scientific people like Thomas Edison or Marie Curie every night. I admired them a lot and often dreamed of being like them. I also love to play video games. I played them a lot, and I think that I am quite good, especially in games like The Legend of Zelda and the Castlevania series. I also love to travel, and perhaps that is why I have a nomad’s journey over the last ten years, starting from Vietnam to Japan, Italy, Ireland, and now Norway. While living in Vietnam, I would often travel to the countryside on my motorbike. Solving puzzles, playing video games, collecting things, and travelling; these tiny things play an essential role in making me who I am today.

Now back to the story. I come from Vietnam, where it is very normal for my generation to grow through endless competitions. My first challenge was a math competition when I was eight. I then became a math student and followed many competitions like the current MIT Mystery Hunt. When I was 12, a friend of my father gave me his old PC as a present. It was a 486 (we called it that since it has an Intel 486 core), and it changed my life. I played with it endlessly. I learned Pascal by myself, and in the last year of my secondary schools (K-9), I proudly won the first rank at both Math and Informatics in the regional contests. Thanks to that, I entered one of the best high schools in Vietnam. I joined the Informatics class, and as you might already guess, we were dealing with programming challenges every day. We learned mainly algorithms and data structures, discrete mathematics, and computational complexity through solving challenging problems from the International Olympiad of Informatics. It is quite similar to Topcoder now. It was tough and very competitive, but it was exciting to me since it was like solving hard puzzles.

Moving to my bachelor’s, I took an honor program in Computer Science, which was one of the best Computer Science programs in Vietnam. In the third year of my bachelor’s in an Image Processing course, I did a project about image annotation. It was a pure K-means for image segmentation based on pixel color values, followed by a k-NN on a pre-trained set of images. It sounds pretty basic now, but this was in 2001, and “I did it my way” so it was a fantastic achievement! It was from this project I became a multimedia researcher.

After my bachelor’s, I continued researching computer vision and image retrieval in my master’s. In my first year as a Ph.D. student, I was working on a multimedia retrieval project, but just three months before the qualifying exam (you need to present your research proposal to continue your Ph.D.), I changed my research topic to Image Forensics, thanks to the course of the same name. I found everything I love in this new research field. It is like solving a puzzle, collecting evidence, and playing a game simultaneously. So, I became an image forensics researcher.

Some people say, “Choose a job you love, and you will never have to work a day in your life”, perhaps they are missing the last part “because no one will hire you”. Yes, it’s just a joke, but it can also be quite true in may circumstances. It was hard to find a job that needs image forensics when I finished my Ph.D. However, since I know image processing, computer vision, and machine learning, it was not that hard for me to find a postdoc in those fields. I was then doing both multimedia forensics and multimedia retrieval. This “evolvement” introduced me to a new field, lifelogging, a research direction that tries to discover insights from personal data archives. At first, it was an “okay” field to me, but later, after digging more into it, I found many interesting challenges that need to be solved. And that was a very long story about how I reach the starting point of my research.

Can you profile your current research, its challenges, opportunities, and implications? Tell us more about your vision and objectives behind your current roles.

Bergen, where I mainly focus on image forensics and lifelogging. Multimedia forensics is about discovering the history of modifications to multimedia content such as videos, images, audio, etc. Mainly, I work with images and have dabbled a bit in video forensics. Audio is nice too, but I mostly enjoy working with the visual side of multimedia. People tend to think about multimedia forensics as a tool to check if an image or a video is real or fake. However, we also try to look at the specifics for the media in question. Some potential questions for an image could, for example, be where was it first posted? What type of camera was it taken with? These are questions that help identify the reliability of the image in questions and give more information than fake or real. Also, I believe that we should also take a further step by considering the context of use (how, where, and when) of the multimedia content. The expectation of truthfulness is radically different if the image is hanging in an art gallery than if it is being used as evidence in a court case.

As previously mentioned, I also work with lifelogging. This work is still in its early stages. We have not proposed any novel approaches yet. Instead, we are building a community by organizing research activities as workshops and bench-marking initiatives. We believe that by holding such events, we are preparing a solid user-base for the next phase when people are more familiar with such technologies, the phase of personal data analytics. We have witnessed great applications of AI during the last decade. Since AI needs data, and people need more personalized solutions, I believe that very soon we will be doing lifelogging in our everyday life. Let’s wait and see if my prediction is becoming true or not.

How would you describe your top innovative achievements in terms of the problems you were trying to solve, your solutions, and the impact it has today and into the future?

In multimedia forensics, I am quite happy that I was among the first to propose an approach for discriminating between computer graphics and natural human faces. People are well aware of “Deepfake”, and many great people are working on this problem. However, when I presented my first study in 2011, many people, including computer graphics researchers, were laughing when I told them that they would soon not be able to distinguish computer-generated faces from the real ones. In image forensics, we try to reveal all traces of the image acquisition history, and since digital images are based on pixels, they are susceptible to changes. For example, many traces of modifications become incredibly hard to find if the image is resized. Most of my approaches are thus physical or geometrical based, which makes them more robust against changes as well as more reliable in terms of decision explanation.

Over your distinguished career, what are your top lessons you want to share with the audience?

I believe that I am still at the start of my career, and perhaps the first and the most important lesson I have is about “causes and effects” or what Steve Jobs described as “Connecting the dots”. There are dots in our life that are very hard to understand or predict how everything is connected, but eventually, when looking back, the connections will reveal themselves over time. Just follow whatever you think is good for you and try very hard to make it a good “dot”. Everyone wants to work with something we love, but finding what we love in our current work is even more important.

What is the best joke you know?

Most of the jokes I love are in Vietnamese, and unless you are Vietnamese, you can’t get them. I am trying to think about some “Western” jokes that share some commonalities with Vietnamese humor and culture. That should be a politics joke. I believe that you can find a similar version with KGB or Stasi. This one was very famous, and surprisingly, it is very well suited to my current research on lifelogging 🙂

“Why do Stasi officers make such good taxi drivers? — You get in the car and they already know your name and where you live.”

A recent photo of Tien.

Bio: Duc-Tien Dang-Nguyen is an associate professor at the University of Bergen. His main research interests are multimedia forensics, lifelogging, and machine learning.

An interview with Associate Professor Hugo L. Hammer

By | | , ,
Leave a comment
Hugo as a Ph.D. student, at the beginning of his research career.

Describe your journey into research from your youth up to the present. What foundational lessons did you learn from this journey? Why were you initially attracted to multimedia?

From an early age, I had the ability to focus and work individually and loved to develop new systems for all sorts of things, which probably was quite annoying for those around me. It turns out that it is these abilities to focus, being curious, and developing new systems is what drives my research today. When I started as a student in mathematics and statistics at the Norwegian University of Science and Technology (NTNU), I didn’t think of research as an alternative and was determined to find a job in the industry. Throughout the studies, I learned how little mathematics and statistics I had actually learned, which is why I decided to become a Ph.D. student. I expected to find a job in the industry after the Ph.D. period but ended up loving research, and that is why I am where I am today.

As a statistician, I have worked a lot with spatial and spatio-temporal data, such as geophysical observations. Such observations have striking similarities to multimedia content, such as images and videos. I have become very interested in machine learning methods used to process and make decisions from multimedia content and the potential for applying such methods towards other applications, such as geophysical applications. I also love working as a statistician within this field. A crucial part of my research is to try to combine methods from machine learning and statistics into new and exciting ways.

Tell us more about your vision and objectives behind your current roles? What do you hope to accomplish, and how will you bring this about? 

In my current position as an associate professor, I do both teaching and research. Teaching and research challenge me in different ways. I continuously try to develop and improve my teaching. I especially focus on how to do high quality, yet resource-efficient, teaching. I have, for example, worked a lot on how to activate students and improve learning when being a single teacher for hundreds of students.

Can you profile your current research, its challenges, opportunities, and implications?

My current research can roughly be divided into three directions. The first direction is about methods for real-time information processing and decision making, for example, from sensory information or video streams. The second direction is based on developing new machine learning models and methods, and as mentioned above, by taking advantage of my background in statistics. The third direction is doing more applied use of machine learning methods toward real-life multimedia data, in particular, medical data. Direction two and three go hand in hand. Having a background in statistics and working more and more with multimedia data is more of an opportunity than a challenge.

How would you describe your top innovative achievements in terms of the problems you were trying to solve, your solutions, and the impact it has today and into the future?

 I am proud of the research we have done on real-time information processing and decision making. Our developed methods are simple but still document state-of-the-art performance. In 2020, we plan to develop software packages to make the methods readily available and hopefully useful for many. We saw the potential of using machine learning, and in particular deep learning, towards geophysical data and problems quite early, and we are now able to operate at the forefront of this research. I’m also proud of our externally funded research projects and, for sure, our rejected research proposals.

Over your distinguished career, what are your top lessons you want to share with the audience?

Here is a lesson from my personal experience. I think it is easy to depend on or have too much respect for other researchers early in the career. Research is of course all about collaboration, but still, for me, it was useful early in the career to create a small research project where I did every step of the process myself (shaping ideas, collecting data, running simulation, writing, finding suitable publishing channels, revisions, etc.). It was hard work, but for sure, it made me a better and more independent researcher.

What is the best joke you know?

Daddy, what are clouds made of?

Linux servers, mostly.

If you were conducting this interview, what questions would you ask, and then what would be your answers?

One suggestion: What do you like to do in your spare time?

Research, right? 🙂 Working every day at an office, I try to find time for physical activity in my spare time. I love to run, bike, or go skiing in Nordmarka (a forest near Oslo, Norway) or in the mountains on the weekends.

A recent photo of Hugo.

Bio: Hugo L. Hammer is an associate professor in statistics at Oslo Metropolitan University. His main research interests are computational statistics, probabilistic forecasting, real-time analytics, and machine learning.

An interview with Professor Roger Zimmermann

By | | , ,
Leave a comment

Roger at the start of his career.

Please describe your journey into research from your youth up to the present. What foundational lessons did you learn from this journey? Why were you initially attracted to multimedia?

I have had an interest in technology early on, though my path to becoming an academic has not been very direct. In high school, I really enjoyed to tinker with electronics, taking radios apart, and learning about digital circuits. My goal was to work in this field, and after high school, I did an apprenticeship with Brown, Boveri & Cie. (BBC), which sometime later became Asea Brown Boveri (ABB). The apprentices were assigned to different company locations, and I was lucky enough to be sent to BBC’s Forschungszentrum (Research Center). The labs, the researchers, and the cutting-edge equipment and projects there left a deep impression on me. Beyond electronics, I really liked microprocessors, computers and how they could be flexibly programmed with software. I decided that I wanted to pursue further studies and I subsequently enrolled in the Höhere Technische Lehranstalt (HTL) Brugg-Windisch in their Informatik program (the HTL program has since changed and the building where I studied is now part of the campus Windisch of the Fachhochschule Nordwestschweiz). Fresh with my HTL degree in hand, I started to work for an engineering company and over the next years, I got the chance to work on some fascinating projects. After five years, I got an itch to study for a Master’s degree and I ended up in California. One of the professors (who became my advisor) encouraged me to go for a Ph.D., and I took him up on his offer to support me. His group worked at the intersection of databases and multimedia. It really fascinated me and we ended up building one of the early streaming media servers. What I still find fascinating about multimedia today is how it brings together many fundamental computer science areas such as networking, graphics, operating system support, signal processing, etc. I also like that multimedia is used by people to express their creativity, humanity and artistic aspirations – it is not only about technology.

My personal lessons looking back are that sometimes you may not know where your journey will take you, but make sure you enjoy and learn from the path to get there.

Tell us more about your vision and objectives behind your current roles? What do you hope to accomplish and how will you bring this about?

I currently work broadly in two areas, namely streaming media systems and data analytics. At this point, one of the main enjoyment I get is from working with my research group and international colleagues from around the world. On the technical side, it is fun if somebody is actually using what we develop. On the human side of things, it is great to see when my students and former students are doing well in various parts of the globe.

Can you profile your current research, its challenges, opportunities, and implications?

In my research group, I have two main themes and those are media systems and multimedia data analytics. In the first cluster, we look at media streaming on the Internet. The main technology in use today is Dynamic Adaptive Streaming over HTTP, also called DASH. Some interesting challenges are in the area of enabling very low latency in live streaming, which is of interest to many large Internet companies. Going forward, I see 5G networks as an interesting challenge. Most people are excited about the very high bandwidth that 5G can offer (in the best case), but I believe one of the major challenges will be the very high variability of 5G networks when a device is moving. On the multimedia, and especially spatial, data analytics side, I am part of a new lab between NUS and the ridesharing company Grab. There is a tremendous amount of data generated (e.g., GPS trajectories) that allow novel data-driven applications such as generating accurate road maps in regions where this information is not readily available or the inference of semantic attributes of roads (e.g., no right turn allowed). The fusion of multiple data types such as trajectories, images, maps, etc., will allow for some exciting new applications.

How would you describe your top innovative achievements in terms of the problems you were trying to solve, your solutions, and the impact it has today and into the future?

One of the areas where my group made innovative contributions was georeferenced mobile video — combining videos with their geo-spatial properties led to a lot of interesting developments. We started with this just about at the same time when the first iPhone came out, and the idea of utilizing all the sensors in a phone in combination with its video was really novel. Nowadays, sensor fusion is common and is used in many machine-learning applications and I am sure there will be even greater break-throughs in the future. Another area where I have been working for decades is media streaming and this whole industry has changed from proprietary networks to the Internet. There have been many people working in this area, but I believe that our own contributions have helped to transform this field.

Over your distinguished career, what are the top lessons you want to share with the audience?

My path to becoming an academic has not been as direct as for some other people. But one of the key things that I have enjoyed along the way was to work with many outstandingly talented and bright people from all around the world. I hope that humanity will keep working together based on facts and science to solve some of the big challenges that are coming our way.

If you were conducting this interview, what questions would you ask, and then what would be your answers?

One issue that concerns me is the apparent trend to not trust facts anymore. So a possible question could be: Do you see a danger when people easily distribute and believe in “alternate facts”?

My answer would be, I definitely see this as a considerable concern in the future. While there may be some technical solutions to combat fake news, etc., it is also increasingly important that people are well educated and think critically, especially in a world where fake information may look very persuasive.

 

What is the best joke you know?

I like many of the weird, but strangely funny comments on life and baseball from Yogi Berra. He was born Lawrence Peter Berra and was a US baseball legend. Two examples:

“When you come to a fork in the road, take it.”

“You should always go to other people’s funerals. Otherwise, they won’t come to yours.”


A current image of Roger.

Short bio:

Roger Zimmermann is an Associate Professor at the School of Computing at the National University of Singapore (NUS). He is also Deputy Director with the Smart Systems Institute (SSI) at NUS. From 2010 to 2016 he co-directed the Centre of Social Media Innovations for Communities (COSMIC), a research institute funded by the National Research Foundation (NRF) of Singapore. Prior to joining NUS he held the positions of Research Area Director with the Integrated Media Systems Center (IMSC) and Research Assistant Professor at the University of Southern California (USC). He earned his M.S. and Ph.D. degrees from the Viterbi School of Engineering at the University of Southern California.

An Interview with Professor Susanne Boll

By | | , ,
Leave a comment

Describe your journey into research from your youth up to the present. What foundational lessons did you learn from this journey? 

My journey into research started with my interest in computers and computer science at school while I was still in my early years at that time. I liked all the STEM subjects and was very good at these in school. I got in touch with programming and the first Mac in high school when my physics teacher started the first basic programming course. After highschool, I continued on this journey and became a Mathematical-Technical1 Assistant and continued studying CS and went on to do a PhD, always driven by the desire that I could learn more, could explore and understand more of this field.

Why were you initially attracted to Multimedia? 

Susanne at

Susanne Boll at the beginning of her research career in 2001

I was initially attracted by multimedia when information systems started to look at novel methods of integrating large amounts of unstructured multimedia and different media types into structured database systems. I joined the GMD Institute for Integrated Publication and Information Systems who were working on multimedia database systems. My PhD was on multimedia document models for representing and replaying multimedia presentations in the context of multimedia information systems. One of the most inspiring early events was a small but very nice IFIP working conference on Database Semantics – Semantic Issues in Multimedia Systems in New Zealand 1999 where I met many researchers from the multimedia community some of whom I still consider my research friends today. I stayed in the field of multimedia but as my work was always relating to the applications of multimedia and the interaction with the user it was not surprising that I moved into the field of Human Computer Interaction and SIGCHI in which I am an active member also today. Over the last three decades I have worked in the field of interactive multimedia and human computer interaction – in different application domains from personal media to health, from mobility to industry 4.0. To cite a much valued friend of mine whom I just met again – “I enjoy when my research makes me smile”, when I can see how research can be translated in applications for a better use.

Why did I volunteer for the role of the director for diversity and outreach? 

Professor Susanne Boll in 2019

Professor Susanne Boll in 2019

Over more than three decades now I was supporting gender equality as a mentor, in different roles, in committees and institutions, by speaking up and by driving actions. Within the multimedia community I observed that there are many individuals supporting and acting for a better gender equality, however, it remained efforts of individuals and we as a community were not able to turn this into a collective understanding. 

There were actually a few recent events related to SIGMM that made truly sad and consider if I should leave this community which I at the same time consider my home community. Some years ago I was observing in a panel in which only men were discussing the future and challenges in multimedia. Observing this was painful for me. I knew and met with each of them individually over the years and they were interesting researchers and great mentors. But that panel it made again obvious that we as a community failed to be inclusive also with regard to the women. Why would there be not an excellent woman would have her say in that panel? Why would not someone organizing the panel consider to be inclusive with regard to gender? Why would not the panelists, when they are invited, ask who else would be on the panel and encourage this?

When I talk about gender equality in these days I almost immediately get the reaction that gender is not diversity. People say that looking at gender equality would be too short sighted and that I should care more about diversity and not gender alone. So let me clearly say that I am well aware that diversity is not gender it is much more than that.  But, don’t let the perfect be the enemy of the good. My personal story starts with gender equality in STEM fields. Looking at women participation in SIGMM, I decided that the actions described in the “25 in 25’’ strategy would be a good starting point for my new role – it is just the beginning.

What are my plans serving in this position?

Within SIGMM, we need to understand and fully embrace the different dimensions of diversity. We should not use the term in the sense of an easy cover-up of a multitude of aspects in which the individual needs get blurred. I sometimes have the feeling as if one aspect of diversity could be traded for another one, and the term was used as if there was a measure that there is “sufficient” diversity in some setting. 

As a  director for diversity and outreach I will be caring about the richness of diversity.  I want to bring the different dimensions of diversity into the multimedia community and make us understand, embrace listen and take action for better diversity and outreach of SIGMM.

1Mathematical-Technical Assistant (MaTA, MA or MTA for short; also: mathematical-technical software developer) is the occupational title of a recognised training occupation according to the Vocational Training Act in Germany, which has existed since the mid-1960s. It is the first non-academic training occupation in data processing.

Bios

Prof. Susanne Boll: 

Susanne Boll is a full professor for Media Informatics and Multimedia Systems at the University of Oldenburg and a member of the board of the OFFIS-Institute for Information Technology. OFFIS belongs to the top 5% research institutes among the non-university institutes in computer science in Germany. Over the last two decades, she has consistently achieved highly competitive research results in the field of multimedia and human–computer interaction. She has actively been driving these fields of research by many scientific research projects and organization of highly visible events in the field. Her scientific results have been published in competitive peer-reviewed international conferences such as Multimedia, CHI, MobileHCI, AutomotiveUI, DIS, and IDC, as well as internationally recognized journals. Her research makes competitive contributions to the field of human computer interaction and ubiquitous computing. Her research projects also have a strong connection to industry partners and application partners and addresses highly relevant challenges in the applications field of automation in transportation systems as well as health care technologies. I am an active member of the scientific community and have co-chaired and organized many international events in my field. Her teaching follows combination of theoretical foundations with team-oriented and research-oriented practical assignments.  She currently leads a highly visible international team of researchers (PhD students, research associates, post docs, senior principal scientists).

An interview with Professor Pål Halvorsen

By | | , ,
Leave a comment

Describe your journey into research from your youth up to the present. What foundational lessons did you learn from this journey? Why were you initially attracted to multimedia?

I remember when I was about 14 years old and had an 8th grade project where we were to identify what we wanted to do in the future and the road to get there. I had just recently discovered the world of computers and so reported several ways to become a computer scientist. After following the identified path to the University of Oslo, graduating with a Bachelor in computer science, my way into research was more by chance, or maybe even by accident. At that time, I spent a lot of time on sports and was not sure what to do for my master thesis. However, I was lucky. I found an interesting topic in the area of system support for multimedia, mainly video. I guess my supervisors liked the work because they later offered me a PhD position (thanks!) where they brought me deeper into the world of multimedia systems research.

My supervisors then helped me to get an associate professor position at the university (thanks again!). I got to know more colleagues, all inspiring me to continue research in the area of multimedia. After a couple of years performing research as a continuation of my PhD and teaching system related courses, I got an opportunity to join Simula Research Laboratory together with Carsten Griwodz. A bit later, we started our own small research group at Simula, and it is still a great place to be.

I think it is safe to say my path has been to a large degree influenced by some of the great people that I have met. You cannot do everything yourself, and I have been blessed with a lot of very good colleagues and friends. As a PhD student, I was told that after a year I should know more about my topic than my supervisors. It sounded not possible, but after having supervised a number of students myself, I believe it is true! Another friend and colleague also said that he had learned everything he knew from his students. Again, very correct – my students (and colleagues) have taught me a lot (thanks!). Thus, my main take home message is to find an area that interests you and nice people to work with! You can accomplish a lot as a good team!  

Regarding my research interests, I initially found an interest in how efficient a computer system could be. I became fascinated by delivery of continuous media early on, and the “system support for multimedia” quickly became my area. After years of reporting an X% improvement of component Y, an interest of the complete end-to-end system rose. I have had a wish to build complete systems. So today, our research group does not only aim to improve individual components but also the entire pipeline in a holistic system – especially in the area of sports and medicine – where we can see the effects of the systems we deploy.

Pål Halvorsen at the beginning of his career

Pål Halvorsen at the beginning of his career as a computer scientist

Tell us more about your vision and objectives behind your current roles? What do you hope to accomplish and how will you bring this about?

Currently, I have several roles. My main position is with SimulaMet, a research center established by Simula Research Laboratory and Oslo Metropolitan University (OsloMet). I also recently moved my main university affiliation to OsloMet while still having a small adjunct professor position at University of Oslo. Both my research and teaching activities are related to my previously stated interests, and the combination of universities and research center is a perfect match for me, enabling a good mix of students and seniors.

I hope to be able to deliver results back into real systems, so that our results are not only published and then forgotten in a dark drawer somewhere. In this respect, we have contact with several real life “problem owners”, mainly in sports and medicine. To bring our results beyond research prototypes, we have also spun off both a sport and a medical company, achieving the vision of having real impact. The fact that we now run our systems for the two top soccer leagues in both Norway and Sweden is an example of our aims being fulfilled. Hopefully, we can soon say similar things in the medical scenario – that medical experts are assisted using our research-based systems!  

Can you profile your current research, its challenges, opportunities, and implications?

Having the end-to-end view, it is hard to make a short answer. We are trying to optimize both single components and the entire pipeline of components. Thus, we are doing a lot of different things. Our challenges are not only related to a specific requirement or a component, but also its integration into a system as a whole. We also address a number of real world applications. As you can see, the variety in our research is large.

However, there are also large opportunities in that the systems are researched and developed with real requirements and wishes in mind. Thus, if we succeed, there is a chance that we might actually have some impact. For example, in sports, we have three deployed systems in use.

How would you describe your top innovative achievements in terms of the problems you were trying to solve, your solutions, and the impact it has today and into the future?

Together with colleagues at Simula, University of Oslo and University of Tromsø, we have been lucky to find some interesting and usable solutions. For example, at the system level, we have solutions (code) included in the Linux kernel, and at the application level, or as efficient complete system providing functionality beyond existing systems, we have running (prototype) systems in both the areas of sport and medicine.

Pål Halvorsen today

Pål Halvorsen in his office in 2019

Over your distinguished career, what are your top lessons you want to share with the audience?

Well, first, I do not think you can call it “distinguished”. This is your description.

The most important thing for me is to have some fun. You must like what you do, and you must find people you enjoy working with. There are a lot of interesting challenges out there. You must just find yours.

What is the best joke you know?

Hehe, I am so bad at jokes. Every ten years, I might have a catchy comment, but I hardly ever tell jokes.

If you were conducting this interview, what questions would you ask, and what would be your answers?

Haha, I am not a man of many words, so I would probably just stick to the set of questions I was given and hoping it would soon be finished 😉

So, maybe this one last question:

Q: Anything to add?

A: No. (Both short since I have to both Q and A)

Bios

Professor Pål Halvorsen: 

Pål Halvorsen is a chief research scientist at SimulaMet, a professor at OsloMet University, an adjunct professor at University of Oslo, Norway, and the CEO of ForzaSys AS. He received his doctoral degree (Dr.Scient.) in 2001.  His research focuses mainly on complete  distributed multimedia systems including operating systems, processing, storage and retrieval, communication and distribution from a performance and efficiency point of view. He is a member of the IEEE and ACM. More information
can be found at http://home.ifi.uio.no/paalh

Pia Helén Smedsrud: 

Pia Helén Smedsrud is a PhD student at Simula Research Laboratory in Oslo, Norway. She has a medical degree from UiO (University of Oslo), and worked as a medical doctor before starting as a research trainee in the field of computer science at Simula. She also has a background from journalism. Her research interests include medical multimedia, clinical implementation and machine learning. Currently, she is doing her PhD in the intersection between informatics and medicine, on machine learning in endoscopy.

An interview with Géraldine Morin

By | | ,
Leave a comment

Please describe your journey into research from your youth up to the present. What foundational lessons did you learn from this journey? Why were you initially attracted to multimedia?

My journey into research was not such a linear path (or ’straight path’ as some French institutions put it —a criteria for them to hire)… I started convinced that I wanted to be a high school math teacher. Since I was accepted in a Math and CS engineering school after a competitive exam, I did accept to study there, working in parallel towards a pure math degree.
The first year, I did manage to follow both curricula (taking two math exams in September), but it was quite a challenge and the second year I gave up on the math degree to keep following the engineering curricula.
I finished with a master degree in applied Math (back then fully included in the engineering curricula) and really enjoyed working on the Master thesis (I did my internship in Kaiserslautern, Germany) so I decided to apply for a Ph.D. grant.
I made it into the Ph.D. program in Grenoble and liked my Ph.D. topic in geometric modelling but had a hard time with my advisor there.
So I decided after two years to give up, (passed a motorcycle driving licence) and went on teaching Math in high school for a year (also passed the teacher examination). Encouraged by my former German Master thesis advisor, I then applied for a Ph.D. program at Rice University in the US to work with Ron Goldman, a researcher whose work and papers I really liked. I got the position and really enjoyed doing research there.
After a wedding, a kid, and finishing the Ph.D. (in that order) I had moved to Germany to live with my husband and found a Postdoc position in Berlin for one year. I applied then to Toulouse, where I have stayed since. In Toulouse, I was hired in a Computer Vision research group, where a subgroup of people were tackling problems in multimedia, and offered me the chance to be the 3D-person of their team 🙂

I learned that a career, or research path, is really shaped by the people you meet on your way, for good or bad. Perseverance for something you enjoy is certainly necessary, and not staying in a context that do not fit you is also important! I am glad I did start again after giving up at first, but also do not regret my choice to give up either.

Research topic, and research areas, are important and a good match with your close collaborators is also very relevant to me. I really enjoy the multimedia community for that matter. The people are open minded and curious, and very encouraging… At multimedia conferences I always feel that my research is valued and relevant to the field (in the other communities, CG or CV, I sometimes get a remark like, ‘oh well, I guess you are not really doing C{G|V}’ …). Multimedia also has a good balance between theory and practice, and that’s fun !

Visit in Chicago during my Ph.D. in the US.

Visit in Chicago during my Ph.D. in the US.

 

Tell us more about your vision and objectives behind your current roles? What do you hope to accomplish and how will you bring this about?

I just took the responsibility of a department, while we are changing the curricula. This is a lot of organisation and administrative work, but also forces me to have a larger vision of how the field of computer science is evolving and what is important to teach. Interestingly, we prepare our student for jobs that do not exist yet ! This new challenge for me, also makes me realise how important it is to keep time for research, and the open-mindedness I get from my research activity.

Can you profile your current research, its challenges, opportunities, and implications?

As I mentioned before, currently, my challenge is to be able to keep on being active in research. I follow up on two paths: first in geometric modeling, trying to bridge the gap between my current interest in skeleton based models and two hot topics that are 3D printing, and machine learning.
The second is to continue working in multimedia, in distributing 3D content in a scalable way.
Concerning my implication, I am also currently co-heading the French geometric modeling group, and I very much appreciate to promote our research community, and contribute to keep it active and recognised.

How would you describe the role of women especially in the field of multimedia?

I have participated in my first women in MM meeting in ACM, and very much appreciated it. I have to admit I was not really interested in women targeted activities before I did participate in my first women workshop (WiSH – Women in SHape) in 2013, that brought groups on women to collaborate during one week… that was a great experience, that made me realise that, despite the fact that I really enjoy working with my -almost all male- colleagues, it was also fun and very inspiring to work with women groups. Moreover, being questioned by younger colleagues about the ability for a woman to have a family and faculty job, I now think that my good experience as a faculty and mother of 3 should be shared when needed.

How would you describe your top innovative achievements in terms of the problems you were trying to solve, your solutions, and the impact it has today and into the future?

My first contributions were in a quite theoretical field : during my Ph.D. I proposed to use analytic functions in a geometric modeling context. That raised some convergence issues that I managed to prove.
Later, I really enjoyed working with collaborators and proposing a shared topic with my colleague Romulus who worked on streaming, we started in 2006 to work on 3D streaming; that led us to collaborating with Wei Tsang Ooi for the National University of Singapore and for more than 12 years, we have been now advancing some innovative solutions for the distribution of 3D content, working on adapted 3D models for me, and system solutions for them… implying along the way new colleagues. Along the way, we won the best paper award for my Ph.D. student paper in the ACM MM in 2008 (I am very proud of that —despite the fact that I could not attend the conference, I gave birth between submission and conference ;).

Over your distinguished career, what are your top lessons you want to share with the audience?

A very simple one: Enjoy what you do! and work will be fun.
For me, I am amazed thinking over new ideas always remain so exciting 🙂

What is the best joke you know? 🙂

hard one !

Jogging in the morning to N Seoul Tower for sunrise, ACM-MM 2018.

Jogging in the morning to N Seoul Tower for sunrise, ACM-MM 2018.

 

If you were conducting this interview, what questions would you ask, and then what would be your answers?

I have heard there are very detailed studies, especially in the US about difference between male and female behaviour.
It seems that being aware of these helps. For example, women tend to judge themselves harder that men do…
(that’s not really a question and answer, more a remark :p )

Another try:
Q: What would make you feel confident/helps you get over challenges ?
A: I think I lack self confidence, and I always ask for a lot of feedback from colleagues (for examples for dry runs).
If I get good feedback, it boosts my confidence, if I get worst feedback, it helps me improve… I win both ways 🙂

 

Bios

Assoc. Prof. Géraldine Morin: 

Je suis Maître de conférences à l’ENSEEIHT, l’une des écoles de l’Institut National Polytechnique de Toulouse de l’Université de Toulouse, et j’effectue ma recherche à l’IRIT (UMR CNRS 5505). Avant de m’installer à Toulouse, j’étais Grenobloise et j’ai été diplomée de l’ENSIMAG (diplôme d’ingénieur) et de l’ Université Joseph Fourier (D.E.A. de mathématiques appliquées) ainsi qu’une licence de maths purs que j’ai suivi en parallèle à ma première année d’école d’ingénieur. J’ai ensuite fait une thèse en Modélisation Géométrique aux Etats-Unis à (Rice University) (“Analytic Functions for Computer Aided Geometric Design”) sous la direction de Ron Goldman. Ensuite, j’ai fait un postdoc d’un an en géométrie algorithmique, à la Freie Universität de Berlin.

An interview with Assoc. Prof. Ragnhild Eg

By | | , ,
Leave a comment

Please describe your journey into research from your youth up to the present. What foundational lessons did you learn from this journey? Why were you initially attracted to multimedia?

In high school, I really had no idea what I wanted to study in university. I liked writing, so I first tried out journalism. I soon discovered that I was too timid for this line of work, and the writing was less creative than I had imagined. So I returned to my favourite subject, psychology. I have always been fascinated by how the human mind works, how we can process all the information that surrounds us – and act on it. This fascination led me from a Bachelor in Australia, back to Norway where I started a Master in cognitive and biological psychology. One of my professors (whom I was lucky to have as a supervisor later) was working on a project on speech perception, and I still remember the first example she used to demonstrate how what we see can alter what we hear. I am delighted that I still encounter new examples of how multi-sensory processes can trick us. Most of all, I am interested by how these complex processes happen naturally, beyond our consciousness. And that is also what interests me in multimedia, how is it that we perceive information conveyed by digital systems in much the same way we perceive information from the physical world? And when we do not perceive it in the same way, what is causing the discrepancy?

My personal lessons are not to let a chosen path lead you in a direction you do not want to go. Moreover, not all of us are driven by a grand master plan. I am very much driven by impulses and curiosity, and this has led me to a line of work where curiosity is an asset.

Ragnhild Eg at the begin of hear research career in 2011

Ragnhild Eg at the beginning of her research career in 2011.

Tell us more about your vision and objectives behind your current roles? What do you hope to accomplish and how will you bring this about?

I currently work at a university college, where I have the opportunity to combine two passions: teaching and research. I wish to continue with both, so my vision relates to my research progression. My objective is pretty basic, I wish to broaden the scope of my research to include more perspectives on human perception. To do that, I want to start with new collaborations that can lead to long-term projects. As mentioned, I often let curiosity guide me, and I do not intend to stop doing just that.

Can you profile your current research, its challenges, opportunities, and implications?

In later years, my research scope has extended from perception of multimedia content to human-computer interactions, and further on to individual factors. Although we investigate perceptual processes in the context of computer systems’ limitations, our original approach was to generalise across a population. Yet, the question of how universal perceptual processes can differ so much between individuals has become more and more intriguing.

How would you describe the role of women especially in the field of multimedia?

I have a love-hate relationship when it comes to stereotypes. Not only are they unavoidable, they are essential for us to process information. Moreover, it can be quite amusing to apply characteristics to stereotypes. On the other hand, stereotypes contribute to preserve, and even strengthen, certain conceptions about individuals. On the topic of women in multimedia, I find it important because we are a minority and I believe any community benefits from diversity. However, I find it difficult to describe our role without falling back on stereotypical gender traits.

How would you describe your top innovative achievements in terms of the problems you were trying to solve, your solutions, and the impact it has today and into the future?

The path that led me to multimedia research started with my studies in psychology, so I came into the field with a different outlook. I use my theoretical knowledge about human cognition and perception, and my experience with psychological research methods, to tackle multimedia challenges. For instance, designing behavioural studies with experimental controls and validity checks. Perhaps not innovative, my first approach to study the perception of multimedia quality was to avoid addressing quality, and rather control it as an experimental factor. Instead, I explored variations in perceptual integration, across different quality levels. Interestingly, I see more and more knowledge introduced from psychology and neuroscience to multimedia research. I regard these cross-overs as an indication that multimedia research has come to be an established field with versatile research methods, and I look forward to seeing what insights come out of it.

Over your distinguished career, what are your top lessons you want to share with the audience?

When I started my PhD, I came into a research environment dominated by computer science. The transition went far smoother than I had imagined, mostly due to open-minded and welcoming colleagues. Yet, working with inter-disciplinary research will lead to encounters where you do not understand the contributions of others, and they may not understand yours. Have respect for the knowledge and expertise others bring with them, and expect the same respect for your own strengths. This type of collaboration can be demanding, but can also bring about the most interesting questions and results.

Another lesson I want to share, is perhaps one that can only come through personal experience. I enjoy collaborating on research projects, but being a researcher also requires a great deal of autonomy. Only at the end of the first year did I realise that no one could tell me what should be the focus of my PhD, even though I was expected to contribute to a larger project. Research is not constrained by clear boundaries, and I believe a researcher must be able to apply their own curiosity even when external forces seem to enforce limits.

Ragnhild Eg in 2018.

Ragnhild Eg in 2018.

If you were conducting this interview, what questions would you ask, and then what would be your answers?

I would ask what is the best joke you know! And my answer would undoubtedly be a knock-knock joke. 
Editor’s note: Officially added to the standard questionnaire!

What is the best joke you know? 🙂

Knock knock

– Who’s there?

A little old lady

– A little old lady who?

Wow, I had no idea you could yodel! 

Bios

Assoc. Prof. Ragnhild Eg: 

Ragnhild Eg is an associate professor at Kristiania University College, where she combines her background and interests in psychology with research and education. She teaches psychology and ethics, and pursue research interests spanning from perception and the effects of technological constraints, to the consequences of online media consumption.

Michael Alexander Riegler: 

Michael is a scientific researcher at Simula Research Laboratory. His research interests are medical multimedia data analysis and understanding, image processing, image retrieval, parallel processing, crowdsourcing, social computing and user intent. 

Interview with Dr. Magda Ek Zarki and Dr. De-Yu Chen: winners of the Best MMsys’18 Workshop paper award

By | | , , , ,
Leave a comment

Abstract

The ACM Multimedia Systems conference (MMSys’18) was recently held in Amsterdam from 9-15 June 2018. The conferencs brings together researchers in multimedia systems. Four workshops were co-located with MMSys, namely PV’18, NOSSDAV’18, MMVE’18, and NetGames’18. In this column we interview Magda El Zarki and De-Yu Chen, the authors of the best workshop paper entitled “Improving the Quality of 3D Immersive Interactive Cloud-Based Services Over Unreliable Network” that was presented at MMVE’18.

Introduction

The ACM Multimedia Systems Conference (MMSys) (mmsys2018.org) was held from the 12-15 June in Amsterdam, The Netherlands. The MMsys conference provides a forum for researchers to present and share their latest research findings in multimedia systems. MMSys is a venue for researchers who explore complete multimedia systems that provide a new kind of multimedia or overall performance improves the state-of-the-art. This touches aspects of many hot topics including but not limited to: adaptive streaming, games, virtual reality, augmented reality, mixed reality, 3D video, Ultra-HD, HDR, immersive systems, plenoptics, 360° video, multimedia IoT, multi- and many-core, GPGPUs, mobile multimedia and 5G, wearable multimedia, P2P, cloud-based multimedia, cyber-physical systems, multi-sensory experiences, smart cities, QoE.

Four workshops were co-located with MMSys in Amsterdam in June 2018. The paper titled “Improving the Quality of 3D Immersive Interactive Cloud-Based Services Over Unreliable Network” by De-Yu Chen and Magda El-Zarki from University of California, Irvine was awarded the Comcast Best Workshop Paper Award for MMSys 2018, chosen from among papers from the following workshops: 

  • MMVE’18 (10th International Workshop on Immersive Mixed and Virtual Environment Systems)
  • NetGames’18 (16th Annual Workshop on Network and Systems Support for Games)
  • NOSSDAV’18 (28th ACM SIGMM Workshop on Network and Operating Systems Support for Digital Audio and Video)
  • PV’18 (23rd Packet Video Workshop)

We approached the authors of the best workshop paper to learn about the research leading up to their paper. 

Could you please give a short summary of the paper that won the MMSys 2018 best workshop paper award?

In this paper we discussed our approach of an adaptive 3D cloud gaming framework. We utilized a collaborative rendering technique to generate partial content on the client, thus the network bandwidth required for streaming the content can be reduced. We also made use of progressive mesh so the system can dynamically adapt to changing performance requirements and resource availability, including network bandwidth and computing capacity. We conducted experiments that are focused on the system performance under unreliable network connections, e.g., when packets can be lost. Our experimental results show that the proposed framework is more resilient under such conditions, which indicates that the approach has potential advantage especially for mobile applications.

Does the work presented in the paper form part of some bigger research question / research project? If so, could you perhaps give some detail about the broader research that is being conducted?

A more complete discussion about the proposed framework can be found in our technical report, Improving the Quality and Efficiency of 3D Immersive Interactive Cloud Based Services by Providing an Adaptive Application Framework for Better Service Provisioning, where we discussed performance trade-off between video quality, network bandwidth, and local computation on the client. In this report, we also tried to tackle network latency issues by utilizing the 3D image warping technique. In another paper, Impact of information buffering on a flexible cloud gaming system, we further explored the potential performance improvement of our latency reduction approach, when more information can be cached and processed.

We received many valuable suggestions and identified a few important future directions. Unfortunately, De-Yu, graduated and decided to pursue a career in the industry. He will not likely to be able to continue working on this project in the near future.

Where do you see the impact of your research? What do you hope to accomplish?

Cloud gaming is an up-and-coming area. Major players like Microsoft and NVIDIA have already launched their own projects. However, it seems to me that there is not a good enough solution that is accepted by the users yet. By providing an alternative approach, we wanted to demonstrate that there are still many unsolved issues and research opportunities, and hopefully inspire further work in this area.

Describe your journey into the multimedia research. Why were you initially attracted to multimedia?

De-Yu: My research interest in cloud gaming system dated back to 2013 when I worked as a research assistant in Academia Sinica, Taiwan. When U first joined Dr. Kuan-Ta Chen’s lab, my background was in parallel and distributed computing. I joined the lab for a project that is aimed to provide a tool that help developers do load balancing on massively multiplayer online video games. Later on, I had the opportunity to participate in the lab’s other project, GamingAnywhere, which aimed to build the world’s first open-source cloud gaming system. Being an enthusiastic gamer myself, having the opportunity to work on such a project was really an enjoyable and valuable experience. That experience came to be the main reason for continuing to work in this area. 

Magda El Zarki: I have worked in multimedia research since the 1980’s when I worked for my PhD project on a project that involved the transmission of data, voice and video over a LAN. It was named MAGNET and was one of the first integrated LANs developed for multimedia transmission. My work continued in that direction with the transmission of Video over IP. In conjunction with several PhD students over the past 20—30 years I have developed several tools for the study of video transmission over IP (MPEGTool) and has several patents related to video over wireless networks. All the work focused on improving the quality of the video via pre and post processing of the signal.

Can you profile your current research, its challenges, opportunities, and implications?

There are quite some challenges in our research. First of all, our approach is an intrusive method. That means we need to modify the source code of the interactive applications, e.g. games, to apply our method. We found it very hard to find a suitable open source game whose source code is neat and clean and easy to modify. Developing our own fully functioning game is not a reasonable approach, alas, due to the complexity involved. We ended up building a 3D virtual environment walkthrough application to demonstrate our idea. Most reviewers have expressed concerns about synchronization issues in a real interactive game, where there may be AI controlled objects, non-deterministic processes, or even objects controlled by other players. We agree with the reviewers that this is a very important issue. But currently it is very hard for us to address it with our limited resources. Most of the other research work in this area faces similar problems to ours – lack of a viable open source game for researchers to modify. As a result, researchers are forced to build their own prototype application for performance evaluation purposes. This brings about another challenge: it is very hard for us to fairly compare the performance of different approaches given that we all use a different application for testing. However, these difficulties can also be deemed as opportunities. There are still many unsolved problems. Some of them may require a lot of time, effort, and resources, but even a little progress can mean a lot since cloud gaming is an area that is gaining more and more attention from industry to increase distribution of games over many platforms.

“3D immersive and interactive services” seems to encompass both massive multi-user online games as well augmented and virtual reality. What do you see as important problems for these fields? How can multimedia researchers help to address these problems?

When it comes to gaming or similar interactive applications, all comes down to the user experience. In the case of cloud gaming, there are many performance metrics that can affect user experience. Identifying what matters the most to the users would be one of the important problems. In my opinion, interactive latency would be the most difficult problem to solve among all performance metrics. There is no trivial way to reduce network latency unless you are willing to pay the cost for large bandwidth pipes. Edge computing may effectively reduce network latency, but it comes with high deployment cost.

As large companies start developing their own systems, it is getting harder and harder for independent researchers with limited funding and resources to make major contributions in this area. Still, we believe that there are a couple ways how independent researchers can make a difference. First, we can limit the scope of the research by simplifying the system, focusing on just one or a few features or components. Unlike corporations, independent researchers usually do not have the resources to build a fully functional system, but we also do not have the obligation to deliver one. That actually enables us to try out some interesting but not so realistic ideas. Second, be open to collaboration. Unlike corporations who need to keep their projects confidential, we have more freedom to share what we are doing, and potentially get more feedback from others. To sum up, I believe in an area that has already attracted a lot of interest from industry, researchers should try to find something that companies cannot or are not willing to do, instead of trying to compete with them.

If you were conducting this interview, what questions would you ask, and then what would be your answers?

 The real question is: Is Cloud Gaming viable? It seems to make economic sense to try to offer it as companies try to reach a broader  and more remote audience. However, computing costs are cheaper than bandwidth costs, so maybe throwing computing power at the problem makes more sense – make more powerful end devices that can handle the computing load of a complex game and only use the network for player interactivity.

Biographies of MMSys’18 Best Workshop Paper Authors

Prof Magda El Zarki (Professor, University of California, Irvine):

Magda El Zarki

Prof. El Zarki’s lab focuses on multimedia transmission over the Internet. The work consists of both theoretical studies and practical implementations to test the algorithms and new mechanisms to improve quality of service on the user device. Both wireline and wireless networks and all types of video and audio media are considered. Recent work has shifted to networked games and massively multi user virtual environments (MMUVE). Focus is mostly on studying the quality of experience of players in applications where precision and time constraints are a major concern for game playability. A new effort also focuses on the development of games and virtual experiences in the arena of education and digital heritage.

De-Yu Chen (PhD candidate, University of California, Irvine):

De-Yu Chen

De-Yu Chen is a PhD candidate at UC Irvine. He received his M.S. in Computer Science from National Taiwan University in 2009, and his B.B.A. in Business Administration from National Taiwan University in 2006. His research interests include multimedia systems, computer graphics, big data analytics and visualization, parallel and distributed computing, cloud computing. His most current research project is focused on improving quality and flexibility of cloud gaming systems.