Image and Video context determination

Student Project (2-3 participants)

Artificial Intelligence (AI) is one of the fastest growing technologies of our time. It is considered by Google CEO Sundar Pichai as “more important to humanity than fire or electricity”. The aim of this project is to evaluate the ability of todays’ available AI tools and techniques to analyze images and videos in order to determine their context.

Your tasks:

  • Research, understand and evaluate related work on AI tools and techniques that enable generation of image and video tags and determination of context
  • Implement a demo application using one of the currently available solutions
  • Evaluate your implementation, its reliability and limitations

Required skills:

  • Python / Lua / NodeJS
  • Basic Knowledge on AI / machine learning / deep learning

Related Links:

Contact:

Mehdi Ben Fadhel

Design and implementation of JSON based media playlists

Student project (2-3 participants)

Adaptive video streaming is ubiquitous. The most popular streaming standards rely on XML-based manifests (MPD in DASH) or text-line-based playlists (M3U8 in HLS) to describe the media to be served including its representations and involved media files. Software media players that consume this media must therefore fetch and parse the manifests/playlists. Interestingly, video platforms like YouTube or Vimeo omit this step and use JSON based descriptions for the adaptive media to be streamed.

Your tasks:

  • Learn and elaborate the feature sets of Media Presentation Descriptions (DASH) and Media Playlists (HLS)
  • Determine and investigate on the components fetching and parsing the Manifests/Playlists in popular open source media streaming players (like dash.js, shaka, hls.js)
  • Design a JSON based manifest/playlist format that covers most of the features of DASH/HLS
  • Refactor the codebases of at least two media players so that that they accept the new common media format and stream the media
  • Evaluate the approach and implementation with regard to stream startup delay, processing need on the client etc


Required skills:

  • Node.js/JavaScript (must)

Related technologies:

Contact:

Martin Lasak

Adaptive Media Streaming

Student Project (1-3 participants), Bachelor- and Master Thesis

Adaptive media streaming solutions adjust the quality of video and audio content depending on the current available bandwidth on the client. For that purpose the content is encoded in different qualities and segmented into small chunks. Adaptive behavior is realized by appending chunks of different qualities.

Dynamic Adaptive Streaming over HTTP (MPEG-DASH) and HTTP Live Streaming (HLS) are standards to enable adaptive media streaming in the browser using conventional HTTP servers.

Participants of this project will work on adaptive media streaming related technology. Potential topics include the design of adaptive streaming algorithms and additions to existing open source players like dash.js and Shaka. Moreover, the implementation of a Node.js based DASH and HLS packager is in the scope of this project.

Tasks:

  • Research & develop adaptive media streaming related technologies
  • Integration with open source MPEG-DASH players (e.g. dash.js, shaka)
  • Extend a Node.js based DASH segmenter to support new features

Related Technologies:

  • MPEG-DASH, HLS
  • JavaScript and Node.js

Contact:

Stefan Pham
Daniel Silhavy

Progressive Web Apps for Media Streaming

Student project (1-3 participants)

Progressive Web Apps (PWA) technology bring apps that are built using Web technologies closer to native apps. Deployment of Web apps across devices is eased, because of standardized APIs.

For example, on mobile platforms (e.g. iOS or Android) PWAs can be added to the home screen with the help of Web App Manifests; on Windows, PWAs can be submitted to the Microsoft Store. Using Service Workers API and Indexed DB, offline playback capability can be enabled. Performance can be improved with the help of Web Workers and WebAssembly.

Your Tasks:

  • Develop an interoperable PWA for media playback, which includes a catalogue and media player (e.g. dash.js or shaka-player)
  • Research and develop download and offline playback functionality for adaptive streams
  • Test in different browsers and devices

Required Skills:

  • Web Technologies: JavaScript, HTML, CSS
  • Basic knowledge in adaptive media streaming technology

Contact:

Stefan Pham

Daniel Silhavy


Related Links:

Experimentation with WebAssembly

Student project (1-3 participants)

WebAssembly or "wasm" is a portable, low-level programming language for in-browser execution to complement JavaScript. Initially, C/C++ is supported. It aims at enabling high performance applications on the Web and is currently being designed as an open standard by a W3C Community Group.

New opportunities arise for media-related use cases such as video transcoding and manipulation in the Web browser using WebAssembly.


Your Tasks:

  • Understand the specification and relationship to asm.js and Emscripten
  • Research requirements and browser support for WebAssembly
  • Research & develop media processing-related use cases for WebAssembly


Required Skills:

  • Web Technologies: JavaScript, HTML, CSS, C/C++
  • Basic knowledge in media streaming


Contact:

Stefan Pham

Daniel Silhavy

Experimentation with Blockchain for Media

Student project (1-3 participants)

Blockchain seems to be a promising technology for rights management and micro payments between users without a central entity. In blockchain architecture, transactions are stored in a public, distributed ledger. When a new transaction is made, it needs to be verified by the decentralized network. This promises to bring transparency, security and efficiency to transactions.


Your Tasks:

  • Understand Blockchain technology and its cryptographic concepts
  • Research & develop with existing Blockchain solutions


Required Skills:

  • Web Technologies: JavaScript, HTML, CSS, C/C++
  • Basic knowledge in crypto and media streaming technology


Contact:

Stefan Pham

Franck Russel Kuipou

Daniel Silhavy

Per-title encoding and dynamic encoding ladders

Student project (1-3 participants), Bachelor- and Master Thesis

A sophisticated video encoding chain is a key factor for the successful delivery of media content over the internet. For a long time the same encoding parameters were used for different types of content, resulting in a single static encoding ladder. However, as video content highly differs in terms of complexity and details, it is recommended to use custom encoding settings for individual media files. Depending on the characteristics of an asset the best encoding parameters are identified. As a result an optimal video quality is achieved, while not wasting bandwidth.

Your tasks:

  • Evaluate different static encoding ladders and identify potential weaknesses
  • Design a stand-alone component that takes an input video and outputs a customized encoding ladder depending on the complexity of the asset
  • Identify and potentially apply machine learning and AI technologies to further improve your component
  • Apply video quality metrics like PSNR and VMAF to evaluate your solution

Required skills:

Basic understanding/Interest in media streaming

Related links:

Contact:

Daniel Silhavy
Stefan Pham

Media Streaming – low latency streaming

Student Project (3-4 participants)

In live streaming there is some delays until the user get the first image of the video. Low latency streaming reduces this delay consequently and you get real time video. The aim is to show the difference between low latency live stream and normal live streaming. For that a camera recording will be streamed via MPEG-DASH and a player will be built to show that stream. The live stream will be provided to you.

Your tasks:

  • Build the player based on an already existing player (dashjs) to play the live stream
  • For that change the xhr request in dahjs to fetchapi
  • Compare low latency live stream and normal live stream

Required skills:

  • JavaScript, NodeJS

Related Links:

Contact:

Franck Russel Kuipou