Projects on Media Streaming
- Image and Video context determination
- Design and implementation of JSON based media playlists
- Progressive Web Apps for Media Streaming
- Adaptive Media Streaming
- Experimentation with WebAssembly
- Experimentation with Blockchain for Media
- Per title encoding and dynamic encoding ladders
- Media Streaming – low latency streaming
Image and Video context determination
Student Project (2-3 participants)
Artificial Intelligence (AI) is one of the fastest growing technologies of our time. It is considered by Google CEO Sundar Pichai as “more important to humanity than fire or electricity”. The aim of this project is to evaluate the ability of todays’ available AI tools and techniques to analyze images and videos in order to determine their context.
- Research, understand and evaluate related work on AI tools and techniques that enable generation of image and video tags and determination of context
- Implement a demo application using one of the currently available solutions
- Evaluate your implementation, its reliability and limitations
- Python / Lua / NodeJS
- Basic Knowledge on AI / machine learning / deep learning
Design and implementation of JSON based media playlists
Student project (2-3 participants) - Please notice: Fully booked for this semester
Adaptive video streaming is ubiquitous. The most popular streaming standards rely on XML-based manifests (MPD in DASH) or text-line-based playlists (M3U8 in HLS) to describe the media to be served including its representations and involved media files. Software media players that consume this media must therefore fetch and parse the manifests/playlists. Interestingly, video platforms like YouTube or Vimeo omit this step and use JSON based descriptions for the adaptive media to be streamed.
- Learn and elaborate the feature sets of Media Presentation Descriptions (DASH) and Media Playlists (HLS)
- Determine and investigate on the components fetching and parsing the Manifests/Playlists in popular open source media streaming players (like dash.js, shaka, hls.js)
- Design a JSON based manifest/playlist format that covers most of the features of DASH/HLS
- Refactor the codebases of at least two media players so that that they accept the new common media format and stream the media
- Evaluate the approach and implementation with regard to stream startup delay, processing need on the client etc
Adaptive Media Streaming
Student Project (1-3 participants), Bachelor- and Master Thesis
Adaptive media streaming solutions adjust the quality of video and audio content depending on the current available bandwidth on the client. For that purpose the content is encoded in different qualities and segmented into small chunks. Adaptive behavior is realized by appending chunks of different qualities.
Dynamic Adaptive Streaming over HTTP (MPEG-DASH) and HTTP Live Streaming (HLS) are standards to enable adaptive media streaming in the browser using conventional HTTP servers.
Participants of this project will work on adaptive media streaming related technology. Potential topics include the design of adaptive streaming algorithms and additions to existing open source players like dash.js and Shaka. Moreover, the implementation of a Node.js based DASH and HLS packager is in the scope of this project.
- Research & develop adaptive media streaming related technologies
- Integration with open source MPEG-DASH players (e.g. dash.js, shaka)
- Extend a Node.js based DASH segmenter to support new features
- MPEG-DASH, HLS
Progressive Web Apps for Media Streaming
Student project (1-3 participants)
Progressive Web Apps (PWA) technology bring apps that are built using Web technologies closer to native apps. Deployment of Web apps across devices is eased, because of standardized APIs.
For example, on mobile platforms (e.g. iOS or Android) PWAs can be added to the home screen with the help of Web App Manifests; on Windows, PWAs can be submitted to the Microsoft Store. Using Service Workers API and Indexed DB, offline playback capability can be enabled. Performance can be improved with the help of Web Workers and WebAssembly.
- Develop an interoperable PWA for media playback, which includes a catalogue and media player (e.g. dash.js or shaka-player)
- Research and develop download and offline playback functionality for adaptive streams
- Test in different browsers and devices
- Basic knowledge in adaptive media streaming technology
Experimentation with WebAssembly
Student project (1-3 participants)
New opportunities arise for media-related use cases such as video transcoding and manipulation in the Web browser using WebAssembly.
- Understand the specification and relationship to asm.js and Emscripten
- Research requirements and browser support for WebAssembly
- Research & develop media processing-related use cases for WebAssembly
- Basic knowledge in media streaming
Experimentation with Blockchain for Media
Student project (1-3 participants)
Blockchain seems to be a promising technology for rights management and micro payments between users without a central entity. In blockchain architecture, transactions are stored in a public, distributed ledger. When a new transaction is made, it needs to be verified by the decentralized network. This promises to bring transparency, security and efficiency to transactions.
- Understand Blockchain technology and its cryptographic concepts
- Research & develop with existing Blockchain solutions
- Basic knowledge in crypto and media streaming technology
Per-title encoding and dynamic encoding ladders
Student project (1-3 participants), Bachelor- and Master Thesis
A sophisticated video encoding chain is a key factor for the successful delivery of media content over the internet. For a long time the same encoding parameters were used for different types of content, resulting in a single static encoding ladder. However, as video content highly differs in terms of complexity and details, it is recommended to use custom encoding settings for individual media files. Depending on the characteristics of an asset the best encoding parameters are identified. As a result an optimal video quality is achieved, while not wasting bandwidth.
- Evaluate different static encoding ladders and identify potential weaknesses
- Design a stand-alone component that takes an input video and outputs a customized encoding ladder depending on the complexity of the asset
- Identify and potentially apply machine learning and AI technologies to further improve your component
- Apply video quality metrics like PSNR and VMAF to evaluate your solution
Basic understanding/Interest in media streaming
- Netflix Technology Blog - Per-Title Encode Optimization
- Netflix Technology Blog - Toward A Pratical Perceptual Video Quality Metric
Media Streaming – low latency streaming
Student Project (3-4 participants)
In live streaming there is some delays until the user get the first image of the video. Low latency streaming reduces this delay consequently and you get real time video. The aim is to show the difference between low latency live stream and normal live streaming. For that a camera recording will be streamed via MPEG-DASH and a player will be built to show that stream. The live stream will be provided to you.
- Build the player based on an already existing player (dashjs) to play the live stream
- For that change the xhr request in dahjs to fetchapi
- Compare low latency live stream and normal live stream