Television in Web Apps with Raspberry Pi, FirefoxOS & JavaScript

Student Project (at least 2 participants)

Everything is the Web. Help bring traditional television to the world of web apps.

Your tasks:

On a Raspberry Pi embedded computer running Firefox OS, you will implement a draft version of a Web API to control a television tuner. The API lists available TV channels on the tuner and allows to switch to such a TV channel. Using this API, you will write a simple television app for FirefoxOS.


Required skills:

  • Linux (must)
  • JavaScript, C (at least one)


Related technologies:


Related FAME Projects:

Contact:

Adaptive Streaming of 360° and panoramic videos

Student project (2 participants), Bachelor- and Master Thesis

MPEG-DASH enables multiple representations with varying bitrate and resolution for media streams. One part of the DASH specification deals with the support for 360 degree and panoramic videos. The goal for that specific type of videos is to selectively deliver certain parts and regions which are of interest for the viewer. A typical use case for panoramic and 360° videos is the live streaming of a sport event where the viewer can dynamically switch between different available areas in the stadium.

Open source players like dash.js and Shaka enable streaming of DASH content in the browser without the need for external plugins. However, they still lack in support and interfaces for the streaming of 360° and panoramic videos.


Your Tasks:

  • Understand the technology behind 360° and panoramic videos in MPEG-DASH
  • Research and develop use cases
  • Extend open source players 
Required skills:

  • Web Technologies: JavaScript, HTML, CSS
  • Basic Knowledge in media streaming


Related technologies:


Related FAME projects:


Contact: 

Stefan Pham

Daniel Silhavy

Extending a 360° Video Cloud Playout

Student Project

The Fraunhofer FOKUS 360° Video Cloud Streaming system (read more here) enables high quality 360° video experience on low capability devices, such as Hybrid TVs (HBBTV), or in cases of constrained network connectivity e.g. on mobile devices. VR-Glasses are a hot topic right now and we believe bringing this immersive experience to Smart-TVs allows to address a much wider audience, since TVs are much more widespread. 

This solution allows content providers and broadcasters to provide an innovative video experience to traditional TV screens. Viewers can experience video content with freely selectable views on their primary video viewing device – the large TV screen. Enriching this technology with a wide range of features addressing viewers as well as broadcasters helps acceptance and spreading of this technology. 

Your tasks:

  • Multiuser: Watch what I watch 
Currently the cloud renderer supports a single user per session. We want you to extend it by allowing other viewers to join an existing session in an observer mode and see what the other viewer sees as a live stream. 

  • Interactive 360° Video
Allowing user to interact with the video with inline links to switch videos.

  • 3D Audio:
Currently the video playout only supports stereo audio. The audio does not adjust according to the view. We want you to implement a way/codec to adjust the audio channel based on the current view perspective.

  • Ad support:
Placing ads (on the fly) inside the 360° video, sensitive to view angles.

  • Landing Page:

Develop a landing page with a grid of videos for users to select. Supported by the HbbTV Application Toolkit (HAT) and the underlying CMS Wordpress.

Required skills:

  • Web Technologies: JavaScript, PHP, HTML
  • Programming Languages: C/C++


Related Technologies:

Related FAME Projects:

Contact

Sascha Braun 

Pure JavaScript binary writer

Student Project (2 participants), Bachelor Thesis

Nowadays JavaScript is capable of amazing feats. The once web browser only language has become a much respected server language as well since the advent of Node.js. Compilers like Emscripten allow for complex native code like AV en-/decoder or even game engines to be recompiled as JavaScript-only apps.

Your tasks:

You will implement support for writing arbitrary binary formats in pure JavaScript in an already existing JavaScript library called binary-parser. You will first familiarize yourself with that library by implementing an app that reads and parses binary input of a simple existing file format. Then you will enhance the library to support writing binary and an app that is able to output the same simple binary file format.

If you choose to apply for this topic as Bachelor Thesis the ultimate goal is being able to read and write MPEG-TS in pure JavaScript.

Required skills:

  • Experience with binary operations and binary arithmetic (must)
  • JavaScript, Node.js, C (at least one)

Related technologies:

Related FAME Projects:


Contact:


Context-sensitive Adaptive Media Streaming

Student Project (1-3 participants), Bachelor- and Master Thesis

MPEG-DASH enables multiple representations with varying bitrate and resolution for media streams. One of the most crucial challenges of using MPEG-DASH is the implementation of a client enabling smooth media playback. The key component is the adaptivity algorithm of the client. The bandwidth that is available defines the maximum possible bitrate for a media stream representation to be used. An approach that is often used for reaching the best quality representation is to start with the lowest bitrate available and gradually increase the quality of the representation. This allows the player to switch to the maximum bitrate. 


At the same time network conditions need to be checked by the client. Network condition can vary everywhere the Web is accessed. The most affected devices are obviously mobile phones and tablets connected via a cellular network. The coverage of a mobile network's base station is limited as well as its amount of clients that can be served. This quickly leads to a bottleneck and bandwidth losses for each user. An advanced algorithm should be aware of its context (e.g. network type, other clients, user history, time etc.).


Tasks:

  • Research & develop context-sensitive adaptive media streaming algorithms
  • Integration with open source MPEG-DASH players (e.g. 
  • dash.jsshaka)
  • Test and compare implementation
Required Skills:

  • Web Technologies: JavaScript, HTML, CSS
  • Basic Knowledge in media streaming
Related technologies:
Related FAME projects:

Contact: 

Stefan Pham


JavaScript-based Media Stream Manipulation

Student Project (1-3 participants), Bachelor- and Master Thesis

With the introduction of HTML5 Video a cross-platform alternative to third-party plugins (e.g. Flash or Silverlight) emerged. It allows video players and applications to become interoperable. Content providers have to consider the multitude of devices available today, ranging from smart phones to smart TVs – all of these devices are used for video streaming. In order to provide the same user experience on different devices, HTML5 offers the means to develop a single application that is compatible with the Web browser on the respective devices.

Open source JavaScript-based frameworks such as codem-isoboxer and mp4box.js enable extraction of MPEG-4/ISOBMFF metadata. This allows stream analysis, processing, transformations and more advanced use cases in a Web browser environment. For example, a Web application could extract images or events from a media stream for further processing.


Your Tasks:

  • Understand MPEG-4/ISOBMFF box structure
  • Analyze JavaScript-frameworks for media stream manipulation
  • Research and develop use cases

Required Skills:

  • Web Technologies: JavaScript, HTML, CSS
  • Basic Knowledge in media formats
Related technologies:


Related FAME projects:


Contact:

Stefan Pham



Optimizing Media Streaming using Next Generation Web protocols

Student Project (1-3 participants), Bachelor- and Master Thesis

With the introduction of HTML5 Video a cross-platform alternative to third-party plugins (e.g. Flash or Silverlight) emerged. It allows video players and applications to become interoperable. Basic media streaming over HTTP 1.1 (e.g. using MPEG-DASH) is based on a client-side pull mechanism. Next Generation Web protocols such as SSEHTTP/2WebSockets or QUIC promise to reduce transmission latency by allowing server push mechanisms. Moreover network load and server resources can be better optimized by giving more control to the server.

Tasks:

  • Research next generation Web protocols
  • Investigate improvements with regard to media streaming latency
  • Test and benchmark browser implementations
Required Skills:

  • Web Technologies: JavaScript, HTML, CSS
  • Basic Knowledge in: HTTP, TCP,  Bandwidth emulation

Related technologies:


Related FAME projects:


Contact: 

Stefan Pham

Cloud Browser

Student Project (4 participants), Master Thesis

The Cloud Browser differs from traditional browsers in that web pages are rendered in the cloud and only the UI output of the web page is streamed to the client using appropriate video streaming technologies. The Client is in general a simple device that is only capable to decode and play a video stream. It doesn't have the capabilities to run a Browser, render web pages or perform computing intensive operations. It only offers capability to interact with the user e.g. using Remote Control. All user interactions need to be sent to the cloud browser and triggered on the corresponding web page. 

One potential use case of cloud browser is a Set-Top-Box that runs in the cloud. The operator provides low performance and cheap devices to costumers that connect to its cloud infrastructure. In this case, the operator doesn't need to supply new kind of devices if he wants to offer new services that requires new hardware or software capabilities. Using the Cloud Browser solution, he needs only to update the cloud browser software stack for all customers at once. Another use case is the rendering of 360° Video in the cloud instead of the local browser. The approach has two main advantages: (1) there is no need to stream the whole 360° video to the client, but only the view requested by the user and (2) there is no need to render the 360° video locally which requires expensive computing operations. 

One important requirement for Cloud Browsers is the zero-latency streaming of the current view. The time between a user interaction and the receiving of the corresponding frames that show the effect of his interaction must not exceed a threshold of few milliseconds. The WebRTC technology could be one potential technology that fulfill this requirement.

Your tasks:

  • You will implement a Cloud Browser framework that includes the following components:
    • Server that offers the Cloud Browser functionality and is capable to serve multiple users and sessions in parallel. The Server should also offer an API for receiving user inputs.
    • Client that offers a player for video stream received from the cloud browser and is capable to open new sessions on the Cloud Browser using appropriate APIs. The Client needs also to intercept user inputs and send them to the server. The Client may be implemented on low capability devices 

Required skills:

  • Web Technologies: JavaScript, HTML, CSS
  • Communication Technologies: WebRTC and/or WebSockets (Optional)

Related technologies:


Related FAME Projects:


Contact:

Louay Bassbouss

Virtual Reality and 360° Video Analytics: User Tracking and View-Field Prediction for Future Video Presentations

Student Project (1-3 participants), Master Thesis

The Fraunhofer FOKUS 360° Video Cloud Streaming solution enables high quality 360° video experience on low capability devices, such as Hybrid TVs (HBBTV), or in cases of constrained network connectivity e.g. on mobile devices. In 360° video the full spherical image of any direction of view is available in every moment while the spectator can freely change her individual perspective of view. Thus for a high quality partial view on the scene the necessary source video material becomes quite large.

This solution allows content providers and broadcasters to provide an innovative video experience to traditional TV screens. Viewers can experience video content with freely selectable views on their primary video viewing device – the large TV screen. The capability of view analytics allows content producers and advertisers to get specific information what viewers are interested in and what part of the scene they are watching, allowing detailed feedback about home user interests that wasn't previously available outside of limited lab trials.

You shall realize the following tasks:

  • View behavior analysis per user (persist the viewed camera angle in a 360° video etc.)
  • Reporting of clustered viewing behaviors on a video player with the help of heat map overlays
  • Integration into the existing FAME infrastructure
  • Live- and on-Demand-Reporting for 360° Live-Streams 
  • Creation of Cam-Tracks: e.g. average viewing hot spots or tracking shots
  • Filtering User-Groups: e.g. by IP (region), User-Agent (device) etc.
  • Prediction and recommendation of current and future camera tracks in real time

Required skills:

  • Good programming/ prototyping skills in HTML/ JS as well as in server technologies of your choice
  • Optional: High-Level Understanding of Data Mining/ User Profiling
  • Creative ideas, analytical skills and autonomous acting

Related technologies:

  • Video Delivery Technologies
  • Predictive Data Mining

Related FAME Projects:


Contact:

Christopher Krauß