Cloud-based 360° Video Playout
The Cloud-based 360° Video Playout allows the viewing of high quality 360° videos on devices with constrained capabilities such as TV sets. Usually the delivery of such video material consumes a large bitrate and results in a considerable processing load to perform the geometrical view projection and rendering. For example, a bitrate of 200 – 300 Mbit/s is required to stream a 24K 360° video and it takes few seconds to render a single 4K FOV frame from the 24K equirectangular frames – which makes a smooth playback impossible even on modern devices with high processing and graphical computing capabilities. The solution addresses both aspects and reduces the required bitrate and processing resources by rendering the field of view in the cloud in advance and by streaming only the selected field of view to the client. By doing so, many features which are applicable to classical videos can be also applied to the FOV videos for example Field of View Adaptive Streaming using DASH and MSE, Efficient Delivery over existing CDNs and DRM Support over EME. The solution also supports synchronization across multiple TVs and other devices (even mixed) so that a shared as well as an extended experience is possible.
To stay updated on the latest developments of Cloud-based 360° Video Playout and news of Fraunhofer FOKUS FAME, please subscribe to our newsletter.
FIFA World Cup Final in 360°
The Cloud-based 360° Video Playout allows content providers and broadcasters to provide an innovative video experience on traditional TV screens. Viewers can experience video content with freely selectable views on their primary video viewing device – the large TV screen. The capability of view analytics allows content producers and advertisers to get specific information about what viewers are interested in and which part of the scene they are watching, allowing detailed feedback about viewers' interests. The viewer can control the experience of the video playback and can interact with dynamic content overlays (e.g. for interactive advertisement).
The Cloud-based 360° Video Playout for HbbTV consists of the following components:
- 360° Video Pre-processing:
The Cloud Rendering Server operates on the source 360° video and calculates the different FOVs depending on the configuration that is also passed as input. A FOV is defined using the four parameters (φ, θ, Ah, Av)
- Storage and Streaming:
The Cloud Streaming Server offers a REST API to control the individual view and playback state for each client. The output FOVs and manifest of pre-processing step need to be available on a streaming server that is accessible from the CDN provider
- 360° Video Player Client:
The Cloud Streaming solution enables Smart TVs and other streaming devices to provide 360° Video experience through usual video playback of the output from the cloud streaming server. Users can control their individual view and playback state via remote control. The player constructs the final video stream to display from the FOV videos. There is no need to process the received video content before playback. The client platform needs only to provide an API that allows applications to send video segments to the video decoder.
- 360° Video Control Mobile App:
The mobile control app acts as a smart remote control for the 360° Video TV Player. It offers an easy-to-use UI to control the video playback and the angle of current view using touch, gestures or device orientation.
As 360° Video is getting more and more popular and commercially relevant, the experience should be possible on all kinds of screens, including the TV screen. However, 360° video material in high (24K and more to achive a UHD field of view) resolution requires very high bitrate (> 200 – 300 Mbit/s) and high processing load for the view projection and rendering. The 360° Video Playout allows the experience of 360° videos on TV screens as well as on any other low capability devices by performing the rendering of the individual field of view at the server-side in a cloud based infrastructure. Only the selected high-resolution view is then streamed to the playback device, which reduces the bitrate dramatically. The viewer can interact with dynamically inserted interactive objects which enrich the video experience further. The solution for example also allows advertisers to dynamically include personalized advertisements (ad-insertion).
Dog Sled Interactive Demo
Caminandes 24K Interactive Demo
360° live picture-in-picture video (PiP)
Features and Assets
- Support of 24K 360° videos with a 4K Field of View
- Processing and delivering of live 360° video input in real time
- Cross-Platform Support: HbbTV, Smart TV, Android TV, Chromecast, Apple TV, Desktop and Mobile Browsers and many others
- Navigable while just using a standard remote control
- Cloud-based video transforming engine
- Efficient bandwidth usage while delivering best possible quality due to the preprocessing approach
- DASH support allows adjusting the playback quality to the bitrate, especially important for large UHD 360° videos
- 360° video playback can be synchronised over multiple TVs and companion devices
- Support for interaction in 360° videos to allow users to control the experience (video playback but also to activate additional content dynamically)
- Picture-in-Picture (PiP) support for showing additional content within a 360° experience and switching between videos in an immersive library style
- PiP allows dynamic ad insertion as well as combining multiple live streams into one experience
- Support for following predefined viewing paths and switching between them
- Editorial Tool to enrich 360° videos with interaction
- Analytics framework to determine popular views, hot spots, regions of user interest
This 360° video was recorded at the 5th FOKUS Media Web Symposium. Please contact us if you like to see such experience on HbbTV enabled TVs. Click on the video to navigate by mouse.
Features and Assets:
- Experience highly synchronized 360° videos shared on multiple TVs and companion devices
- Cross-Platform Support: HbbTV, Smart TV, Android TV, Chromecast, Apple TV, Desktop and Mobile Browsers and many others.
- DASH support allows to adjust the playback quality to your bandwidth, especially important for large UHD 360° videos.
- Support of 24K 360° videos with a 4K Field of View.
- Add interaction to 360° videos and allow users to control the experience. Try our Editorial Tool and easily enrich your videos with interaction.
- Unlike other 360° video playouts our system saves bandwidth while delivering best possible quality due to the preprocessing approach.