Summary

Support additional HTTP Live Streaming streams characteristics described by the EXT-X-MEDIA tag.

Problem

HTTP Live Streaming protocol is features rich protocol and implementing this protocol 100% is challenging and many features are not widely used. Thus JavaFX Media does not implements this protocol 100% and we adding additional functionality as needed. Before JDK-8282999 we supported media streams with audio and video data multiplexed as one stream. HLS protocol has ability to provide audio and video data is two separate streams via EXT-X-MEDIA and it is mostly used to save space by providing only one audio stream for multiple video streams with different resolutions and bitrates. Now days such streams becomes more popular and without support of EXT-X-MEDIA tag JavaFX Media will only play video without audio for such streams.

Solution

Add support for EXT-X-MEDIA tag for audio only streams to JavaFX Media. We will only select matching audio stream to video streams if it is DEFAULT or AUTOSELECT stream. Bitrate switching will be supported as well and we will switch audio stream if video stream switched due to bitrate change. We will not support audio streams via EXT-X-MEDIA which not marked as DEFAULT or AUTOSELECT, since based on specification we need to provide APIs to user to make selection of such streams and JavaFX Media does not have such APIs. All unsupported streams provided via EXT-X-MEDIA will be dropped without errors and if non exist MediaPlayer will attempt to play main video only stream if supported.

Specification

javafx/scene/media/package-summary.html

JavaDoc patch: https://bugs.openjdk.org/secure/attachment/108798/package-info.java.diff.txt

Following will be added to "HTTP Live Streaming (HLS)" section as new characteristic:

 - Additional audio renditions via #EXT-X-MEDIA tag:
  - MP2T streams with one H.264/AVC video track and elementary AAC audio stream via #EXT-X-MEDIA tag.
  - fMP4 streams with one H.264/AVC or H.265/HEVC video track and elementary AAC audio stream via #EXT-X-MEDIA tag.
  - fMP4 streams with one H.264/AVC or H.265/HEVC video track and fMP4 streams with one AAC audio track via #EXT-X-MEDIA tag.