Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8103341

Provide API to stream raw frames of supported codecs into JavaFX media in memory

XMLWordPrintable

    • Icon: Enhancement Enhancement
    • Resolution: Duplicate
    • Icon: P4 P4
    • None
    • 7u40
    • javafx
    • None
    • Any

      The current version of JavaFX Media only accepts "url" as a source. You cannot simply take an in-memory byte stream and feed it into JavaFX Media for playback. Contrast it with all other other JavaFX. They all take their input in multiple ways.

      For example, there is an API to take images from file as well as to programmaticaly produce them in memory. It means that I can read from my own file format or otherwise programmatically pruduce it and have JavaFX display it. JavaFX media does not currently have this kind of capability.

      *API discussion*

      The API user shall be able to stream data into JavaFX media at the low lower of raw encoded audio and video packets. So, if JavaFX media supports h264 or VP6 for video and mp3 or aac for audio, it should be possible to stream raw h264 or VP6 video frames and/or raw mp3 or aac audio frames into JavaFX media for playback.

      The ability to encode/decode/transcode frames of different formats is not needed (out of scope) and is much harder due to licensing issues.

      *Rationale*

      Rationale for this requiest if multifold. First of all, in proprietary enterprise applications of the kind we are writing, data typically travels in our custom propritary/non-standard streams. It whould be desirable to stream multimedia data in our applications inside our proprietary data streams, extract raw frames from our proprietary streams and send them to JavaFX media on the desktop for playback.

      Second rationale is that it opens door for 3rd party developers to extend JavaFX media to support more types of containers. JavaFX becomes more flexible in the kinds of content it can consume without JavaFX media team having to worry about supporting all combinations of containers and codecs which are often non-documented and/or specific to certain products.

      See, for example, RT-33271. Suppose that there is an FLV with h264 and mp3. JavaFX media supports both h264 and mp3 and plays the video and audio perfectly if I place the corresponding raw video and audio frames into mp4 container. FLV is a trivial container format to extract the raw frames from. My code to do it is about one page long, but wrapping those raw frames it into mp4 container for JavaFX media is a very non-trivial task that requires a lot of code. So, because I cannot just provide my raw frames to JavaFX media, I'm forced to do a lot of extra work:
      * Wrap my raw frames into mp4 container.
      * Start http server on desktop just for the purpuse of supplying frames to JavaFX Media.

      So, yes, because JavaFX Media can play http there is "an API to do it". I just start http server in the same JVM and pass http://localhost:port/file url to JavaFX Media. It is a realy ugly solution to a simple problem of playing a certain stream of audio and video frames. It should be much easier.

            stayer Kirill Kirichenko (Inactive)
            relizarovjfx Roman Elizarov (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved:
              Imported: