-
Enhancement
-
Resolution: Unresolved
-
P5
-
internal
This is a follow-up issue/enhancement that came up during JDK-8342690.
Currently Prism processes MediaFrame-s by splitting each plane into 4 separate 1-component textures (see ex. D3DResourceFactory.java). This was possibly done for better compatibility on old APIs and older hardware that might not support Media formats natively. With this approach any Shaders using these textures have 4 separate Texture uniforms that are sampled separately and combined.
Modern APIs like D3D12 internally support most commonly used Media texture formats. However, in order to use them we would have to rewrite Media-related JSL shaders a bit by updating how they sample Textures. For example, in D3D12 for DXGI_FORMAT_NV12 (representing 4:2:0 YCbCr-style texture) we would have to have one 1-component Texture uniform for Y component (luminance) and one 2-component Texture uniform for CbCr components (chrominance). For more formats and reference see: https://learn.microsoft.com/en-us/windows/win32/api/dxgiformat/ne-dxgiformat-dxgi_format
This would have potential benefits in easier Texture updates and maybe slightly improved Texture memory allocations (one instead of four). However, this also might break old API compatibility, so check if that is even viable, or shelve this task until old rendering APIs are deprecated/removed. Also it is worth checking how this would behave on hardware that potentially does not support Media formats.
Currently Prism processes MediaFrame-s by splitting each plane into 4 separate 1-component textures (see ex. D3DResourceFactory.java). This was possibly done for better compatibility on old APIs and older hardware that might not support Media formats natively. With this approach any Shaders using these textures have 4 separate Texture uniforms that are sampled separately and combined.
Modern APIs like D3D12 internally support most commonly used Media texture formats. However, in order to use them we would have to rewrite Media-related JSL shaders a bit by updating how they sample Textures. For example, in D3D12 for DXGI_FORMAT_NV12 (representing 4:2:0 YCbCr-style texture) we would have to have one 1-component Texture uniform for Y component (luminance) and one 2-component Texture uniform for CbCr components (chrominance). For more formats and reference see: https://learn.microsoft.com/en-us/windows/win32/api/dxgiformat/ne-dxgiformat-dxgi_format
This would have potential benefits in easier Texture updates and maybe slightly improved Texture memory allocations (one instead of four). However, this also might break old API compatibility, so check if that is even viable, or shelve this task until old rendering APIs are deprecated/removed. Also it is worth checking how this would behave on hardware that potentially does not support Media formats.