ADDITIONAL SYSTEM INFORMATION :
http://java.oracle.com/' Version '1.8.0_161'
MacOS High Sierra 10.13.4

A DESCRIPTION OF THE PROBLEM :
I have noticed that if HttpURLConnection received a response with the following headers:
Transfer-Encoding: chunked, gzip
Content-Encoding: gzip

It fails to unchunk the request. If the Transfer-Encoding header is:
Transfer-Encoding: chunked
Content-Encoding: gzip

It does.

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
On the Mac I installed netcat using homebrew to allow me to create a test server to send back responses with the different header configurations to my test program.

If you run the test program supplied and the run the following on the command line, the test program will generate a file on your desktop with the contents of the response. You may have to ctrl-c to terminate the netcat command.

The first uses Transfer-Encoding: chunked and the second Transfer-Encoding: chunked, gzip

{ printf 'HTTP/1.0 200 OK\r\nTransfer-Encoding: chunked\r\nContent-Encoding: gzip\r\nContent-Type: application/xml\r\n\r\n'; cat ~/Desktop/sitemap-gzip.xml; } | nc -l -p 12346

{ printf 'HTTP/1.0 200 OK\r\nTransfer-Encoding: chunked, gzip\r\nContent-Encoding: gzip\r\nContent-Type: application/xml\r\n\r\n'; cat ~/Desktop/sitemap-gzip.xml; } | nc -l -p 12346




EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
If you look at the contents of the file in a hex editor:

I would expect to see an unchunked gzip file when running the 2nd netcat command which uses Transfer-Encoding: chunked, gzip.

Instead we can see it remains chunked.
ACTUAL -
If you look at the contents of the file in a hex editor:

You can see that the file generated by the first netcat command is an unchunked gzip file. You can tell because the first 2 bytes have the GZIPInputStream.GZIP_MAGIC = 0x8b1f.

However, In the second netcat example where we have Transfer-Encoding: chunked, gzip, the content of the file has failed to be unchunked and you can see it is chunked by looking for the /r/n(0xOD, OXA) pairs.

---------- BEGIN SOURCE ----------
import java.io.File;
import java.io.IOException;
import java.net.HttpURLConnection;
import java.net.URL;

import uk.co.screamingfrog.seospider.data.ResponseCodes;
import uk.co.screamingfrog.utils.io.SFFileUtils;

public class TransferEncodingChunkedGzipTest
{
    private final static String CONTENT_PATH = "/Users/USER_NAME/Desktop/content";
    
    public static void main(
        String[] args) throws IOException
    {
        testMe();
    }
    
    public static void testMe() throws IOException
    {
        final URL url = new URL("http://127.0.0.1:12346");
        final HttpURLConnection httpConn = (HttpURLConnection) url.openConnection();
        
        int responseCode = ResponseCodes.getResponseCode(httpConn.getResponseCode());
        if (responseCode == ResponseCodes.SUCCESS)
        {
            SFFileUtils.copyInputStreamToFile(
                    httpConn.getInputStream(),
                    new File(CONTENT_PATH));
            
            System.out.println("finished");
        }
        else
        {
            System.out.println("Connection failed");
        }
    }
}

---------- END SOURCE ----------

FREQUENCY : always