Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8286171

HttpClient/2 : Expect:100-Continue blocks indefinitely when response is not 100

    XMLWordPrintable

Details

    • b01
    • generic
    • generic
    • Verified

    Description

      ADDITIONAL SYSTEM INFORMATION :
      Windows 10
      JDK 17

      A DESCRIPTION OF THE PROBLEM :
      When HttpClient sends an Expect:100-Continue header via http 2 protocol. If the server responds with an 417[Expectation Code] or any other error code besides code 100. The client hangs forever. Issue does not exist when using http 1.1 protocol

      STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
      1) Create an very basic HttpServer which does not parse any bytes but instead sends http 2 frames at each stage of the request.

      2)Send an POST request with expectContinue enabled

      3)Send one request to first upgrade the client first to http 2 and then on the 2nd request send POST with expectContinue enabled. Client exits normally if response code is 100 but hangs forever if code is anything else

      EXPECTED VERSUS ACTUAL BEHAVIOR :
      EXPECTED -
      HttpClient should finish the Expect 100 request either as an normal response or should throw an exception if response code is not 100
      ACTUAL -
      Client hangs forever when Expect : 100-Continue response code is not 100

      ---------- BEGIN SOURCE ----------
      The problem does not persist with http 1.1 protocol hence we need to first upgrade the protocol on the client and server to http 2 after exchanging Connection: Upgrade & 101 Switching protocol packets and then on the 2nd request we send an PUT request with an expect 100 header enabled.

      To keep this example as simple as possible an very oversimplified version of HPack was implemented and the server does not attempt to parse any bytes of the client but from debugging experience i create stages where each stage decides what response to send to the client.

      The most basic version of HPACK

      private static final class OverSimplifiedHPack
       {
        /*
             0 1 2 3 4 5 6 7
         +---+---+---+---+---+---+---+---+
         | 0 | 0 | 0 | 0 | Index (4+) |
         +---+---+-----------------------+
         | H | Value Length (7+) |
         +---+---------------------------+
         | Value String (Length octets) |
         +-------------------------------+
       */
        private static byte[] encode_literal_non_indexed_headers(String[] index_values)throws Exception
        {
         try(ByteArrayOutputStream encoding=new ByteArrayOutputStream();
             ByteArrayOutputStream buffer=new ByteArrayOutputStream())
         {
          for(int i=0;i<=index_values.length-2;i+=2)
          {
           String value=index_values[i+1];

           byte[]
           header_index=encodePrefix(encoding,Integer.parseInt(index_values[i]),(byte)4),
           valueBytes=value.getBytes(),
           valueLength=encodePrefix(encoding,valueBytes.length,(byte)7);
          
           encoding.reset();
           
           encoding.write(header_index[0]&0b00001111);
           if(header_index.length>1){encoding.write(header_index,1,header_index.length-1);}
           
           encoding.write(valueLength[0]&0b01111111);
           if(valueLength.length>1){encoding.write(valueLength,1,valueLength.length-1);}
           encoding.write(valueBytes);
           
           buffer.write(encoding.toByteArray());
          }
          return buffer.toByteArray();
         }
        }
        
        private static byte[] encodePrefix(ByteArrayOutputStream encoding,int value,byte prefix)throws Exception
        {
         encoding.reset();
         
         int max=(int)Math.pow(2,prefix)-1;
         if(value<max){encoding.write(value);}
         else
         {
          encoding.write(max);

          value-=max;
          while(value>128)
          {
           encoding.write(((value%128)+128));

           value/=128;
          }

          encoding.write(value);
         }

         return encoding.toByteArray();
        }
       }

      The HttpServer

      The server gives some predefined responses and is designed to parse exactly 2 requests.

      1)The 1st request is an random get request. This is designed to 1st upgrade the client to Http 2

      2)The 2nd request is the actual Expect 100 header where there is one important boolean flag

             boolean doContinue=true;

      Setting this flag to true writes 100 which produces expected result on the client side. Setting to false however writes code 417 indicating expectation failed which hangs the client

      8 = index of the header :status in the static encoding table of HPACK[RFC 7541]
      28 = index of the header content-length in the static encoding table of HPACK[RFC 7541]

      public static void main2(String[] args)throws Exception
       {
        try(ServerSocket socket=new ServerSocket(5000))
        {
         try(Socket client=socket.accept())
         {
          try(InputStream input=client.getInputStream();
              OutputStream output=client.getOutputStream())
          {
           byte[] array=new byte[8196];
           
           int
           stage=1,
           requestID=0;
           /*Please change this variable for both test cases*/
           boolean doContinue=true;
           
           while(input.read(array)>0)
           {
            if(stage==1)
            {
             try(ByteArrayOutputStream serverPreface=new ByteArrayOutputStream())
             {
              serverPreface.write
              (
               (
                "HTTP/1.1 101 Switching Protocols\r\n"+
                "Connection:Upgrade\r\n"+
                "Upgrade:h2c\r\n\r\n"
               ).getBytes()
              );
              
              byte[] settings={};
              serverPreface.write
              (
               new byte[]
               {
                (byte)((settings.length>>16) & 0xFF),
                (byte)((settings.length>>8) & 0xFF),
                (byte)(settings.length & 0xFF),
                (byte)0x4,
                (byte)0x1,
                0,0,0,0
               }
              );
              serverPreface.write(settings);
              
              output.write(serverPreface.toByteArray());
              output.flush();
             }
            
             if(requestID==0){stage=3;}
            }
            else if(stage==2)
            {
            //if we are at the 2nd request skip any frames which is not an Header Frame
             if(requestID==1 && array[3]!=0x1){continue;}
            
             try(ByteArrayOutputStream frameBytes=new ByteArrayOutputStream())
             {
              byte[] headerBytes=OverSimplifiedHPack.encode_literal_non_indexed_headers
              (
               new String[]
               {
                "8",doContinue?"100":"407",
                "28","0"
               }
              );
              frameBytes.write
              (
               new byte[]
               {
                (byte)((headerBytes.length>>16) & 0xFF),
                (byte)((headerBytes.length>>8) & 0xFF),
                (byte)(headerBytes.length & 0xFF),
                (byte)0x1,
                (byte)0x4,
                0,0,0,3
               }
              );
              frameBytes.write(headerBytes);
              
              output.write(frameBytes.toByteArray());
              output.flush();
             }
            
             if(doContinue){stage=3;}
             else{stage=-1;}
            }
            else if(stage==3)
            {
             //for the 2nd request we respond only after the client has sent us all the header and
            //data frames. For any other frame types we don't respond
             if(requestID==1 && !(array[3]==0x1 || array[3]==0x0)){continue;}
         
             try(ByteArrayOutputStream frameBytes=new ByteArrayOutputStream())
             {
              byte[] content=(requestID==0?"Request 1 Done":"Test Complete").getBytes();
              
              byte[] headerBytes=OverSimplifiedHPack.encode_literal_non_indexed_headers
              (
               new String[]
               {
                "8","202",
                "28",""+content.length
               }
              );
              frameBytes.write
              (
               new byte[]
               {
                (byte)((headerBytes.length>>16) & 0xFF),
                (byte)((headerBytes.length>>8) & 0xFF),
                (byte)(headerBytes.length & 0xFF),
                (byte)0x1,
                (byte)0x4,
                0,0,0,(byte)(requestID==0?1:3)
               }
              );
              frameBytes.write(headerBytes);
              
            
              frameBytes.write
              (
               new byte[]
               {
                (byte)((content.length>>16) & 0xFF),
                (byte)((content.length>>8) & 0xFF),
                (byte)(content.length & 0xFF),
                (byte)0x0,
                (byte)0x1,
                0,0,0,(byte)(requestID==0?1:3)
               }
              );
              frameBytes.write(content);
              
              output.write(frameBytes.toByteArray());
              output.flush();
             }
             
             if(requestID==0)
             {
              requestID=1;
              stage=2;
             }
             else{stage=-1;}
            }
           }
          }
         }
        }
       }

      Output

      When doContinue=true

      Status code: 200
      Headers: {:status=[200], content-length=[14]}
      Body: Request 1 Done
      =======================
      Status code: 202
      Headers: {:status=[202], content-length=[13]}
      Body: Test Complete

      When do Continue = false

      Status code: 200
      Headers: {:status=[200], content-length=[14]}
      Body: Request 1 Done
      =======================

      /*HANGS FOREVER*/
      ---------- END SOURCE ----------

      CUSTOMER SUBMITTED WORKAROUND :
      No workaround. Issue does not exist for http 1.1. When upgrading to version 2 response 100 must be written before the 101 Switching protocol response and then both test cases work else client throws error.

      FREQUENCY : always


      Attachments

        Issue Links

          Activity

            People

              ccleary Conor Cleary
              webbuggrp Webbug Group
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: