Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-7009998

JMX synchronization during connection restart is faulty

XMLWordPrintable

    • b71
    • x86
    • linux_redhat_5.0

      FULL PRODUCT VERSION :
      java version "1.6.0_21"
      Java(TM) SE Runtime Environment (build 1.6.0_21-b06)
      Java HotSpot(TM) 64-Bit Server VM (build 17.0-b16, mixed mode)


      ADDITIONAL OS VERSION INFORMATION :
      Linux elsoxfappp102 2.6.18-194.3.1.el5 #1 SMP Sun May 2 04:17:42 EDT 2010 x86_64 x86_64 x86_64 GNU/Linux


      A DESCRIPTION OF THE PROBLEM :
      We constantly get the following error when using JMX:
      Dec 25, 2010 7:55:49 PM ClientCommunicatorAdmin restart
      WARNING: Failed to restart: java.io.IOException: Illegal state.

      I instrumented the method com.sun.jmx.remote.internal.ClientNotifForwarder.preReconnection to print out a debug message when entered and got the following:

      19:55:49.714 ClientListenerInfo [Thread-3] state = 3, beingReconnected = false
      19:55:49.722 ClientListenerInfo [RMI TCP Connection(516)-145.36.209.222] state = 3, beingReconnected = false
      19:55:49.726 ClientListenerInfo [RMI TCP Connection(518)-145.36.209.222] state = 3, beingReconnected = true

      Note that three threads enter this code in quick succession (within 12ms). The last thread enters in an illegal state.

      In reading the code there is a scenario where the synchronization does the wrong thing if 3 threads attempt to go through the code at the same time. Consider the code in com.sun.jmx.remote.internal.ClientCommunicatorAdmin.restart, the first thread will set the state to RE_CONNECTING and then leave the sychronization block, the second thread will find the state to be RE_CONNECTING and wait on the lock. When thread 1 finishes and sets the state to CONNECTED, then thread 2 can leave the synchronization block - but fails to set the state to RE_CONNECTING because that code is incorrectly in the else branch. Thus thread 2 starts the reconnecting and thread 3 wakes to find the state not RE_CONNECTING so it believes it can safely start the reconnect and it also starts reconnecting. The bad mode is discovered in the preReconnection method.

      I cannot provide a reproducible case because I don't know how to cause the connection to close, thus forcing the reconnect. But it is clear that the code will fail if 3 threads try to simultaneously execute the restart method.


      STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
      Cannot provide a way to reproduce this error.


      ERROR MESSAGES/STACK TRACES THAT OCCUR :
      Dec 25, 2010 7:55:49 PM ClientCommunicatorAdmin restart
      WARNING: Failed to restart: java.io.IOException: Illegal state.


      REPRODUCIBILITY :
      This bug can be reproduced always.

            jbachorik Jaroslav BachorĂ­k
            emcmanus Eamonn McManus
            Votes:
            1 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved:
              Imported:
              Indexed: