This is caused by a JDK 6 (and earlier) bug that doesn't properly close opened resources. This does occur when the reloaded jar has the exact same file path.
Users editing custom component using the SB will be impacted by this issue. Any change to resources located in the Custom component jar (e.g.:Image/FXML/CSS/Properties file/...) will be not reflected in the SB when reloading the jar files. To have these changes reflected, one needs to quit the tool and restart it.

The good news is that this is fixed in JDK 7 with the addition of a new method added to URLClassloader.