The OS X specific command line parameter -Xdock:name=Title doesn't fully work in OS X 10.9, unlike in earlier releases. It does set the name in the menu bar, but not the name of the dock tile, as observed in the dock or in the Cmd-Tab app switcher. Instead, the dock name for unbundled applications is the name of the executable, typically "java".

This behavior is consistent among all JDKs I've tested, including the latest Apple JDK (1.6.0_65-b14-462). This suggest that this is a limitation of the JavaRuntimeSupport.framework in OS X 10.9. Note that this name cannot be changed once a dock entry has been created, so it is possible that something earlier in the startup triggers the creation of a dock entry, freezing the name. However, based on some experimentation, and the lack of documentation regarding the framework, this seems to be a VFR.