The following methods
http://download.oracle.com/javase/7/docs/api/javax/swing/AbstractButton.html#checkHorizontalKey%28int,%20java.lang.String%29
http://download.oracle.com/javase/7/docs/api/javax/swing/AbstractButton.html#checkVerticalKey%28int,%20java.lang.String%29

do not mention what do they return in case if the check was successful and no IllegalArgumentException was thrown.

They say just:

protected int checkVerticalKey(int key,
                   String exception)

Verify that the key argument is a legal value for the vertical properties. Valid values are:
...
Parameters:
    key - the property value to check
Throws:
    IllegalArgumentException - if key is not one of the legal values listed above