For non ImmutableCollections.AbstractImmutableSets the current copyOf-method allocates both a HashSet and subsequently an empty array when the input collection is empty.

A check for isEmpty and returning the empty immutable Set should be just fine and avoid needless allocations.