This issue is reproducible on OS X with command below:

$JDK_HOME/bin/jpackage create-app-image \
--output output \
--module 'me.mymodule' \
--module-path '../input-mods/mods' \

In output ".app" folder will be created (empty application name) and command will fail with error below:
java.nio.file.FileAlreadyExistsException: output/.app/Contents/MacOS