At Adoptium we build the tarballs and then rename the result to include the version: https://github.com/adoptium/jmc-build/blob/main/.github/workflows/build.yml#L170

For example:
Rename org.openjdk.jmc-linux.gtk.x86_64.tar.gz to org.openjdk.jmc-9.0.0-linux.gtk.x86_64.tar.gz

Can it be fixed in JMC upstream to produce tarballs with the version as part of the name so our rename step is not needed?

Distributing the upstream tarball as is would be annoying for users; after downloading you do not know what version the tarball is without further examination and the names could collide.