Test Methodology

We will test jpackage with a mix of automated jtreg tests, partially automated tests and manual tests. Some of the manual tests could later become automated tests (either jtreg tests or SQE tests). Partially automated tests are jtreg based tests which will require user interaction to install and uninstall generated installers. Installer generation, install and uninstall verification is done automatically by running jtreg tests with special arguments.

Test Inventory (optional)

Automated tests

JUnit tests

• AppImageFileTest
• ApplicationLayoutTest
• CompareDottedVersionTest
• DeployParamsTest
• DottedVersionTest
• InvalidDottedVersionTest
• OverridableResourceTest
• PathGroupTest
• PlatformVersionTest
• ToolValidatorTest

CLI and miscellaneous tests

• AppVersionTest
• BasicTest
• ErrorTest
• JavaOptionsEqualsTest
• JavaOptionsTest
• MainClassTest
• ModulePathTest
• ModulePathTest2
• ModulePathTest3
• MultipleJarAppTest
• NonExistentTest
• UnicodeArgsTest

Create App Image

• ArgumentsTest
• EmptyFolderTest

Create package tests (partially automated)

• AddLauncherTest
• AppImagePackageTest
• EmptyFolderPackageTest
• FileAssociationsTest
• IconTest
• InstallDirTest
• LicenseTest
• MultiLauncherTwoPhaseTest
• RuntimePackageTest
• SimplePackageTest

Windows specific

Create App Image

• WinConsoleTest

Create package tests (automated)

• WinScriptTest

Create package tests (partially automated)

• WinDirChooserTest
• WinL10nTest
• WinMenuGroupTest
• WinMenuTest
• WinPerUserInstallTest
• WinResourceTest
• WinShortcutTest
• WinUpgradeUUIDTest

MacOS specific

Create App Image

• MacFileAssociationsTest
• MacPropertiesTest
• SigningAppImageTest

Create package tests (partially automated)

• NameWithSpaceTest
• SigningPackageTest

Linux specific

Create package tests (automated)

• LinuxResourceTest

Create package tests (partially automated)

• UsrTreeTest
• AppCategoryTest
• LicenseTypeTest
• LinuxBundleNameTest
• MaintainerTest
• PackageDepsTest
• ReleaseTest
• ShortcutHintTest

Test Configurations (optional)

Windows

Wix, a third-party tool, is required to run MSI and EXE installer tests. For EXE we using Wix to generate MSI installer and then embedding it to EXE.

MacOS

MacOS should have required tools by default.

Linux

Might require additional packages to build RPM or DEB.

SQE Manual Tests:

JPKG001- Windows Testing

• JPKG001-001: AddLauncherTest
• JPKG001-002: FileAssociationsTest
• JPKG001-003: InstallDirTest
• JPKG001-004: LicenseTest
• JPKG001-005: RuntimePackageTest
• JPKG001-006: SimplePackageTest
• JPKG001-007: WinDirChooserTest
• JPKG001-008: WinMenuGroupTest
• JPKG001-009: WinMenuTest
• JPKG001-010: WinPerUserInstallTest
• JPKG001-011: WinShortcutTest
• JPKG001-012: WinUpgradeUUIDTest
• JPKG001-013: Reinstall Test Same
• JPKG001-014: Reinstall Test Different
• JPKG001-015: Run JPackage Bundle on x86 machine

JPKG002- Mac Testing

• JPKG002-001: AddLauncherTest
• JPKG002-002: FileAssociationsTest
• JPKG002-003: InstallDirTest
• JPKG002-004: LicenseTest
• JPKG002-005: SimplePackageTest
• JPKG002-006: SigningPackageTest
• JPKG002-007: SigningPackageTest-InvalidCert
• JPKG002-008: NameWithSpaceTest
• JPKG002-009: Reinstall Test

JPKG003- Linux Testing - RPM

• JPKG003-001: AppCategoryTest
• JPKG003-002: ReleaseTest
• JPKG003-003: BundleNameTest
• JPKG003-004: LicenseTypeTest
• JPKG003-005: PackageDepsTest
• JPKG003-006: FileAssociationsTest
• JPKG003-007: InstallDirTest
• JPKG003-008: LicenseTest
• JPKG003-009: RuntimePackageTest
• JPKG003-010: SimplePackageTest
• JPKG003-011: BasicShortcutHintTest
• JPKG003-012: AddLauncherTest

JPKG004- Linux Testing - Debian

• JPKG004-001: AppCategoryTest
• JPKG004-002: ReleaseTest
• JPKG004-003: BundleNameTest
• JPKG004-004: MaintainerTest
• JPKG004-005: PackageDepsTest
• JPKG004-006: FileAssociationsTest
• JPKG004-007: InstallDirTest
• JPKG004-008: LicenseTest
• JPKG004-009: RuntimePackageTest
• JPKG004-010: SimplePackageTest
• JPKG004-011: BasicShortcutHintTest
• JPKG004-012: AddLauncherTest