"Not enough space" IOException occur while invoke "runtime.exec(cmds)"

The frameowrk of Solaris Certification Test Suite "HCTS" prompt the "Not enough space" IOException while executing the "runtime.exec(cmds)". And it always fail if the first invocation of "runtime.exec(cmds)" fail. Seems the resource can not be released properly.
The memeory is 1G and swap is 2G. The solaris version is "snv64a". It can be reproduced every time on the machine.
The issue occur on Solaris snv64a and snv70b, jre version of snv64a is 1.5.0_11.
The 'cmds' is a shell script that are used to clean the test environment. Actually, the exception always occur after the first IOException been thrown. The "cmds" of any command can due to this exception. This error occur on several machines, after adding the swap space, this IOException can disappear.

This issue does not occur on Solaris 10, S10U1, S10U2 and S10U3 on the same machine.