jmap is not able to attach to a process to dump its heap:

Solaris SPARC (Niagara):

[2006-11-25T08:46:15.24] Executing jmap -d64 -dump:format=b,file=heap.bin 17433
[2006-11-25T08:46:15.24] 17433: Unable to open door: target process not responding or HotSpot VM not loaded
[2006-11-25T08:46:15.24] The -F option can be used when the target process is not responding

Solaris AMD64:

[2006-11-26T14:40:23.84] Executing jmap -d64 -dump:format=b,file=heap.bin 18393
[2006-11-26T14:40:23.84] Dumping heap to heap.bin ...
[2006-11-26T14:40:23.84] Exception in thread "main" java.io.IOException: Unable to enqueue command to target VM: Internal error
[2006-11-26T14:40:23.84] at sun.tools.attach.SolarisVirtualMachine.enqueue(Native Method)
[2006-11-26T14:40:23.84] at sun.tools.attach.SolarisVirtualMachine.execute(SolarisVirtualMachine.java:107)
[2006-11-26T14:40:23.84] at sun.tools.attach.HotSpotVirtualMachine.executeCommand(HotSpotVirtualMachine.java:195)
[2006-11-26T14:40:23.84] at sun.tools.attach.HotSpotVirtualMachine.dumpHeap(HotSpotVirtualMachine.java:162)
[2006-11-26T14:40:23.84] at sun.tools.jmap.JMap.dump(JMap.java:223)
[2006-11-26T14:40:23.84] at sun.tools.jmap.JMap.main(JMap.java:122)

3. Linux AMD64:

[2006-11-26T06:53:10.92] Executing jmap -dump:format=b,file=heap.bin 24451
[2006-11-26T06:53:10.92] 2006-11-25 22:53:10
[2006-11-26T06:53:10.92] Full thread dump Java HotSpot(TM) 64-Bit Server VM (20061122112410.ik199011.rt_merge mixed mode):
[2006-11-26T06:53:10.92]
[2006-11-26T06:53:10.92] "process reaper" daemon prio=10 tid=0x00000000401f6c00 nid=0x5fa1 runnable [0x0000002aa512a000..0x0000002aa512a550]
[2006-11-26T06:53:10.92] java.lang.Thread.State: RUNNABLE
[2006-11-26T06:53:10.92] at java.lang.UNIXProcess.waitForProcessExit(Native Method)
[2006-11-26T06:53:10.92] at java.lang.UNIXProcess.access$900(UNIXProcess.java:20)
[2006-11-26T06:53:10.92] at java.lang.UNIXProcess$1$1.run(UNIXProcess.java:132)
[2006-11-26T06:53:10.92]
[2006-11-26T06:53:10.92] "Low Memory Detector" daemon prio=10 tid=0x00000000401e4c00 nid=0x5f8b runnable [0x0000000000000000..0x0000000000000000]
[2006-11-26T06:53:10.92] java.lang.Thread.State: RUNNABLE
[2006-11-26T06:53:10.92]
[2006-11-26T06:53:10.92] "CompilerThread1" daemon prio=10 tid=0x00000000401e2c00 nid=0x5f8a waiting on condition [0x0000000000000000..0x0000000000000000]
[2006-11-26T06:53:10.92] java.lang.Thread.State: RUNNABLE
[2006-11-26T06:53:10.92]
[2006-11-26T06:53:10.92] "CompilerThread0" daemon prio=10 tid=0x00000000401e1400 nid=0x5f89 waiting on condition [0x0000000000000000..0x0000000000000000]
[2006-11-26T06:53:10.92] java.lang.Thread.State: RUNNABLE
[2006-11-26T06:53:10.92]
[2006-11-26T06:53:10.92] "Signal Dispatcher" daemon prio=10 tid=0x00000000401dfc00 nid=0x5f88 waiting on condition [0x0000000000000000..0x0000000000000000]
[2006-11-26T06:53:10.92] java.lang.Thread.State: RUNNABLE
[2006-11-26T06:53:10.92]
[2006-11-26T06:53:10.92] "Finalizer" daemon prio=10 tid=0x00000000401bd400 nid=0x5f86 in Object.wait() [0x0000002aa4a22000..0x0000002aa4a22450]
[2006-11-26T06:53:10.92] java.lang.Thread.State: WAITING (on object monitor)
[2006-11-26T06:53:10.92] at java.lang.Object.wait(Native Method)
[2006-11-26T06:53:10.92] - waiting on <0x0000002a9ba70aa0> (a java.lang.ref.ReferenceQueue$Lock)
[2006-11-26T06:53:10.92] at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
[2006-11-26T06:53:10.92] - locked <0x0000002a9ba70aa0> (a java.lang.ref.ReferenceQueue$Lock)
[2006-11-26T06:53:10.92] at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
[2006-11-26T06:53:10.92] at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
[2006-11-26T06:53:10.92]
[2006-11-26T06:53:10.92] "Reference Handler" daemon prio=10 tid=0x00000000401bc800 nid=0x5f85 in Object.wait() [0x0000002aa4922000..0x0000002aa49225d0]
[2006-11-26T06:53:10.92] java.lang.Thread.State: WAITING (on object monitor)
[2006-11-26T06:53:10.92] at java.lang.Object.wait(Native Method)
[2006-11-26T06:53:10.92] - waiting on <0x0000002a9ba70b80> (a java.lang.ref.Reference$Lock)
[2006-11-26T06:53:10.92] at java.lang.Object.wait(Object.java:485)
[2006-11-26T06:53:10.92] at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
[2006-11-26T06:53:10.92] - locked <0x0000002a9ba70b80> (a java.lang.ref.Reference$Lock)
[2006-11-26T06:53:10.92]
[2006-11-26T06:53:10.92] "main" prio=10 tid=0x0000000040112000 nid=0x5f83 waiting on condition [0x0000002a9674d000..0x0000002a9674d710]
[2006-11-26T06:53:10.92] java.lang.Thread.State: TIMED_WAITING (sleeping)
[2006-11-26T06:53:10.92] at java.lang.Thread.sleep(Native Method)
[2006-11-26T06:53:10.92] at heapdump.share.EatMemory.sleepSome(EatMemory.java:155)
[2006-11-26T06:53:10.92] at heapdump.share.EatMemory.execute(EatMemory.java:167)
[2006-11-26T06:53:10.92] at heapdump.share.EatMemory.run(EatMemory.java:189)
[2006-11-26T06:53:10.92] at heapdump.share.EatMemory.main(EatMemory.java:236)
[2006-11-26T06:53:10.92]
[2006-11-26T06:53:10.92] "VM Thread" prio=10 tid=0x00000000401b7c00 nid=0x5f84 runnable
[2006-11-26T06:53:10.92]
[2006-11-26T06:53:10.92] "VM Periodic Task Thread" prio=10 tid=0x00000000401e6c00 nid=0x5f8c waiting on condition
[2006-11-26T06:53:10.92]
[2006-11-26T06:53:10.92] JNI global references: 618
[2006-11-26T06:53:10.92]
[2006-11-26T06:53:10.92] Heap
[2006-11-26T06:53:10.92] def new generation total 25856K, used 25821K [0x0000002a99cd0000, 0x0000002a9b8d0000, 0x0000002a9b8d0000)
[2006-11-26T06:53:10.92] eden space 23040K, 100% used [0x0000002a99cd0000, 0x0000002a9b350000, 0x0000002a9b350000)
[2006-11-26T06:53:16.49] from space 2816K, 98% used [0x0000002a9b350000, 0x0000002a9b6075a8, 0x0000002a9b610000)
[2006-11-26T06:53:16.49] to space 2816K, 0% used [0x0000002a9b610000, 0x0000002a9b610000, 0x0000002a9b8d0000)
[2006-11-26T06:53:16.49] tenured generation total 57344K, used 57290K [0x0000002a9b8d0000, 0x0000002a9f0d0000, 0x0000002a9f0d0000)
[2006-11-26T06:53:16.49] the space 57344K, 99% used [0x0000002a9b8d0000, 0x0000002a9f0c2a00, 0x0000002a9f0c2a00, 0x0000002a9f0d0000)
[2006-11-26T06:53:16.49] compacting perm gen total 21248K, used 2605K [0x0000002a9f0d0000, 0x0000002aa0590000, 0x0000002aa44d0000)
[2006-11-26T06:53:16.49] the space 21248K, 12% used [0x0000002a9f0d0000, 0x0000002a9f35b768, 0x0000002a9f35b800, 0x0000002aa0590000)
[2006-11-26T06:53:16.49] No shared spaces configured.
[2006-11-26T06:53:16.49]
[2006-11-26T06:53:16.49] 24451: Unable to open socket file: target process not responding or HotSpot VM not loaded
[2006-11-26T06:53:16.49] The -F option can be used when the target process is not responding
[2006-11-26T06:53:16.49] Java exited with exit status 1
The "java.io.IOException: Bad file number" failure mode has
also been seen in the following MM_REGRESSION test:

    sun/management/jmxremote/bootstrap/LocalManagementTest.sh

according to Pavel. The failure mode is intermittent.
No, it's not "Bad file number" it's
Exception in thread "main" java.io.IOException: Permission denied
       at sun.tools.attach.SolarisVirtualMachine.open(Native Method)
But there is plenty of "Bad file number" in our nightly jstack test results
Sorry for the confusion