-
Bug
-
Resolution: Duplicate
-
P3
-
None
-
1.4.2_05
-
generic
-
generic
We have a problem when we are accessing a huge file (~6GB) using NIO and
MappedByteBuffers. Since file is huge, I tried to map only portions of
the file - no more than 16 portions 1MB each. And eventually I got an
IO exception "Can not allocate memory". I was not able to find any leaks
or other problems in my program. It works fine with relatively small
files. Then I searched news groups and found posting about similar
issue. So, I ran a simple test program on the same file and reproduced
the problem - once I set the portion size to 1MB, I am getting an exception.
Below is an output from successfull run (when portion is 100KB) and failing run (with 1MB). You can also see the version of JVM I am running. I am able to reproduce this on Linux and on Windows 2000.
polly:/tmp 3> java -version
java version "1.4.2_03"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2_03-b02)
Java HotSpot(TM) Client VM (build 1.4.2_03-b02, mixed mode)
polly:/tmp 4> java -cp . Test $RDB/adt040803.dat 10000 100000
Mapping /usr/local/instinet/RTS/var/db/2004_08_03/adt040803.dat
size of file: 6337593344
size of page: 100000
iterations : 10000
polly:/tmp 5> java -cp . Test $RDB/adt040803.dat 10000 1000000 | less 2>&1
Mapping /usr/local/instinet/RTS/var/db/2004_08_03/adt040803.dat
size of file: 6337593344
size of page: 1000000
iterations : 10000
Exception occurred at count 1927 trying to map from 3557000000 to 3558000000
(1927 faults so far)
Exception in thread "main" java.io.IOException: Cannot allocate memory
at sun.nio.ch.FileChannelImpl.map0(Native Method)
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:705)
at Test.main(Test.java:53)
polly:/tmp 6>
MappedByteBuffers. Since file is huge, I tried to map only portions of
the file - no more than 16 portions 1MB each. And eventually I got an
IO exception "Can not allocate memory". I was not able to find any leaks
or other problems in my program. It works fine with relatively small
files. Then I searched news groups and found posting about similar
issue. So, I ran a simple test program on the same file and reproduced
the problem - once I set the portion size to 1MB, I am getting an exception.
Below is an output from successfull run (when portion is 100KB) and failing run (with 1MB). You can also see the version of JVM I am running. I am able to reproduce this on Linux and on Windows 2000.
polly:/tmp 3> java -version
java version "1.4.2_03"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2_03-b02)
Java HotSpot(TM) Client VM (build 1.4.2_03-b02, mixed mode)
polly:/tmp 4> java -cp . Test $RDB/adt040803.dat 10000 100000
Mapping /usr/local/instinet/RTS/var/db/2004_08_03/adt040803.dat
size of file: 6337593344
size of page: 100000
iterations : 10000
polly:/tmp 5> java -cp . Test $RDB/adt040803.dat 10000 1000000 | less 2>&1
Mapping /usr/local/instinet/RTS/var/db/2004_08_03/adt040803.dat
size of file: 6337593344
size of page: 1000000
iterations : 10000
Exception occurred at count 1927 trying to map from 3557000000 to 3558000000
(1927 faults so far)
Exception in thread "main" java.io.IOException: Cannot allocate memory
at sun.nio.ch.FileChannelImpl.map0(Native Method)
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:705)
at Test.main(Test.java:53)
polly:/tmp 6>
- duplicates
-
JDK-6417205 (fc) Out of address space because mapped ByteBuffers are not freed
- Resolved
- relates to
-
JDK-4797189 Finalizers not called promptly enough
- Closed
-
JDK-5072161 OutOfMemoryError while using GZIPOutputStream
- Closed
-
JDK-6751792 OutOfMemory error from native leak in PNGEnoder class?
- Closed