FULL PRODUCT VERSION :
java version "1.5.0_06"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_06-b05)
Java HotSpot(TM) Client VM (build 1.5.0_06-b05, mixed mode, sharing)
ADDITIONAL OS VERSION INFORMATION :
Microsoft Windows XP [Version 5.1.2600]
EXTRA RELEVANT SYSTEM CONFIGURATION :
Intel 3.06GHz P4 --- hyperthreaded
A DESCRIPTION OF THE PROBLEM :
If a FileChannel.write(ByteBuffer b, long position) or FileChannel.read(ByteBuffer b, long position) takes place at the same time as a RandomAccessFile.setLength call on another thread where the channel is the one associated with the RandomAccessFile, then the read or write may take place at an unexpected position or the setLength may result in an unexpected file length.
These read and write methods are supposed to be independent of methods which affect the file position (and setLength is not declared as such anyway although that is probably how it is implemented).
STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Just run the attached code. As with most multithreading problems the result is somewhat variable but on my machine it does hit one of the failure points every time (which one varies).
Changing the code to use a common lock for read/write and setLength operations prevents the problem occuring.
EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
A successful run should just print "Test completed"
ACTUAL -
Invalid data in file: 0,0 != 3,15
Test completed
REPRODUCIBILITY :
This bug can be reproduced often.
---------- BEGIN SOURCE ----------
import java.io.RandomAccessFile;
import java.io.File;
import java.io.IOException;
import java.nio.channels.FileChannel;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.util.Random;
/** Test concurent read, write and length changes.
* It appears that RandomAccessFile.setLength interacts with FileChannel.write
* @author mThornton
* @since 22-Dec-2005 11:23:52
*/
public class TestConcurrency implements Runnable
{
private static final int PAGE_SIZE = 1024;
private int group=8;
private File file;
private RandomAccessFile raf;
private FileChannel channel;
private int size;
private int maximumSize;
private volatile boolean complete;
private Object lock = new Object();
private Object extendLock = new Object();
// private Object extendLock = lock; //new Object();
public static void main(String[] args)
{
TestConcurrency test = new TestConcurrency();
try
{
test.init(10, 1000);
new Thread(test).start();
test.modifyFileContent();
}
catch (IOException e)
{
e.printStackTrace();
}
test.cleanup();
System.out.println("Test completed");
}
void init(int initialSize, int maxSize) throws IOException
{
file = File.createTempFile("testConcurrency", "tmp");
raf = new RandomAccessFile(file, "rw");
channel = raf.getChannel();
size = initialSize;
raf.setLength(size*(long)PAGE_SIZE);
maximumSize = maxSize;
}
void cleanup()
{
if (raf != null)
{
try
{
raf.close();
}
catch (IOException e)
{
}
}
if (channel != null)
{
try
{
channel.close();
}
catch (IOException e)
{
}
}
if (file != null)
{
if (!file.delete())
System.err.println("Unable to delete temporary file: "+file);
}
}
private synchronized int size()
{
return size;
}
private void extendFile()
{
try
{
while (size < maximumSize)
{
synchronized (this)
{
wait();
}
if (complete)
break;
int n = size + 20;
synchronized (extendLock)
{
raf.setLength(n*(long)PAGE_SIZE);
}
synchronized (this)
{
size = n;
}
}
}
catch (IOException e)
{
e.printStackTrace();
}
catch (InterruptedException e)
{
}
complete = true;
}
private synchronized void trigger()
{
notify();
}
private void modifyFileContent()
{
ByteBuffer buffer = ByteBuffer.allocateDirect(PAGE_SIZE);
buffer.order(ByteOrder.nativeOrder());
Random rand = new Random();
int sequence = 0;
try
{
test:
while (!complete)
{
sequence++;
int limit = size();
int index = rand.nextInt(limit-group);
// first write data
long position = index*(long)PAGE_SIZE;
trigger();
for (int i=0; i<group; i++, position += PAGE_SIZE)
{
buffer.clear();
buffer.putInt(sequence);
buffer.putInt(index+i);
buffer.clear();
int n;
synchronized (lock)
{
n = channel.write(buffer, position);
}
if (n != buffer.capacity())
{
System.err.println("Incomplete write: "+n);
break test;
}
}
// now verify data
position = index*(long)PAGE_SIZE;
for (int i=0; i<group; i++, position += PAGE_SIZE)
{
buffer.clear();
int n;
synchronized (lock)
{
n = channel.read(buffer, position);
}
if (n != buffer.capacity())
{
System.err.println("Incomplete read: "+n);
break test;
}
buffer.flip();
int s = buffer.getInt();
int x = buffer.getInt();
if (s != sequence || x != index+i)
{
System.err.println("Invalid data in file: "+s+","+x+" != "+sequence+","+(index+i));
break test;
}
}
}
}
catch (IOException e)
{
e.printStackTrace();
}
complete = true;
trigger();
}
public void run()
{
extendFile();
}
}
---------- END SOURCE ----------
CUSTOMER SUBMITTED WORKAROUND :
Synchronize around read/write and setLength on a common object.
J2SE Version (please include all output from java -version flag):
jre-7-ea-bin-b142-windows-i586-12_may_2011.exe
Does this problem occur on J2SE 1.5.x or 6ux? Yes / No (pick one)
Yes, it occurs for java 5, and 6 too
Operating System Configuration Information (be specific):
Microsoft Windows Operating System Vista
Bug Description:
Find a very bad thread bug in RandomAccessFile. We share the RandomAccessFile
instance
over multiple threads and have synchronized the IO operations.
But it worked like
without synchronized.
The only workaround seems that after every seek(long), have to verify it
and set repeat the seek(long).
Steps to reproduce:
It can be reproduced with the attached test file TestThreadFileIO.java.
If all is ok then there is no output on the console.
You can see it if you use only one thread.
There is also a test result that we receive.
CAP members change the sample code to use a FileChannel.
but it still sahow th esame thread problem.
Attached the modified test case TestThreadFileIOnew.java
For the previous test case, they did not mixed RandomAccessFile and FileChannel,
only use RandomAccessFile. The new sample is more a mix because need the
RandomAccessFile to create the FileChannel.
Not see what is wrong in both samples, not access the file at the sam time.
The access is synchronized. It is a sequencial call of more as one thread
to the same file.
The updated test case uses raf.length() which isn't thread safe. If the test is changed to use channel.size() instead of raf.length() then it work as expected. For a future release we will nede to re-examine RandomAccessFile's implementation. Another thing to point out is that FileChannel defines read/write methods that take a position parameter. These methods do not change the global position and are designed for cases where multiple threads need to operation on the same file concurrently. There is also AsynchronousFileChannel in jdk7.
java version "1.5.0_06"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_06-b05)
Java HotSpot(TM) Client VM (build 1.5.0_06-b05, mixed mode, sharing)
ADDITIONAL OS VERSION INFORMATION :
Microsoft Windows XP [Version 5.1.2600]
EXTRA RELEVANT SYSTEM CONFIGURATION :
Intel 3.06GHz P4 --- hyperthreaded
A DESCRIPTION OF THE PROBLEM :
If a FileChannel.write(ByteBuffer b, long position) or FileChannel.read(ByteBuffer b, long position) takes place at the same time as a RandomAccessFile.setLength call on another thread where the channel is the one associated with the RandomAccessFile, then the read or write may take place at an unexpected position or the setLength may result in an unexpected file length.
These read and write methods are supposed to be independent of methods which affect the file position (and setLength is not declared as such anyway although that is probably how it is implemented).
STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Just run the attached code. As with most multithreading problems the result is somewhat variable but on my machine it does hit one of the failure points every time (which one varies).
Changing the code to use a common lock for read/write and setLength operations prevents the problem occuring.
EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
A successful run should just print "Test completed"
ACTUAL -
Invalid data in file: 0,0 != 3,15
Test completed
REPRODUCIBILITY :
This bug can be reproduced often.
---------- BEGIN SOURCE ----------
import java.io.RandomAccessFile;
import java.io.File;
import java.io.IOException;
import java.nio.channels.FileChannel;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.util.Random;
/** Test concurent read, write and length changes.
* It appears that RandomAccessFile.setLength interacts with FileChannel.write
* @author mThornton
* @since 22-Dec-2005 11:23:52
*/
public class TestConcurrency implements Runnable
{
private static final int PAGE_SIZE = 1024;
private int group=8;
private File file;
private RandomAccessFile raf;
private FileChannel channel;
private int size;
private int maximumSize;
private volatile boolean complete;
private Object lock = new Object();
private Object extendLock = new Object();
// private Object extendLock = lock; //new Object();
public static void main(String[] args)
{
TestConcurrency test = new TestConcurrency();
try
{
test.init(10, 1000);
new Thread(test).start();
test.modifyFileContent();
}
catch (IOException e)
{
e.printStackTrace();
}
test.cleanup();
System.out.println("Test completed");
}
void init(int initialSize, int maxSize) throws IOException
{
file = File.createTempFile("testConcurrency", "tmp");
raf = new RandomAccessFile(file, "rw");
channel = raf.getChannel();
size = initialSize;
raf.setLength(size*(long)PAGE_SIZE);
maximumSize = maxSize;
}
void cleanup()
{
if (raf != null)
{
try
{
raf.close();
}
catch (IOException e)
{
}
}
if (channel != null)
{
try
{
channel.close();
}
catch (IOException e)
{
}
}
if (file != null)
{
if (!file.delete())
System.err.println("Unable to delete temporary file: "+file);
}
}
private synchronized int size()
{
return size;
}
private void extendFile()
{
try
{
while (size < maximumSize)
{
synchronized (this)
{
wait();
}
if (complete)
break;
int n = size + 20;
synchronized (extendLock)
{
raf.setLength(n*(long)PAGE_SIZE);
}
synchronized (this)
{
size = n;
}
}
}
catch (IOException e)
{
e.printStackTrace();
}
catch (InterruptedException e)
{
}
complete = true;
}
private synchronized void trigger()
{
notify();
}
private void modifyFileContent()
{
ByteBuffer buffer = ByteBuffer.allocateDirect(PAGE_SIZE);
buffer.order(ByteOrder.nativeOrder());
Random rand = new Random();
int sequence = 0;
try
{
test:
while (!complete)
{
sequence++;
int limit = size();
int index = rand.nextInt(limit-group);
// first write data
long position = index*(long)PAGE_SIZE;
trigger();
for (int i=0; i<group; i++, position += PAGE_SIZE)
{
buffer.clear();
buffer.putInt(sequence);
buffer.putInt(index+i);
buffer.clear();
int n;
synchronized (lock)
{
n = channel.write(buffer, position);
}
if (n != buffer.capacity())
{
System.err.println("Incomplete write: "+n);
break test;
}
}
// now verify data
position = index*(long)PAGE_SIZE;
for (int i=0; i<group; i++, position += PAGE_SIZE)
{
buffer.clear();
int n;
synchronized (lock)
{
n = channel.read(buffer, position);
}
if (n != buffer.capacity())
{
System.err.println("Incomplete read: "+n);
break test;
}
buffer.flip();
int s = buffer.getInt();
int x = buffer.getInt();
if (s != sequence || x != index+i)
{
System.err.println("Invalid data in file: "+s+","+x+" != "+sequence+","+(index+i));
break test;
}
}
}
}
catch (IOException e)
{
e.printStackTrace();
}
complete = true;
trigger();
}
public void run()
{
extendFile();
}
}
---------- END SOURCE ----------
CUSTOMER SUBMITTED WORKAROUND :
Synchronize around read/write and setLength on a common object.
J2SE Version (please include all output from java -version flag):
jre-7-ea-bin-b142-windows-i586-12_may_2011.exe
Does this problem occur on J2SE 1.5.x or 6ux? Yes / No (pick one)
Yes, it occurs for java 5, and 6 too
Operating System Configuration Information (be specific):
Microsoft Windows Operating System Vista
Bug Description:
Find a very bad thread bug in RandomAccessFile. We share the RandomAccessFile
instance
over multiple threads and have synchronized the IO operations.
But it worked like
without synchronized.
The only workaround seems that after every seek(long), have to verify it
and set repeat the seek(long).
Steps to reproduce:
It can be reproduced with the attached test file TestThreadFileIO.java.
If all is ok then there is no output on the console.
You can see it if you use only one thread.
There is also a test result that we receive.
CAP members change the sample code to use a FileChannel.
but it still sahow th esame thread problem.
Attached the modified test case TestThreadFileIOnew.java
For the previous test case, they did not mixed RandomAccessFile and FileChannel,
only use RandomAccessFile. The new sample is more a mix because need the
RandomAccessFile to create the FileChannel.
Not see what is wrong in both samples, not access the file at the sam time.
The access is synchronized. It is a sequencial call of more as one thread
to the same file.
The updated test case uses raf.length() which isn't thread safe. If the test is changed to use channel.size() instead of raf.length() then it work as expected. For a future release we will nede to re-examine RandomAccessFile's implementation. Another thing to point out is that FileChannel defines read/write methods that take a position parameter. These methods do not change the global position and are designed for cases where multiple threads need to operation on the same file concurrently. There is also AsynchronousFileChannel in jdk7.
- relates to
-
JDK-4823133 RandomAccessFile.length() is not thread-safe
-
- Resolved
-