Name: boT120536 Date: 12/11/2000


Applies to 1.2.2 and 1.3.0 JDKs from sun, blackdown & ibm, both on linux and
solaris, as detailed below

java.net.Socket.setSendBufferSize() and java.net.Socket.sendReceiveBufferSize()
misbehave partially or completely on Linux and Solaris, both in JDK 1.2.2 and
1.3.0.
They work as expected on Windows. And they work in C based tcp test programs
like `iperf`.

If the underlying operating system is configured to allow large buffer sizes,
these calls should use them just as C setsockopt() does.
It is known that there are no guarantees as to which buffer size the OS will
actually use. This is fine.
The data below seems to imply that the JDK is itself assigning funny (and too
small) buffer sizes, whereas this really is the job of the OS.
The JDK used to call setsockopt() through JNI, which is precisely the expected
behaviour. Something funny is happening in more recent releases.

Please find attached a tiny program that reproduces the behaviour.
It tries to set socket buffer size to 256K, but only gets much less.
OS level TCP tuning parameters are set appropriately on all used machines (and
can be dumped with the attached script).
Please consider that not being able to properly tune buffer sizes means that
high performance data intensive applications running over Gigabit Ethernet LAN,
and/or high bandwidth * delay WAN path can easily be one order of magnitude
slower than under ideal settings (as can easily be verified with `iperf`).
See http://www-didc.lbl.gov/tcp-wan.html for more material explaining why these
things matter.

This may perhaps be related to bug #4292944, but the "evaluation" given to
#4292944 does not apply here.

public class SocketBufferBug {

public static void main(String args[]) throws java.io.IOException {
String host = "localhost";
int port = 514; // syslog
int SO_SNDBUF = 256 * 1024;
int SO_RCVBUF = 256 * 1024;
if (args.length > 0) host = args[0];
if (args.length > 1) port = Integer.parseInt(args[1]);
if (args.length > 2) SO_SNDBUF = Integer.parseInt(args[2]);
if (args.length > 3) SO_RCVBUF = Integer.parseInt(args[3]);

java.net.Socket s = new java.net.Socket(host,port);
System.out.println("before: sendBufSize=" + s.getSendBufferSize());
System.out.println("before: recvBufSize=" + s.getReceiveBufferSize());

s.setSendBufferSize(SO_SNDBUF);
s.setReceiveBufferSize(SO_RCVBUF);

System.out.println("after : sendBufSize=" + s.getSendBufferSize());
System.out.println("after : recvBufSize=" + s.getReceiveBufferSize());

s.close();
}
}

#!/bin/sh
# dumpparams.sh
# Dump TCP related tuning parameters and limits
os=`uname`
if [ $os = "SunOS" ]; then
echo 'tcp_max_buf=\c'
ndd /dev/tcp tcp_max_buf
echo 'tcp_xmit_hiwat=\c'
ndd /dev/tcp tcp_xmit_hiwat
echo 'tcp_recv_hiwat=\c'
ndd /dev/tcp tcp_recv_hiwat
echo 'tcp_cwnd_max=\c'
ndd /dev/tcp tcp_cwnd_max
elif [ $os = "Linux" ]; then
echo "wmem_max=`cat /proc/sys/net/core/wmem_max`"
echo "rmem_max=`cat /proc/sys/net/core/rmem_max`"
echo "wmem_default=`cat /proc/sys/net/core/wmem_default`"
echo "rmem_default=`cat /proc/sys/net/core/rmem_default`"
fi

##########################################################################
iperf verifies that we can actually use large buffer sizes on linux and solaris
(256K and 512K)
##########################################################################

[monarc01 /afs/cern.ch/user/h/hoschek/tmp/socketbug] iperf -s -p 8000 -w 256K -m
------------------------------------------------------------
Server listening on TCP port 8000
TCP window size: 256 KByte
------------------------------------------------------------
[ 4] local 137.138.137.126 port 8000 connected with 137.138.181.215 port 1138
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-10.0 sec 472 MBytes 378 Mbits/sec
[ 4] MSS size 1448 bytes (MTU 1500 bytes, ethernet)
[ 4] Read lengths occurring in more than 5% of reads:
[ 4] 2896 bytes read 4066 times (5.57%)
[ 4] 6744 bytes read 3784 times (5.18%)
[ 4] 8192 bytes read 48088 times (65.9%)

[inssrv1 /afs/cern.ch/user/h/hoschek/tmp/socketbug] iperf -c monarc01 -p 8000 -w
256K -f K
------------------------------------------------------------
Client connecting to monarc01, TCP port 8000
TCP window size: 512 KByte (WARNING: requested 256 KByte)
------------------------------------------------------------
[ 3] local 137.138.181.215 port 1138 connected with 137.138.137.126 port 8000
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.0 sec 483536 KBytes 48344 KBytes/sec



#########################################################
Windows NT 4.0: OK
#########################################################
R:\tmp\socketbug>q:\win32\jdk\sun-1.2.2\bin\java -version
java version "1.2.2"
Java HotSpot(TM) Server VM (2.0fcs, mixed mode, build E)

R:\tmp\socketbug>q:\win32\jdk\sun-1.2.2\bin\java SocketBufferBug monarc01 514
before: sendBufSize=8192
before: recvBufSize=8192
after : sendBufSize=262144
after : recvBufSize=262144

R:\tmp\socketbug>q:\win32\jdk\ibm-1.2.2\bin\java -version
java version "1.2.2"
Classic VM (J2RE 1.2.2 IBM build cn122-20000127 (JIT enabled: jitc))

R:\tmp\socketbug>q:\win32\jdk\ibm-1.2.2\bin\java SocketBufferBug monarc01 514
before: sendBufSize=8192
before: recvBufSize=8192
after : sendBufSize=262144
after : recvBufSize=262144

R:\tmp\socketbug>q:\win32\jdk\sun-1.3.0\bin\java -version
java version "1.3.0_01"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.3.0_01)
Java HotSpot(TM) Client VM (build 1.3.0_01, mixed mode)

R:\tmp\socketbug>q:\win32\jdk\sun-1.3.0\bin\java SocketBufferBug monarc01 514
before: sendBufSize=8192
before: recvBufSize=8192
after : sendBufSize=262144
after : recvBufSize=262144

R:\tmp\socketbug>q:\win32\jdk\ibm-1.3.0\bin\java -version
java version "1.3.0"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.3.0)
Classic VM (build 1.3.0, J2RE 1.3.0 IBM build cn130-20001018 (JIT enabled:
jitc))

R:\tmp\socketbug>q:\win32\jdk\ibm-1.3.0\bin\java SocketBufferBug monarc01 514
before: sendBufSize=8192
before: recvBufSize=8192
after : sendBufSize=262144
after : recvBufSize=262144



#########################################################
SOLARIS 2.6 and 2.7: FAILED
#########################################################

[monarc01 /afs/cern.ch/user/h/hoschek/tmp/socketbug] uname -a
SunOS monarc01 5.6 Generic_105181-12 sun4u sparc SUNW,Ultra-4

[monarc01 /afs/cern.ch/user/h/hoschek/tmp/socketbug] dumpparams.sh
tcp_max_buf=1048576
tcp_xmit_hiwat=8192
tcp_recv_hiwat=8192
tcp_cwnd_max=262144

[monarc01 /afs/cern.ch/user/h/hoschek/tmp/socketbug]
/afs/cern.ch/sw/java/sparc_solaris26/jdk/sun-1.2.2/bin/java -version
java version "1.2.2"
Solaris VM (build Solaris_JDK_1.2.2_06, native threads, sunwjit)

[monarc01 /afs/cern.ch/user/h/hoschek/tmp/socketbug]
/afs/cern.ch/sw/java/sparc_solaris26/jdk/sun-1.2.2/bin/java SocketBufferBug
before: sendBufSize=8192
before: recvBufSize=32768
after : sendBufSize=262144
after : recvBufSize=57344



[monarc01 /afs/cern.ch/user/h/hoschek/tmp/socketbug]
/afs/cern.ch/sw/java/sparc_solaris26/jdk/sun-1.3.0/bin/java -version
java version "1.3.0_01"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.3.0_01)
Java HotSpot(TM) Client VM (build 1.3.0_01, mixed mode)

[monarc01 /afs/cern.ch/user/h/hoschek/tmp/socketbug]
/afs/cern.ch/sw/java/sparc_solaris26/jdk/sun-1.3.0/bin/java SocketBufferBug
before: sendBufSize=8192
before: recvBufSize=32768
after : sendBufSize=65536
after : recvBufSize=57344



#########################################################
LINUX REDHAT6.1
#########################################################

[inssrv1 /afs/cern.ch/user/h/hoschek/tmp/socketbug] uname -a
Linux inssrv1 2.2.17-7.4.0smp #1 SMP Thu Nov 16 10:17:16 CET 2000 i686 unknown

[inssrv1 /afs/cern.ch/user/h/hoschek/tmp/socketbug] dumpparams.sh
wmem_max=514288
rmem_max=514288
wmem_default=65535
rmem_default=65535


[inssrv1 /afs/cern.ch/user/h/hoschek/tmp/socketbug]
/afs/cern.ch/sw/java/i386_redhat61/jdk/blackdown-1.2.2/bin/java -version
java version "1.2.2"
Classic VM (build Linux_JDK_1.2.2_FCS, native threads, sunwjit)
[inssrv1 /afs/cern.ch/user/h/hoschek/tmp/socketbug]
/afs/cern.ch/sw/java/i386_redhat61/jdk/blackdown-1.2.2/bin/java SocketBufferBug
before: sendBufSize=65535
before: recvBufSize=65535
after : sendBufSize=524288
after : recvBufSize=524288



[inssrv1 /afs/cern.ch/user/h/hoschek/tmp/socketbug]
/afs/cern.ch/sw/java/i386_redhat61/jdk/blackdown-1.3.0/bin/java -version
java version "1.3.0"
Java(TM) 2 Runtime Environment, Standard Edition (build Blackdown-1.3.0-FCS)
Java HotSpot(TM) Client VM (build Blackdown-1.3.0-FCS, mixed mode)

[inssrv1 /afs/cern.ch/user/h/hoschek/tmp/socketbug]
/afs/cern.ch/sw/java/i386_redhat61/jdk/blackdown-1.3.0/bin/java SocketBufferBug
before: sendBufSize=65535
before: recvBufSize=65535
after : sendBufSize=131072
after : recvBufSize=131072



[inssrv1 /afs/cern.ch/user/h/hoschek/tmp/socketbug]
/afs/cern.ch/sw/java/i386_redhat61/jdk/sun-1.3.0/bin/java -version
java version "1.3.0_01"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.3.0_01)
Java HotSpot(TM) Client VM (build 1.3.0_01, mixed mode)

[inssrv1 /afs/cern.ch/user/h/hoschek/tmp/socketbug]
/afs/cern.ch/sw/java/i386_redhat61/jdk/sun-1.3.0/bin/java SocketBufferBug
before: sendBufSize=65535
before: recvBufSize=65535
after : sendBufSize=131072
after : recvBufSize=131072



[inssrv1 /afs/cern.ch/user/h/hoschek/tmp/socketbug]
/afs/cern.ch/sw/java/i386_redhat61/jdk/ibm-1.3.0/bin/java -version
java version "1.3.0"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.3.0)
Classic VM (build 1.3.0, J2RE 1.3.0 IBM build cx130-20001025 (JIT enabled: jitc)
)

[inssrv1 /afs/cern.ch/user/h/hoschek/tmp/socketbug]
/afs/cern.ch/sw/java/i386_redhat61/jdk/ibm-1.3.0/bin/java SocketBufferBug
before: sendBufSize=65535
before: recvBufSize=65535
after : sendBufSize=131072
after : recvBufSize=131072
(Review ID: 112793)
======================================================================