FULL PRODUCT VERSION :
1.5.0_01 to 1.5.0_05

ADDITIONAL OS VERSION INFORMATION :
Linux 2.6 + glibc 2.3

A DESCRIPTION OF THE PROBLEM :
gethostbyaddr_r() and gethostbyname_r() in glibc are unstable under load. Concurrent calls through Inet4AddressImpl methods cause malloc corruption, stack corruption, etc. resulting in a SEGV crash of the JVM.

See discussion thread here for multiple reports and more details:
http://forum.java.sun.com/thread.jspa?threadID=643360

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
All reported crashes involve running tomcat under load in production or pre-production environments.

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
No crash.
ACTUAL -
Crash.

ERROR MESSAGES/STACK TRACES THAT OCCUR :
I have a big collection of these. This is representative:

#
# An unexpected error has been detected by HotSpot Virtual Machine:
#
# SIGSEGV (0xb) at pc=0xb7f24900, pid=31459, tid=705854384
#
# Java VM: Java HotSpot(TM) Client VM (1.5.0_06-b05 mixed mode)
# Problematic frame:
# C [libc.so.6+0x79900] memchr+0x70
#

--------------- T H R E A D ---------------

Current thread (0x085c9c50): JavaThread "http-80-Processor196" daemon [_thread_
in_native, id=32032]

siginfo:si_signo=11, si_errno=0, si_code=1, si_addr=0xb26b307c

Registers:
EAX=0xb26b307c, EBX=0xb7fdce80, ECX=0x00000a0a, EDX=0x0a0a0a0a
ESP=0x2a1253f4, EBP=0x2a125424, ESI=0x000000e7, EDI=0xb26b307c
EIP=0xb7f24900, CR2=0xb26b307c, EFLAGS=0x00010206

  Top of Stack: (sp=0x2a1253f4)
0x2a1253f4: b26b307c 000000f7 b7f0d1fe b26b307c
0x2a125404: 0000000a 000000f7 00000000 b26b307c
0x2a125414: 2a1254e8 b7fdce80 28c2c738 fbad2418
0x2a125424: 2a125448 b7f0d181 28c2c738 2a1254e8
0x2a125434: 000003e7 0000000a 00000001 00000000
0x2a125444: b7fdce80 2a125474 b7f1661d 28c2c738
0x2a125454: 2a1254e8 000003e7 0000000a 00000001
0x2a125464: 00000000 b7e851dc 2a1254d0 000003e8

Instructions: (pc=0xb7f24900)
0xb7f248f0: 00 40 4e e9 8e 00 00 00 90 8d b4 26 00 00 00 00
0xb7f24900: 8b 08 bf ff fe fe fe 31 d1 01 cf 0f 83 04 01 00

Stack: [0x2a0a8000,0x2a128000), sp=0x2a1253f4, free space=500k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C [libc.so.6+0x79900] memchr+0x70
C [libc.so.6+0x62181] _IO_getline+0x41
C [libc.so.6+0x6b61d] fgets_unlocked+0x5d
C [libnss_files.so.2+0x3e31]
C [libnss_files.so.2+0x3471] _nss_files_gethostbyname_r+0x211
C [libc.so.6+0xeb19f] gethostbyname_r+0x10f
C [libnet.so+0x4d61] Java_java_net_Inet4AddressImpl_getLocalHostName+0x91
j java.net.Inet4AddressImpl.getLocalHostName()Ljava/lang/String;+0
j java.net.InetAddress.getLocalHost()Ljava/net/InetAddress;+7
j com.sun.mail.smtp.SMTPTransport.getLocalHost()Ljava/lang/String;+122
j com.sun.mail.smtp.SMTPTransport.protocolConnect(Ljava/lang/String;ILjava/lang
/String;Ljava/lang/String;)Z+254
j javax.mail.Service.connect(Ljava/lang/String;ILjava/lang/String;Ljava/lang/St
ring;)V+378


REPRODUCIBILITY :
This bug can be reproduced occasionally.

CUSTOMER SUBMITTED WORKAROUND :
This is a patch which I have applied which has eliminated this problem.

A better solution would be to cache the results for concurrent reads by Java threads in InetAddress, and serialize reads to fill the cache through the safer non-reentrant functions in glibc.

--- ./java/net/Inet4AddressImpl.java.orig 2005-11-11 02:20:32.000000000 +0800
+++ ./java/net/Inet4AddressImpl.java 2005-12-30 17:20:27.000000000 +0800
@@ -13,10 +13,10 @@
  * @since 1.4
  */
 class Inet4AddressImpl implements InetAddressImpl {
- public native String getLocalHostName() throws UnknownHostException;
- public native byte[][]
+ public synchronized native String getLocalHostName() throws UnknownHostException;
+ public synchronized native byte[][]
         lookupAllHostAddr(String hostname) throws UnknownHostException;
- public native String getHostByAddr(byte[] addr) throws UnknownHostException;
+ public synchronized native String getHostByAddr(byte[] addr) throws UnknownHostException;
     private native boolean isReachable0(byte[] addr, int timeout, byte[] ifaddr, int ttl) throws IOException;
 
     public synchronized InetAddress anyLocalAddress() {