ServiceUnavailableException caught even the secondary DNS is available

Customer is HDE and holds JFB contract

http://javaweb.sfbay.sun.com/cgi-bin/jpg/customer?d=hde.co.jp

Cu is using JFB 1.5.0_24 on Linux 2.6.18.

They reported JVM returns ServiceUnavailableException or OperationNotSupportedException even if secondary DNS responds normally.

Their program query DNS record by using JNDI.

They understand Java DNS query works based on OS configuration /etc/resolv.conf
http://java.sun.com/j2se/1.5.0/docs/guide/jndi/jndi-dns.html#URL

They configured /etc/resolv.conf as follows:

nameserver 127.0.0.1
nameserver 192.168.145.1

If JVM query to the DNS of 127.0.0.1 by DirContext#getAttributes and unavailable, they think JVM query to the DNS of 192.168.145.1 and get the result. but JVM returns ServiceUnavailableException after a few seconds.

When they got the network capture, JVM seems to query the DNS of 192.168.145.1
few times after 127.0.0.1 Server failure.

470.143601 127.0.0.1 -> 127.0.0.1 DNS Standard query MX sun.com
470.143736 127.0.0.1 -> 127.0.0.1 DNS Standard query response, Server failure
470.144350 192.168.145.8 -> 192.168.145.1 DNS Standard query MX sun.com
470.144688 192.168.145.1 -> 192.168.145.8 DNS Standard query response MX 20 mx3.sun.com MX 20 mx4.sun.com MX 5 btmx4.sun.com MX 5 btmx6.sun.com
471.145261 192.168.145.8 -> 192.168.145.1 DNS Standard query MX sun.com
471.145609 192.168.145.1 -> 192.168.145.8 DNS Standard query response MX 20 mx3.sun.com MX 20 mx4.sun.com MX 5 btmx4.sun.com MX 5 btmx6.sun.com
473.146082 192.168.145.8 -> 192.168.145.1 DNS Standard query MX sun.com
473.146438 192.168.145.1 -> 192.168.145.8 DNS Standard query response MX 20 mx3.sun.com MX 20 mx4.sun.com MX 5 btmx4.sun.com MX 5 btmx6.sun.com
477.146045 192.168.145.8 -> 192.168.145.1 DNS Standard query MX sun.com
477.146418 192.168.145.1 -> 192.168.145.8 DNS Standard query response MX 20 mx4.sun.com MX 5 btmx4.sun.com MX 5 btmx6.sun.com MX 20 mx3.sun.com

Here is sample code and reproducible steps.

import java.util.*;
import javax.naming.*;
import javax.naming.directory.*;
public class DnsQuery
{
/**
* args[0] domain
* args[1] query type
*/
public static void main(String args[])
throws Exception
{
if(args.length < 2){
System.err.println("java DnsQuery [domain] [query-type]");
System.exit(1);
}
Hashtable<String, String> env = new Hashtable<String, String>();
env.put("java.naming.factory.initial", "com.sun.jndi.dns.DnsContextFactory");
env.put("java.naming.provider.url", "dns:");
DirContext ctx = new InitialDirContext(env);
try{
Attributes attrs = ctx.getAttributes(args[0], new String[] {args[1]});
Attribute attr = attrs.get(args[1]);
System.out.println(attr);
}
finally{
ctx.close();
}
}
}


<<Steps>>
(1) configure /etc/resolv.conf like following
nameserver 127.0.0.1
nameserver 192.168.145.1

(2) They intentionally configure primary DNS recursive-clients and max-cache-ttl value
    of named.conf like following as to response code 2(server failure).
    (They use bind9.)
options {
directory "/var/named";
pid-file "/var/run/named/named.pid";
recursion yes;
recursive-clients 1;
max-cache-ttl 1;
};

(3) compile sample code and execute sample code at the same time like following
$java DnsQuery sun.com mx & java DnsQuery sun.co.jp mx

They also tried JFB6u20 and result is the same.


We have managed to reproduce the problem in our lab machine

# cat /etc/resolv.conf
nameserver 10.14.4.197
nameserver 129.158.31.142
 
10.14.4.197's DNS server is following setting
 
# cat /etc/named.conf
options {
directory "/var/named";
pid-file "/var/run/named/named.pid";
recursion yes;
recursive-clients 1;
max-cache-ttl 1;
};
 
-----------------------------
[root@s4lab223 dnsquery]# jdk1.5.0_24/bin/java DnsQuery sun.com mx
MX: 10 nwk-avmta-1.sfbay.sun.com., 10 nwk-avmta-2.sfbay.sun.com., 10 brm-avmta-1.central.sun.com. <---- normal dns record

[root@s4lab223 dnsquery]# jdk1.5.0_24/bin/java DnsQuery sun.com mx
javax.naming.ServiceUnavailableException: DNS server failure [response code 2]; remaining name 'sun.com'
        at com.sun.jndi.dns.DnsClient.checkResponseCode(DnsClient.java:594)
        at com.sun.jndi.dns.DnsClient.isMatchResponse(DnsClient.java:553)
        at com.sun.jndi.dns.DnsClient.doUdpQuery(DnsClient.java:399)
        at com.sun.jndi.dns.DnsClient.query(DnsClient.java:186)
        at com.sun.jndi.dns.Resolver.query(Resolver.java:64)
        at com.sun.jndi.dns.DnsContext.c_getAttributes(DnsContext.java:413)
        at com.sun.jndi.toolkit.ctx.ComponentDirContext.p_getAttributes(ComponentDirContext.java:213)
        at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.getAttributes(PartialCompositeDirContext.java:121)
        at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.getAttributes(PartialCompositeDirContext.java:109)
        at javax.naming.directory.InitialDirContext.getAttributes(InitialDirContext.java:123)
        at DnsQuery.main(DnsQuery.java:26)
-----------------------------
 
 dig command also returns error when DNS server respond server fail.
 dig command returns error immediately without trying 129.158.31.142.
-----------------------------
[root@s4lab223 dnsquery]# dig sun.com mx

; <<>> DiG 9.3.6-P1-RedHat-9.3.6-4.P1.el5 <<>> sun.com mx
;; global options: printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 14000
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;sun.com. IN MX

;; Query time: 3 msec
;; SERVER: 10.14.4.197#53(10.14.4.197)
;; WHEN: Tue May 25 12:44:33 2010
;; MSG SIZE rcvd: 25
-----------------------------
 
 if Java works like dig command behavior, Java should not query to secondary server.
 Or if Java query secondary DNS server and get response, Java should return dns record instead of Exception.
 
this behavior can be confirmed both linux and solaris env.
 
you can confirm by setting /etc/resolv.conf as follows:

[root@s4lab223 dnsquery]# cat /etc/resolv.conf
nameserver 10.14.4.197
nameserver 129.158.31.142


Thanks