FULL PRODUCT VERSION :
java version "1.6.0_20"
Java(TM) SE Runtime Environment (build 1.6.0_20-b02)
Java HotSpot(TM) 64-Bit Server VM (build 16.3-b01, mixed mode)

ADDITIONAL OS VERSION INFORMATION :
Linux dbnu099 2.6.16.60-0.42.4-smp #1 SMP Fri Aug 14 14:33:26 UTC 2009 x86_64 x86_64 x86_64 GNU/Linux

A DESCRIPTION OF THE PROBLEM :
Every request on webserver port which is not http conform for example just open und direclty close a socket on webserver produce memory dust.

it looks like a memory leak.

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
start a

com.sun.net.httpserver.HttpServer;

Webserver with or without context. simply start any kind of these webserver.

open a telnet connection on the webserver port and directly close it in a while loop.




EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
when i fininish these test i expected a normally running webserver.
ACTUAL -
acutally there is a memory leak. Every open socket connection on webserver produce a object that never cleaned up in memory.

This only appears when the request is non http conform request. such from a port scanner or similar.

when i make a heap dump i saw a huge amount of:

sun.nio.ch.SelektionKeyImpl
and
sun.net.httpserver.HTTPConnection

Objects, which are not cleaned up by the garbage collector. so there still is a reference.

ERROR MESSAGES/STACK TRACES THAT OCCUR :
FEIN: RMI TCP Connection(1863)-10.1.102.144: close connection
Exception in thread "RMI TCP Connection(idle)" java.lang.OutOfMemoryError: Java heap space

REPRODUCIBILITY :
This bug can be reproduced always.

---------- BEGIN SOURCE ----------
package TestServer;
import java.io.IOException;
import java.io.OutputStream;
import java.net.InetSocketAddress;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.logging.Logger;

import com.sun.net.httpserver.Headers;
import com.sun.net.httpserver.HttpExchange;
import com.sun.net.httpserver.HttpHandler;
import com.sun.net.httpserver.HttpServer;


public class HTTPServer extends Thread implements HttpHandler {

private static Logger logger = Logger.getLogger("Test");
private InetSocketAddress address;
private HttpServer server;
private String hostname = "localhost";
private int port=6686;

private Object waiter = new Object();

private boolean isRunning=false;

//private Executor exec = ThreadPoolExecutor.;


private boolean isRunning() {
return isRunning;
}

private void setRunning(boolean isRunning) {
this.isRunning = isRunning;
}

public HTTPServer() {

        try {
         address = new InetSocketAddress(hostname, port);
server = HttpServer.create(address, 10);

//server.createContex

logger.info("starting http server("+address+")");

server.createContext("/", this);
//server.setExecutor(Executors.newCachedThreadPool());
server.start();

this.performAction();

} catch (Exception e) {
e.printStackTrace();
logger.warning(e.toString());
}
        
}

private void performAction() {
setRunning(true);
        Runtime.getRuntime().addShutdownHook(this);
           
     while (isRunning()) {
     try {
synchronized (waiter) {
waiter.wait(5000);
}
     } catch (InterruptedException ie) {
     }
     }
     server.stop(3);
     //TODO unpublishAllEndpoints();
}

@Override
public void handle(HttpExchange xchg) throws IOException {

Headers headers = xchg.getRequestHeaders();
logger.info("receive request from: "+xchg.getRemoteAddress());
Set<Map.Entry<String, List<String>>> entries = headers.entrySet();

StringBuffer response = new StringBuffer();
for (Map.Entry<String, List<String>> entry : entries)
response.append(entry.toString() + "\n");

xchg.sendResponseHeaders(200, response.length());
OutputStream os = xchg.getResponseBody();
os.write(response.toString().getBytes());
os.close();
}


public void run() {
logger.info("HTTP Test Server ShutdownHook has been aktivated.");

     setRunning(false);
     synchronized (waiter) {
     waiter.notifyAll();
     }
    }

}



package TestServer;
import java.util.logging.Logger;


public class MainTest {

private static Logger logger = Logger.getLogger("Test");

/**
* @param args
*/
public static void main(String[] args) {
logger.info("create HTTPServer...");
new HTTPServer();
}

}

---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
dont send non http conform requests to server