Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8332154

Memory leak in SynchronousQueue

XMLWordPrintable

    • b24
    • generic
    • generic
    • Verified

        ADDITIONAL SYSTEM INFORMATION :
        OpenJDK 21.0.2, reproducible on Linux/MacOS. It is likely reproducible on Windows too, although our reproducer fails for unrelated reasons there.

        A DESCRIPTION OF THE PROBLEM :
        Since the introduction of this commit https://github.com/openjdk/jdk/commit/8d1ab57065c7ebcc650b5fb4ae098f8b0a35f112, SynchronousQueue sometimes appears in a state where it has only request nodes (i.e. it is empty) and a reference to an item it stored before.

        We believe this is a memory leak: one consequence of this bug is an Executor which holds a reference to a terminated runnable. This reference prevents the runnable from being garbage-collected.

        REGRESSION : Last worked in version 21

        STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
        We wrote a test case.
        This test uses a SycnhronousQueue in an environment with a lot of threads, eventually leaving it in a state with no data nodes and multiple request nodes. Then the test checks the presence of references to the previously stored objects.

        EXPECTED VERSUS ACTUAL BEHAVIOR :
        EXPECTED -
        Since the SynchronousQueue is empty, it should not reference the data that was added to it previously.
        ACTUAL -
        Sometimes this test fails on OpenJDK 21.0.2.

        ---------- BEGIN SOURCE ----------
        import java.lang.ref.WeakReference;
        import java.lang.reflect.Field;
        import java.lang.reflect.Modifier;
        import java.util.*;
        import java.util.concurrent.*;
        import java.util.concurrent.atomic.AtomicInteger;

        /**
         * If `SynchronousQueue` is empty, then it should not reference any objects that were in it before.
         */
        public class Main {

            // There are fewer suppliers that waiters, which means that all the supplied items should be consumed
            private static final int SUPPLIERS = 490;
            private static final int WAITERS = 500;
            private static final AtomicInteger waitersLocked = new AtomicInteger(0);
            private static final AtomicInteger threadsFinished = new AtomicInteger(0);
            private static final Set<Object> visitedItems = new HashSet<>();

            private static final class Leak {}

            private static final class Waiter extends Thread {
                private final BlockingQueue<Leak> queue;

                private Waiter(BlockingQueue<Leak> queue) {
                    this.queue = queue;
                }

                public void run() {
                    for (int i = 0; i < 100; ++i) {
                        try {
                            waitersLocked.incrementAndGet();
                            // the object collected here is not reachable anymore, it should be garbage-collected
                            queue.take();
                            waitersLocked.decrementAndGet();
                        } catch (InterruptedException ignored) {
                        }
                    }
                    threadsFinished.incrementAndGet();
                }
            }

            private static class Supplier extends Thread {
                private final BlockingQueue<Leak> queue;

                private Supplier(BlockingQueue<Leak> queue) {
                    this.queue = queue;
                }

                public void run() {
                    for (int i = 0; i < 100; ++i) {
                        try {
                            Leak leak = new Leak();
                            queue.put(leak);
                        } catch (InterruptedException ignored) {
                        }
                    }
                    threadsFinished.incrementAndGet();
                }
            }

            private static List<Field> getAllFields(Class<?> clazz) {
                ArrayList<Field> fields = new ArrayList<>();
                Collections.addAll(fields, clazz.getDeclaredFields());
                var superClass = clazz.getSuperclass();
                if (superClass != null) {
                    fields.addAll(getAllFields(superClass));
                }
                return fields;
            }

            private static void introspect(Object object) throws IllegalAccessException {
                for (Field field : getAllFields(object.getClass())) {
                    if (Modifier.isStatic(field.getModifiers())) {
                        continue;
                    }
                    field.setAccessible(true);
                    Object item = field.get(object);
                    if (item instanceof Leak) {
                        System.out.println("Found leak");
                        System.exit(1);
                    }
                    if (item instanceof Thread) {
                        continue;
                    }
                    if (item != null && visitedItems.add(item)) {
                        introspect(item);
                    }
                }
            }

            public static void main(String[] args) throws InterruptedException, IllegalAccessException {
                SynchronousQueue<Leak> queue = new SynchronousQueue<>(false);
                for (int i = 0; i < WAITERS; ++i) {
                    new Waiter(queue).start();
                    if (i < SUPPLIERS) {
                        new Supplier(queue).start();
                    }
                }
                while (threadsFinished.get() + waitersLocked.get() != SUPPLIERS + WAITERS) {
                    // spin wait
                }
                if (!queue.isEmpty()) {
                    System.out.println("Queue is not empty");
                    System.exit(1);
                }
                introspect(queue);
                System.out.println("Successful!");
                System.exit(0);
            }
        }
        ---------- END SOURCE ----------

        FREQUENCY : often


              vklang Viktor Klang
              webbuggrp Webbug Group
              Votes:
              1 Vote for this issue
              Watchers:
              8 Start watching this issue

                Created:
                Updated:
                Resolved: