Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8177653

clarify restrictions on Iterator.forEachRemaining

XMLWordPrintable

        FULL PRODUCT VERSION :
        java version "1.8.0_121"
        Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
        Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)


        ADDITIONAL OS VERSION INFORMATION :
        Linux andreas-ThinkPad-T460p 4.4.0-66-generic #87-Ubuntu SMP Fri Mar 3 15:29:05 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux


        A DESCRIPTION OF THE PROBLEM :
        Using the method Iterator#forEachRemaining(Consumer action) of ArrayList and LinkedList leads to unexpected results.

        Both implementations of this method doesn't consider any changes made by ListIterator#add(T newelement) leading to wrong order or in one case event to completely wrong results.


        STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
        Execute the provided junit tests.

        EXPECTED VERSUS ACTUAL BEHAVIOR :
        EXPECTED -
        Described in the comments of each test.
        ACTUAL -
        4 test cases work as expected.
        3 test cases show proper elements in wrong order
        1 test case produces wrong elements

        REPRODUCIBILITY :
        This bug can be reproduced always.

        ---------- BEGIN SOURCE ----------
        /*****************************************************************************
         * Copyright (C) Compart AG, 2017 - Compart confidential
         *
         *****************************************************************************/

        package com.compart.java.util.test;

        import java.util.ArrayList;
        import java.util.Arrays;
        import java.util.Iterator;
        import java.util.LinkedList;
        import java.util.List;
        import java.util.ListIterator;
        import java.util.function.Consumer;

        import org.junit.Assert;
        import org.junit.Test;
        import org.junit.internal.ArrayComparisonFailure;

        /**
         * JUnit test to show bugs in implementations of {@link Iterator#forEachRemaining(Consumer)}
         *
         * Observed behavior applies to
         *
         * openjdk version "1.8.0_121"
         *
         * OpenJDK Runtime Environment (build 1.8.0_121-8u121-b13-0ubuntu1.16.04.2-b13)
         *
         * OpenJDK 64-Bit Server VM (build 25.121-b13, mixed mode)
         *
         * -
         *
         * java version "1.8.0_45"
         *
         * Java(TM) SE Runtime Environment (build 1.8.0_45-b14)
         *
         * Java HotSpot(TM) Server VM (build 25.45-b02, mixed mode)
         *
         * -
         *
         * java version "1.8.0_121"
         *
         * Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
         *
         * Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)
         *
         */
        public class ArrayListIteratorTest {

            @SuppressWarnings("boxing")
            private static final List<Integer> INPUT = Arrays.asList(new Integer[] { 1, 2, 3, 4 });

            @SuppressWarnings("boxing")
            private static final Integer[] EXPECTED = new Integer[] { 1, 11, 2, 12, 3, 13, 4, 14 };

            /**
             * Cute small consumer that adds 10 to each element and adds id to the list using the list iterator.
             */
            public class IncrementAction implements Consumer<Integer> {

                private ListIterator<Integer> listIterator;

                public IncrementAction(ListIterator<Integer> listIterator) {
                    this.listIterator = listIterator;
                }

                @Override
                public void accept(Integer t) {
                    this.listIterator.add(Integer.valueOf(10 + t.intValue()));
                }
            }

            /**
             * matches my expectation
             */
            @Test
            public void testArrayMethodAddAll() {
                List<Integer> list = createArrayListWithAddAll();
                ListIterator<Integer> listIterator = list.listIterator();
                performWithMethods(listIterator);
                assertions(list);
            }

            /**
             * matches my expectation
             */
            @Test
            public void testArrayMethodConstructor() {
                List<Integer> list = createArrayListWithConstructor();
                ListIterator<Integer> listIterator = list.listIterator();
                performWithMethods(listIterator);
                assertions(list);
            }

            /**
             * matches my expectation
             */
            @Test
            public void testLinkedMethodAddAll() {
                List<Integer> list = createLinkedWithAddAll();
                ListIterator<Integer> listIterator = list.listIterator();
                performWithMethods(listIterator);
                assertions(list);
            }

            /**
             * matches my expectation
             */
            @Test
            public void testLinkedMethodConstructor() {
                List<Integer> list = createLinkedWithConstructor();
                ListIterator<Integer> listIterator = list.listIterator();
                performWithMethods(listIterator);
                assertions(list);
            }

            /**
             * As we used {@link ArrayList#addAll(java.util.Collection)} the resulting capacity is big enough to fit all
             * elements to be added during the test. All calls to {@link ListIterator#add(Object)} will not cause the ArrayList
             * to grow. This causes the ArrayList#Itr implementaiton of the array list to run on the same list as the add method
             * of the {@link ListIterator} implementation in the array list. The add will shift all element in the elemntData
             * array. This causes the iterator to read the same elemnt four times.
             *
             * Observed: [11, 11, 11, 11, 1, 2, 3, 4], wrong elements in wrong order.
             */
            @Test
            public void testArrayForEachRemainingAddAll() {
                List<Integer> list = createArrayListWithAddAll();
                ListIterator<Integer> listIterator = list.listIterator();
                performWithForEachRemaining(listIterator);
                assertions(list);
            }

            /**
             * As we used the constructor pointing to INPUT the array list capacity fits perfect. The fist call to
             * {@link ListIterator#add(Object)} causes the ArrayList to grow. This causes the ArrayList#ListItr to run on an out
             * dated local copy causing all new elements to be added to the beginning.
             *
             * Observed: [11, 12, 13, 14, 1, 2, 3, 4], correct elements in wrong order.
             */
            @Test
            public void testArrayForEachRemainingConstructor() {
                List<Integer> list = createArrayListWithConstructor();
                ListIterator<Integer> listIterator = list.listIterator();
                performWithForEachRemaining(listIterator);
                assertions(list);
            }

            /**
             * LinkedList's {@link ListIterator#forEachRemaining(Consumer)} implementation first calls accept on the consumer
             * and then does the necessary steps to mimic a call to {@link ListIterator#next()} leading in a wrong oder.
             *
             * Observed: [11, 1, 12, 2, 13, 3, 14, 4], correct elements in wrong order.
             */
            @Test
            public void testLinkedForEachRemainingAddAll() {
                List<Integer> list = createLinkedWithAddAll();
                ListIterator<Integer> listIterator = list.listIterator();
                performWithForEachRemaining(listIterator);
                assertions(list);
            }

            /**
             * LinkedList's {@link ListIterator#forEachRemaining(Consumer)} implementation first calls accept on the consumer
             * and then does the necessary steps to mimic a call to {@link ListIterator#next()} leading in a wrong oder.
             *
             * Observed: [11, 1, 12, 2, 13, 3, 14, 4], correct elements in wrong order.
             */
            @Test
            public void testLinkedForEachRemainingConstructor() {
                List<Integer> list = createLinkedWithConstructor();
                ListIterator<Integer> listIterator = list.listIterator();
                performWithForEachRemaining(listIterator);
                assertions(list);
            }

            private void performWithMethods(ListIterator<Integer> listIterator) {
                Consumer<Integer> action = new IncrementAction(listIterator);

                // this piece of code is taken from the java doc of Iterator#forEachRemaining(Consumer<? super E> action) where
                // it is stated that forEachemaining should behave as this two lines.
                while (listIterator.hasNext()) {
                    action.accept(listIterator.next());
                }
            }

            private void performWithForEachRemaining(ListIterator<Integer> listIterator) {
                listIterator.forEachRemaining(new IncrementAction(listIterator));
            }

            private List<Integer> createLinkedWithConstructor() {
                List<Integer> list = new LinkedList<>(INPUT);
                return list;
            }

            private List<Integer> createLinkedWithAddAll() {
                List<Integer> list = new LinkedList<>();
                list.addAll(INPUT);
                return list;
            }

            private List<Integer> createArrayListWithConstructor() {
                List<Integer> list = new ArrayList<>(INPUT);
                return list;
            }

            private List<Integer> createArrayListWithAddAll() {
                List<Integer> list = new ArrayList<>();
                list.addAll(INPUT);
                return list;
            }

            private void assertions(List<Integer> list) throws ArrayComparisonFailure {
                Assert.assertArrayEquals(Arrays.asList(EXPECTED) + " is not " + list, EXPECTED, list.toArray());
            }

        }

        ---------- END SOURCE ----------

        CUSTOMER SUBMITTED WORKAROUND :
        avoid Iterator#forEachRemaining(Consumer action)

          1. JI9048292.java
            7 kB
            Pallavi Sonal
          2. output_JI9048292.log
            10 kB
            Pallavi Sonal

              smarks Stuart Marks
              webbuggrp Webbug Group
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Created:
                Updated:
                Resolved: