Look here:

Symbol* SymbolTable::lookup_common(const char* name,
                            int len, unsigned int hash) {
  Symbol* sym;
  if (_lookup_shared_first) {
    sym = lookup_shared(name, len, hash);
    if (sym == NULL) {
      _lookup_shared_first = false; // <--- switches here
      sym = lookup_dynamic(name, len, hash);
    }
  } else {
    sym = lookup_dynamic(name, len, hash);
    if (sym == NULL) {
      sym = lookup_shared(name, len, hash);
      if (sym != NULL) {
        _lookup_shared_first = true;
      }
    }
  }
  return sym;
}

It would seem that the switch to dynamic table is too eager: it switches to dynamic table even if the lookup from there would be NULL.

Changing it to a symmetric:
 
  if (_lookup_shared_first) {
    sym = lookup_shared(name, len, hash);
    if (sym == NULL) {
      sym = lookup_dynamic(name, len, hash);
      if (sym != NULL) {
        _lookup_shared_first = false;
      }
    }
  }

...seems to yield measurable improvements in Hello World time.

 Performance counter stats for 'build/baseline/bin/java -Xms128m -Xmx128m Hello' (1000 runs):

             28.34 msec task-clock # 1.170 CPUs utilized ( +- 0.10% )
        72,330,965 cycles # 2.552 GHz ( +- 0.09% )
        62,625,584 instructions # 0.87 insn per cycle ( +- 0.00% )

         0.0242176 +- 0.0000301 seconds time elapsed ( +- 0.12% )


 Performance counter stats for 'build/linux-x86_64-server-release/images/jdk/bin/java -Xms128m -Xmx128m Hello' (1000 runs):

             27.95 msec task-clock # 1.171 CPUs utilized ( +- 0.10% )
        71,334,133 cycles # 2.552 GHz ( +- 0.09% )
        62,698,685 instructions # 0.88 insn per cycle ( +- 0.00% )

         0.0238646 +- 0.0000282 seconds time elapsed ( +- 0.12% )

It would be interesting to consider a more hysteresis-based approach for this switching.

Draft PR:
  https://github.com/openjdk/jdk/pull/2335