Update test/java/lang/instrument/Re(transform|define)BigClass.sh test to use NMT for memory leak detection

The two tests currently use /proc/self/stat to read the memory consumption for the process. This only works on linux and only gives one value. NMT gives a lot more detail about where the memory is consumed and works on all platforms.