CODETOOLS-7901237 highlighted the problem with regenerating the benchmark list with
every benchmark generation session. While we have plunged the leak with benchmark generator
re-reading the benchmark list and merging it, it might be cleaner to split the metadata file on per-benchmark
basis, so that per-benchmark configuration would be normally overwritten without messing with anything else.