The emails get generated and the bot attempts to store them in the mailing list archive repo, which we are storing in a GitLab server. The PUT call returns 400, which the bot interprets as a failure and aborts the WorkItem. On the next run of that WorkItem, the emails are found in the archive, which the bot interprets as having been sent, so no new attempt is made. The issue here is that GitLab returns a failure code but still stores the data.

Using a Git repository as the database for this was never a good idea. Rewriting this to use some other data store would be a pretty major undertaking. So far this, and other instability issues with this model, has only been observed for the jdk repository, which is by far the one with the most pull requests. The file structure we currently use ('openjdk/<repo>/<pr>.mbox') creates a huge directory of mbox files. Perhaps some of these issues could be alleviated with a different file structure in the repository, which limits the number of files in a single directory.