The mailinglist bridge bot currently polls the mailing list archives quite frequently, which may be causing issues for the hosting server. We need to look into ways of reducing the polling frequency.

We already use e-tag to avoid requesting full responses when nothing has changed. We also traverse the mboxes month by month starting with the latest. Once we find one that returns 304 (indicating that nothing has changed) we stop going further back and just use cached results. This scheme is clever, but misses one crucial situation, when no mails exist in a certain mbox. In that case we get 404 back, and this response is not cached.

Caching 404 is trickier as we will need to be aware of how old the particular archive is and define a cutoff for when 404 should not be retried anymore.

I also think we need to investigate more exactly how frequent calls to the server are in normal operation.