Uploaded image for project: 'Skara'
  1. Skara
  2. SKARA-1030

Lots of "uncleanable" repos found in scratch dirs

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: P3
    • Resolution: Fixed
    • Affects Version/s: 0.9
    • Fix Version/s: 0.9
    • Component/s: bots
    • Labels:

      Description

      After fixing SKARA-1012, a new similar problem has appeared. Now we frequently get "uncleanable" repos in the bot scratch dirs. The error messages look like this:

      Invalid local repository detected (uncleanable) - preserved in: /home/bots/scratch/scratch-26/mlbridge-mergebase/openjdk/jfx-uncleanable-859b698e-e593-4621-ad94-5de33032ba9c

      The failing git command is "git reset --hard" which fails with:

      fatal: Unable to create '/home/bots/scratch/scratch-58/mlbridge-mergebase/openjdk/panama-foreign-uncleanable-6863adc2-f779-4910-9e5c-19e56ebc881d/.git/index.lock': File exists.

      Tracing back logs when a repo like this was last used typically finds the initial clone of the repo, which is then interrupted by a bot runner restart. My theory here is that since we added --dissociate, cloning takes much longer, so the probability of getting interrupted by a restart goes up quite a bit.

      One way of mitigating this would be to always clone to a different dir name (add .tmp or something), then move the new repo to the final name. By doing that we know that any existing dir with a .tmp suffix is known to be unfinished and can just be deleted, with no need to send any alarms or preserve the contents.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              erikj Erik Joelsson
              Reporter:
              erikj Erik Joelsson
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: