Uploaded image for project: 'Skara'
  1. Skara
  2. SKARA-1529

Optimize Census namespace instantiation

    XMLWordPrintable

Details

    • Enhancement
    • Resolution: Fixed
    • P3
    • 1.0
    • None
    • libraries
    • None

    Description

      The Skara bots are not always very responsive. I have started investigating various reasons for this. Instantiating and initializing certain common data structures very often is a pretty big source of unnecessary work. One of those is the Census, which is used in many bots. This is stored in a Git repository that we have to clone/fetch into a scratch dir before reading all the files from disk. When every WorkItem needs to do this, and there are many of them scheduled, it adds a lot of execution time, which delays other WorkItems.

      In most bots, there isn't anything we can easily do about this, but in certain cases, only a subset of the Census data is needed, and this subset could be instantiated directly from the HostedRepository by reading a small number of files. This is much faster than managing a local clone of the complete repository, and can also leverage the caching mechanisms we have for REST calls. The simplest such case is the IssueNotifier in the notify bot, which only needs a single namespace. There is also a non public bot where this applies, and that is where the massive performance gains are realized from this change. I doubt we will see much of a difference in the notifier. Even so, I will include updating the notifier to use the new mechanism to demonstrate that it works.

      In order to add a factory method for the Census Namespace from a HostedRepository, we needed to first clean up the module dependency graph a bit to avoid circular dependencies between the modules 'forge' and 'census'. Fortunately, this was pretty easily done. Only one method needed to move. All the rest of the offending dependencies weren't in use.

      Attachments

        Issue Links

          Activity

            People

              erikj Erik Joelsson
              erikj Erik Joelsson
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: