Feb 6, 2016

Creating new test jobs in OpenStack CI

Reviewing patches for the OpenStack CI infrastructure, there's one piece that often confuse contributors: The question how Zuul and Jenkins configuration are working together.

While we have the Infra Manual with a whole page on how to create a project - and I advise everyone to read it - , let me try to tackle the specific topic of adding new jobs from a different angle.

What we're discussing here are job, or tests, that are run. Jenkins actually runs these jobs. Zuul watches for changes in gerrit (URL for OpenStack is review.openstack.org) to trigger the appropriate jobs so that Jenkins runs them.


To understand the relationship between these two systems, let's try as an analogy programming languages: As a developer, you create a library of functions that do a variety of actions. You also write a script that uses this library to execute them. Jenkins can be considered the library of test functions. But just defining these is not enough, you have to call them. Zuul takes care of calling them, so in the analogy is your script.

So, to actually get a job running for a repository, you first need to define it in the Jenkins "library", and then you trigger its invocation in Zuul. You can also add certain conditions to limit when the job runs or whether it is voting.

If you dig deeper into Jenkins and Zuul, keep in mind that these are two different programming languages, even if both use YAML as format. Jenkins runs jobs and these are defined as text files using the Jenkins job builder. To define them, you can write a job, or use a job-template and instantiate it, or group several job-template in a job-group and instantiate that job-group to create with a few lines many jobs. Zuul uses these jobs and has as syntactic sugar templates to reuse jobs and the queues they run in.

Let's look at a simple examples, adding a new docs job to your repository called amazing-repo:

  1. Check out the project-config repository and make it ready for patch submission like creating a branch where you work on.
  2. Since for the docs job already a template exists, you can reuse it. It is called'gate-{name}-docs', so add it to your repository in file jenkins/jobs/projects.yaml:
    - project:
      name: amazing-repo
      node: bare-trusty
      jobs:
          - gate-{name}-docs
  3. Now define how to trigger the job. Edit file zuul/layout.yaml and update your repository entry to add the job:

    - name: openstack/amazing-repo
      template:
        - name: merge-check
      check:
        - gate-amazing-repo-docs
      gate:
        - gate-amazing-repo-docs

    This adds the job to both the check and gate queue. So, it will notonly be run when a patch is initially submitted for review in the check queue but also after a patch gets approved in the gate queue. Since your tree might be different when you submitted a change and when it merges, we run jobs in both situations so that the tree istested exactly as it merges.
  4. Let's go one step back: Your repository is not ready yet to havethe docs job voting, so you only want to run it as non-voting.
    In that case add a condition in the jobs section of zuul/layout.yaml:

    - name: gate-amazing-repo-docs
        voting: false


    And in your repository, only add it to the check queue. Non-voting jobs should not be in the gate, they get ignored completely and just waste resources:

      - name: openstack/amazing-repo
        template:
          - name: merge-check
        check:
          - gate-amazing-repo-docs
        gate:
          - ...

So, these are simple jobs. Stay tuned for a followup article that will cover how to use templates in Zuul - and how to modify your repository in the context of templates.

Thanks to Doug Fish for reviewing the text and giving suggestions on how to improve it - and urging me to write a follow-up.

P.S. Follow-up article is called "Templates in OpenStack's Zuul".