Now we have one new tool with a few different options for fine-grained control. It is run as part of the gates and can be run locally as well. If you used
tools/validate.pyto test your changes locally, you should now use
Instead, of a single gate job, there are now four checking gates, you'll see now something like the following from Jenkins:
Build succeeded. gate-openstack-manuals-validate-niceness SUCCESS in 4s (non-voting) gate-openstack-manuals-validate-syntax SUCCESS in 2s gate-openstack-manuals-validate-deletions SUCCESS in 4s gate-openstack-manuals-validate-build SUCCESS in 8m 47s
The advantage of all of these changes are that a writer submitting a patch gets quickly an overview about what fails. This is due to the separate jobs. These jobs only check what has changed and thus do not build all books but only those impacted by the change under review.
Description of ChecksNote that all checks parse the git output to record the modified files in a review and only look at these files. If you want to run checks locally on all files, use the
Niceness checkThis tests for extra whitespace. It is non-voting for now and thus not run as part of the final gate.
Syntax checkThis check validates the docbook XML code for each file against DocBook 5.1 (actually candidate release 1 - 5.1CR1).
Deletions checkThis check tests whether any files have been removed and if those are still referenced anywhere. It handles images and include files.
Build CheckThis check will build all books that might be modified by the commit. It checks which files are modified and which books include these files and runs a check of all books where one file was changed. The books are build in parallel to speed up building.
For the Install Guide, it builds all three variants of the Guide - the Fedora/RHEL/Centos Install Guide, the Ubuntu Install Guide and the openSUSE Install Guide. It now also builds the High Availability Guide (which is not using DocBook but asciidoc) if one of its files has changed.Thus, everything is build the same way as we do for publishing the manuals to the OpenStack docs site.
Since we only build the modified books and build them in parallel, you might get now a comment from Jenkins in a minute or two while you had to work 30 minutes in August for this. Jenkins currently needs around 10 minutes to build all books.
OptionsThe tool is
tools/test.pyand has the following options:
tools/test.py --help usage: test.py [-h] [--force] [--check-build] [--check-syntax] [--check-deletions] [--check-niceness] [--check-all] [--non-voting] [--verbose] [path] Validate XML files against the DocBook 5 RELAX NG schema positional arguments: path Root directory that contains DocBook files, defaults to `git rev-parse --show-toplevel`/doc optional arguments: -h, --help show this help message and exit --force Force the validation of all files and build all books --check-build Try to build books using modified files --check-syntax Check the syntax of modified files --check-deletions Check that deleted files are not used. --check-niceness Check the niceness of files, for example whitespace. --check-all Run all checks (default if no arguments are given) --non-voting Do not exit on failures --verbose Verbose execution
I'd like to thank Christian Berendt for starting this with the creation of test.py (as a modified copy of validate.py) and all the reviewers that helped moving this forward.