This document describes the main features of the NICOS
(NIghtly COntrol System) tool that facilitates nightly builds of large
software projects on UNIX-like
platforms. It is currently used for
Please refer to
this page
for short instructions how to download, configure, and use the NICOS tool.
Other related documents:
The NICOS tool is a set of PERL and Bourne Shell scripts.
The NICOS tool controls the nightly builds of
software products with large number of packages
managed by different developers. It monitors
build processes on multiple platforms trying to
identify and compensate short term system problems,
analyzes the build results, allows to run tests, and
sends notifications about problems to responsible developers.
The NICOS tool uses the CVS repository to get the
code and imposes virtually no requirements on
the release build system (can be configured to work with
SRT, CMT, or SCRAM). The NICOS code consists of PERL and
Bourne Shell scripts.
The NICOS tool is able to perform build processes on different
platforms or with different compiling options in certain order.
The information about the build progress is reflected on the
nightly builds web page. There could be one
nightly release for daily rebuilds, or
several nightly releases with lifetimes.
For instance, in the ATLAS project there
are seven nightly releases with a lifetime of one
week. The ATLAS nightlies include
linked builds that
produce binaries on different Linux platforms and with
different compiling options, debug and optimized.
The linked builds have the separate web pages of build results
with links pointing to each other.
The NICOS implies the following stages of the build process:
- The code is updated from a CVS repository.
Several options can be used to determine the versions
of software packages of a project:
- use the latest CVS tag.
- use the versions defined in some database (MySQL table
or simple ASCII file). This option is useful for package
oriented release tools such as CMT.
- use an external script provided by a user.
NICOS can read the list of packages and its tags
directly from a text or xml file. This list can
be created by an external script if the tags are
stored in a database. NICOS has its own web based tag
collection tool, Version Administrator, that allows to handle
the projects with small number of packages
(demo page
is available). The Version Administrator is a PHP script
so it should be installed on web server with a PHP module.
- The project is configured with the set of commands
relevant for particular release tool (saved in the
NICOS configuration file).
- The code is compiled. It is assumed that
the release tool allows to make it with a fixed set of
commands run from one directory (true for known release tools).
- The build output is checked for errors.
The problems are identified if a certain patterns
are found (such as 'No rule to make target',
'Symbol referencing errors'). NICOS contains
the default set of such patterns, and a user can modify
it.
- QA checks are performed.
They are considered unsuccessful if non-zero value
is returned or a certain patterns (indicated in
nicos configuration file) are found.
- If desired, the packages are tested individually
with
make <check target>
or similar command.
- The integration tests are run.
In the ATLAS nightlies the
ATN tool is used for
organization and validation of integration tests.
ATN determines a success by the exit code of a test,
and it can verify that certain patterns are present
and/or absent in the output.
- The build results information is posted on the
build summary web page. This web page shows the build
and individual test results for each package as well
the integration test results in a separate table.
- Developers are notified about problems with their
packages via e-mail. The list of e-mails addresses for each
package is stored in NICOS database. Authorized
developers can modify these lists with the web interface.
Separate address list is used for the integration tests.
No automatic e-mail notifications about errors are generated
if
NICOS
determines that
there were hardware or network problems.
Send comments or questions