README.md 20.1 KB
Newer Older
Alex Iribarren's avatar
Alex Iribarren committed
1
2
# CI integration for building RPMs

3
This repo exists to help you build RPMs using Koji and Gitlab's CI infrastructure. You have here an [example of how to use it](https://gitlab.cern.ch/linuxsupport/myrpm).
Alex Iribarren's avatar
Alex Iribarren committed
4
5
6
7


## Prerequirements:

Alex Iribarren's avatar
Alex Iribarren committed
8
9
1. A [Koji tag](https://cern.service-now.com/service-portal?id=sc_cat_item&name=Koji-tag&fe=IT-RPM) where your packages will end up.
2. A [service account](https://account.cern.ch/account/Management/NewAccount.aspx) with [access to Koji](https://cern.service-now.com/service-portal?id=sc_cat_item&name=request&fe=IT-RPM), and in particular to your Koji tag. We'll use the account `kojici` as an example, but you must have your own. This account can be used for as many RPMs as you want.
Alex Iribarren's avatar
Alex Iribarren committed
10
3. A [Gitlab](https://gitlab.cern.ch/) repository with Pipelines (ie. continuous integration) enabled. You can enable Pipelines by going to *General Settings* for your repo and toggling the *Pipelines* feature under *Visibility, project features, permissions*. More info in [KB0003690](https://cern.service-now.com/service-portal?id=kb_article&n=KB0003690).
Alex Iribarren's avatar
Alex Iribarren committed
11
4. Source code with a `Makefile` (or a `Makefile.koji`, which will take precedence if both exist) that defines two mandatory targets, `srpm` and `rpm`. Those targets should build at least one source RPM and an RPM, respectively. Check our [RPM starter](https://gitlab.cern.ch/linuxsupport/myrpm) repo for a working example.
Alex Iribarren's avatar
Alex Iribarren committed
12

Alex Iribarren's avatar
Alex Iribarren committed
13
Note: RPMCI uses [interruptible](https://docs.gitlab.com/ee/ci/yaml/#interruptible) jobs in order to stop the execution of a pipeline if a newer pipeline has made it redundant. In order to take advantage of this feature, [Auto-cancel redundant pipelines](https://docs.gitlab.com/ee/ci/pipelines/settings.html#auto-cancel-redundant-pipelines) has to be enabled for your project.
Alex Iribarren's avatar
Alex Iribarren committed
14
15
16

## Procedure:

Alex Iribarren's avatar
Alex Iribarren committed
17
1. Create your new empty repo. *Protip: Create a subgroup for all the RPMs of your service/group, and create your new repo inside it.*
Alex Iribarren's avatar
Alex Iribarren committed
18
2. Copy **all** the files from our [RPM starter](https://gitlab.cern.ch/linuxsupport/myrpm) repo.
19
3. Edit the variables in `.gitlab-ci.yml` (see below) and fill in the name of your service account, your Koji tag, etc. **Don't modify the include!**
20
4. In Gitlab, go to *Settings > CI / CD > Variables*. Add two new variables called `KOJICI_USER` and `KOJICI_PWD` and set them to the username and password of your service account. `KOJICI_PWD` should be set as a [masked variable](https://gitlab.cern.ch/help/ci/variables/README#mask-a-custom-variable) as an extra precaution but do note that it [must comply with some requirements](https://gitlab.cern.ch/help/ci/variables/README#masked-variable-requirements). *Protip: If you followed the protip in #1, you can set these variables at the subgroup level and all repos within it will be able to use them automatically!*
21
5. From *lxplus* or *aiadm*, manually add your package to your Koji tag. For example:
22
23
24
25
26
    ```bash
    ~ > koji add-pkg --owner=$USER mytag7-testing myrpm
    ~ > koji add-pkg --owner=$USER mytag7-qa myrpm
    ~ > koji add-pkg --owner=$USER mytag7-stable myrpm
    ```
Alex Iribarren's avatar
Alex Iribarren committed
27
   Note that you **don't need to add your package to your `-build` tag**, and it may in fact break your builds if you do so.
Alex Iribarren's avatar
Alex Iribarren committed
28
6. Place all your code in the `src/` directory. Don't forget to update the version number and the changelog in your spec file!
Alex Iribarren's avatar
Alex Iribarren committed
29
7. Commit all your changes to your new repo.
Alex Iribarren's avatar
Alex Iribarren committed
30
8. Treat yourself to a coffee while Gitlab runs your new pipelines! A Koji scratch (ie. temporary) build will be created, allowing you to test if your build succeeds or not. If your build takes more than an hour, you may need to [increase the timeout](#ci-jobs-timing-out) for your Gitlab project.
Alex Iribarren's avatar
Alex Iribarren committed
31
9. Once you're satisfied with your RPM and the Gitlab CI pipelines are passing, you can [create a tag](https://git-scm.com/book/en/v2/Git-Basics-Tagging) in your git repo. When you create a tag, Gitlab will run the tests again and, if they pass, will then launch a (non-scratch, ie. permanent) build job in Koji. *Protip: use the version of the new RPM as the tag name, future you will thank you. In any case, RPMCI will [add a VCS](#vcs-tag-in-spec-files) tag to your spec file automatically.*
Alex Iribarren's avatar
Alex Iribarren committed
32
10. When that build succeeds, you'll be able to manually promote it to QA of your Koji tag, and from there to Stable.
Alex Iribarren's avatar
Alex Iribarren committed
33
34
35
36


### Available Configuration Variables:

37
Here are the basic configuration options you can set in your `.gitlab-ci.yml` file, and their defaults:
Alex Iribarren's avatar
Alex Iribarren committed
38

39
```yaml
40
41
  KOJI_TAG: ''
  BUILD_8s: 'False'
Alex Iribarren's avatar
Alex Iribarren committed
42
  BUILD_8: 'False'
Alex Iribarren's avatar
Alex Iribarren committed
43
  BUILD_7: 'False'
44
  DIST_8s: '.el8s'
Alex Iribarren's avatar
Alex Iribarren committed
45
46
  DIST_8: '.el8'
  DIST_7: '.el7'
Alex Iribarren's avatar
Alex Iribarren committed
47
48
```

Alex Iribarren's avatar
Alex Iribarren committed
49
50
51
If your Koji tag is configured with a custom disttag (such as `.el7.cern`), make
sure you add it to the appropriate `DIST_*` variable.

52
Your Koji tag will usually be configured in the `KOJI_TAG` variable and it will automatically
53
get the OS' release appended to it (ie. `KOJI_TAG: config` will become `config8`, etc.).
54
If you need to configure a different tag for a particular OS, you can do so using the `KOJI_TAG_*`
55
variables, but you will have to specify the OS release yourself (ie. `KOJI_TAG_7: ai7`):
56

57
```yaml
58
  KOJI_TAG: 'mytag'
59
60
  KOJI_TAG_7: 'myoldtag7'
  BUILD_8s: 'True'
61
62
63
  BUILD_8: 'True'
  BUILD_7: 'True'
```
64

Alex Iribarren's avatar
Alex Iribarren committed
65
66
## Convenience Functions

67
In order to simplify common operations, RPMCI includes a series of [convenience functions](functions/)
Alex Iribarren's avatar
Alex Iribarren committed
68
69
70
71
72
73
which are built into the builder (and kojicli) docker images. These functions can
be executed from your gitlab-ci scripts and allow you to easily perform common
operations such as adding your Koji tag as a repo without having to worry about internal
RPMCI variables and details. This also allows you to future-proof your scripts in case
new versions of RPMCI change implementation details.

74
Here is a partial list of the available functions, please check [functions/](functions/) for the full list:
Alex Iribarren's avatar
Alex Iribarren committed
75
76
77
78
79
80
81
82
83
84
85
86

* `rpmci_add_repo_kojitag_stable`: add your stable Koji tag as a repo. If you add a parameter
  (ie: `rpmci_add_repo_kojitag_stable config8`), then it adds the stable repo for the specified tag.
* `rpmci_add_repo <name> <url>`: add a custom repo.
* `rpmci_is_OS7`, `rpmci_is_OS8`, etc: test the OS a particular job is running on.
* `rpmci_install_builddeps`: install the build dependencies specified in the spec file.
* `rpmci_install_koji_rpms`: install the RPMs built by Koji (for the testing stages).
* `get_kojitag`: get your full configured Koji tag name in a running job.
* `echo_error`: print a message in bold red.
* `echo_info`: print a message in green.
* `echo_command`: print a message in bold green.

Alex Iribarren's avatar
Alex Iribarren committed
87
88
Note that these convenience functions are _only_ available in the RPMCI docker images, `builder-*` and `kojicli`.

Alex Iribarren's avatar
Alex Iribarren committed
89
90
91
92
93
94
95
96
97
## Building for a new OS

If you are already using RPMCI and want to add a build for a new OS, here's
a simple procedure you can follow:

1. Verify if you already have Koji tags for the new OS, or
[request them](https://cern.service-now.com/service-portal?id=sc_cat_item&name=Koji-tag&fe=IT-RPM) if necessary.
2. Add your package to your new tags.
3. Update your `.gitlab-ci.yml` to enable builds (via the `BUILD_*` variables)
Alex Iribarren's avatar
Alex Iribarren committed
98
99
for the new OS. Don't forget to adjust the `DIST_*` variables as necessary.
Commit and push [your changes](https://gitlab.cern.ch/linuxsupport/rpms/cern-config-users/-/commit/fca61bc1e082fdd752eabb85832a09cdc53439c8).
Alex Iribarren's avatar
Alex Iribarren committed
100
101
102
103
104
105
4. Now is a good time to stop builds for deprecated OSes. You can remove `BUILD_6`
(and any other `*_6` variables) to stop builds for SLC6, which may stop working at any time.
5. Now is also a great time to [add some tests](#adding-your-own-tests) to your pipeline!
6. If there haven't been any code changes to your actual RPM since the last release,
you can rebuild it for the new OS:
   1. Find the git tag you created for your last release: `git tag -l -n1`
Alex Iribarren's avatar
Alex Iribarren committed
106
   2. Create a new tag with the same name but append something to it, like `.cs8`: `git tag v2.45-1.cs8`
Alex Iribarren's avatar
Alex Iribarren committed
107
   3. Push your new tag to Gitlab.
Alex Iribarren's avatar
Alex Iribarren committed
108
109
110
   4. RPMCI will start the new release process. Since the RPM version hasn't changed, the
   `Koji_build` steps for your previous OSes [will all fail](https://gitlab.cern.ch/linuxsupport/rpms/cern-config-users/-/pipelines/2352412)
   because the build already exists, but you can ignore them.
Alex Iribarren's avatar
Alex Iribarren committed
111
112
113
114
115
116
117
118
   5. Once the `Koji_build` step for the new OS succeeds, you can now test and tag
   your build as normal.
   6. You will now have the same version on all OSes without having to artificially bump the release.
1. If you did make code changes to your RPM, you will have to create a new release for all OSes:
   1. Update you spec file to increase the RPM version
   2. git tag the new release, RPMCI will build it as usual.
   3. Continue with the usual procedure.

Alex Iribarren's avatar
Alex Iribarren committed
119

Alex Iribarren's avatar
Alex Iribarren committed
120
121
122
## Customizations

### Adding your own tests
Alex Iribarren's avatar
Alex Iribarren committed
123

Alex Iribarren's avatar
Alex Iribarren committed
124
125
126
127
This CI pipeline includes a `test` stage that is run after building the scratch RPM
on Koji. There's a "base" test job called `.test_install` that installs your RPMs
in the `before_script` section. This job is then extended by OS-specific jobs
(`test_install6`, `test_install7` or `test_install8`), which are the ones actually executed.
Alex Iribarren's avatar
Alex Iribarren committed
128

129
If you want to run extra actions (functional tests, etc.) once your RPMs are installed,
Alex Iribarren's avatar
Alex Iribarren committed
130
131
132
133
134
135
136
137
138
139
140
141
142
you can do so in the `script` section:

```yaml
.test_install:
  script:
    - /usr/bin/client --selftest
    - printf "[config]\nserver = https://myserver.cern.ch\n" > /etc/client.conf
    - /usr/bin/client --debug login
```

You can add any necessary dependencies in the `before_script` section, but you will then have to
also include a command to install your RPMs. You can modify the `.test_install` job directly to
affect all OSes, or modify only the OS-specific jobs.
Alex Iribarren's avatar
Alex Iribarren committed
143
144
145

You can also add your own tests by adding another job that extends one of
the OS-specific jobs.
Alex Iribarren's avatar
Alex Iribarren committed
146

Alex Iribarren's avatar
Alex Iribarren committed
147
Here are a few examples:
148

Alex Iribarren's avatar
Alex Iribarren committed
149
### Adding a dependency for all OSes and tests
150
151

```yaml
Alex Iribarren's avatar
Alex Iribarren committed
152
.test_install:
153
  before_script:
154
    - yum install -y --nogpgcheck --enablerepo=cernonly oracle-instantclient-tnsnames.ora
Alex Iribarren's avatar
Alex Iribarren committed
155
    - rpmci_install_koji_rpms
Alex Iribarren's avatar
Alex Iribarren committed
156
```
157

158
### Adding your own Koji tag's repo (for extra dependencies)
Alex Iribarren's avatar
Alex Iribarren committed
159

Alex Iribarren's avatar
Alex Iribarren committed
160
161
162
This makes use of the `rpmci_add_repo_kojitag_stable` convenience function, which
will add your Koji tag's stable repo. You will still need to install your Koji-built
RPMs, and for that you can use the `rpmci_install_koji_rpms` function.
Alex Iribarren's avatar
Alex Iribarren committed
163
164
165

```yaml
.test_install:
166
  before_script:
Alex Iribarren's avatar
Alex Iribarren committed
167
168
    - rpmci_add_repo_kojitag_stable
    - rpmci_install_koji_rpms
169
```
Alex Iribarren's avatar
Alex Iribarren committed
170
171
172
173

### New job with extra tests

```yaml
Alex Iribarren's avatar
Alex Iribarren committed
174
175
test_nose7:
  extends: test_install7
176
  script:
Alex Iribarren's avatar
Alex Iribarren committed
177
178
179
180
181
182
183
    # These deps are only needed to run the unit tests
    - yum install --nogpgcheck -y python-nose python2-mock python-suds
    - cd t
    - nosetests

test_nose6:
  extends: test_install6
184
185
  before_script:
    - yum install -y python2-future
Alex Iribarren's avatar
Alex Iribarren committed
186
    - rpmci_install_koji_rpms
187
  script:
Alex Iribarren's avatar
Alex Iribarren committed
188
189
190
191
192
193
194
195
196
197
198
    # These deps are only needed to run the unit tests
    - yum install --nogpgcheck -y python-nose python2-mock python-suds
    - cd t
    - nosetests
```

### Replacing a particular OS' test

You may not want to install all the RPMs that you've built:

```yaml
Alex Iribarren's avatar
Alex Iribarren committed
199
test_install7:
200
  before_script:
Alex Iribarren's avatar
Alex Iribarren committed
201
    - yum install -y --nogpgcheck koji/myrpm-client*.rpm
Alex Iribarren's avatar
Alex Iribarren committed
202
```
Alex Iribarren's avatar
Alex Iribarren committed
203

Alex Iribarren's avatar
Alex Iribarren committed
204
205
206
207
208
209
210
211
212
### Configuring rpmlint

You can create a custom `.rpmlint` configuration file to disable some tests:

```
addFilter("E: hardcoded-library-path")
addFilter("myrpm.spec:6: W: non-standard-group")
```

213
214
215
216
217
218
### Ignoring rpmlint

If you don't care about rpmlint results, you can turn it into a manual job
so it's not run automatically:

```yaml
Alex Iribarren's avatar
Alex Iribarren committed
219
.test_rpmlint:
220
221
222
223
224
225
  when: manual
```

You can also let it run, but not have it stop the pipeline if there are errors:

```yaml
Alex Iribarren's avatar
Alex Iribarren committed
226
.test_rpmlint:
227
228
229
  allow_failure: true
```

Alex Iribarren's avatar
Alex Iribarren committed
230
231
232
233
234
235
236
237
### Adding build dependencies

If your local builds (`srpm` and `rpm` stages) require dependencies from other repos,
such as from your Koji tag, you can add them for all OSes as follows:

```yaml
.rpm_deps:
  before_script:
Alex Iribarren's avatar
Alex Iribarren committed
238
239
    - rpmci_add_repo_kojitag_stable
    - rpmci_install_builddeps
Alex Iribarren's avatar
Alex Iribarren committed
240
241
242
243
244
245
246
```

You can also add different dependencies per OS, like this:

```yaml
.rpm_deps8:
  before_script:
Alex Iribarren's avatar
Alex Iribarren committed
247
    - rpmci_add_repo_kojitag_stable "dbclients8"
Alex Iribarren's avatar
Alex Iribarren committed
248
    - yum install -y oracle-instantclient
Alex Iribarren's avatar
Alex Iribarren committed
249
    - rpmci_install_builddeps
Alex Iribarren's avatar
Alex Iribarren committed
250

251
252
253
254
255
256
.rpm_deps8s:
  before_script:
    - rpmci_add_repo_kojitag_stable "dbclients8s"
    - yum install -y oracle-instantclient
    - rpmci_install_builddeps

Alex Iribarren's avatar
Alex Iribarren committed
257
258
259
.rpm_deps7:
  before_script:
    - yum install -y --enablerepo=cernonly oracle-instantclient
Alex Iribarren's avatar
Alex Iribarren committed
260
    - rpmci_install_builddeps
Alex Iribarren's avatar
Alex Iribarren committed
261
262
```

Alex Iribarren's avatar
Alex Iribarren committed
263
You may also perform conditional actions depending on the OS by using the `rpmci_is_*` convenience functions:
Alex Iribarren's avatar
Alex Iribarren committed
264
265
266
267
268

```yaml
.rpm_deps:
  before_script:
    - |
Alex Iribarren's avatar
Alex Iribarren committed
269
270
      if rpmci_is_OS8; then
        rpmci_add_repo_kojitag "dbclients8-stable"
Alex Iribarren's avatar
Alex Iribarren committed
271
      fi
Alex Iribarren's avatar
Alex Iribarren committed
272
    - rpmci_install_builddeps
Alex Iribarren's avatar
Alex Iribarren committed
273
274
```

Alex Iribarren's avatar
Alex Iribarren committed
275
Don't forget to include the `rpmci_install_builddeps` line in your `before_script`, otherwise
Alex Iribarren's avatar
Alex Iribarren committed
276
no dependencies will actually be installed!
277

Alex Iribarren's avatar
Alex Iribarren committed
278
### Adding your own CI stages
Alex Iribarren's avatar
Alex Iribarren committed
279

Alex Iribarren's avatar
Alex Iribarren committed
280
There are some unused stages (`prebuild`, `prekoji`, `pretest`, `posttest`, `postkoji`,
Alex Iribarren's avatar
Alex Iribarren committed
281
`docker_prepare`, `docker_build` and `docker_test`) which you can use to define your own custom jobs.
Alex Iribarren's avatar
Alex Iribarren committed
282
283
284

If these are insufficient, or you wish to add/modify/reorder the stages of the CI process,
you will have to redefine the ones included in [rpm-ci.yml](https://gitlab.cern.ch/linuxsupport/rpmci/blob/master/rpm-ci.yml)
Alex Iribarren's avatar
Alex Iribarren committed
285
and add your own in your own `.gitlab-ci.yml` file. Note, however, that there are
Alex Iribarren's avatar
Alex Iribarren committed
286
287
`dependencies` and `needs` clauses between some jobs that you may need to modify.

288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
### Skipping the Gitlab RPM build step

RPMCI builds the source and the binary RPM, but only the source RPM is really needed for
building in Koji later. The binary RPM is built in Gitlab as a sanity check and to prevent
broken RPMs from being submitted to Koji. There are situations (such as upstream packages
that take a really long time to build) where this step is overkill, so it can be disabled:

```yaml
.build_rpm:
  when: manual
```

The Gitlab pipelines will still contain jobs in the `rpm` stage and you can run them
manually if you want, but they will not be run automatically and they won't delay the
rest of the pipeline.

304
305
306
307
308
309
310
311
312
313
314
315
### Skipping the download of Koji build artifacts

If your packages are very large and you don't want them added to the Gitlab pipeline,
you can tell RPMCI to not download them from Koji:

```yaml
variables:
  SKIP_KOJI_DOWNLOAD: 'True'
```

You will always be able to download them directly from Koji and you'll have direct links
to do so as part of the "Koji build summary" at the end of the job.
Alex Iribarren's avatar
Alex Iribarren committed
316

317
318
319
Note that disabling Koji downloads will also disable the check to see if the DIST tag is correct
and the testing stages (as there are no packages available to test).

320
321
## Tips and Tricks

Alex Iribarren's avatar
Alex Iribarren committed
322
323
324
325
326
327
328
329
330
331
332
### Error: No source RPMs found

If you get this error, it probably means your Makefile does not conform to what
RPMCI requires or expects. The [RPM starter](https://gitlab.cern.ch/linuxsupport/myrpm)
repo provides a good [Makefile](https://gitlab.cern.ch/linuxsupport/myrpm/-/blob/master/Makefile) to follow.

In particular, make sure the [DIST](https://gitlab.cern.ch/linuxsupport/myrpm/-/blob/master/Makefile#L6) variable
is conditionally assigned (it will normally be assigned by RPMCI) and that this
variable is [passed to rpmbuild](https://gitlab.cern.ch/linuxsupport/myrpm/-/blob/master/Makefile#L18).
Make sure that `_topdir` is also set correctly.

333
334
335
336
337
338
339
### Debugging builds locally

The best way to debug your builds locally is to run them within the same Docker images RPMCI uses. You can do something like this:

```bash
~ > git clone https://:@gitlab.cern.ch:8443/linuxsupport/myrpm.git
...
340
~ > docker run --rm -it -v ~/myrpm:/build:z gitlab-registry.cern.ch/linuxsupport/rpmci/builder-c8:latest
341
342
343
344
345
346
347
348
349
...
[root@99369f9e7a22 /]# cd /build/
[root@99369f9e7a22 build]# yum-builddep -y *.spec
...
[root@99369f9e7a22 build]# make rpm
...
```

If it builds here, it will work in Gitlab CI and it has a high chance of success in Koji.
Alex Iribarren's avatar
Alex Iribarren committed
350
351
352
353
354
355

### CI jobs timing out

Gitlab CI's default timeout for jobs is 1 hour. If one of your jobs takes longer than that,
it will be terminated and the pipeline won't continue. You can change this timeout for your
project in `Settings` -> `CI / CD` -> `General pipelines` -> `Timeout`.
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372

### Building Go binaries on CC7

If you're building golang binaries on CC7 and you want the EPEL golang, you'll have to do the following:

```yaml
build_srpm7:
  before_script:
    - yum-builddep -y --disableplugin=protectbase,priorities *.spec

build_rpm7:
  before_script:
    - yum-builddep -y --disableplugin=protectbase,priorities *.spec
```

Your spec file should also contain a `BuildRequires: golang`.

Alex Iribarren's avatar
Alex Iribarren committed
373
Lastly, the `git` version available on CC7 can produce errors like the following while fetching Go dependencies:
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389

```
fatal: git fetch-pack: expected shallow list
```

One solution to this issue is to use `rh-git29` software collection. To do that, you should define a `before_script` similar to the following one:

```
  before_script:
    (...)
    - yum-config-manager --add-repo "http://linuxsoft.cern.ch/cern/centos/7/sclo/x86_64/rh"
    - yum install -y --nogpgcheck rh-git29
    - export PATH=/opt/rh/rh-git29/root/usr/bin:$PATH
    - export LD_LIBRARY_PATH=/opt/rh/httpd24/root/usr/lib64:$LD_LIBRARY_PATH
    - export MANPATH=/opt/rh/rh-git29/root/usr/share/man:$MANPATH
    - export PERL5LIB=/opt/rh/rh-git29/root/usr/share/perl5/vendor_perl
Alex Iribarren's avatar
Alex Iribarren committed
390
    - rpmci_install_builddeps
391
```
Alex Iribarren's avatar
Alex Iribarren committed
392
393

This could be simplified by calling `source /bin/scl_source enable rh-git29`, but unfortunately it seems to fail.
394

395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
### Building Go packages that have external dependencies

Fetching external dependencies will fail in Koji builds as the builders don't have internet access. Instead, you should bundle all dependencies in the srpm.

You can do that easily with the `go mod vendor` command which creates a `vendor` directory with a copy of all dependencies. You can choose to include these with the project or only download them when creating the srpm.

To do the second, you simply need to run the command when creating the .tgz archive in your `Makefile`. For example:

```
sources: vendor
  go mod vendor
	@tar cvzf $(TARFILE) --exclude-vcs --transform 's,^,$(PKGNAME)/,' src vendor
```

Then, when `go build` is executed, it will find and use the vendored versions of the dependencies.

Nikolaos Filippakis's avatar
Nikolaos Filippakis committed
411
You can also add the following to `.gitlab-ci.yml` to download the dependencies once rather than in each stage.
412
413
414
415
416
417
418
419
420
421
422

```yaml
.rpm_deps:
  before_script:
    - rpmci_install_builddeps
    - go mod vendor
  cache:
    paths:
      - vendor
```

423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
### Building Maven packages

Normal Maven builds will not work in Koji because Koji builders don't
have internet access. The solution to this is to cache all the dependencies
before sending the job to Koji.

Edit your `Makefile` to download all the dependencies to the `.m2` directory and add it to the package:

```Makefile
sources:
    mvn -Dmaven.repo.local="$(PWD)/.m2" surefire:help dependency:go-offline
    tar -zcvf $(TARFILE) --transform 's,^,$(SPECFILE_NAME)-$(SPECFILE_VERSION)/,' src .m2 pom.xml
```

Edit your spec file to add `maven-local` and build from `.m2`:

```
...
BuildRequires:  maven-local
...

%setup -q
%mvn_file : %{name}/%{name} %{name}

%build
%mvn_build -f -j -- -Dmaven.repo.local=".m2"

%install
%mvn_install
...
```

You can find a full example of a Maven build [here](https://gitlab.cern.ch/ComputerSecurity/cert-flume-extra/-/tree/master).

457
458
### VCS tag in Spec files

Alex Iribarren's avatar
Alex Iribarren committed
459
RPMCI will automatically add a `VCS` tag to your spec file that
460
461
462
463
464
465
466
467
468
469
points to your gitlab repo at the commit hash the RPM is built from. This gives
you another way of tracing the source code that was used to build your package.

You can see the value of this tag on a built package like this:

```bash
~ > rpm -qp --qf "%{VCS}\n" ~/myrpm-1.1-6.el8.x86_64.rpm
https://gitlab.cern.ch/linuxsupport/myrpm/-/tree/60f909134d2ec5895cec23eabc75084502d091d7
```

Alex Iribarren's avatar
Alex Iribarren committed
470
471
You will also find the `VCS` tag in Koji by looking at a information page for a [build](https://koji.cern.ch/buildinfo?buildID=47852)
or an [RPM](https://koji.cern.ch/rpminfo?rpmID=287030).
472

473
474
475
476
477
478
By default, the `VCS` tag is `"${CI_PROJECT_URL}/-/tree/${CI_COMMIT_SHA}"`, but you can
customize this by supplying your own format in the `_VCS_TAG` variable.

The `VCS` tag won't be added if one already exists in the spec file. You can also
stop adding this tag altogether by setting the variable `ADD_VCS_TAG` to `False`.

479
480
481
## Getting Help

If you run into issues on how to use RPMCI, you can contact us on the [Linux Mattermost channel](https://mattermost.web.cern.ch/it-dep/channels/linux)
Alex Iribarren's avatar
Alex Iribarren committed
482
or by opening a [ServiceNOW ticket](https://cern.service-now.com/service-portal?id=sc_cat_item&name=incident&fe=IT-RPM).