service accounts and push-type models?

How do we allow the sync to have access to private projects on gitlab? I assume there's a service account associated that we can add as a reporter to give read-access to the project's gitlab registry.

Additionally, is there a way to force a sync whenever a new image is pushed to the gitlab registry?

Hi Giordon,

Since unpacked.cern.ch is itself publicly available, we didn't implement authenticated access to registries in the image converter. We can reconsider if needed. Just to understand, is your use case that you'd like to keep the code private but the container build product public?

As for updates, unpacked.cern.ch automatically finds out when then an upstream image changes and updates the content on cvmfs accordingly. If you'd like to convert all or a subset of tags for an image, you can use a wildcard specification (like imagename:*).

@smosciat Please feel free to add/correct if needed.

Cheers, Jakob

Hi Giordon @gstark ,

in theory the service account cvmfsunpacker can be added, and this should allow to have access to private images. This is not how usually the repository operates, so we can expect some hiccup.

As Jakob was mentioning, the images will be public when they are published. So everybody will be able to see what is inside.

There is no way to programmatically issue a "sync operation", however one is done every 10 minutes, then it takes some time to actually publish the final image.

If you push a new version of an image already inside the repository we update also the one in the repository, the old one get deleted after a grace period of 30 days.

I am not sure I was able to express myself clearly, feel free to come back me!

Cheers, Simone

So if code is private, that means the registry is private as well. I'd like to be able to add a private gitlab registry with :* to sync up all tags as well -- and I guess the 10minute sync means that new tags pushed to the registry will get picked up within 10 minutes.

I would like to use the cvmfsunpacker and add it as a service account to the private registries and give it read-only access so it can grab the images and make them public. The alternative is to additionally push to docker hub and sync from docker hub rather than the gitlab registry.

The 10 minutes means that the code runs every 10 minutes. If there is a big influx of images, or the images are particularly big, it can take more time.

Again, it might not work, and if doesn't work we will fix it.

I suggest that you can go ahead and add the user to your registry and the images to the wish list.

I can follow up for a while, here, or live.

How many images are we talking about? BTW

10 or 20 backlogged, but we only explicitly tag an image once every few months. The problem is that we also have non-tagged images for each branch and that's probably a lot more. I'm not sure if there's a way to add a pattern-matching expression or not... but maybe I'll just submit MRs manually by hand for now. I'll try adding the service account.

Pattern matching is allowed, with the * acting like the one in the bash shell.

For instance you could do something like:

image:v* to match image:v0.0.1 and image:v1.0.2, but not image:branch-features-whatever.

So it kinda depends on your structure.

(Underneath there is a regex engine that convert * to .* and it support all the standard regex features, like + or [a-zA-Z], but please avoid to rely on this knowledge)

mentioned in merge request !37 (merged)

closed