News for hackers. Stuff that matters.
Re:GitLab 8.10 Released with Wildcard Branch Protection and ... by dominotw on Friday July 22, 2016 @04:25PM (#12146507)
hey sytse, how does gitlab ci handle caching of maven,ivy, gem s...artifacts? Does each new build have to download it ?

I see some issues discussions, workarounds and seems like it does allow -cache directories but it is unclear how it all works.

for us( unfortunately) its 40 min vs 5 min build times, with/without caches.

Re:GitLab 8.10 Released with Wildcard Branch Protection and ... by sytse on Friday July 22, 2016 @04:46PM (#12146641)
I think there are a couple of ways to cache:

1. Build artifacts (recommended) http://docs.gitlab.com/ce/ci/build_artifacts/README.html

2. Cross Runner Caching (not implemented yet) https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/issues/...

3. Single Runner Caching https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/blob/ma... Edit: a better explanation is in http://docs.gitlab.com/ce/ci/yaml/README.html#cache

4. Container registry (not possible for Maven, etc.) https://about.gitlab.com/2016/05/23/gitlab-container-registr...

I agree it is confusing and my knowledge might not be accurate. I've asked CI experts to improve the documentation in https://gitlab.com/gitlab-org/gitlab-ce/issues/20155

Re:GitLab 8.10 Released with Wildcard Branch Protection and ... by Snappy on Friday July 22, 2016 @08:41PM (#12147779)
First off, I'll admit our documentation for caching needs improvement.

We do support caching of gem's and other file artifacts. There are tricks about file locations, but once you get the hang of it, it works well. See https://gitlab.com/gitlab-org/gitlab-ci-yml/blob/master/Ruby... for an example for Ruby. It's actually one of the sources for our new CI configuration templates!

But sometimes, you don't really want caching, you want artifacts. Caching is an optimization, but isn't guaranteed to always work, so you need to be prepared to regenerate any cached files in each job that needs them.

Artifacts, on the other hand, are guaranteed to be available. It's sometimes confusing because the name `artifact` sounds like something that is only useful outside of the build, like for downloading a final image. But artifacts are also available in between stages within a build. So if you "build" your application by downloading all the required modules, you might want to declare them as `artifacts` so that each subsequent stage can depend on them being there. There are some optimizations like declaring an expiry time so you don't keep artifacts around too long, and using `dependencies` to control exactly where artifacts are passed around. Again, complicated subject that is poorly documented. I'd be happy to help you figure it out for your use case (and then publish the learnings).

Re:GitLab 8.10 Released with Wildcard Branch Protection and ... by snuxoll on Saturday July 23, 2016 @11:41AM (#12149785)
Have you looked at using a caching proxy or maven repo in front of GitLab? For our TeamCity builds we have ProGet sitting in front of the public NuGet and Maven repos, it caches the downloads so any machine on our local network can reuse them instead of downloading artifacts every time.