News for hackers. Stuff that matters.
Re:GitLab Container Registry by connorshea on Monday May 23, 2016 @02:45PM (#11755899)
Kind of unrelated, but I'm super happy to be working at GitLab.

Nearly everything we do is out in the open, and most features are available in CE, which is fully open source (https://gitlab.com/gitlab-org/gitlab-ce). I've wanted great, open source tools that don't look like dung for a long long time, and GitLab is definitely reaching that goal. Can't help but gush about the product we're building. :D

Here are some of my favorite upcoming features: https://gitlab.com/gitlab-org/gitlab-ce/issues/17575, https://gitlab.com/gitlab-org/gitlab-ce/issues/14661, https://gitlab.com/gitlab-org/gitlab-ce/issues/15337

Re:GitLab Container Registry by Cyph0n on Monday May 23, 2016 @04:59PM (#11756939)
Damn, you are one lucky person! Getting paid to work on OSS is a dream come true.

Keep up the awesome work everyone :)

Re:GitLab Container Registry by sytse on Monday May 23, 2016 @05:10PM (#11757026)
Thanks! And I think Connor created his own luck. People were assuming he already worked at GitLab since he was so active on social media and on our issue trackers. If you consistently contribute to GitLab it is very hard for us not to hire you.
Re:GitLab Container Registry by Cyph0n on Monday May 23, 2016 @06:01PM (#11757403)
That's a pretty effective hiring method in my opinion. You guys should publish a blog post or something on your hiring practices and results - I'm sure the HN crowd would enjoy it.
Re:GitLab Container Registry by sytse on Monday May 23, 2016 @07:06PM (#11757725)
Thanks, our hiring practices are detailed on https://about.gitlab.com/handbook/hiring/ what kinds of results should we include?

I'm also thinking about writing about an open organization, where the handbooks and issue trackers are public. Would that be interesting?

Re:GitLab Container Registry by broodbucket on Tuesday May 24, 2016 @12:52AM (#11759055)
Yeah, definitely. It's an interesting concept, I didn't know about the handbooks being public, seeing employees interacting on issues in the open and seeing the company's strategy be posted publicly is very cool.
Re:GitLab Container Registry by agibsonccc on Monday May 23, 2016 @08:52PM (#11758186)
Not to jump on to the gitlab bandwagon here. Adam from Skymind (YC W16) here

I just want to say the last 4 hires we've made (we're only 11 soon 12) have been from around the world (we're in 4 countries) and this has worked great for us. A thriving open source community is key to hiring for us. A lot of our team does some sort of traveling.

I myself will be working from tokyo (currently in korea actually..) moving from SF.

We make the time zones work :D. Key to that for us has been gitter where we can collaborate/support the open source community while also being able to use it for collaborating with partners and running things like training classes.

Just wanted to provide another example to prove gitlab isn't an anomaly. The same is true for a lot of the larger open source organizations (eg: canonical,redhat,automattic)

Re:GitLab Container Registry by sytse on Tuesday May 24, 2016 @02:07PM (#11764040)
Thanks for chiming in Adam. Good to hear you're making it work. I think having a HQ is overrated. Technology is shrinking distance and making it easier to avoid the commute.
Re:GitLab Container Registry by radicalbyte on Monday May 23, 2016 @05:50PM (#11757333)
It's such a crazy effective way to hire people. Luckily most companies don't understand it ;-)
Re:GitLab Container Registry by vonklaus on Monday May 23, 2016 @06:06PM (#11757431)
I want to work on growth for Gitlab. Where are you guys seeing problems in market penetration so I can fill out an application[0].

[0] pun intented.

edit: To anyone interested, this is the public haml file that generates their website, to which they encourage contributions. So a pull request job application it is.

Re:GitLab Container Registry by rudedogg on Monday May 23, 2016 @04:01PM (#11756462)
I'm running GitLab CE off a NAS to host my projects, and it works really well. Thank you all for making it freely available.
Re:GitLab Container Registry by sytse on Monday May 23, 2016 @04:31PM (#11756704)
Glad to hear that, thanks for using GitLab! For other people, I think there are Synology packages on https://www.synology.com/en-us/dsm/app_packages/Docker-GitLa...
Re:GitLab Container Registry by dalanmiller on Wednesday May 25, 2016 @12:55PM (#11771211)
Unfortunately only for the Intel processor units. I didn't realize this until my ARM-powered device arrived and the package wasn't available.
Re:GitLab Container Registry by stp-ip on Monday May 23, 2016 @06:14PM (#11757480)
Not to mention the awesome team at GitLab and the Core Team from the community. Awesome progress and quite transparent and with an amazingly open approach. Buffer with a product one wants to work on.
Re:GitLab Container Registry by dblessing on Monday May 23, 2016 @11:58PM (#11758899)
Thanks for recognizing our Core Team, too. They're an integral part of our community. :)
Re:GitLab Container Registry by kaishiro on Monday May 23, 2016 @11:57PM (#11758891)
Sorry if this is the wrong venue to ask this, but any idea if/when Codeship will natively support you guys?

It's honestly my one hangup since we do all of our deploys there.

Re:GitLab Container Registry by moritzplassnig on Tuesday May 24, 2016 @07:02AM (#11760447)
Codeship founder here :) Getting a lot of requests for a GitLab integration recently and we're looking into it and are hopefully able to build it soon (I wish it would already be there)
Re:GitLab Container Registry by tonytamps on Thursday May 26, 2016 @07:06PM (#11782417)
I want GitLab + CodeShip so badly but I won't be holding my breath.

This has been a very standard response for the last 2 years.

https://twitter.com/codeship/status/454327470402322433

Re:GitLab Container Registry by moritzplassnig on Friday May 27, 2016 @06:23PM (#11789461)
I know :/ Wish we would have built it already
Re:GitLab Container Registry by jobvandervoort on Tuesday May 24, 2016 @07:14AM (#11760533)
Let me know if there's any way we can help! job at gitlab!
Re:GitLab Container Registry by lukebennett on Monday May 23, 2016 @01:49PM (#11755503)
Fantastic, Gitlab are absolutely nailing it at the moment. Don't think there's any other hosted service offering so many CI/CD features for free under one roof.

It feels like what Github did for SCM, Gitlab are doing for CI.

Re:GitLab Container Registry by sytse on Monday May 23, 2016 @02:10PM (#11755665)
Thanks Luke! We'll try to keep this going. We're working on deployment environments for our next release so you can see what is deployed in each environment. After that we can do deployments that need to be confirm manually.
Re:GitLab Container Registry by jstsch on Monday May 23, 2016 @04:53PM (#11756887)
Wow, that's great news. Exactly what we are looking for. I was investigating to use GitLab CI to build, and then only deploy commits with a certain tag, but a dedicated workflow for deployments is much more convenient and less error-prone.
Re:GitLab Container Registry by sytse on Monday May 23, 2016 @05:11PM (#11757035)
Glad to hear that. By the way, the 'only deploy tags' strategy is already possible today if you want to do it that way.
Re:GitLab Container Registry by FireBeyond on Monday May 23, 2016 @02:54PM (#11755971)
I agree, this is awesome. Their post yesterday hinted at something, and I honestly didn't think it would be this exciting. I can't wait to use it.
Re:GitLab Container Registry by sytse on Monday May 23, 2016 @03:19PM (#11756139)
Thanks, glad you like it. BTW I wonder how many people understood our hint yesterday: "We almost can't _contain_ our excitement about how our announcement on Monday will _register_ with you!".
Re:GitLab Container Registry by lukebennett on Monday May 23, 2016 @03:31PM (#11756236)
Heh, I didn't read the hint until after the registry announcement, but saw what you did and appreciated it with the benefit of hindsight!
Re:GitLab Container Registry by pritambarhate on Tuesday May 24, 2016 @02:17AM (#11759315)
These days it seems that Docker is everywhere. I am new to docker and I seem to find it as one more additional complicated system that the developer now needs to learn in order to deploy his or her application.

How useful do you find docker for application which can be deployed on Heroku or Beanstalk? I can understand using docker for a Language ecosystem which is not supported on Public PaaS. Or for people for whom public PaaS is not an option.

I would like to know about the experience of using Docker in day to day development from people who have used Docker in team environments. How was your experience of converting the team to use Docker instead of their regular development/deployment? For example, at our company, For LAMP or MEAN or Java Stacks, we have a defined a procedure of setting up the dev machines to include all these tools and developers know how to manage these on their own machine. Once the code is pushed to Jenkins, there are deploy scripts written by tech leads which take care of deployment to various servers or to Heroku.

In your Docker driven development process, did everybody started using docker on their development machine too? Or everybody just kept using their local setup, and only one/two people handled the task of creating final docker images? Or do you just use you CI server to handle the docker image creation and deployment from that point onwards?

Re:GitLab Container Registry by trey-jones on Tuesday May 24, 2016 @08:40AM (#11761043)
I am also pretty new to docker and it took a few days to get my head into it, but I am using it mainly as a contained dev environment, similar to the way we use vagrant. I am finding that docker is much better suited to complex environments, and a lot easier to update.

Contrary to your fears of everyone having to learn docker to develop, I see it as a tool that allows every developer on a team to be brought into new technology without having to learn all about a new toolchain. We are using docker-compose for this. Everyone has to install docker, and have some cursory knowledge of docker-compose commands, but that's it.

As an example, I recently shoehorned an angular2 front end into an existing project. Without docker, this means every developer needs node and all the dependencies before they begin to work. With docker-compose, the next time they start up the environment, the npm container installs all of their dependencies, and the typescript compiler container listens and compiles all changes to the webroot. The other developers don't even need to know what node is, or that it's being used!

Docker is a very powerful tool with a lot of untapped potential I'm sure. I've no experience with docker in production, however.

Re:GitLab Container Registry by brown9-2 on Tuesday May 24, 2016 @08:42AM (#11761058)
In your Docker driven development process, did everybody started using docker on their development machine too?

Yes. The build scripts produce an image as well as a JAR.

It doesn't do anyone any good to have the knowledge of how to package your code for production locked in just a few people's heads.

Re:GitLab Container Registry by Tiksi on Monday May 23, 2016 @03:39PM (#11756292)
It seems like every time I set up some supporting infrastructure it ends up rolled into the next gitlab release a week later, not that I'm complaining (much).

If you haven't checked out gitlab in a while, you definitely should. It's been moving fast and come a long way lately.

Thank you Gitlab team for making an open source, self hosted platform and all the recent improvements you've made.

Re:GitLab Container Registry by sytse on Monday May 23, 2016 @05:15PM (#11757058)
Thanks for your kind words. May I ask what supporting infrastructure you'll set in 3 weeks from now? We'll have to add it to the schedule for 8.9
Re:GitLab Container Registry by Tiksi on Monday May 23, 2016 @05:52PM (#11757350)
Well since you asked, I'm working on a system that writes perfect bug-free code for me, but if you can implement that I'll save myself the effort.

But really, the next thing on the roadmap is to get tighter integration between our CFM (saltstack/salt-api) and gitlab. Tie it into deploys, auto-open issues on failed states, etc., though that's gonna be more on the salt side than the gitlab side and likely wouldn't make sense for you guys to implement anyways.

Im sure I'll end up setting up whatever does make it into 8.9 a week before it's released though.

Re:GitLab Container Registry by sytse on Monday May 23, 2016 @07:16PM (#11757763)
It would be nice to upload Salt modules directly from GitLab.

Since you can deploy from GitLab too, auto-opening issues for failed builds makes sense. I opened an issue on https://gitlab.com/gitlab-org/gitlab-ce/issues/17771

I predict you'll be setting up different environments a week before 8.9 since that is what we'll ship.

Re:GitLab Container Registry by nikolay on Monday May 23, 2016 @02:03PM (#11755611)
At this pace of innovation, GitHub will soon be a part of the history! GitHub has done a tremendous job with forking and pull requests, but, at least recently, they've acted disorientedly! I hope they get back into shape!
Re:GitLab Container Registry by r3bl on Monday May 23, 2016 @04:14PM (#11756572)
I really don't know what to hope for.

All that I do now that I really enjoy watching what looks like a full on feature battle between GitLab and GitHub going on at this moment. As a guy who uses both (GitLab for my job, GitHub personally), I feel like I'm getting the best of both worlds.

Although, to be perfectly honest, it feels to me like, at this point of the "feature battle", GitLab started embracing and actually improving every single thing I liked about GitHub to the point where I'm questioning my stay on GitHub for personal projects.

Re:GitLab Container Registry by zanny on Monday May 23, 2016 @10:14PM (#11758543)
Considering Gitlab is the more open source of the two, the answer has to always be gitlab.

Albeit, it still sucks most free software communities cannot use self-hosted gitlab because too many features are stuck in the enterprise edition, but its way better than what github is offering, which is nothing.

Re:GitLab Container Registry by andromeduck on Monday May 23, 2016 @04:27PM (#11756668)
now if only they could get gitlab.com page loads consistently under 1 second...

x_x

Re:GitLab Container Registry by recuter on Monday May 23, 2016 @05:12PM (#11757040)
This is my first week at Gitlab. I've joined along with a few other people (one of whom has a great big bushy beard) to specifically make you happy.

There are interesting challenges that currently make this none trivial but expect things to improve greatly in the near future.

Re:GitLab Container Registry by sytse on Monday May 23, 2016 @04:37PM (#11756751)
Yes, we should fix the slow page loads on GitLab.com. We're working on it in https://gitlab.com/gitlab-com/operations/issues/42 I'm sorry for the current slowness.
Re:GitLab Container Registry by edwinksl on Monday May 23, 2016 @07:04PM (#11757718)
Yeah the site is taking a ridiculous amount of time to load.
Re:GitLab Container Registry by nikolay on Monday May 23, 2016 @05:28PM (#11757164)
Yup! It supposedly is written in a much more efficient language, but it severely slower than GitHub and Bitbucket.
Re:GitLab Container Registry by ymse on Monday May 23, 2016 @05:37PM (#11757246)
It's written in Ruby like Github, but the latter have considerably more experience running high-traffic systems.

Just perused their job page and will apply this week. Sound like they could use some Varnish and Ceph talent :)

Re:GitLab Container Registry by sytse on Monday May 23, 2016 @07:08PM (#11757736)
For sure we're interested in people that have experience with Ceph.
Re:GitLab Container Registry by nikolay on Monday May 23, 2016 @05:43PM (#11757293)
Oh, I confused it with Gogs, sorry!
Re:GitLab Container Registry by andrewmunsell on Monday May 23, 2016 @02:24PM (#11755771)
Everyone at GitLab has been doing an amazing job shipping some really cool features.

I'm really looking forward to trying out the new container registry and move away from my hack-y solution that I use right now to build Docker images on my own VM, and move back to the shared runners :)

Re:GitLab Container Registry by sytse on Monday May 23, 2016 @02:36PM (#11755846)
Thanks Andrew, I hope your move goes smoothly. We want to allow everyone to go frictionless from idea to production. We're extending GitLab so this can be done simply without having to script things together yourself.
Re:GitLab Container Registry by diggan on Monday May 23, 2016 @03:00PM (#11755998)
I'm excited for this, seems like GitLab is moving more into a all-in-one solution, compared to Github that focuses on "social coding", whatever that now means.

So to try out this new feature (together with the pipelines), I tried setting up a simple project that uses a docker image to serve a simple html page.

However, it seems like it's not possible to build/push from the CI system (unless you setup a self-hosted runner) which kind of leaves this "Container Registry" without value, because I still need to manually build/push my images from one machine...

Re:GitLab Container Registry by sytse on Monday May 23, 2016 @03:08PM (#11756057)
Thanks, our idea is indeed to build an all-in-one solution. For more information see https://about.gitlab.com/direction/#scope

You should be able to build/push from the CI system, see the example GitLab CI configuration file in the blog post.

EDIT: For GitLab.com users using shared runners this is waiting on the Runner upgrade that we'll do in the coming days. If you use your own runner that runs 1.2 it should be fine.

Re:GitLab Container Registry by drewcrawford on Monday May 23, 2016 @06:28PM (#11757541)
Does this work for autoscale runners? Are those privileged by default?
Re:GitLab Container Registry by sytse on Monday May 23, 2016 @07:18PM (#11757771)
It is about the Runners needing to be on version 1.2. We'll upgrade the .com autoscaling ones in the coming days. If you run your own autoscaling runners you can upgrade yourself.
Re:GitLab Container Registry by diggan on Monday May 23, 2016 @04:06PM (#11756500)
Ah, thanks to sytse, Snappy and tmaczukin for the replis. I missed that.

My configuration looks like this currently (and I'm guessing I'll hold of for a few days for the shared workers to get updated):

  image: docker:latest
  services:
  - docker:dind
  stages:
    - build
    - deploy
  build:
    stage: build
    script:
      - docker build -t registry.gitlab.com/victorbjelkholm/deploy-html-test:latest .
    only:
      - master
  deploy:
    stage: deploy
    script:
      - docker push registry.gitlab.com/victorbjelkholm/deploy-html-test:latest
    only:
      - master
Re:GitLab Container Registry by Snappy on Monday May 23, 2016 @04:48PM (#11756842)
If you're going to use the shared runners, I don't think that configuration is going to work. The build step will build it locally, and even though you named it to use the registry, it won't actually push to the registry automatically. By breaking up the push to another step, you're actually going to run that deploy stage on a fresh Digital Ocean instance, so your previously-built image will be lost.

That's why the example pushed the image right after the build step.

Having said that, if you're literally using that script, you don't even have tests, so you may as well just put both steps into a single build stage like the simplified example. :)

For now, you'll need the explicit `docker login` as well. We'll work to remove that.

Re:GitLab Container Registry by sytse on Monday May 23, 2016 @04:35PM (#11756741)
Thanks, you didn't miss anything in the blog post, we updated the blog post. https://gitlab.com/gitlab-com/www-gitlab-com/commit/c758f6b2...
Re:GitLab Container Registry by tmaczukin on Monday May 23, 2016 @03:11PM (#11756089)
Our shared runners should be updated within a week. Then you will be able to use Container Registry on GitLab.com without self-hosted runner.
Re:GitLab Container Registry by Snappy on Monday May 23, 2016 @03:17PM (#11756126)
Yeah, sorry about that. We'll update the blog post to clarify. We're updating the CI runners this week to support this.
Re:GitLab Container Registry by sytse on Monday May 23, 2016 @04:36PM (#11756743)
Re:GitLab Container Registry by cridenour on Monday May 23, 2016 @02:48PM (#11755921)
After GitHub moved to unlimited repositories, I started to question if our small team should stay on GitLab - and this just triple-confirmed that we will.

Awesome feature for an already awesome product. Great job!

Re:GitLab Container Registry by sytse on Monday May 23, 2016 @03:20PM (#11756145)
Thank you very much much. I hope your switch goes smoothly.
Re:GitLab Container Registry by yoavm on Monday May 23, 2016 @04:10PM (#11756533)
Up until recently I never even considered using CI in my projects because I just didn't have the time. The way GitLab are implementing all these tools makes it not only easy to use, but fun.
Re:GitLab Container Registry by sytse on Monday May 23, 2016 @04:33PM (#11756720)
Glad to hear that! This was our intention. CI and CD is something everyone knows they should do but it isn't always done. By making it easier to set up we try to get more people to do it and feel good about themselves.
Re:GitLab Container Registry by Fenntrek on Monday May 23, 2016 @01:41PM (#11755445)
Interesting move.

Seems Gitlab are moving fairly fast (compared to competition) and going for that all in one ecosystem (If you want it) from SCM to deployment with the range of products they've introduced.

Re:GitLab Container Registry by ayufan on Monday May 23, 2016 @02:07PM (#11755632)
We are thinking big. The integrated Container Registry is part of our plan. We will soon have an integrated deployments with easy to access environments. We also plan to introduce manual actions on pipelines allowing you to execute arbitary things, ex. promote to production, but also merge it back to the master.

https://gitlab.com/gitlab-org/gitlab-ce/issues/17009 https://gitlab.com/gitlab-org/gitlab-ce/issues/17010

Re:GitLab Container Registry by sotojuan on Monday May 23, 2016 @04:46PM (#11756817)
Other good news for GitLab: Zach Holman just joined them as an advisor!

https://mobile.twitter.com/holman/status/734842346278244352

Re:GitLab Container Registry by sytse on Monday May 23, 2016 @05:17PM (#11757072)
HN discussion of that tweet in https://news.ycombinator.com/item?id=11756674 BTW that post and this post are 1 and 2 on HN right now
Re:GitLab Container Registry by avitzurel on Monday May 23, 2016 @04:05PM (#11756488)
Gitlab is what Github was 3 years ago.

They are on a roll, rolling out features left and right and seem to nail the need for every engineer.

Kudos to the team! Keep rocking!

Re:GitLab Container Registry by sytse on Monday May 23, 2016 @04:41PM (#11756779)
Thank you, we're very determined to keep shipping. Please let us know if there is something you can use that we're not doing yet.
Re:GitLab Container Registry by iyn on Monday May 23, 2016 @04:36PM (#11756744)
I'm really impressed by the hard work of GitLab team, keep it up!

If anybody from GitLab is reading: any plans for rkt support?

Re:GitLab Container Registry by Snappy on Monday May 23, 2016 @04:39PM (#11756763)
Thanks!

No current plans for rkt, but feel free to create an issue to discuss it.

Re:GitLab Container Registry by hackbinary on Monday May 23, 2016 @04:51PM (#11756870)
For those who don't know what rkt is, I am presuming that it is the App container runtime: https://coreos.com/rkt/ https://github.com/coreos/rkt
Re:GitLab Container Registry by flexd on Tuesday May 24, 2016 @03:50AM (#11759660)
I am also interested in rkt support, so I created an issue like you suggested. https://gitlab.com/gitlab-org/gitlab-ce/issues/17784
Re:GitLab Container Registry by nhumrich on Tuesday May 24, 2016 @12:13AM (#11758948)
Can I use gitlab container registry with gitlab.com? Can I use private images in said registry with public gitlab.com ci builders? I haven't found a way to use private containers with gitlab.com ci yet without spinning up a worker for every project. Maybe a way to register a "team wide" builder?
Re:GitLab Container Registry by ayufan on Tuesday May 24, 2016 @12:38AM (#11759016)
Currently not. The limitations can be found here: http://docs.gitlab.com/ce/container_registry/README.html#lim.... We plan to improve that in one of upcoming releases.
Re:GitLab Container Registry by alfg on Monday May 23, 2016 @09:45PM (#11758425)
So awesome! I've been invested into the Docker ecosystem lately and one of the next things I've wanted to setup is a decent CI workflow.

I wish Github would also ship more features at the pace Gitlab is. The only issue I've had with Gitlab are the response times on their UI. Hopefully they sort that out soon.

Re:GitLab Container Registry by jobvandervoort on Tuesday May 24, 2016 @02:47AM (#11759430)
I suppose you are using GitLab.com. We're working on improving the speed [0]. If you are willing to run your own instance, it should be fast.

[0]: https://gitlab.com/gitlab-com/operations/issues/42

Re:GitLab Container Registry by znpy on Monday May 23, 2016 @10:28PM (#11758595)
It's happening: GitLab has basically filled the feature gap with GitHub and it is surpassing it.
Re:GitLab Container Registry by voltagex_ on Tuesday May 24, 2016 @12:53AM (#11759059)
Once the performance gap closes it'll be even more exciting.

I just worry about 10, 15, 20 years in the future when ArchiveTeam has to back up GitHub and GitLab and whatever else... there's so much data!

Re:GitLab Container Registry by znpy on Tuesday May 24, 2016 @02:08AM (#11759279)
There will also be bigger storage :)
Re:GitLab Container Registry by educar on Tuesday May 24, 2016 @03:41AM (#11759625)
I wish Gitlab had an easy clean way to import GitHub. The current method involves setting up oauth information in the config file and restarting gitlab. Not nice :/ I fail to see why they cannot just work with GitHub API tokens for import?
Re:GitLab Container Registry by mikewhy on Monday May 23, 2016 @03:05PM (#11756038)
This is great! But I'm curious if anyone at Gitlab can comment on what this gives over running the existing docker registry? Can we configure this to use S3 to store the resulting images?

Our runner image has docker, we run our tests with docker-compose, if they pass we push them to our existing registry. In fact our .gitlab-ci.yml looks very similar to the example under "elaborate example" in the blog post.

Just wondering what we'd be missing out on if we didn't switch to Gitlab Container Registry.

Re:GitLab Container Registry by ayufan on Monday May 23, 2016 @03:15PM (#11756113)
The biggest plus of using integrated registry is that you have integrated authentication and authorization of GitLab that follow your groups and members assigned to your GitLab projects, making it really easy to have private container repositories stored on registry.

Second the built-in registry is really easy to configure and maintain. You have to specify the address and provide a certificate to start using it. You also use the same backups mechanism as you use for your GitLab installation.

In the future we may make it possible to use external storage for container images, like S3. This is something that is already supported by docker/distribution.

Re:GitLab Container Registry by fapjacks on Monday May 23, 2016 @03:57PM (#11756433)
I am super stoked to see this and will be using the crap out of it and pointing others to it (the registry is kind of a pain to get going)! It looks like this is just the v2 registry (from Distribution) integrated into Gitlab, so I'm wondering what's stopping me from backing this registry with S3? Is it just not supported by the Gitlab config yaml? I back my private registry with S3 and it's just a couple of config options to enable it. Or am I misunderstanding some fundamental concept here? Thanks for the awesome work!
Re:GitLab Container Registry by sytse on Monday May 23, 2016 @04:44PM (#11756799)
Glad to hear you're super stoked! I think you're on the money regarding the s3 backup. I think it is making the configuration accessible. I expect you can work around that by doing it yourself.
Re:GitLab Container Registry by Snappy on Monday May 23, 2016 @03:21PM (#11756155)
It really comes down to cost and convenience. We don't charge anything additional to run the container registry (including unlimited private projects for personal or business use), and it's already installed with GitLab.

Having said that, we do love deep integration, so we'll continue to improve it going forward. If you have any ideas for improvements, please do create an issue!

Re:GitLab Container Registry by allan_s on Monday May 23, 2016 @03:42PM (#11756323)
In the same idea, has someone already worked integration with a debian repository (with aptly or similar) and has some linked to share on how to do it the smartway?

We're thinking about using FPM to create the debian package Package which is the retrieve as the artifacts of the gitlab-ci build stage

And then to have a separate service that will receive a webhook at the end of the (successful) build, to retrieve the artifacts and update a repository using aptly.

Does it seems the right way or is there some much simpler solution ?

Re:GitLab Container Registry by tostaki on Monday May 23, 2016 @06:37PM (#11757590)
At my work, we have a setup close to this. On a successful build of jenkins (after build action), a deb is generated with FPM (awesome piece of software btw) and uploaded to an s3 bucket. Then on another machine, a cron that run every minutes sync the s3 bucket to a local aptly repository and publish it to another s3 bucket which is the real debian repository.

This works well but the s3+cron part is probably not the smartest way and publication can take some time (like 5-10min).

Re:GitLab Container Registry by voltagex_ on Tuesday May 24, 2016 @12:54AM (#11759064)
I know it probably doesn't matter too much for internal use, but what does Lintian say about your packages?
Re:GitLab Container Registry by tostaki on Tuesday May 24, 2016 @07:38AM (#11760680)
Many bad things! I didn't check until now but it's a bunch of "non-standard-dir-perm" and "dir-or-file-in-var-www" and things like "maintainer-name-missing". I think it's possible to build correct debian package with FPM but for our use, we just want to build easily native deb package.
Re:GitLab Container Registry by dozzie on Tuesday May 24, 2016 @07:50AM (#11760742)
> I think it's possible to build correct debian package with FPM [...]

Of course it is possible, it's just ridiculous amount of work. It's much easier to build package with proper tools in the first place.

Re:GitLab Container Registry by mikepurvis on Monday May 23, 2016 @04:08PM (#11756512)
Aptly is a piece of cake compared to reprepro, because of the REST API. I just run it on a separate box, and make curl invocations from Jenkins jobs to post debs to it at the end of the build.

Depending what it is you're building, using the proper debian tooling (dpkg-buildpackage, etc) is not that much harder than FPM, and cooperating with the system gives you a lot of goodies for free (sourcedebs, cowbuilder, etc).

Re:GitLab Container Registry by thebouv on Monday May 23, 2016 @04:50PM (#11756857)
I've tried several times to figure out how to get Docker working in a situation like mine. And we've been considering GitLab as well. So this is likely a good time to experiment.

Doing all the research on how to integrate Docker into my particular situation has been daunting. I really need to track down some online courses or something. The articles just aren't cutting it. Or need to find a mentor just to ask stupid questions to.

Re:GitLab Container Registry by Tomdarkness on Monday May 23, 2016 @04:50PM (#11756864)
Looks great. One thing that does not appear to be clear though is if you build multiple docker images from the same project.

Our deployment workflow at the moment builds two docker images, one for the web app and another for the background services. Both share the same code.

It would appear you can only have one docker image per project?

Re:GitLab Container Registry by hackerboos on Monday May 23, 2016 @04:53PM (#11756884)
AFAIK you can just have 2 stages to build each image and push both under different tags.
Re:GitLab Container Registry by sytse on Monday May 23, 2016 @05:13PM (#11757048)
You can do this but currently you can have only one image per project. We did this to keep it simple. You can use a dummy project to work around it. Or consider making web app and the background services separate projects.
Re:GitLab Container Registry by netcraft on Monday May 23, 2016 @03:05PM (#11756041)
am I correct that this is only for on-prem deployments, that they aren't going to be offering a public container registry a la docker hub or quay?
Re:GitLab Container Registry by tmaczukin on Monday May 23, 2016 @03:17PM (#11756131)
You can enable Container Registry for your projects on GitLab.com :)

And in a few days our shared runners should be updated to support the full docker workflow with the Container Registry.

Re:GitLab Container Registry by netcraft on Monday May 23, 2016 @05:29PM (#11757183)
thats really amazing - are there size limits?
Re:GitLab Container Registry by sytse on Monday May 23, 2016 @07:20PM (#11757776)
Our soft limit is 10GB per project for GitLab.com https://about.gitlab.com/gitlab-ci/
Re:GitLab Container Registry by ayufan on Monday May 23, 2016 @03:16PM (#11756119)
The Container Registry is available on our free GitLab.com.
Re:GitLab Container Registry by SEJeff on Monday May 23, 2016 @03:06PM (#11756053)
This might sound like a silly question, but does this require gitlab, or can it work standalone?
Re:GitLab Container Registry by sytse on Monday May 23, 2016 @03:09PM (#11756068)
You install it by installing the GitLab Omnibus package. But you push and pull to it from other application too, no need to use GitLab.
Re:GitLab Container Registry by btashton on Monday May 23, 2016 @05:45PM (#11757307)
When would we expect to see this on githost.io?
Re:GitLab Container Registry by dblessing on Monday May 23, 2016 @06:34PM (#11757575)
We are investigating what it will take to enable this on GitHost.io. At first glance, it doesn't seem to be too difficult. I created https://gitlab.com/gitlab-com/githost/issues/12 to track this feature request.
Re:GitLab Container Registry by skylan_q on Monday May 23, 2016 @04:24PM (#11756654)
These guys are f*cking killing it. Way to go!
Re:GitLab Container Registry by davexunit on Monday May 23, 2016 @02:22PM (#11755757)
Awesome, a new place to host your unpatched, opaque disk images!
Re:GitLab Container Registry by sytse on Monday May 23, 2016 @03:25PM (#11756192)
I think the patching of Docker container images is a big problem. We try to provide a completely automated flow in GitLab using CI, CD, and the container registry. I would love to have a big button that says 'update all my containers'. I've made an issue https://gitlab.com/gitlab-org/gitlab-ee/issues/592
Re:GitLab Container Registry by davexunit on Monday May 23, 2016 @03:49PM (#11756387)
The problem is that Docker, and more generally using stacks of binary disk images, is fundamentally flawed with respect to security and reproducibility. It's nontrivial to inspect these images for vulnerabilities because there is nothing that specifies the precise set of software that is in that image. Some stuff is compiled from source, some stuff is installed via one or more package managers, each of which may bundle additional software that one may not know about if they didn't inspect each piece of software carefully. Furthermore, one cannot even verify the image reasonably because the results are different depending on when it was built.

In short, container images as popularized by Docker are insecure by design.

Re:GitLab Container Registry by moondev on Monday May 23, 2016 @03:58PM (#11756439)
Don't you think that gitlab adding container registry support will encourage more people to build their own images, as opposed to trusting "black-box" images from elsewhere? Combining a base image with the deployable artifact is much more efficient than baking amis or other images. You then know exactly what your image contains and nothing more, because you built it yourself.

Of course I'm looking at this from the perspective of deploying micro-services and immutable infrastructure. Your use-case may be different.

Re:GitLab Container Registry by davexunit on Monday May 23, 2016 @04:33PM (#11756731)
Any nontrivial image relies on a large number of other images as a base, and just because you built it yourself doesn't mean that you didn't just download and install software with known security vulnerabilities.
Re:GitLab Container Registry by snuxoll on Monday May 23, 2016 @04:48PM (#11756837)
Yes, but you will ALWAYS have that issue with software you build yourself. This is primarily why I maintain an in-house yum repository for anything that we use out of our distribution repositories, it takes a little more effort to build RPM's but it's worth it from a maintainability and security aspect (as I type this I'm working on a salt-based package management tool somewhat akin to Katello/Sattelite that I can use to manage upgrades that Puppet doesn't).
Re:GitLab Container Registry by moondev on Monday May 23, 2016 @05:16PM (#11757062)
The thought is you use a vanilla LTS image as a base. If that needs to updated then your ci can easily do it.

You then combine that with your installable artifact (deb, rpm). It's the same exact process using classic images, so i'm not sure what the complaint about docker specifically would be.

Re:GitLab Container Registry by voltagex_ on Tuesday May 24, 2016 @01:26AM (#11759152)
At least by building images by myself I'm taking responsibility in a way for the security of them - rather than relying on the (in)security of whatever base image.

I did see a feature somewhere for scanning container images for security vulnerabilities, but I think something closer to FreeBSD's pkg-audit is needed.

Re:GitLab Container Registry by sytse on Monday May 23, 2016 @04:48PM (#11756839)
I agree that building your own images is the future. We want to make this as easy as possible. I think it is essential for security.
Re:GitLab Container Registry by justinsaccount on Monday May 23, 2016 @05:10PM (#11757024)
Can I play too?

The problem is that unix, and more generally any operating system, is fundamentally flawed with respect to security and reproducibility. It's nontrivial to inspect these installations for vulnerabilities because there is nothing that specifies the precise set of software that is in that installation. Some stuff is compiled from source, some stuff is installed via one or more package managers, each of which may bundle additional software that one may not know about if they didn't inspect each piece of software carefully. Furthermore, one cannot even verify the installation reasonably because the results are different depending on when it was installed.

In short, operating systems are insecure by design.

Re:GitLab Container Registry by lfam on Monday May 23, 2016 @06:37PM (#11757586)
Indeed. I recommend you look into one of the projects that is trying to mitigate this problem by allowing you to precisely specify, inspect, and change the dependency graph of software on your computer. That is, Guix or Nix.

Guix (and I presume Nix) can then take advantage of that transparency and control to build containers, virtual machines, and bare-metal systems to your specifications.

Re:GitLab Container Registry by tracker1 on Monday May 23, 2016 @04:42PM (#11756787)
if you're building go or rust packages, all you need is the output executable alone a lot of the time. :-) The host OS and docker version provide most of the low-level bits.
Re:GitLab Container Registry by ymse on Monday May 23, 2016 @07:24PM (#11757794)
The registry is just that: a registry. You can push a Guix-generated image to it if you wish and use with any container runtime. rkt supports the same API; the Docker relation is by name only.

In fact, being able to push static images directly to such a registry with Guix would be a great feature. Both for continuous integration (without needing Hydra) and for coupling with a cluster scheduler such as Kubernetes.

Re:GitLab Container Registry by moondev on Monday May 23, 2016 @03:02PM (#11756019)