How To Securely Manage Secrets with HashiCorp Vault on Ubuntu 16.04
0

Introduction

Containerization is quickly becoming probably the most method that is accepted of and deploying applications in cloud environments. The standardization it provides, along with its resource efficiency (when compared to full virtual machines) and flexibility, make it a enabler that is great of contemporary DevOps mind-set. Numerous interesting cloud native implementation, orchestration, and monitoring techniques become feasible if your applications and microservices are completely containerized.

Docker containers are probably the most container type today that is common. A service to build and store your images, or run your own software to do so.( though public Docker image repositories like Docker Hub are full of containerized open source software images that you can docker pull and use today, for private code you’ll need to either pay**********)

GitLab Community Edition is a software that is self-hosted that provides Git repository hosting, project tracking, CI/CD services, and a Docker image registry, among other features. In this tutorial we shall utilize GitLab’s constant integration solution to construct Docker pictures from an illustration Node.js software. These pictures will be tested and uploaded to your very own personal Docker registry.

Prerequisites

(we need to set up a secure GitLab server, and a GitLab CI runner to execute continuous integration tasks*******)Before we begin. The parts below provides links and much more details.

A GitLab Server Secured with SSL

To shop our supply rule, run CI/CD tasks, and host the Docker registry, we truly need a GitLab example installed on an Ubuntu 16.04 host. GitLab presently suggests a host with about 2 cores that are CPU 4GB of RAM. Furthermore, we are going to secure the host with SSL certificates from let us Encrypt. To take action, you will need a domain title pointed during the host.

You can finish these necessity needs using the after tutorials:

A GitLab CI Runner

How to setup constant Integration Pipelines with GitLab CI on Ubuntu 16.04 will provide you with a synopsis of GitLab’s CI solution, and demonstrate just how to set a CI runner up to process jobs. We will build on top of the demo runner and app infrastructure produced inside guide.

Step 1 — starting a Privileged GitLab CI Runner

In the necessity GitLab integration that is continuous, we create a GitLab runner making use of sudo gitlab-runner register as well as its interactive setup procedure. This runner is with the capacity of operating builds and tests of computer software within separated Docker containers.

However, to build Docker pictures, our runner requires access that is full a Docker solution it self. Advised option to configure this will be to utilize Docker’s formal docker-in-docker image to perform the jobs. This involves giving the runner a unique privileged execution mode, therefore we are going to produce another runner with this particular mode enabled.

Note: giving the runner privileged mode fundamentally disables every one of the safety benefits of making use of containers. Unfortuitously, others types of allowing Docker-capable runners additionally carry comparable safety implications. Please go through the formal GitLab documents on Docker develop to find out more in regards to the various runner choices and that will be perfect for your circumstances.

Because you will find safety implications to making use of a privileged runner, we intend to produce a project-specific runner which will just accept Docker jobs on our hello_hapi task (GitLab admins can invariably by hand include this runner to many other jobs later). From your own hello_hapi task web page, simply click Settings at the end associated with left-hand menu, then click CI/CD inside submenu:

GitLab project settings menu

Now click on the Expand switch beside the Runners settings part:

GitLab "Runners settings" expand button

There may be some details about starting a Specific Runner, including an enrollment token. Pay attention to this token. It to register a new runner, the runner will be locked to this project only.( when we use**********)

GitLab project-specific runners options

Although we’re with this web page, click on the Disable provided Runners switch. You want to ensure our Docker jobs constantly operate on our privileged runner. If a non-privileged provided runner had been available, GitLab might go for this 1, which may end up in create mistakes.

Log into the host which has your CI that is current runner it. The GitLab CI Runner Service
section of the prerequisite tutorial before proceeding.( if you don’t have a machine set up with runners already, go back and complete the Installing**********)

Now, operate these demand to create the privileged runner:( that is project-specific**********)

  • sudo gitlab-runner register -n
  • --url https://gitlab.example.com/
  • --registration-token your-token
  • --executor docker
  • --description "docker-builder"
  • --docker-image "docker:latest"
  • --docker-privileged

Output

Registering runner... succeeded runner=61SR6BwV Runner registered effectively. Take a moment to start out it, however, if it is operating currently the config should really be immediately reloaded!

Be certain to replace your very own information. We set our runner choices regarding demand line rather than utilising the interactive prompts, as the prompts do not allow united states to specify --docker-privileged mode.

Your runner is currently create, registered, and operating. To confirm, switch back once again to your web browser. Click on the wrench symbol however GitLab menu club, then simply click Runners inside menu that is left-hand. Your runners will ( be listed**********)

GitLab runner listing

Now for it to push images to.( that we have a runner capable of building Docker images, let’s set up a private Docker registry**********)

Step 2 — starting GitLab’s Docker Registry

Setting your very own Docker registry enables you to push and pull pictures from your server that is private increasing safety and reducing the dependencies your workflow is wearing outside solutions.

GitLab will create a personal Docker registry with only a configuration that is few. First we’ll set the URL up in which the registry will live. Then we shall (optionally) configure the registry to utilize an object that is s3-compatible solution to keep its information.

SSH into the GitLab host, then start the GitLab configuration file up:

  • sudo nano /etc/gitlab/gitlab.rb

Scroll down seriously to the Container Registry settings part. We are going to uncomment the registry_external_url line and set it to a port number to our GitLab hostname of 5555:

/etc/gitlab/gitlab.rb

registry_external_url 'https://gitlab.example.com:5555'

Next, include these two lines to share with the registry how to locate our let us Encrypt certificates:

/etc/gitlab/gitlab.rb

registry_nginx['ssl_certificate'] = "/etc/letsencrypt/live/gitlab.example.com/fullchain.pem"
registry_nginx['ssl_certificate_key'] = "/etc/letsencrypt/live/gitlab.example.com/privkey.pem"

Save and shut the file, then reconfigure GitLab:

  • sudo gitlab-ctl reconfigure

Output

. . . gitlab Reconfigured!

Update the firewall allowing traffic towards registry slot:

Now change to another device with Docker installed, and get on the Docker that is private registry. As it has Docker installed already:( if you don’t have Docker on your local development computer, you can use whichever server is set up to run your GitLab CI jobs,**********)

  • docker login gitlab.example.com:5555

You may be prompted for the account. Make use of your GitLab qualifications to sign in.

Output

Login Succeeded

Success! The registry is initiated and working. At this time it shall keep files regarding GitLab host's neighborhood filesystem. If you wish to utilize an object storage space solution as an alternative, carry on with this particular part. Or even, skip down seriously to move 3.

To create an item storage space backend the registry, we must understand the information that is following our item storage space solution:

  • Access Key
  • Secret Key
  • Region (us-east-1) including, if making use of Amazon S3, or Region Endpoint if making use of an S3-compatible solution (https://nyc.digitaloceanspaces.com)
  • Bucket Name

If you are making use of DigitalOcean Spaces, you will find away just how to create a fresh area and acquire the information that is above reading just how to produce a DigitalOcean area and API Key.

if you have your item storage space information, start the GitLab setup file:

  • sudo nano /etc/gitlab/gitlab.rb

Once once more, scroll down seriously to the container registry part. Try to find the registry['storage'] block, uncomment it, and upgrade it towards after, once more making certain to replace your very own information in which ( that is appropriate**********)

/etc/gitlab/gitlab.rb

registry['storage'] = {
  's3' => {
    'accesskey' => 'your-key',
    'secretkey' => 'your-secret',
    'bucket' => 'your-bucket-name',
    'region' => 'nyc3',
    'regionendpoint' => 'https://nyc3.digitaloceanspaces.com'
  }
}

If you are making use of Amazon S3, you simply require region rather than regionendpoint. If you are making use of an service that is s3-compatible Spaces, you will need regionendpoint. In this instance region does not in fact configure such a thing plus the value you enter does not matter, however it nevertheless must show up rather than blank.

Save and shut the file.

Note: there was presently a bug in which the registry will power down after thirty moments if the item storage space bucket is empty. To prevent this, place a file within bucket before operating the step that is next. You can remove it later, after the registry has added its objects that are own

If you might be making use of DigitalOcean Spaces, it is possible to drag and drop to upload a file utilising the control interface software.

Reconfigure GitLab again:

  • sudo gitlab-ctl reconfigure

On your other Docker device, get on the registry once more to ensure all is well:

  • docker login gitlab.example.com:5555

You should get a Login Succeeded message.

Now that individuals've got our Docker registry create, let us upgrade our application's CI setup to construct and test our software, and push Docker pictures to your personal registry.

Step 3 — Updating gitlab-ci.yaml and Building a Docker Image

Note: in the event that you don't finish the article that is prerequisite GitLab CI you will have to duplicate throughout the instance repository towards GitLab host. Proceed with the Copying the Example Repository From GitHub part to take action.

To get our software building in Docker, we must upgrade the .gitlab-ci.yml file. You are able to modify this file right in GitLab by simply clicking it from project that is main, then clicking the Edit button. Alternately, you could clone the repo to your machine that is local the file, then git push it back once again to GitLab. That could seem like this:

  • git clone [email protected]:sammy/hello_hapi.git
  • cd hello_hapi
  • # edit the file w/ your chosen editor
  • git commit updating that is-am setup"
  • git push

First, delete every thing inside file, then paste inside configuration that is following**********)

.gitlab-ci.yml

image: docker:latest
solutions:
- docker:dind

phases:
- build
- test
- launch

factors:
  TEST_IMAGE: gitlab.example.com:5555/sammy/hello_hapi:$CI_COMMIT_REF_NAME
  RELEASE_IMAGE: gitlab.example.com:5555/sammy/hello_hapi:latest

before_script:
  - docker login -u gitlab-ci-token $ that is-p gitlab.example.com:5555

build:
  phase: build
  script:
    - docker develop --pull -t $TEST_IMAGE .
    - docker push $TEST_IMAGE

test:
  phase: test
  script:
    - docker pull $TEST_IMAGE
    - docker run $TEST_IMAGE npm test

launch:
  phase: launch
  script:
    - docker pull $TEST_IMAGE
    - docker label $TEST_IMAGE $RELEASE_IMAGE
    - docker push $RELEASE_IMAGE
  just:
    - master

Be certain to upgrade the URLs that is highlighted and with your own information, then save with the Commit changes button in GitLab. If you're updating the file outside of GitLab, commit the noticeable modifications and git push back once again to GitLab.

This brand new config file informs GitLab to utilize the most recent docker image (image: docker:latest) and connect it towards docker-in-docker solution (docker:dind). After that it describes build, test, and release phases. The build phase develops the Docker image utilising the Dockerfile supplied inside repo, then uploads it to your Docker image registry. If that succeeds, the test phase will install the image we simply built and run the npm test demand within it. In the event that test phase works, the release phase will pull the image, tag it as hello_hapi:latest and push it back once again to the registry.

Depending in your workflow, you might like to include extra test phases, and sometimes even deploy phases that push the software to a staging or manufacturing environment.

Updating the setup file need triggered a build that is new. Go back to the hello_hapi task in GitLab and then click regarding CI status indicator the commit:

GitLab commit notification with pipeline status icon

On the page that is resulting are able to click the phases to see their progress:

GitLab pipeline detail

GitLab pipeline stage progress

Eventually, all phases should suggest they certainly were effective by showing check that is green icons. We could discover the Docker pictures which were simply built by pressing the Registry product inside left-hand menu:

GitLab container registry image list

If you click the"document that is little symbol beside the image title, it'll duplicate the right docker pull ... demand towards clipboard. After that you can pull and run your image:

  • docker pull gitlab.example.com:5555/sammy/hello_hapi:latest
  • docker run -it --rm -p 3000:3000 gitlab.example.com:5555/sammy/hello_hapi:latest

Output

> [email protected] begin /usr/src/app > node app.js Server operating at: http://56fd5df5ddd3:3000

The image is drawn down from registry and were only available in a container. Change to your web browser and hook up to the software on slot 3000 to check. In this instance we are operating the container on our neighborhood device, vialocalhost at the following URL:( so we can access it**********)

http://localhost:3000/hello/test

Output

hi, test!

Success! You are able to stop the container with CTRL-C. To any extent further, each and every time we push brand new rule towards master branch of our repository, we are going to immediately build and test a fresh hello_hapi:latest image.

Conclusion

In this guide we create a GitLab that is new runner build Docker images, created a private Docker registry to store them in, and updated a Node.js App to be tested and built within Docker containers.

To find out more about the many elements found in this setup, it is possible to see the documentation that is official of CE, GitLab Container Registry, and Docker.

Information Healing For Linux Most Readily Useful 5 OS Open Supply Distro Linux

Previous article

Just how to Install Webmin Panel in Ubuntu 14.04 Server

Next article

You may also like

Comments

Leave a reply

Your email address will not be published. Required fields are marked *

More in DigitalOcean