How To Securely Manage Secrets with HashiCorp Vault on Ubuntu 16.04

Introduction

Duplicacy is a cross-platform back-up device that gives numerous functionalities — including incremental backups, concurrent back-up, and client-side encryption — which make an effort to streamline the entire process of burning information towards the cloud. The CLI (command-line software) Linux variation is free for individual usage but calls for a paid permit for commercial users. In addition, Duplicacy can be obtained for MacOS and Windows with a interface that is graphical which variation calls for both individual and commercial users to cover a license.

Built regarding concept of lock-free deduplication, Duplicacy ended up being made to properly handle backups to a range that is wide of storage services. When a Duplicacy client creates and stores a chunk that is new other customers utilising the exact same storage space bucket is able to see your amount currently exists therefore won’t upload it once again. This permits split customers to generally share and backup identical information without involving any effort that is additional monitor backups.

This guide provides a high-level breakdown of how exactly to install the CLI form of Duplicacy and make use of it to handle a data that is typical process with a DigitalOcean Space. We’ll also demonstrate how to back up a shared repository from multiple Droplets to the Space that is same well as how exactly to backup snapshots to numerous areas for greater information safety.

Prerequisites

To follow additionally guide, you’ll need:

  • Two Ubuntu 16.04 Droplets configured making use of our server setup that is initial guide. It is possible to name your servers whatever you’d like, but to help keep things clear they’ll be known as server-01 and server-02 throughout this guide.
  • Two DigitalOcean Spaces. See our Introduction to DigitalOcean areas for guidelines on the best way to get these installed and operating.
  • An Access key and a key that is secret your areas. To build these, go to the API web page in DigitalOcean control interface.

With these prerequisites in position, you might be willing to install Duplicacy.

Installing Duplicacy

The latest Duplicacy CLI variation are installed (without permit needed for individual users) from Duplicacy GitHub repository making use of wget.

Run the next commands on both server-01 and server-02 to down load Duplicacy onto all of them (replacing the install website link for that the most recent launch):

  • sudo wget -O /opt/duplicacy https**********************************************)( that is://github.com/gilbertchen/duplicacy/releases/download/v2.064_2.0.10

Next, create a link that is symbolic the down load location and a fresh directory within /usr/local/bin:

  • sudo ln -s /opt/duplicacy /usr/local/bin/duplicacy

Finally, make duplicacy executable:

  • sudo chmod 0755 /opt/duplicacy

Duplicacy should now be set up for each of the Droplets and you’re now willing to configure it to utilize your room.

Initializing Your Repository and Configuring Duplicacy

Duplicacy backs up information from directory degree, therefore it must be associated with a specific directory or repository on both of your Droplets before you can begin uploading files to your Space. To do this, you will need to create a project repository and initialize it Duplicacy’s that is using demand.

The init demand takes the syntax that is following*****)

duplicacy init repository_id s3://region@endpoint/space_name
  • repository_id: this is actually the label employed by Duplicacy to tell apart between various repositories. As we will in the next step of this tutorial), the repository ID should be the same on both Droplets.( if you plan to back up the same repository from multiple sources
  • region: The region may be the title for the area which your Droplet is found.
  • endpoint: An endpoint is a location that is static by server-side web APIs to specify where certain resources are found. The endpoint will be the region followed by .digitaloceanspaces.com, as in nyc3.digitaloceanspaces.com for DigitalOcean Spaces. Your Space’s endpoint title may also be copied straight from areas control interface underneath the “Settings” tab.
  • space_name: this is actually the title of the room that has been specified during its creation. Remember that this is simply not your Space’s Address. If for example the Space’s URL is https://example_space.nyc3.digitaloceanspaces.com, its title would you should be example_space.

If you’d such as your backups to visit a certain folder inside your room, include the title for the folder following the title of the room whenever you operate the init demand. Whenever doing this, don’t forget to stick to the folder’s title with a slash:( that is trailing*****)

  • duplicacy init repository_id s3://region@endpoint/space_name/folder_name/

Once you have got those precisely hand, you might be willing to produce a repository directory on every one of your Droplets utilising the mkdir demand. From then on, navigate into the repositories that are new cd:

  • mkdir project-repository
  • cd project-repository/

With project-repository/ as your working directory, run the next init demand on server-01. Make sure to change the values that are highlighted yours details:

  • duplicacy init project_01 s3://nyc3@nyc3.digitaloceanspaces.com/example_space

It will probably be worth noting that you could elect to allow encryption with Duplicacy utilizing the -e choice using the init demand, the following:

whenever encryption is enabled, Duplicacy will prompt one to enter your encryption password any time you make use of it to connect to your room.

Once the init demand runs, Duplicacy will prompt you for the Access and Secret tips, which are often copied over from Control Panel’s API web page.

Output

Enter S3 Access Key ID:ExampleAccessKeyGBBI Enter S3 Secret Access Key:ExampleSecretKeyEC1wnP2YiHobVcSBaZvLoFXXlnA

And it’s going to output the ( that is following*****)

Output

/home/sammy/project-repository is going to be copied to s3://nyc3@nyc3.digitaloceanspaces.com/example_space with id project_01

whenever you operate the init demand, it generates a directory inside your repository called .duplicacy/ which holds a file known as preferences. This file contains information regarding your room and any storage or encryption options you’ve specified. If at a point that is later choose to make modifications towards Duplicacy setup, you’ll either modify the preferences file straight or delete it. The file is going to be recreated the time that is next operate the init demand for the reason that repository.

Now perform the init demand on your Droplet that is second(:

  • duplicacy init project_01 s3://nyc3@nyc3.digitaloceanspaces.com/example_space

After including your Access tips once again, you will notice a output that is slightly different on your own very first Droplet:

Output

The storage space 's3://nyc3@nyc3.digitaloceanspaces.com/example_space' was already initialized Compression degree: 100 Typical amount size: 4194304 Optimum amount size: 16777216 Minimal amount size: 1048576 Chunk seed: 6475706c6963616379 /home/sammy/project-repository is going to be copied to s3://nyc3@nyc3.digitaloceanspaces.com/example_space with id project_01

Both your servers’ repositories are now actually initialized, but there’s an additional action you might want to decide to try configure Duplicacy. For your Access and Secret keys every time you back up your data, which would become tedious rather quickly as it stands, Duplicacy will prompt you. To avoid this, you can use Duplicacy’s set command to write your Space’s credentials to Duplicacy’s preferences file. Run the commands that are following**********)on every one of your servers to own Duplicacy save your valuable Access and key tips, correspondingly:

  • duplicacy set -key s3_id -value ExampleAccessKeyGBBI
  • duplicacy set -key s3_secret -value ExampleSecretKeyEC1wnP2YiHobVcSBaZvLoFXXlnA

You are now actually willing to make use of Duplicacy to backup every one of your Droplets’ repositories to at least one of the areas!

Backing Up One Repository from Numerous Sources

Distributed groups can take advantage of unobtrusive cloud backup solutions that prevent file disputes and information loss. By firmly taking a snapshot of an repository that is entire uploading it to an area with an individual demand, Duplicacy streamlines backups while avoiding file disputes across numerous devices.

To try out Duplicacy’s back-up functionality, usage touch to populate the project-repository on every one of your Droplets with a couple of dummy files:

  • touch /project-repository/file-1.txt
  • touch /project-repository/file-2.txt

Next, on server-01, usage Duplicacy’s backup demand to generate a snapshot of the repository and upload it towards room. As you’ve initiated your repository with just one storage space location, you won’t should specify any choices to straight back your files up:

The ensuing production should look something similar to this:

Output

No past back-up discovered Indexing /home/mark/project-repository Detailing all chunks Loaded file-1.txt (0) Loaded file-2.txt (0) Backup for /home/sammy/project-repository at modification 1 finished

Now take to burning your repository from server-02:

Output

Last back-up at modification 1 discovered Indexing /home/sammy/project-repository Backup for /home/sammy/project-repository at modification 2 finished

You’ll realize that as the repositories on server-01 and server-02 had been identical, Duplicacy didn’t pack any files you ran the backup command on your first Droplet like it did when. To see what will happen when you back up a snapshot that is slightly different start one of many dummy files on server-02 and atart exercising . text to it:

project-repository/file-1.txt

The fast fox that is brown within the sluggish dogs.

Save and shut the file by entering CTRL - X, Y, then ENTER, after which run the backup demand once more:

Output

Storage set to s3://[email protected]/example_space Final back-up at modification 2 discovered Indexing /home/sammy/project-repository Loaded file-1.txt (45) Backup for /home/sammy/project-repository at modification 3 finished

Because there have been changes that are new one of many files within repository, Duplicacy stuffed that file and uploaded it included in modification 3.

You may use the restore demand to return your repository back into a revision that is previous using the -r option and specifying the revision number. Note that it will not overwrite files that are existing the -overwrite choice is specified, such as this:

  • duplicacy restore -overwrite -r 2

After operating the restore demand, you'll concur that Duplicacy did certainly rewrite file-1.txt by checking whether it offers any articles:

If this demand does not create any production, then file-1.txt has returned to being a clear file along with effectively rolled straight back your repository towards the past modification.

Backing around several space stores

Storing backups at mulitple locations that are offsite been a common data security practice for many years. However, the process of backing up files to destinations that are multiple show tiresome and cause a drop in efficiency. There are numerous of third-party back-up tools, however, that may offer a solution that is quick backup information to numerous places in cloud.

To show this functionality in Duplicacy, include your next room towards the repository on server-01. You shall not be able to do this by running the init command again because that repository has already been initiated by Duplicacy and associated with your first Space. For these scenarios, you will need to use the add command which connects an repository that is already-initialized another storage space bucket.

Duplicacy’s add demand makes use of the syntax that is following

  • duplicacy add storage_id repository_id s3://region@endpoint/example_space_02

This appears mostly like the init demand utilized previously, using the main distinction being you to specify an ID for the new storage location that it requires. It will send backups when you ran the init command above, Duplicacy assigned the default ID to your first storage bucket, since that is the default location where. The storage space title you give your next room are whatever you’d like, nonetheless it could be helpful so you remember which Space it represents.( for it to be something descriptive*****)

With that information in your mind, include your space that is second to repository:

  • duplicacy include space_02 project_01 s3://nyc3@nyc3.digitaloceanspaces.com/example_space_02

You are now actually ready to straight back your repository up to your second Space. It’s recommended that you do this by first backing up your repository to your default storage location, and then using Duplicacy’s copy command to copy an backup that is identical towards 2nd storage space location:

  • duplicacy back-up
  • duplicacy content -from standard -to room_02

This will duplicate over each amount and snapshot from your own space that is first over your second. It’s important to note that the copy command is non-destructive, and it will not write over any files that are existing

Conclusion

whenever coupled with DigitalOcean Spaces, Duplicacy enables users to handle cloud backups with freedom. You need to back up one repository to multiple places in the cloud, Duplicacy could become an integral part of your backups solution.( if you need to back up the same repository from multiple computers or*****)

If you’re thinking about learning more on how to make use of Duplicacy, you can examine out of the task wiki on GitHub. As an alternative, if you’d always find out more about backup techniques generally speaking, see our guide on how best to Select a highly effective Backup technique for your VPS or our contrast between Object space vs. Block space Services.

LEAVE A REPLY

Please enter your comment!
Please enter your name here