Never Ending Security

It starts all here

Installing and Configuring Remote Backups Using Bacula on an Ubuntu 12.04 VPS

Installing and Configuring Bacula on an Ubuntu 12.04 VPS

Introduction

Bacula is a sophisticated backup solution that can handle local and remote backups by leveraging a client-server model. It has very flexible configuration options that provide complete control over the backup environment.

In this guide, we will be backing up the entire root filesystem of our Ubuntu 12.04 VPS.

Installing MySQL

Bacula uses an SQL database to manage its information. We will be using MySQL for this tutorial.

For an Ubuntu 12.04 VPS, this setup must be installed in advance, or else the following error will occur during installation of the bacula suite:

An error occurred while installing the database:

ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)

To avoid this situation, we can simply install MySQL prior to installing the bacula components:

sudo apt-get update
sudo apt-get install mysql-server

You will be prompted to provide and confirm a database administrator password. Enter a password to continue.

Installing Bacula

Next, we will install the bacula components:

sudo apt-get install bacula-server bacula-client

You will be prompted to set up a database for bacula. Select “yes” to allow dbconfig-common to automatically configure the database.

When prompted, enter the database administrator password that you selected when installing MySQL. It will then ask for an application password for bacula-director-mysql. Press “Enter” or “Return” to generate a random password.

Filesystem Configuration

Next, we need to create some directories to act as our backup and restore points.

Using the “-p” flag with the “mkdir” command creates any necessary parent directories along with the target directory:

sudo mkdir -p /bacula/backup /bacula/restore

We need to change the file permissions so that only the bacula process can access these locations.

sudo chown -R bacula:bacula /bacula
sudo chmod -R 700 /bacula

Configuring Bacula

Bacula has several components that must be configured independently in order to function correctly. The configuration files can all be found in the /etc/bacula/ directory.

Configuring bacula-dir.conf

Start by editing the bacula-dir.conf file:

sudo nano /etc/bacula/bacula-dir.conf

A few different sections must be modified for the backups that we will be performing in this guide.

First, search for the “Standard Restore template” section. Under the “Job” settings, change the Whereparameter to point to the restore point we created:

Job {
  Name = "RestoreFiles"
  Type = Restore
  Client=Blank-fd
  FileSet="Full Set"
  Storage = File
  Pool = Default
  Messages = Standard
  Where = /bacula/restore
}

Next, search for the “List of files to be backed up” section. Under the “FileSet” settings, we will add an option to use gzip to compress our archives.

We will then specify exactly what we would like to backup with the “File =” parameter. You can have multiple “File =” declarations in this section, each with a different path.

For this guide, we will be backing up the entire root (/) file system. Change the parameters to reflect that:

Include {
    Options {
      signature = MD5
      compression = GZIP
    }
# 
#  Put your list of files here, preceded by 'File =', one per line
#    or include an external list with:
#
#    File = file-name
#
#  Note: / backs up everything on the root partition.
#    if you have other partitions such as /usr or /home
#    you will probably want to add them too.
#
#  By default this is defined to point to the Bacula binary
#    directory to give a reasonable FileSet to backup to
#    disk storage during initial testing.
#
    File = /
  }

Finally, add file paths to be excluded from the backup. This is done in the Exclude section of the FileSet directory by using the same “File =” syntax that we used in the Include section.

The defaults are generally good here, but we need to change the path to the archive directory. We do not want to backup the backup folder. Change the second “File =” to reflect our root bacula file path:

Exclude {
    File = /var/lib/bacula
    File = /bacula
    File = /proc
    File = /tmp
    File = /.journal
    File = /.fsck
  }

Save and close the file.

Configuring bacula-sd.conf

The next file we need to edit is the bacula-sd.conf file. This will define the area where bacula stores its backups.

Open the bacula-sd.conf file with sudo privileges:

sudo nano /etc/bacula/bacula-sd.conf

Under the “Devices supported by this Storage daemon” section, in the “Device” configuration, change theArchive Device to reflect the backup location we set up earlier:

Device {
  Name = FileStorage
  Media Type = File
  Archive Device = /bacula/backup
  LabelMedia = yes;                   # lets Bacula label unlabeled media
  Random Access = Yes;
  AutomaticMount = yes;               # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = no;
}

Save and close the file.

Checking Configuration Syntax

Before continuing, check that the configuration options that we used are recognized by bacula. We can use bacula’s internal testing utilities to ensure that there are no syntax errors in the files we modified.

First, check the bacula director configuration:

sudo bacula-dir -tc /etc/bacula/bacula-dir.conf

If the command executes without returning any output, the configuration file is syntactically correct. Next, check the bacula storage configuration:

sudo bacula-sd -tc /etc/bacula/bacula-sd.conf

Again, no output signifies that our configuration file is valid.

At this time, restart the bacula services we modified to make the changes propagate:

sudo service bacula-sd restart
sudo service bacula-director restart

We are now ready to test our backups.

Testing the Backup

We will be interacting with bacula through its internal console. This can be accessed with the “bconsole” command:

sudo bconsole

You will be dropped into a bacula console with an asterisk (*) for a prompt. Begin by issuing the “label” command. You will be prompted to enter a name for the archive file that will be created:

label
Automatically selected Catalog: MyCatalog
Using Catalog "MyCatalog"
Automatically selected Storage: File
Enter new Volume name: MyArchiveName

Select the kind of storage “pool” for the backup. Since our backup will be stored as a file, select #2:

2: File

Bacula now knows how we want to write the data for our backup. We can now run our backup to test that it works correctly:

run
A job name must be specified.
The defined Job resources are:
     1: BackupClient1
     2: BackupCatalog
     3: RestoreFiles
Select Job resource (1-3):

Select #1 to choose the backup that we configured.

1: BackupClient1

At the prompt, type “yes” to perform the backup:

yes

Bacula will tell you that you have messages. The messages will be the output generated by the backup. Check the messages for errors by typing:

messages

You will see some output lines. Check again until you see a summary showing the results of the backup. It might take a few minutes. It should say “Termination: Backup OK”. Check that no errors were generated.

Testing the Restore

Now that a backup has been created, it is important to check that it can be restored properly. In the bacula console, type:

restore all

A selection menu will appear with many different options. We would like to restore the most recent backup, so select Option 5:

5: Select the most recent backup for a client

This will drop you into a virtual file tree with the entire directory structure that you backed up. This interface allows for simple commands to mark and unmark files to be restored.

Because we specified that we wanted to “restore all”, every backed up file is pre-selected for restoration.

If you would like to fine-tune your selection, you can navigate and list files with the “ls” and “cd” commands, mark files for restoration with “mark”, and unmark files with “unmark”. A full list of commands is available by typing “help” into the console.

help

When you are finished, exit the file selection by typing:

done

Confirm that you would like to restore:

yes

Again, check the messages for errors when the restoration is complete.

messages

The summary should have a line that reads “Termination: Restore OK”. When you are finished, type “exit” to leave the bacula console:

exit

Checking the Filesystem

To check that the backup was completed as expected, list the contents of the backup directory. You will need to use sudo since the directory structure is owned by the “bacula” user:

sudo ls /bacula/backup

You should see a file with the archive name you chose.

Next, we should check our restore point. List the contents of the restore point directory:

sudo ls /bacula/restore

You should see a mirror of the root file structure, excluding the files and directories that were listed in the “Exclude” section of the bacula-dir.conf file.

You now have a basic bacula setup that can backup your local filesystem. Bacula is also a powerful solution for remote backups

How to Configure Remote Backups Using Bacula in an Ubuntu 12.04 VPS

Introduction

Bacula is a sophisticated backup solution that can handle local and remote backups by leveraging a client-server model. It has very flexible configuration options that provide complete control over the backup environment.

In this guide, we will be setting up a remote backup involving a backup cloud server and a client machine that will be completely backed up.

For the purposes of this guide, the backup VPS has a hostname of BackupServer and the client machine’s host name is ClientMachine. Change these values to reflect the server names of the machines you are using.

Table of Contents

  1. Client Installation
  2. Client Machine Configuration
  3. Backup Server Configuration
    • Job Configuration
    • Client Configuration
    • Storage Configuration
    • Pool Configuration
    • Editing bacula-sd.conf
  4. Checking the Configuration and Restarting Services
  5. Testing Remote Backups
  6. Testing Remote Restore
  7. Checking the Filesystem

Client Installation

No local backups will be stored on the remote client machine, so not all of the bacula components need to be installed.

Install the the bacula-fd (file-daemon) and the bconsole (bacula console) on this machine with apt-get using the bacula-client metapackage:

sudo apt-get update
sudo apt-get install bacula-client

The necessary components are now installed and ready to be configured.

Client Machine Configuration

The configuration of the client environment is relatively straightforward. We will only be editing the bacula file daemon configuration file. Open the file with root privileges with the following command:

sudo nano /etc/bacula/bacula-fd.conf

We need to change a few items and save some information that we will need for our server configuration. Begin by finding the Director section.

The bacula director is located on the backup VPS. Change the “Name” parameter to the hostname of your backup server followed by “-dir”.

You also need to copy the password that bacula generated for your client file daemon to some place that you’ll have available when you are configuring your backup server settings:

Director {
  Name = BackupServer-dir
  Password = "u2LK-yBrQzfiEsc6NWftHEhymmdPWsklN"  # Copy this password for later reference!
}

Next, we need to adjust one parameter in the FileDaemon section. We will change the “FDAddress” parameter to match the IP address or domain name of our client machine. The “Name” parameter should already be populated correctly with the client file daemon name:

FileDaemon {                          # this is me
  Name = ClientMachine-fd
  FDport = 9102                  # where we listen for the director
  WorkingDirectory = /var/lib/bacula
  Pid Directory = /var/run/bacula
  Maximum Concurrent Jobs = 20
  FDAddress = ClientMachine.DomainName.com
}

We also need to configure this daemon to pass its log messages to the backup cloud server. Find theMessages section and change the “director” parameter to match your backup cloud server’s name.

Messages {
  Name = Standard
  director =  BackupServer-dir = all, !skipped, !restored
}

Save the file and exit.

Check that your configuration file has the correct syntax with the following command:

sudo bacula-fd /etc/bacula/bacula-fd.conf

If the command returns no output, the configuration file has valid syntax. Restart the file daemon to use the new settings:

sudo service bacula-fd restart

The client machine is now correctly configured.

In this example, we would like to restore to a folder on this same machine. Create the file structure and lock down the permissions and ownership for security with the following commands:

sudo mkdir -p /bacula/restore
sudo chown -R bacula:bacula /bacula
sudo chmod -R 700 /bacula

The client machine is now configured correctly. Next, we will configure the backup cloud server to pull the file data from the client.

Backup Server Configuration

Log into the backup cloud server to complete this stage of the configuration.

The bulk of the configuration is actually done on the backup server. That is because the bacula “director” manages all other bacula processes and must be able to communicate correctly with a number of different components.

To start, open the “bacula-dir.conf” file with administrator privileges:

sudo nano /etc/bacula/bacula-dir.conf

Job Configuration

Begin by finding the Job Section. The current configuration is named “BackupClient1” and is used for the backup server’s local backup. We need to change the name to reflect this:

Job {
  Name = "LocalBackup"
  JobDefs = "DefaultJob"
}

Now that we have identified the first job as backing up on the local machine, we want to create a similar job for backup up our remote client. To do this, copy and paste the job definition below the one you just modified.

Change the name to reflect the fact that this is a remote backup scenario. The “Client” parameter identifies our remote client file daemon as the target for our backup. The Pool parameter allows bacula to store its remote backups separate from our local backups. We will define the pool we’re referencing later in the file:

Job {
  Name = "RemoteBackup"
  JobDefs = "DefaultJob"
  Client = ClientMachine-fd
  Pool = RemoteFile
}

Next, define a place for the remote backups to restore. We will use the directory that we created on the client machine to restore remote backups.

Find the “RestoreFiles” job definition. Copy the current entry and paste it below. We will then modify some entries to label it accurately and work with client machine:

Job {
  Name = "RestoreRemote"
  Type = Restore
  Client=ClientMachine-fd
  FileSet="Full Set"
  Storage = File     
  Pool = Default
  Messages = Standard
  Where = /bacula/restore
}

Client Configuration

Find the Client definition. We will change the “Address” parameter to reflect our actual backup cloud server IP address instead of using localhost. The password should already be set correctly for the local machine.

Client {
  Name = BackupServer-fd
  Address = BackupServer.DomainName.com
  FDPort = 9102
  Catalog = MyCatalog
  Password = "CRQF7PW-mJumFtENX2lqGvJ6gixPTyRQp"          # password for Local FileDaemon
  File Retention = 30 days            # 30 days
  Job Retention = 6 months            # six months
  AutoPrune = yes                     # Prune expired Jobs/Files
}

The next step is to actually define the client machine that we’ve been referencing in our configuration. Copy the Client entry we just modified paste it below the current definition. This new definition will be for the remote machine that we are backing up.

Match the name to your client machine’s hostname followed by “-fd”. The “Address” line needs to match the client machine’s IP address or domain name as well.

Finally, this is where you enter the password that you copied from the remote client’s file daemon configuration file. Make sure that you modify this password value, or else bacula will not function correctly.

Client {
  Name = ClientMachine-fd
  Address = ClientMachine.DomainName.com
  FDPort = 9102 
  Catalog = MyCatalog
  Password = "u2LK-yBrQzfiEsc6NWftHEhymmdPWsklN"          # password for Remote FileDaemon
  File Retention = 30 days            # 30 days
  Job Retention = 6 months            # six months
  AutoPrune = yes                     # Prune expired Jobs/Files
}

Storage Configuration

Next, change the “Address” parameter in the Storage section with the IP address or domain name of the backup VPS. Once again, the password should already be correct here:

Storage {
  Name = File
# Do not use "localhost" here   
  Address = BackupServer.DomainName.com                # N.B. Use a fully qualified name here
  SDPort = 9103
  Password = "097dnj3jw1Yynpz2AC38luKjy5QTnGoxS"
  Device = FileStorage
  Media Type = File
}

Pool Configuration

Find the Pool definitions section. We will first add a parameter to the “File” pool definition. Add the “Label Format” parameter to the definition and choose a prefix to name local file backups. For this guide, local backups will have “Local-” as a prefix.

Pool {
  Name = File
  Pool Type = Backup
  Recycle = yes                       # Bacula can automatically recycle Volumes
  Label Format = Local-
  AutoPrune = yes                     # Prune expired volumes
  Volume Retention = 365 days         # one year
  Maximum Volume Bytes = 50G          # Limit Volume size to something reasonable
  Maximum Volumes = 100               # Limit number of Volumes in Pool
}

Next, we need to copy the section we just modified and paste it below the current entry. This will be set up for remote backup storage.

Change the name of the new pool to reflect its job of storing remote backups. Also, change the prefix by adjusting the “Label Format” parameter to be “Remote-”

Pool { 
  Name = RemoteFile
  Pool Type = Backup
  Recycle = yes                       # Bacula can automatically recycle Volumes
  Label Format = Remote-
  AutoPrune = yes                     # Prune expired volumes
  Volume Retention = 365 days         # one year
  Maximum Volume Bytes = 50G          # Limit Volume size to something reasonable
  Maximum Volumes = 100               # Limit number of Volumes in Pool
}

Save and close the file.

Editing bacula-sd.conf

Next, open the “bacula-sd.conf” file with root privileges:

sudo nano /etc/bacula/bacula-sd.conf

Change the “SDAddress” parameter to reflect the backup server’s IP address or domain name:

Storage {                             # definition of myself
  Name = BackupServer-sd
  SDPort = 9103                  # Director's port
  WorkingDirectory = "/var/lib/bacula"
  Pid Directory = "/var/run/bacula"
  Maximum Concurrent Jobs = 20
  SDAddress = BackupServer.DomainName.com
}

Save and close the file.

Checking the Configuration and Restarting Services

Check the configuration with the following commands:

sudo bacula-dir /etc/bacula/bacula-dir.conf
sudo bacula-sd /etc/bacula/bacula-sd.conf

If no output is returned, the configuration files have valid syntax. If this is the case, restart the daemons to use the new settings:

sudo service bacula-director restart
sudo service bacula-sd restart

Testing Remote Backups

Log into the bacula console to test the backup functionality.

sudo bconsole

Test that the bacula director can connect to the remote machine by typing the following:

status
Status available for:
     1: Director
     2: Storage
     3: Client
     4: All
Select daemon type for status (1-4):

Choose #3 to check on the client connection and then select the remote machine:

3: Client
2: ClientMachine-fd

It should return a summary with some statistics, confirming that we can connect to the remote file daemon.

Run a test backup of the remote system by typing the following command:

run
Automatically selected Catalog: MyCatalog
Using Catalog "MyCatalog"
A job name must be specified.
The defined Job resources are:
     1: LocalBackup
     2: RemoteBackup
     3: BackupCatalog
     4: RestoreFiles
     5: RestoreRemote
Select Job resource (1-5):

Select the “RemoteBackup” option to run a backup of the remote machine. Type “yes” to begin the backup:

2: RemoteBackup

The director will send the backup task to the remote file daemon and which will pass its information to the backup server’s storage daemon. You can check the status of the job using the “status” command as we did above. You should also check the messages using the “messages” command.

messages

If you continue to check messages, eventually you will receive a summary of the backup operation. It should contain the line “Termination: Backup OK” if everything went as expected.

Testing Remote Restore

Now, test the restore functionality:

restore all

Choose the “Select the most recent backup for a client” option. Select the remote client that we have just backed up:

5: Select the most recent backup for a client
2: ClientMachine-fd

You will be dropped into a file tree where you are able to select the files you would like to restore with the “mark” and “unmark” commands.

We have chosen to restore everything, so we can just type “done” to move on. Select the job that we defined for remote restoration and type “yes” to run the restoration:

done
2: RestoreRemote

Again, you can check the restoration with the “status” and “messages” commands. You should eventually get a summary in the messages that contains the line “Termination: Restore OK”. This means the restoration was successful. Type “exit” to leave the bacula console.

exit

Checking the Filesystem

We can check that our remote backup file has the correct file format with the following command:

sudo ls /bacula/backup
LocalBackup   Remote-0002

As you can see, our backup file for the remote system has adapted the naming conventions we supplied. The local backup is not named according to our convention because it is from before our changes.

If we log into our remote client machine, we can check our restore with the following line:

sudo ls /bacula/restore
bin   dev  home        lib    media  opt   run   selinux  sys  var
boot  etc  initrd.img  lost+found  mnt    root  sbin  srv      usr  vmlinuz

As you can see, we have restored the filesystem to this folder correctly.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s