Docker BitBucket Server and BitBucket Server Backup Client

March 24, 2019   

Hosting your own BitBucket Server instance is an easy way to manage your important code and projects, and implementing an automated backup process is an important component to this setup. The guide below describes the setup of the BitBucket Server Backup Client to create snapshots of your BitBucket home directory and databases.

Github Repo

The sample configuration files for Docker and Docker Compose can be found at the following URL: [https://github.com/rshaltry/docker-bitbucket-backup-client]

Initial BitBucket Server Build

If you do not already have a BitBucket Server instance in your Docker environment, you should first clone my docker-bitbucket-backup-client repository:

$ git clone [email protected]:rshaltry/docker-bitbucket-backup-client.git

Edit the env.bitbucket and env.bitbucket-backup files to set your authentication values:

POSTGRES_USER=bitbucket
POSTGRES_PASSWORD=b1tbuck3t
POSTGRES_DB=bitbucket
POSTGRES_ENCODING=UTF8
bitbucket.home=/source
bitbucket.user=backup
bitbucket.password=BackupPass
bitbucket.baseUrl=http://dockerhost:7990

You should also create the directory for storing the BitBucket server backup files:

$ mkdir -p /backup/bitbucket

NOTE: You can change the destination of the backups on the Docker host, but make sure to also update the volume configuration in the docker-compose.yml file.

Then use the provided docker-compose.yml compose file to setup a new BitBucket Server instance:

$ docker-compose up -d

After a few minutes, you should be able to access your new environment on port 7990 using the IP address or hostname of the Docker host, e.g.:

http://dockerhost:7990

Running an Initial Backup

To initialize and test the backup process, use Docker Compose to run the bitbucket-backup container, which automatically bootstraps the backup process using its call to Java from the Docker entrypoint:

$ docker-compose run --rm bitbucket-backup

The backup process will start running within a few seconds:

2019-03-24 19:13:55,430 INFO         Initializing
2019-03-24 19:13:58,365 INFO         Using Bitbucket 6.1.1
2019-03-24 19:13:58,565 INFO         Contacting Bitbucket

During the backup process, your BitBucket Server instance will show the backup progress:

BitBucket Server status screen while Backup Client is executing

If the backup is successful, you should see messages that look something like:

2019-03-24 19:13:55,430 INFO         Initializing
2019-03-24 19:13:58,365 INFO         Using Bitbucket 6.1.1
2019-03-24 19:13:58,565 INFO         Contacting Bitbucket
2019-03-24 19:13:58,934 INFO         Bitbucket has been locked for maintenance. It may be unlocked with token: fe167a0ed89753edf97753718db638c82dddeeae
2019-03-24 19:13:59,216 INFO         Starting database backup on Bitbucket. It may be cancelled with token: 3e0251abd710a8029f70ef3655g6a27c1f93a823
2019-03-24 19:14:02,862 INFO   (12%) Now scanning /source
2019-03-24 19:14:03,434 INFO   (12%) Verifying files in Bitbucket home "/source"
2019-03-24 19:14:03,462 INFO   (12%) Verifying Bitbucket home
2019-03-24 19:14:04,015 INFO   (12%) Backing up Bitbucket home
2019-03-24 19:14:12,367 INFO   (36%) Backing up Bitbucket home
2019-03-24 19:14:12,556 INFO   (58%) Backing up Bitbucket home
2019-03-24 19:14:15,368 INFO   (58%) Waiting on Bitbucket database backup to complete
2019-03-24 19:14:17,490 INFO   (98%) Waiting on Bitbucket database backup to complete
2019-03-24 19:14:22,491 INFO  (100%) Bitbucket has completed backing up the database
2019-03-24 19:14:22,491 INFO  (100%) Downloading database backup zip from Bitbucket
2019-03-24 19:14:22,703 INFO  (100%) Backup complete: /root/bitbucket-backup-client-3.5.0/backups/bitbucket-20190324-191422-702.tar
2019-03-24 19:14:22,708 INFO  (100%) Unlocking Bitbucket using token: faaa7a0987dd03edf977537438b638c829e6afae
2019-03-24 19:14:22,766 INFO  (100%) Total backup time: 25.25 s

Scheduling Automated BitBucket Server Backups

Using a simple crontab entry that runs the bitbucket-backup Docker container, the BitBucket Server can automatically be backed up to the host system:

$ crontab -e

Add the following line to the crontab to backup every night at midnight:

0 0 * * * /usr/bin/docker-compose -f /docker/docker-compose/bitbucket/docker-compose.yml run --rm bitbucket-backup 2>&1 > /dev/null

Automated Backups with Remote S3 Sync

Automating the BitBucket Server backup process is just one part of the backup strategy, and ideally backup archives should be stored in a location outside the Docker instance. Some basic backup location suggestions might include:

The bitbucket-backup.sh script from the repo you cloned earlier shows a simple example of syncing BitBucket Server backup archives to an Amazon S3 bucket:

#!/bin/bash

BACKUP_PATH=/backup/bitbucket    # Path to BitBucket backups
DAYS_TO_KEEP=3                   # Days to keep old backups
AWS_PROFILE=default                  # AWS CLI profile name
AWS_S3_BUCKET=mybucket-backups       # AWS S3 bucket name
AWS_S3_DESTINATION=bitbucket         # Directory name for backup sync
AWS_CONFIG_FILE=/path/to/.aws/config # AWS configuration file path

# Determine paths for script and executables
PATH=/bin:/usr/bin:/usr/local/bin
DIR="$( cd "$( /usr/bin/dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && /bin/pwd )"
DOCKER_COMPOSE=docker-compose
FIND=find
GZIP=gzip
AWS=aws

$DOCKER_COMPOSE -f $DIR/docker-compose.yml run --rm bitbucket-backup \
  && $FIND $BACKUP_PATH -mtime +$DAYS_TO_KEEP -exec rm -f "{}" \; \
  && $GZIP $BACKUP_PATH/*.tar \
  && $AWS s3 sync $BACKUP_PATH s3://$AWS_S3_BUCKET/$AWS_S3_DESTINATION --profile $AWS_PROFILE --delete --quiet

The crontab entry to kick off this backup script at midnight every night is below:

0 0 * * * /docker/docker-compose/bitbucket/bitbucket-backup.sh >> /tmp/bitbucket-backup.log 2>&1

If you are using the example script above, you should have a separate IAM user created specifically for backups, with the following Permission Policy (where mybucket-backups is the name of your S3 bucket):

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "s3:ListAllMyBuckets",
            "Resource": "arn:aws:s3:::*"
        },
        {
            "Effect": "Allow",
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::mybucket-backups",
                "arn:aws:s3:::mybucket-backups/*"
            ]
        }
    ]
}