yaronr/backup-volume-container repository overview

Over 900 people downloaded this image within days of its creation, so I guess I have to say something :)

What this image does

1. When you execute this image, it will first pull the latest backup from the underlying persistent storage.
2. Next, it will monitor the data folder's file system for changes (create, modify, add, remove) - recursively.
3. If a change is detected, it will wait for a 'quiet period' of N seconds, before initiating an incremental backup to the underlying store. Quiet period == no changes.

Features

• Secure: Encrypted and signed archives, and transport
• Bandwidth and space efficient
• Standard file format: tar + rdiff
• Choice of remote protocol: scp/ssh, ftp, rsync, HSI, WebDAV, Tahoe-LAFS, and Amazon S3
• Choice of backend: S3 Amazon Web Services, Google Cloud Storage, Rackspace Cloud, Dropbox, copy.com, ftp, ftps, gdocs, gio, rsync, mega, swift...

I'm using it under CoreOS, so this is a part of my unit file that shows how you could use it:

ExecStartPre=-/usr/bin/rm -rf ${DATA_DIR} 
ExecStartPre=-/usr/bin/mkdir ${DATA_DIR}

ExecStart=/bin/bash -c  ' \ 	
  /usr/bin/docker run \ 	
    -v  ${DATA_DIR}:/var/backup \  
    --name=${CONTAINER_NAME} \  
    --rm \ 
    -e  AWS_ACCESS_KEY_ID=<ID> -e AWS_SECRET_ACCESS_KEY=<SECRET> \
    --privileged  \
    yaronr\backup-volume-container s3://s3.amazonaws.com/<bucket_name>/backup 60'

60 is the quiet period. It's important to keep this value high enough, else you will be creating a lot of backups.

Backups are incremental, and can be encrypted. Underlying persistent stores include:

This image uses Duplicity for backup and restore. Go to the Duplicity⁠ docs to see more.

The way I use it, is to map the container's data dir to a host dir, and map host data/subdirs to relevant containers. You could also map it directly to your various docker containers without the host.

I prefer having the host as an intermediary, because it increases robustness and is a cleaner approach in my view.

I built this image for my own use, but the source code is available and I will be very to receive pull requests that enhance / improve this solution, as well as comments whether or not you find this useful and how you use it.