Getting the latest stable release¶

pip install django-dbbackup

Getting the latest release from trunk¶

In general, you should not be downloading and installing stuff directly off repositories – especially not if you are backing up sensitive data.

Security is important, bypassing PyPi repositories is a bad habbit, because it will bypass the fragile key signatures authentication that are at least present when using PyPi repositories.

pip install -e git+https://github.com/mjs7231/django-dbbackup.git#egg=django-dbbackup

DBBACKUP_DATABASES¶

List of key entries for settings.DATABASES which shall be used to connect and create database backups.

Default: list(settings.DATABASES.keys()) (keys of all entries listed)

DBBACKUP_TMP_DIR¶

Directory to be used in local filesystem for temporary files.

Default: tempfile.gettempdir()

DBBACKUP_TMP_FILE_MAX_SIZE¶

Maximum size in bytes for file handling in memory before write a temporary file in DBBACKUP_TMP_DIR.

Default: 10*1024*1024

DBBACKUP_CLEANUP_KEEP and DBBACKUP_CLEANUP_KEEP_MEDIA¶

When issueing dbbackup and mediabackup with --clean option, the number of old backup files are looked for and removed.

Default: 10 (backups)

DBBACKUP_CLEANUP_FILTER¶

A callable that takes a filename (of an old backup, to be cleaned) and returns a boolean indicating whether the backup should be kept (True) or deleted (False).

Default: lambda filename: False

This can be used to keep monthly backups, for example.

DBBACKUP_DATE_FORMAT¶

Date format to use for naming files. It must contain only alphanumerical characters, '_', '-' or '%'.

Default: '%Y-%m-%d-%H%M%S'

DBBACKUP_HOSTNAME¶

Used to identify backup by server name in their file name..

Default: socket.gethostname()

DBBACKUP_FILENAME_TEMPLATE¶

The template to use when generating the backup filename. By default this is '{databasename}-{servername}-{datetime}.{extension}'. This setting can also be made a function which takes the following keyword arguments:

def backup_filename(databasename, servername, datetime, extension):
    pass

DBBACKUP_FILENAME_TEMPLATE = backup_filename

This allows you to modify the entire format of the filename, for example, if you want to take advantage of Amazon S3’s automatic expiry feature, you need to prefix your backups differently based on when you want them to expire.

{datetime} is rendered with DBBACKUP_DATE_FORMAT.

DBBACKUP_MEDIA_FILENAME_TEMPLATE¶

Same as DBBACKUP_FILENAME_TEMPLATE but for media files backups.

PGP¶

You can encrypt a backup with the --encrypt option. The backup is done using GPG.

python manage.py dbbackup --encrypt

...or when restoring from an encrypted backup:

python manage.py dbrestore --decrypt

Requirements:

• Install the python package python-gnupg: pip install python-gnupg.
• You need GPG key. (GPG manual)
• Set the setting DBBACKUP_GPG_RECIPIENT to the name of the GPG key.

DBBACKUP_GPG_ALWAYS_TRUST¶

The encryption of the backup file fails if GPG does not trust the public encryption key. The solution is to set the option ‘trust-model’ to ‘always’. By default this value is False. Set this to True to enable this option.

DBBACKUP_GPG_RECIPIENT¶

The name of the key that is used for encryption. This setting is only used when making a backup with the --encrypt or --decrypt option.

DBBACKUP_SEND_EMAIL¶

Controls whether or not django-dbbackup sends an error email when an uncaught exception is received.

Default: True

DBBACKUP_SERVER_EMAIL¶

The email address that error messages come from, such as those sent to DBBACKUP_ADMINS.

Default: django.conf.settings.SERVER_EMAIL

DBBACKUP_ADMINS¶

A list of all the people who get code error notifications. When DEBUG=False and an operation raises an exception, DBBackup will email these people with the full exception information. This should be a tuple of (Full name, email address).

Default: django.conf.settings.ADMINS

DBBACKUP_EMAIL_SUBJECT_PREFIX¶

Subject-line prefix for email messages sent by DBBackup.

Default: '[dbbackup] '

CONNECTOR¶

Absolute path to a connector class by default it is:

• dbbackup.db.sqlite.SqliteConnector for 'django.db.backends.sqlite3'
• dbbackup.db.mysql.MysqlDumpConnector for django.db.backends.mysql
• dbbackup.db.postgresql.PgDumpConnector for django.db.backends.postgresql
• dbbackup.db.postgresql.PgDumpGisConnector for django.contrib.gis.db.backends.postgis
• dbbackup.db.mongodb.MongoDumpConnector for django_mongodb_engine

All supported built-in connectors are listed below.

EXCLUDE¶

Tables to exclude from backup as list. This option can be unavailable for connectors making snapshots.

EXTENSION¶

Extension of backup file name, default 'dump'.

DUMP_CMD¶

Path to the command used for create a backup, default is the appropriate command supposed to be in your PATH, for example: 'mysqldump' for MySQL.

This setting is useful only for connectors using command line tools (children of dbbackup.db.base.BaseCommandDBConnector)

RESTORE_CMD¶

Same as DUMP_CMD but for restoring action.

DUMP_PREFIX and RESTORE_PREFIX¶

String to include as prefix of dump or restore command. It will be add with a space between launched command and its prefix.

DUMP_SUFFIX and RESTORE_PREFIX¶

String to include as suffix of dump or restore command. It will be add with a space between launched command and its suffix.

ENV, DUMP_ENV and RESTORE_ENV¶

Environment variables used during command running, default are {}. ENV is used for every command, DUMP_ENV and RESTORE_ENV override the values defined in ENV during the dedicated commands.

USE_PARENT_ENV¶

Specify if the connector will use its parent’s environment variables. By default it is True to keep PATH.

SqliteConnector¶

It is in pure Python and copy the behavior of .dump command for create a SQL dump.

SqliteCPConnector¶

You can also use dbbackup.db.sqlite.SqliteCPConnector for make simple raw copy of your database file, like a snapshot.

In-memory database aren’t dumpable with it.

SINGLE_TRANSACTION¶

When doing a restore, wrap everything in a single transaction, so errors cause a rollback.

This corresponds to --single-transaction argument of psql and pg_restore.

Default: True

DROP¶

With PgDumpConnector, it includes tables dropping statements in dump file. PgDumpBinaryConnector drops at restoring.

This corresponds to --clean argument of pg_dump and pg_restore.

Default: True

PSQL_CMD¶

Path to psql command used for administration tasks like enable PostGIS for example, default is psql.

PASSWORD¶

If you fill this settings PGPASSWORD environment variable will be used with every commands. For security reason, we advise to use .pgpass file.

ADMIN_USER¶

Username used for launch action with privileges, extension creation for example.

ADMIN_PASSWORD¶

Password used for launch action with privileges, extension creation for example.

OBJECT_CHECK¶

Validate documents before insert in database (option --objcheck in command line), default is True.

DROP¶

Replace objects that are already in database, (option --drop in command line), default is True.

Setup¶

To store your backups on the local file system, simply setup the required settings below.

DBBACKUP_STORAGE = 'django.core.files.storage.FileSystemStorage'
DBBACKUP_STORAGE_OPTIONS = {'location': '/my/backup/dir/'}

Available settings¶

location

Absolute path to the directory that will hold the files.

base_url

URL that serves the files stored at this location.

file_permissions_mode

The file system permissions that the file will receive when it is saved.

directory_permissions_mode

The file system permissions that the directory will receive when it is saved.

See FileSystemStorage’s documentation for a full list of available settings.

Setup¶

In order to backup to Amazon S3, you’ll first need to create an Amazon Webservices Account and setup your Amazon S3 bucket. Once that is complete, you can follow the required setup below.

pip install boto django-storages

Add the following to your project’s settings:

DBBACKUP_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
DBBACKUP_STORAGE_OPTIONS = {
    'access_key': 'my_id',
    'secret_key': 'my_secret',
    'bucket_name': 'my_bucket_name'
}

Available settings¶

access_key - Required

Your AWS access key as string. This can be found on your Amazon Account Security Credentials page.

secret_key - Required

Your Amazon Web Services secret access key, as a string.

bucket_name - Required

Your Amazon Web Services storage bucket name, as a string. This directory must exist before attempting to create your first backup.

host - Default: 's3.amazonaws.com' (boto.s3.connection.S3Connection.DefaultHost)

Specify the Amazon domain to use when transferring the generated backup files. For example, this can be set to 's3-eu-west-1.amazonaws.com'.

use_ssl - Default: True

default_acl - Required

See Django Storage S3 storage official documentation for more informations about available settings.

Setup your Dropbox account¶

1. Login to Dropbox and navigate to Developers » MyApps. https://www.dropbox.com/developers/apps
2. Click the button to create a new app and name it whatever you like. For reference, I named mine ‘Website Backups’.
3. After your app is created, note the options button and more importantly the ‘App Key’ and ‘App Secret’ values inside. You’ll need those later.

Setup your Django project¶

pip install dropbox django-storages

...And make sure you have the following required settings:

DBBACKUP_STORAGE = 'storages.backends.dropbox.DropBoxStorage'
DBBACKUP_STORAGE_OPTIONS = {
    'oauth2_access_token': 'my_token',
}

Available settings¶

oauth2_access_token - Required

Your OAuth access token

root_path

Jail storage to this directory

Setup¶

pip install django-storages

DBBACKUP_STORAGE = 'storages.backends.ftp.FTPStorage
DBBACKUP_STORAGE_OPTIONS = {
    'location': 'ftp://user:pass@server:21'
}

Settings¶

location - Required

A FTP URI with optional user, password and port. example: 'ftp://anonymous@myftp.net'

base_url

URL that serves with HTTP(S) the files stored at this location.

Setup¶

We use FTP backend from Django-Storages (again).

pip install django-storages

Here a simple configuration:

DBBACKUP_STORAGE = 'storages.backends.ftp.FTPStorage'
DBBACKUP_STORAGE_OPTIONS = {'location': ftp://myftpserver/}

Setup¶

This backend is from Django-Storages with paramiko under.

pip install paramiko django-storages

The next configuration admit SSH server grant a the local user:

DBBACKUP_STORAGE = 'storages.backends.sftpstorage.SFTPStorage'
DBBACKUP_STORAGE_OPTIONS = {'host': 'myserver'}

Available settings¶

host - Required

Hostname or adress of the SSH server

root_path - Default ~/

Jail storage to this directory

params - Default {}

Argument used by method:paramikor.SSHClient.connect(). See paramiko SSHClient.connect() documentation for details.

interactive - Default False

A boolean indicating whether to prompt for a password if the connection cannot be made using keys, and there is not already a password in params.

file_mode

UID of the account that should be set as owner of the files on the remote.

dir_mode

GID of the group that should be set on the files on the remote host.

known_host_file

Absolute path of know host file, if it isn’t set "~/.ssh/known_hosts" will be used.

Help¶

usage: manage.py dbbackup [-h] [--version] [-v {0,1,2,3}]
                          [--settings SETTINGS] [--pythonpath PYTHONPATH]
                          [--traceback] [--no-color] [--noinput] [-q] [-c]
                          [-d DATABASE] [-s SERVERNAME] [-z] [-e]
                          [-o OUTPUT_FILENAME] [-O OUTPUT_PATH]

Backup a database, encrypt and/or compress and write to storage.

optional arguments:
  -h, --help            show this help message and exit
  --version             show program's version number and exit
  -v {0,1,2,3}, --verbosity {0,1,2,3}
                        Verbosity level; 0=minimal output, 1=normal output,
                        2=verbose output, 3=very verbose output
  --settings SETTINGS   The Python path to a settings module, e.g.
                        "myproject.settings.main". If this isn't provided, the
                        DJANGO_SETTINGS_MODULE environment variable will be
                        used.
  --pythonpath PYTHONPATH
                        A directory to add to the Python path, e.g.
                        "/home/djangoprojects/myproject".
  --traceback           Raise on CommandError exceptions
  --no-color            Don't colorize the command output.
  --noinput             Tells Django to NOT prompt the user for input of any
                        kind.
  -q, --quiet           Tells Django to NOT output other text than errors.
  -c, --clean           Clean up old backup files
  -d DATABASE, --database DATABASE
                        Database(s) to backup specified by key separated by
                        commas(default: all)
  -s SERVERNAME, --servername SERVERNAME
                        Specify server name to include in backup filename
  -z, --compress        Compress the backup files
  -e, --encrypt         Encrypt the backup files
  -o OUTPUT_FILENAME, --output-filename OUTPUT_FILENAME
                        Specify filename on storage
  -O OUTPUT_PATH, --output-path OUTPUT_PATH
                        Specify where to store on local filesystem

Help¶

usage: manage.py dbrestore [-h] [--version] [-v {0,1,2,3}]
                           [--settings SETTINGS] [--pythonpath PYTHONPATH]
                           [--traceback] [--no-color] [--noinput] [-q]
                           [-d DATABASE] [-i INPUT_FILENAME] [-I INPUT_PATH]
                           [-s SERVERNAME] [-c] [-p PASSPHRASE] [-z]

Restore a database backup from storage, encrypted and/or compressed.

optional arguments:
  -h, --help            show this help message and exit
  --version             show program's version number and exit
  -v {0,1,2,3}, --verbosity {0,1,2,3}
                        Verbosity level; 0=minimal output, 1=normal output,
                        2=verbose output, 3=very verbose output
  --settings SETTINGS   The Python path to a settings module, e.g.
                        "myproject.settings.main". If this isn't provided, the
                        DJANGO_SETTINGS_MODULE environment variable will be
                        used.
  --pythonpath PYTHONPATH
                        A directory to add to the Python path, e.g.
                        "/home/djangoprojects/myproject".
  --traceback           Raise on CommandError exceptions
  --no-color            Don't colorize the command output.
  --noinput             Tells Django to NOT prompt the user for input of any
                        kind.
  -q, --quiet           Tells Django to NOT output other text than errors.
  -d DATABASE, --database DATABASE
                        Database to restore
  -i INPUT_FILENAME, --input-filename INPUT_FILENAME
                        Specify filename to backup from
  -I INPUT_PATH, --input-path INPUT_PATH
                        Specify path on local filesystem to backup from
  -s SERVERNAME, --servername SERVERNAME
                        If backup file is not specified, filter the existing
                        ones with the given servername
  -c, --decrypt         Decrypt data before restoring
  -p PASSPHRASE, --passphrase PASSPHRASE
                        Passphrase for decrypt file
  -z, --uncompress      Uncompress gzip data before restoring

Help¶

usage: manage.py mediabackup [-h] [--version] [-v {0,1,2,3}]
                             [--settings SETTINGS] [--pythonpath PYTHONPATH]
                             [--traceback] [--no-color] [--noinput] [-q] [-c]
                             [-s SERVERNAME] [-z] [-e] [-o OUTPUT_FILENAME]
                             [-O OUTPUT_PATH]

Backup media files, gather all in a tarball and encrypt or compress.

optional arguments:
  -h, --help            show this help message and exit
  --version             show program's version number and exit
  -v {0,1,2,3}, --verbosity {0,1,2,3}
                        Verbosity level; 0=minimal output, 1=normal output,
                        2=verbose output, 3=very verbose output
  --settings SETTINGS   The Python path to a settings module, e.g.
                        "myproject.settings.main". If this isn't provided, the
                        DJANGO_SETTINGS_MODULE environment variable will be
                        used.
  --pythonpath PYTHONPATH
                        A directory to add to the Python path, e.g.
                        "/home/djangoprojects/myproject".
  --traceback           Raise on CommandError exceptions
  --no-color            Don't colorize the command output.
  --noinput             Tells Django to NOT prompt the user for input of any
                        kind.
  -q, --quiet           Tells Django to NOT output other text than errors.
  -c, --clean           Clean up old backup files
  -s SERVERNAME, --servername SERVERNAME
                        Specify server name to include in backup filename
  -z, --compress        Compress the archive
  -e, --encrypt         Encrypt the backup files
  -o OUTPUT_FILENAME, --output-filename OUTPUT_FILENAME
                        Specify filename on storage
  -O OUTPUT_PATH, --output-path OUTPUT_PATH
                        Specify where to store on local filesystem

Help¶

usage: manage.py mediarestore [-h] [--version] [-v {0,1,2,3}]
                              [--settings SETTINGS] [--pythonpath PYTHONPATH]
                              [--traceback] [--no-color] [--noinput] [-q]
                              [-i INPUT_FILENAME] [-I INPUT_PATH]
                              [-s SERVERNAME] [-e] [-p PASSPHRASE] [-z] [-r]

Restore a media backup from storage, encrypted and/or compressed.

optional arguments:
  -h, --help            show this help message and exit
  --version             show program's version number and exit
  -v {0,1,2,3}, --verbosity {0,1,2,3}
                        Verbosity level; 0=minimal output, 1=normal output,
                        2=verbose output, 3=very verbose output
  --settings SETTINGS   The Python path to a settings module, e.g.
                        "myproject.settings.main". If this isn't provided, the
                        DJANGO_SETTINGS_MODULE environment variable will be
                        used.
  --pythonpath PYTHONPATH
                        A directory to add to the Python path, e.g.
                        "/home/djangoprojects/myproject".
  --traceback           Raise on CommandError exceptions
  --no-color            Don't colorize the command output.
  --noinput             Tells Django to NOT prompt the user for input of any
                        kind.
  -q, --quiet           Tells Django to NOT output other text than errors.
  -i INPUT_FILENAME, --input-filename INPUT_FILENAME
                        Specify filename to backup from
  -I INPUT_PATH, --input-path INPUT_PATH
                        Specify path on local filesystem to backup from
  -s SERVERNAME, --servername SERVERNAME
                        If backup file is not specified, filter the existing
                        ones with the given servername
  -e, --decrypt         Decrypt data before restoring
  -p PASSPHRASE, --passphrase PASSPHRASE
                        Passphrase for decrypt file
  -z, --uncompress      Uncompress gzip data before restoring
  -r, --replace         Replace existing files

Help¶

usage: manage.py listbackups [-h] [--version] [-v {0,1,2,3}]
                             [--settings SETTINGS] [--pythonpath PYTHONPATH]
                             [--traceback] [--no-color] [--noinput] [-q]
                             [-d DATABASE] [-z] [-Z] [-e] [-E]
                             [-c CONTENT_TYPE]

optional arguments:
  -h, --help            show this help message and exit
  --version             show program's version number and exit
  -v {0,1,2,3}, --verbosity {0,1,2,3}
                        Verbosity level; 0=minimal output, 1=normal output,
                        2=verbose output, 3=very verbose output
  --settings SETTINGS   The Python path to a settings module, e.g.
                        "myproject.settings.main". If this isn't provided, the
                        DJANGO_SETTINGS_MODULE environment variable will be
                        used.
  --pythonpath PYTHONPATH
                        A directory to add to the Python path, e.g.
                        "/home/djangoprojects/myproject".
  --traceback           Raise on CommandError exceptions
  --no-color            Don't colorize the command output.
  --noinput             Tells Django to NOT prompt the user for input of any
                        kind.
  -q, --quiet           Tells Django to NOT output other text than errors.
  -d DATABASE, --database DATABASE
                        Filter by database name
  -z, --compressed      Exclude non-compressed
  -Z, --not-compressed  Exclude compressed
  -e, --encrypted       Exclude non-encrypted
  -E, --not-encrypted   Exclude encrypted
  -c CONTENT_TYPE, --content-type CONTENT_TYPE
                        Filter by content type 'db' or 'media'

runtests.py¶

You can test code in local machine with the runtests.py script:

python runtests.py

But if argument are provided, it acts as manage.py so you can simply launch other command to test deeply, example:

# Enter in Python shell
python runtests.py shell

# Launch a particular test module
python runtests.py test dbbackup.tests.test_utils

All tests are stored in dbbackup.tests.

functional.sh¶

It tests at the higher level if backup/restore mechanism is alright. It becomes powerful because of the configuration you can give to it. See the next chapter for explanation about it.

Configuration¶

DBBackup contains a test Django project at dbbackup.tests and its settings module. This configuration takes care of the following environment variables:

DATABASE_URL - Default: 'sqlite:///%s' % tempfile.mktemp()

A URL representing the used database, see DJ-Database-URL for all the available format.

MEDIA_ROOT - Default= tempfile.mkdtemp()

Django’s MEDIA_ROOT, useful if you want test media backup from filesystem

STORAGE - Default: dbbackup.tests.utils.FakeStorage

Storage used for backups

STORAGE_OPTIONS

Options for instanciate the chosen storage. It must be in format "key1=foo,key2=bar" and will be convert into a dict.

dbrestore¶

--backup-extension has been removed, DBBackup should automaticaly know the appropriate file.

Listing from this command, --list, has been removed in favor of listbackups command.

mediabackup¶

mediabackup‘s --no-compress option has been replaced by --compress for keep consistency with other commands.

Now this command can backup remote storage, not only filesystem’s DBBACKUP_BACKUP_DIRECTORY.

mediarestore¶

You are now able to restore your media files backups. Unfortunately you’ll not be able to restore old backup files.

SQLite¶

Previously backup was made by copy the database file, now you have the choice between make a raw snaphot or make a real SQL dump. It can be useful to exclude tables or don’t overwrite data.

If you want to restore your old backups choose dbbackup.db.sqlite.SqliteCPConnector.