Incremental Backup and Restore

As explained in the Database Backup and Restore page, eXtremeDB provides incremental backup and restore APIs for both in-memory and persistent databases as well as a file backup capability for persistent databases. (Please see the File Backup page for further details.)

An incremental backup can be either a full snapshot of the entire database (backup type FULL) or an incremental backup (partial image) of the database containing only differences in the database content between two consecutive backup records (backup type INCREMENTAL). Note that a FULL backup (snapshot) must be done at least once before an INCREMENTAL backup can be done successfully. If backup type AUTO is specified a FULL or INCREMENTAL backup will be created depending on the content of the backup file. If there is no snapshot in the file yet, the backup process will create one; otherwise a partial backup is created.

Incremental backup must be enabled by the application at runtime; this introduces minimal overhead because the eXtremeDB runtime tracks all modifications to the database by keeping track of the modified pages. The next call for a backup has all the necessary information to make an incremental backup that contains only the pages that were changed since the previous backup. For example, if an application creates only a couple of objects/rows between two backups, the incremental backup will contain only some data pages, so-called "backup clusters", containing the modified pages and some affected data. The incremental backup procedure does not block normal database processing. The eXtremeDB runtime tracks all modifications in the background. Then the backup procedure simply scans those modification markers and copies the content of the data pages without affecting transaction processing or blocking indexes.

Recommended Practice

To assure that a database can be restored with minimal loss of data it is important to do a full backup (snapshot) when the database has sufficient initial contents; then perform regular incremental backups at appropriate intervals. The backup procedure can be run from a dedicated task or process. In pseudo code the procedure could be as follows:

 
    connect to the database 
    while (!stop)
    {
        backup( connection, file, label )
        sleep with appropriate granularity
    } 
    disconnect from the database
     

This code will produce a number of backups until the stop variable is set by some external logic. Later, it will be possible to restore the database if necessary. The code above may be run as a dedicated task in the main database application, as a separate process or even as a batch file using xSQL to control the backup process.

To restore the database, the procedure could be as follows:

 
    create the database
    connect to the database
    restore( connection, file, label )
    perform normal database operations
    disconnect from the database
     

Once restored, the backup cycle should then be run to produce new backups.

Notes on database backup

1. The size of the backup file could be equal to or exceed the actual size of the database. This is because backup frames contain overhead -- the increments can be larger than the actual size of the modified data.

2. While the database is being backed up, it is available for both read and write operations with the exception of a short period of time at the end of the backup process. At the end of each backup, the database is locked and becomes unavailable for updates. This is transparent to applications; a write transaction would wait for the database to become available.

3. The restore from a backup requires exclusive access to the database.

4. The backup protocol maintains database integrity and does not allow restoring the database from an out-of-order backup.

Native Language APIs

The incremental backup/restore APIs are specific to the application language. Please use the links below to view detailed explanations and examples for your development environment:

C	Incremental backup/restore in C
C++	Incremental backup/restore in C++
SQL	Incremental backup/restore using SQL
xSQL	Incremental backup/restore using xSQL

The other native language APIs (Python, Java and C#) allow incremental backup via the SQL statements.