Incremental Backup and Restore

As explained in the Database Backup and Restore page, eXtremeDB provides incremental backup and restore APIs for both in-memory and persistent databases as well as a file backup capability for persistent databases. (Please see the File Backup page for further details.)

An incremental backup can be either a full snapshot of the entire database (backup type FULL) or an incremental (partial image) of the database containing only differences in the database content between two consecutive backup records (backup type INCREMENTAL). Note that a FULL backup (snapshot) must be done at least once before an INCREMENTAL backup can be done successfully. If backup type AUTO is specified a FULL or INCREMENTAL record will be created depending on the content of the backup file. If there is no snapshot in the file yet the backup process will create one; otherwise a partial backup record is created.

Incremental backup must be enabled by the application at runtime and this introduces minimal additional overhead as the eXtremeDB runtime tracks all modifications to the database by saving the modified pages. The next call for a backup has all the necessary information to make an incremental backup record that contains only the pages that were changed since the previous backup. For example, if an application creates only a couple of objects (instances of database classes) in between of two backups, the incremental backup record will contain only some data pages, so-called "backup clusters", containing the modified pages and some affected data. The incremental backup procedure does not block normal database processing. The eXtremeDB runtime tracks all modifications in the background. Then the backup procedure simply scans those modification markers and copies the content of the data pages without affecting transaction processing or blocking indexes.

An incremental backup can be run in synchronous or asynchronous mode.

Recommended Practice

To assure that a database can be restored with minimal loss of data it is important to do a full backup (snapshot) when the database has sufficient initial contents; then perform regular incremental backups at appropriate intervals. The backup procedure can be run from a dedicated task or process. In pseudo code the procedure could be as follows:

    connect to the database 
    while (!stop)
        backup( connection, file, label )
        sleep with appropriate granularity
    disconnect from the database

This code will produce a number of the database backups until the stop variable is set by some external logic. Later it will be possible to restore the database when necessary. The code above may be run as a dedicated task in the main database application, as a separate process or even as a batch file using xSQL to control the database.

To restore the database, the procedure could be as follows:

    create the database
    connect to the database
    restore( connection, file, label )
    perform normal database operations
    disconnect from the database

Once restored, the backup cycle should then be run to produce more backups.

Notes on database backup

1. The size of the backup file could be equal to or exceed the actual size of the database. This is because backup frames contain overhead -- the increments can be larger than the actual size of modified data.

2. While the database is being backed up, it is available for both read and write operations with the exception of the short period of time at the end of the backup process. At the end of each backup, the database is locked and becomes unavailable for updates. This is transparent to applications; a write transaction would wait for the database to become available.

3. The restore from a backup requires exclusive access to the database.

4. The backup protocol maintains database integrity and does not allow restoring the database from an out-of-order backup.

Native Language APIs

The incremental backup/restore APIs are specific to the application language. Please use the links below to view detailed explanations and examples for your development environment:

C Incremental backup/restore in C
C++ Incremental backup/restore in C++
SQL Incremental backup/restore using SQL
xSQL Incremental backup/restore using xSQL

The other native language APIs (Python, Java and C#) allow incremental backup via the SQL statements.


Send feedback on this topic to McObject.