McObject IMDS Persistent Storage Benchmark

In the past decade, in-memory database systems (IMDSs) have emerged as the database solution for many embedded applications. In-memory databases work with data entirely in main memory, eliminating the file I/O, cache management, data transfer and other overhead that is hard-wired into traditional “on-disk” database management systems (DBMSs).

But some applications require data durability beyond what RAM-based storage can provide. As a solution, IMDS vendors offer transaction logging, which keeps a record of changes to the database, enabling recovery in the event of system failure. This logging, however, requires writes to persistent storage. Do IMDSs with transaction logging retain their performance advantage?

In one of the first benchmark tests of its kind, McObject examined on-disk database system performance vs. that of an IMDS operating with transaction logging. To highlight the effects on performance of today’s diverse data storage options, we ran separate tests using hard disk drive (HDD), solid state drive (SSD) and state-of-the-art flash-based devices as storage for on-disk database records, and for the IMDS transaction log.

The link below will trigger McObject’s free white paper containing the complete benchmark test results.

Read, In Search of Data Durability and High Performance: Benchmarking In-Memory & On-Disk Databases With Hard-Disk, SSD and Memory-Tier NAND Flash.

Test hardware and software

McObject’s testing platform consisted of the following off-the-shelf components:

• Server. Dell PowerEdge T110 Tower Server with 4GB of 1333mhz memory. The original equipment hard-disk storage was not used.
• Hard Disk Drive. Western Digital VelociRaptor, 10,000 RPM, 600 GB.
• Solid State Drive. SanDisk Extreme Solid State Drive, 240 GB.
• Memory-tier storage. Fusion ioDrive2, 1.2 TB, installed via PCI Express within the test server.
• In-memory database system. McObject eXtremeDB IMDS.
• Disk-based DBMS. McObject eXtremeDB Fusion – with this hybrid DBMS, the developer can combine in-memory and on-disk database storage, or specify entirely on-disk storage, with notations in the database design (schema). The 100% on-disk storage used in these tests implements caching, file I/O and all other aspects of a disk-based DBMS that affect performance.

Methodology

The test application examined five database operations: record inserts, record updates, record deletes, index searches and table traversals. It measured performance on each operation while looping, with each loop comprising a database transaction, and each loop repeating the database operation at least once, as follows:

Insert records test – 2 record inserts/loop
Updates test – 2 record updates/loop
Deletes test – 2 record deletes/loop
Index search test – 8 searches/loop
Table traversals test – 8 traversals/loop

Total loops executed for each test ranged from 20,000 to 400,000. The benchmark application recorded the number of loops accomplished per millisecond for each of the two database types (On-disk DBMS and IMDS+TL) with each of the three storage devices (HDD, SSD and Fusion ioDrive2).

Results

Inserts, Updates & Deletes

The test showed that for insert, update and delete operations, the IMDS performing transaction logging maintained its speed advantage over the traditional on-disk database system. This advantage increased as the IMDS moved from using hard disk, to SSD, and finally to the Fusion ioDrive2 as storage for the transaction log.

Figure 1 shows results in loops/ms for each of the configurations, as well as the performance multiple exhibited by each of the IMDS+TL configurations over the baseline on-disk DBMS with hard disk storage. For example, in the test of database deletes, the IMDS+TL using the solid state drive was 11.87 times faster than the on-disk DBMS using HDD. For database inserts, the IMDS+TL using Fusion ioDrive2 for storage was 20.05 times faster than the on-disk DBMS using HDD.

Figure 1.

Index Searches & Table Traversals

Database index searches and table traversals showed no significant performance change when moving from on-disk DBMS to IMDS+TL, or when changing storage medium from hard disk to either SSD or the Fusion ioDrive2. This result was expected, because such database “reads” are typically much less costly, in performance terms, than insert, update and delete operations. Because they do not change the database contents, reads impose little overhead, and hence have less to gain from moving from disk to main memory for data storage, or from disk to SSD or to the Fusion ioDrive2 to store the transaction log. In these tests, the on-disk DBMS cache was large enough to hold the database. Had it not been, the cache misses would have exhibited different performance characteristics for HDD, SSD and Fusion ioDrive2.