In Search of Data Durability and High Performance: Benchmarking In-Memory & On-Disk Databases With Hard-Disk, SSD and Memory-Tier NAND Flash.

In the past decade, in-memory database systems (IMDSs) have emerged as the database solution for many embedded applications. In-memory databases work with data entirely in main memory, eliminating the file I/O, cache management, data transfer and other overhead that is hard-wired into traditional “on-disk” database management systems (DBMSs).

But some applications require data durability beyond what RAM-based storage can provide. As a solution, IMDS vendors offer transaction logging, which keeps a record of changes to the database, enabling recovery in the event of system failure. This logging, however, requires writes to persistent storage. Do IMDSs with transaction logging retain their performance advantage?

In one of the first benchmark tests of its kind, McObject examined on-disk database system performance vs. that of an IMDS operating with transaction logging. To highlight the effects on performance of today’s diverse data storage options, we ran separate tests using hard disk drive (HDD), solid state drive (SSD) and state-of-the-art flash-based devices as storage for on-disk database records, and for the IMDS transaction log.