Select Page

Will The Real In-memory Database Please Stand Up

From the in-memory database system experts at McObject

 

A summary of the white paper by McObject, “Will The Real In-memory Database Please Stand Up?” on the advantages of a real in-memory database (IMDS) for data management.

When you hatch a database management system, it will, by design and implementation, be either an in-memory database system or an on-disk database management system. The choice affects the fundamental optimization strategies that will be baked into the database system code. To optimize an on-disk database is to minimize disk I/O, so its designers and developers will use extra CPU cycles and extra memory if doing so will reduce or eliminate I/O. Conversely, IMDSs by definition eliminate all disk I/O; their optimization is all about delivering the highest performance at a given level of processing power, thus reducing demand for CPU cycles is a key objective. And since memory is storage for an IMDS, reducing memory overhead (i.e. RAM consumed for anything other than storing data) is also a key objective.

Those optimizations are diametrically opposed. So it follows that you cannot, 5, 10, 15 or 20 years after an on-disk DBMS was designed and developed, suddenly turn it into an in-memory database system and expect the same performance or efficient memory use as a database system written to be in-memory in the first place. The on-disk design goals mentioned above were baked in a long time ago, and while you will end up with something that is faster than the original, it won’t be as fast or as efficient in its use of memory as a true IMDS, created from scratch with the appropriate set of design goals.

When it comes to evolving a product to meet market demands, an in-memory database vendor has an advantage. They can always choose to use more CPU cycles and/or memory in order to add a feature. So an IMDS can evolve into a hybrid in-memory/on-disk database system and the on-disk implementation can be every bit as good as the database system that was originally written to be on-disk. In other words, it’s easy to add ingredients to your pie (CPU cycles and memory consumption). But once they’re in the pie mix, you can’t really take those ingredients out, short of a rewrite.
Read the white paper
"In other words, it’s easy to add ingredients to your pie (CPU cycles and memory consumption). But once they’re in the pie mix, you can’t really take those ingredients out, short of a rewrite."

A well-designed hybrid database system, based on a true in-memory database, is more efficient than one that has bolted on in-memory capability to a persistent database system.

Bolting in-memory capacity onto a persistent database system isn't as efficient as starting from an in-memory DBMS

The Port Authority Building in Antwerp is a fascinating example of architectural design, but not the most efficient use of resources.

i

For more information about in-memory database performance, review the white paper, Gaining an Extreme Performance Advantage.

i

Read more about in-memory database management systems in the white paper, In-Memory Database Systems: Myths and Facts.

“Those optimizations are diametrically opposed. So it follows that you cannot, 5, 10, 15 or 20 years after an on-disk DBMS was designed and developed, suddenly turn it into an in-memory database system and expect the same performance or efficient memory use as a database system written to be in memory in the first place.”

Steve Graves
Co-founder and CEO, McObject

Learn more about eXtremeDB

eXtremeDB is a database development tool that is ideally suited for systems that are safety-critical or have stringent constraints. The type-safe native C API, comprehensive target-side debugging capabilities, host-side diagnostic tools, optimized data layouts, integrated small footprint embedded web-server, and dozens of supported toolchains all ensure maximum flexibility for developers and minimize time-to-market.

eXtremeDB features that maximize database reliability
$

The in-memory database designed specifically for embedded systems

$

Watch the Webinar: Fault-tolerant Memory Management for Mission Critical Embedded Systems

$

Learn how eXtremeDB reduces memory consumption

$

Learn more about flexible eXtremeDB for embedded and client/server

$

We created the eXtremeDB in-memory database for professional developers, and offer more features to chose from than any other DBMS.  

The eXtremeDB reliable in-memory database system is found in systems that cannot afford to fail

Operator Interface for the transportation subsystem, Radioactive Waste Treatment Facility, Smolensk Nuclear Power Plant. Real-time data populating the BISMARC interface is drawn from the eXtremeDB in-memory database.

Hybrid data storage

Unlike other IMDS, eXtremeDB can combine the strengths of on-disk and in-memory database systems. In other words, eXtremeDB databases can be all-in-memory, all-persistent, or have a mix of in-memory tables and persistent tables. This unparalleled flexibility enables developers to tailor data management in order to optimize applications for speed and persistence.  System developers can make intelligent trade-offs between performance, cost-efficiency, power consumption, and physical space-conserving data storage hardware.

Learn more about hybrid data storage
Chart of the hybrid storage of the eXtremeDB in-memory database system

Related Resources

Watch this and other Webinars from the DBMS experts at McObject

Webinars for Professional Developers

Watch to on-demand Webinars, hosted by experts, about proven database management system practices.  Watch “Multi-Core & Embedded Software: Optimize Performance by Resolving Resource Contention“.  Or, “Embedded Databases: Make or Break Technology Choices for High Performance Applications” and others.

I

Watch Embedded Databases: Building In Always On High Availability
This Webinar highlights the issue of operational continuity: how can a database system survive the failure of the software or hardware environment in which it operates?

I

Watch What Makes a Database System ‘In-Memory’?
Join McObject CEO Steve Graves to explore this topic, including the limitations (and burden) of database caching; data transfer and duplication; volatility and recoverability, and more. Gain ideas and techniques for building better, faster software.

I

Watch Eliminating Database Corruption
Why corruption occurs and provides strategies to prevent it, focusing on hidden dangers – like storage device settings that can undermine data consistency – as well as more recognizable risks, such as passing wrongly typed data to a database run-time.

 Review our list of Webinars

More than just IoT database Webinars, find our experts in respected journals

Articles for Professional Developers

See a list of articles

White Papers for Professional Developers

We have been testing, improving on, and retesting our software from the beginning in 2001 in order to provide our clients with the best possible data management solutions. Read “Database Persistence, Without The Performance Penalty” and more.

Review our research

White paper: Database Persistence, Without The Performance Penalty

Some applications require higher data durability than in-memory storage provides. What if DRAM could be made persistent? AgigA Tech’s AGIGARAM non-volatile DIMM (NVDIMM) delivers that capability. McObject benchmarked the eXtremeDB In-Memory Database System using AGIGARAM as storage, including “pulling the plug” mid-execution, and comparing the NVDIMM to transaction logging as a solution for data durability/recoverability. This paper presents the benchmark tests and results.