Will The Real In-memory Database Please Stand Up?
A summary of the white paper by McObject, “Will The Real In-Memory Database Please Stand Up?” on the advantages of a real in-memory database (IMDS) for data management.
When you hatch a database management system, it will, by design and implementation, be either an in-memory database system or an on-disk database management system. The choice affects the fundamental optimization strategies that will be baked into the database system code. To optimize an on-disk database is to minimize disk I/O, so its designers and developers will use extra CPU cycles and extra memory if doing so will reduce or eliminate I/O. Conversely, IMDSs by definition eliminate all disk I/O; their optimization is all about delivering the highest performance at a given level of processing power, thus reducing demand for CPU cycles is a key objective. And since memory is storage for an IMDS, reducing memory overhead (i.e. RAM consumed for anything other than storing data) is also a key objective.
Those optimizations are diametrically opposed. So it follows that you cannot, 5, 10, 15 or 20 years after an on-disk DBMS was designed and developed, suddenly turn it into an in-memory database system and expect the same performance or efficient memory use as a database system written to be in memory in the first place. The on-disk design goals mentioned above were baked in a long time ago, and while you will end up with something that is faster than the original, it won’t be as fast or as efficient in its use of memory as a true IMDS, created from scratch with the appropriate set of design goals.
When it comes to evolving a product to meet market demands, an in-memory database vendor has an advantage. They can always choose to use more CPU cycles and/or memory in order to add a feature. So an IMDS can evolve into a hybrid in-memory/on-disk database system and the on-disk implementation can be every bit as good as the database system that was originally written to be on-disk. In other words, it’s easy to add ingredients to your pie (CPU cycles and memory consumption). But once they’re in the pie mix, you can’t really take those ingredients out, short of a rewrite.
The following link will trigger a download of the complete white paper Will The Real In-Memory Database Please Stand Up? from McObject. For more on this topic, please download In-Memory Database Systems: Myths and Facts.