(22-page full report - registration required)
Terabyte-Plus In-Memory Database System (IMDS) Benchmark Brief
(4-page synopsis - no registration required)
In-memory database systems (IMDSs) hold out the promise of breakthrough performance for time-sensitive, data-intensive tasks. Yet IMDSs’ compatibility with very large databases (VLDBs) has been largely uncharted. This benchmark analysis fills the information gap and pushes the boundaries of IMDS size and performance. Using McObject’s 64-bit eXtremeDB-64, the application creates a 1.17 Terabyte, 15.54 billion row database on a 160-core Linux-based SGI® Altix® 4700 server. It measures time required for database provisioning, backup and restore. In SELECT, JOIN and SUBQUERY tests, benchmark results range as high as 87.78 million query transactions per second. The report also examines efficiency in utilizing all of the test bed system’s 160 processors. The full report includes complete database schema, relevant application source code and additional analysis.
In-Memory vs. RAM-Disk Databases: A Linux-based Benchmark
A new type of DBMS, the in-memory database system (IMDS), claims breakthrough performance and availability via memory-only processing. But doesn't database caching, or using a RAM-disk, achieve the same result with a traditional (disk-based) database? This benchmark tests eXtremeDB against a widely used embedded database, in both disk-based and RAM-disk modes. Deployment on RAM-disk boosts the traditional database by as much as 74 percent, but it still lags the IMDS substantially. Read about the architectural reasons for this disparity.
Benchmarking In-Memory & On-Disk Databases With Hard-Disk, SSD and Memory-Tier NAND Flash
In-memory database systems (IMDSs) accelerate data management. To provide data durability, IMDSs offer transaction logging, in which changes to the database are recorded on persistent media. But critics object that logging re-introduces the storage-related latency of on-disk DBMSs. Will an IMDS with transaction logging still outperform a traditional DBMS? Will type of storage—hard disk drive vs. solid state drive vs. state-of-the-art memory-tier products—affect the results? McObject’s original research answers these and related questions.
Pipelining Vector-Based Statistical Functions for In-Memory Analytics
Columnar data handling accelerates time series analysis (including market data analysis) by maximizing the proportion of relevant data brought into CPU cache with each fetch. As explained in this white paper, McObject's eXtremeDB Financial Edition delivers columnar data handling, and builds on it with pipelining technology that enables multiple vector-based statistical functions to work on a given data sequence (time series) within CPU cache, without the need to "materialize" interim results as output in main memory. This eliminates the latency caused by back-and-forth transfers between CPU cache and memory.
Database Persistence, Without the Performance Penalty
Some applications require higher data durability than in-memory storage provides. What if DRAM could be made persistent? AgigA Tech’s AGIGARAM non-volatile DIMM (NVDIMM) delivers that capability. McObject benchmarked the eXtremeDB In-Memory Database System using AGIGARAM as storage, including “pulling the plug” mid-execution, and comparing the NVDIMM to transaction logging as a solution for data durability/recoverability. This paper presents the benchmark tests and results.
NoSQL, Object Caching & IMDSs: Alternatives for Highly Scalable Data Management
Has the traditional relational database management system (RDBMS) reached its limits in today's high volume, highly scalable real-time applications? Arguably the RDBMS imposes a bottleneck in such environments; this widely held view can be seen in current enthusiasm over NoSQL solutions. McObject's white paper examines RDBMS limits and the technologies that are suggested to replace or supplement it, including NoSQL (actually an umbrella term for numerous software categories), object caching solutions (such as Memcached), and in-memory database systems (IMDSs). Characteristics discussed include persistence, performance, scalability, recoverability, data integrity, and database developer tools.
In-Memory Database Systems: Myths and Facts
In the past decade, software vendors have emerged to offer in-memory database system (IMDSs), described as accelerating data management by holding all records in main memory. But is this new? For years, database management systems have employed caching. Several vendors offer something called “memory tables.” RAM-disks and -- more recently -- Flash-based solid state drives (SSDs) are available for use with databases. Do IMDSs really add anything unique? In fact, the distinction between these technologies and true in-memory database systems is significant, and can be critical to project success. This paper explains the key differences, replacing IMDS myths with facts.
Data Management in Set-Top Box Electronic Programming Guides
The electronic programming guide (EPG) enables digital television users to search, filter and customize program listings and even control access to content. These capabilities entail significant real-time data management, and a handful of vendors have incorporated commercial, off-the-shelf (COTS) databases in their set-top boxes. This report presents lessons learned in such projects, mapping emerging digital TV standards, set-top box data management requirements, and typical data objects and interrelationships. Sample code and embedded database schema focus on efficiencies gained by implementing EPG data management using an in-memory database.
Data Management for Military and Aerospace Embedded Systems
This white paper examines the data management needs of military and aerospace embedded systems, and focuses on existing and emerging data management technology and its suitability to meet these requirements.
Will The Real IMDS Please Stand Up?
In-memory database systems (IMDSs) have changed the software landscape, enabling "smarter" real-time applications and sparking mergers and acquisitions involving the largest technology companies. But IMDSs’ popularity has sparked a flurry of products falsely claiming to be in-memory database systems. Understanding the distinction is critical to determining the performance, cost and ultimately the success or failure of a solution. This white paper examines specific products, seeking to answer the question, “is it really an in-memory database system?”
Re-Inventing Data Management For Intelligent Devices
Intelligent devices such as set-top boxes, consumer electronics, and networking gear are adding software "smarts" and managing larger volumes of more complex data –a challenge typically met with embedded database management systems (DBMS). But traditional databases, with roots in business processing, present CPU and memory requirements that are too expensive for price-sensitive high-tech gear. This paper examines the emerging on-device database requirements, and looks at one in-memory database, eXtremeDB, developed in response to these needs.
SQL or Navigational Database APIs: Which Best Fits Embedded Systems?
For embedded systems developers, the choice of database application programming interfaces (APIs) often boils down to the high-level SQL language and Call Level Interface, and navigational APIs integrated with C++ and other languages. Which API is best? This paper examines the familiarity and ease-of-use often cited as benefits of SQL. A sample application is implemented with SQL and then with a navigational API, to explore the issues of programming ease, maintainability, determinism and learning curve. Special attention is given to the significance of SQL optimizers in evaluating embedded database APIs.
The Role of In-Memory Database Systems for Routing Table Management in IP Routers
Core Internet bandwidth grows at triple the rate of CPU power, but high-value applications depend on managing much more data traffic at the network's edge. This requires rapid evolution of routing table management (RTM) software within IP routers. This paper examines using in-memory database systems (IMDS) to add RTM development flexibility, data integrity and fault tolerance. It provides performance examples on Linux and Windows 2000. This embedded database solution adds to vendors’ ability to produce new generations of routers faster and at less cost, improving their competitive position.