Features for a Faster Database
Database management system features to look for if speed matters to your next project.
McObject has specialized in database management systems since 2001. We compiled this list of features you might want to search for if fast data management is a priority for your next project.
eXtremeDB was designed from the beginning to maximize database speed, flexibility and reliability for professional developers. We hope you find this list of value.
In-memory database systems (IMDSs) offer superior performance and the possibility of very small RAM, CPU and storage demands. IMDSs (or RAM databases) boost speed and reduce footprint by eliminating logical and physical disk I/O, multiple data copies, and irrelevant tasks, such as caching.
In contrast, on-disk databases (HDD/SSD) cache frequently requested data in memory, for faster access, but write database updates, insertions and deletes through the cache to be stored to disk. On the plus side, byte-for-byte, disk storage can be cheaper than RAM, and can also take less physical space.
A hybrid database like eXtremeDB can be all-in-memory, all-persistent, or have a mix of in-memory tables and persistent tables with a simple database schema.
Multiple Database Index Types
The right index can boost lookup speed logarithmically as well as reduce RAM and CPU demands. While the B-Tree is the best known index, many others can be more efficient in specific circumstances, such as geospatial/mapping and telecom/networking applications. Less well-known indexes include Hash table, R-Tree, Patricia trie, Trigram and others.
Persistent databases keep recently-used records in cache, so they can be accessed without physical I/O. But managing the cache is itself logic that requires substantial memory and CPU cycles, so even a “cache hit” underperforms an in-memory database. In-memory databases do not cache data.
As a hybrid database, eXtremeDB can be used all in-memory, all persistent or in combination. Specifying one set of data as transient (managed in memory), while choosing persistent storage for other record types, requires a simple database schema declaration.
Normally, a database cache starts out empty and fills up as data is written to and/or fetched from the database. This is called a “cold” cache, and while the cache is cold, every search request will result in a cache-miss. Some database systems, including eXtremeDB, allow you to save the cache before shutting down a database, and to reload (or “pre-warm”) the cache on restart of the database.
A few database systems, including eXtremeDB, provide some control over the least-recently-used (LRU) algorithm that is used to determine what data to eject from the cache when it is necessary to bring new data into an already-full cache. Generically, this is called cache prioritization.
Pipelining is the technique in eXtremeDB that accelerates processing by combining the database system’s vector-based statistical functions into assembly lines of processing for time series data, with the output of one function becoming input for the next. Calculations are pipelined in order to keep data within CPU cache during its transformation by multiple functions. Without pipelining, interim results from each function’s transformation would be transferred back and forth between CPU cache and main memory, imposing significant latency.
Learn more about Pipelining from eXtremeDB. Pipelining utilizes a built-in library of vector-based statistical functions, with the output of one function becoming input for the next to keep the interim results in CPU cache and reduce database latency.
Resource Conflict Resolution
Threads running in parallel often contend for system resources, actually reducing overall performance. Achieving multi-core’s promised linear performance gains hinges on resolving this issue. This Webinar, Multi-core and Embedded Software: Optimize Performance by Resolving Resource Contention looks at two such conflicts — contention in updating a shared data store, and threads vying for access to the C/C++ memory manager — in-depth, and addresses them with techniques that can provide the basis to solve other multi-core resource conflicts.
eXtremeDB offers a “pessimistic” MURSIW (MUltiple Reader, SIngle Writer) transaction manager for all in-memory systems. Generally, the exceptional speed of an in-memory database benefits from the greater simplicity of MURSIW relative to MVCC, but either transaction manager can be used in any circumstance, simply by choosing which library to link with the application code.
Database pages can be stored in a compressed format and decompressed when accessed by the application. This can maximize available storage space, which could be important for an in-memory database, or even a persistent database stored on a relatively small SSD.
Run-length encoding (RLE) compression can be applied to time series data (i.e. fields defined as the ‘sequence’ data type). In McObject’s tests, activating this feature reduced storage space requirements by 90% and increased speed in reading the database by 21%. Compression and RLE are separate features that can be used individually.
Data can also be compressed between server and device in eXtremeDB’s Active Replication Fabric for IoT networks. Operating in low-bandwidth networks requires techniques to compress the network traffic. To solve this problem for all network components of eXtremeDB at once, the compression is implemented at the system abstraction layer (SAL).
Learn more about eXtremeDB Active Replication Fabric and how it offers reliable data management for IoT by solving 5 challenges for IoT system developers.
- Bidirectional Replication
- Intermittent connections
- Multi-tier replication (e.g. edge-to-gateway-to-gateway-to-cloud)
- Compression to maximize limited bandwidth networks
Flexible Data Layout
Relational databases typically store data in tables consisting of rows and columns, and transfer data between storage, main memory and CPU cache by organizing the data into pages and transferring whole pages of rows.
Time series data benefits from organizing data in a column-wise manner, wherein a database page contains only values of a single column of data. In other words, with a columnar approach, tables’ columns (instead of rows) are the basic unit for constructing the database pages used for DBMS input/output.
eXtremeDB implements columnar data layout for fields of type ‘sequence’. Sequences can be combined to form a time series, ideal for working with tick streams, historical quotes, IoT sensor data and other sequential/chronological data.
eXtremeDB is proven, reliable database management, used in systems that cannot afford to fail from nuclear power plants to military aircraft to financial technology (FinTech).
eXtremeDB sets the standard in database management system reliability. Industry leaders around the world depend on eXtremeDB for reliable embedded database management.
Articles for Professional Developers
- “On Time Series Analysis and Big Data. Interview with Andrei Gorine” ODBMS.org
- “What is a distributed database system?” IoT Agenda, TechTarget.com
- “What’s Changing in the Satellite Industry?” SatMagazine
Webinars for Professional Developers
Watch to on-demand Webinars, hosted by experts, about proven database management system practices. We invite you to watch:
- Multi-core and Embedded Software: Optimize Performance by Resolving Resource Contention
- Embedded Databases: Building In Always On High Availability
Review our list of Webinars
White Papers for Professional Developers
McObject is continually researching, testing, improving on, and retesting our software in order to provide our clients with the best possible data management solutions. We invite you to read “Data Management for Military and Aerospace Embedded Systems” and more.
Review our research
Network & Telecom
Network gear developers build on proven eXtremeDB speed and reliability, combined in-memory and persistent data layouts, optimized access methods and unmatched flexibility.
JVC, DirecTV, GoPro and others all discovered that eXtremeDB’s small code size, portable data format and efficient use of compute & storage can reduce component cost while also supporting data-hungry new features.
eXtremeDB’s sophisticated event notification systems, time series data processing and high availability have powered its wide-spread adoption in SCADA, fleet management, smart building automation and other verticals.
Aerospace & Defense
Northrop Grumman, Lockheed Martin, British Aerospace, EADS and others depend on eXtremeDB’s reliability, unmatched performance and broad platform support.
eXtremeDB optimization technology can dramatically boost utilities’ power generation yields. Distribution networks can become self-healing and bi-directional, enabling end-users to contribute power back to the grid.
eXtremeDB’s unique hybrid row- and columnar-layout (OLTP and time series) couples with pipelined functions for statistical analysis and scalable distributed database architecture power record-setting STAC-M3 benchmark results.
Hard Real-time Systems
eXtremeDB/rt is the first and only deterministic embedded DBMS for mission- and safety-critical hard real-time applications. It is built to satisfy strict requirements for timely execution of avionics and aircraft navigation systems, driver assistance systems, critical medical equipment and similar systems’ data management.
Big Data & Analytics
Columnar and row-oriented storage, a SQL engine infused with advanced analytics, and a scalable distributed architecture deliver speed and reliability to complex data-driven applications.
From inception, eXtremeDB’s resilient and optimized storage capabilities uniquely qualified it to fuel database management for safety-critical and resource-constrained embedded applications.
Internet of Things
eXtremeDB’s Active Replication Fabric™ creates a data management foundation for the IoT ecosystem that seamlessly integrates edge and cloud applications’ storage and ensuring data safety all around.
List of eXtremeDB features
No other database management system offers more to developers.
Active Replication Fabric
Active Replication Fabric™ offers reliable IoT database management by solving 5 key challenges for developers.
Pipelining combines built-in vector-based statistical functions into assembly lines of processing for time series data, with the output of one function becoming input for the next.