Features that Optimize a Time Series Database
eXtremeDB for HPC is the ideal time series database for Big Data, capital markets and IoT.
Conventional relational database management systems (RDBMSs) cannot deliver the speed, reliability and flexibility systems’ and IoT applications’ generated time series demand.
McObject’s eXtremeDB for HPC provides an alternative: a low latency database system for time series data such as that found in financial applications and the Internet of Things. Built on a core in-memory database system (IMDS) design with advanced features to deliver high performance, scalability and reliability, eXtremeDB for HPC has features that optimize time series data such as trades and quotes, sensor data, and all other types of streaming data.
Flexible data layout
eXtremeDB for HPC implements columnar data layout for fields of type ‘sequence’. Sequences can be combined to form a time series, ideal for working with tick streams, historical quotes and sequential data from IoT devices. This technology supports database designs that combine row-based and column-based layouts, in order to best leverage the CPU cache speed. Learn how.
“eXtremeDB leverages its sequence storage format to be best-in-class in filtering, aggregations, and window operations.” TSM-Bench: Benchmarking Time Series Database Systems for Monitoring Applications
Learn what makes eXtremeDB ideal for Big Data & Analytics and Financial Systems
Read about eXtremeDB elastic scalability
The lowest possible latency
Pipelining a rich library of vector-based statistical functions cuts latency in time series analytics and maximizes efficiency of L1/L2 cache use. GUI-based database performance monitoring enables the user to view key metrics such as transaction time and throughput when fine-tuning to reduce latency.
For streaming data, eXtremeDB delivers low latency database management via a highly efficient in-memory database system (IMDS) design that removes the I/O, cache management, data transfer and other sources of DBMS latency.
For historical or OLTP data, offers a wide array of performance-enhancing features, such as pre-warming the cache, cache prioritization, and many more.
When deployed as an embedded database system, its in-process architecture eliminates costly (in performance terms) inter-process communication.
In addition, eXtremeDB is optimized with internal transaction and memory managers to provide maximum efficiency working with multi-threaded applications on multi-processor systems. Support for clustering accelerates processing by enabling multiple servers to share the workload.
Throughput between main memory and CPU cache is 3x to 4x slower than the CPU can process data. Traditional DBMSs traverse this bottleneck frequently (twice per function), with the CPU handing off results of each step of a multi-step calculation to temporary tables in main memory.
Learn how eXtremeDB pipelines vector-based statistical functions, reducing data transfers between CPU cache and main memory for low latency.
In-database analytics and more
eXtremeDB for HPC offers a rich library of over 150 vector-based statistical functions. In-database analytics reduces latency-inducing transfers between the CPU cache and main memory.
Examples include: Boolean, add, subtract, multiply, divide, compare, not, and, or, xor, conversion, weighted sum, weighted average, covariance, correlation, conditional operations, difference, concatenation, max, min, sum, product, count, average, variance, standard deviation, user-defined functions, and more. See a complete list.
Scalable processing
Time series data management systems frequently require large volumes of information to be available continuously. eXtremeDB excels in this capability. Review McObject’s benchmark which demonstrates nearly linear scalability as database size grows to 1.17 Terabytes (15.54 billion rows) on a 160-core Linux server.
Interoperability
eXtremeDB ’s fast, native C/C++ API interoperates fully with its SQL API. The native interface is ideal for time-sensitive operations, while the SQL API (with its JDBC & ODBC support) permits higher level access and interfacing with external systems. eXtremeDB Data Relay enables open, highly selective replication between a system’s low latency database and external systems such as enterprise DBMSs.
Persistence and availability
Applications must ensure data integrity, persistence and availability. eXtremeDB for HPC achieves these goals through ACID transactions, optional transaction logging, and selective volatile or persistent storage via the product’s hybrid storage capability. High availability support enables deployment of non-stop fault-tolerant systems based on multiple, synchronized database copies, with application-directed failover.
Proof of performance
TSM-Bench: Benchmarking Time Series Database Systems for Monitoring Applications
An independent benchmark tailored for time series database systems used in monitoring applications, the key contributions consist of representative queries that meet the requirements that we collected from a water monitoring use case, and a new scalable data generator method based on Generative Adversarial Networks (GAN) and Locality Sensitive Hashing (LSH). We demonstrate, through an extensive set of experiments, how TSM Bench provides a comprehensive evaluation of the performance of seven leading time series database systems while offering a detailed characterization of their capabilities and trade-offs.
Open the TSM-Bench: Benchmarking Time Series Database Systems for Monitoring Applications.
eXtremeDB has repeatedly set speed records, then shattered its own records. Read summaries of our audited benchmark tests.
“eXtremeDB leverages its sequence storage format to be best-in-class in filtering, aggregations, and window operations.”
How did NSE.iT – the 100%-owned IT consulting subsidiary of India’s National Stock Exchange – achieve sub-millisecond latency in its algorithmic trading solution? Download the case study (PDF)