Proven the fastest analytics for time series data.

Once you’ve read our statistics below, we invite you to learn more about deploying eXtremeDB for HPC in financial systems, and pipelining for blazingly fast analytics of time series data.

In contrast, pipelining vector-based statistical functions with eXtremeDB for HPC avoids these hand offs, keeping interim results in CPU cache to reduce database latency.

TSM-Bench: Benchmarking Time Series Database Systems for Monitoring Applications

A benchmark tailored for time series database systems used in monitoring applications, the key contributions consist of representative queries that meet the requirements that we collected from a water monitoring use case, and a new scalable data generator method based on Generative Adversarial Networks (GAN) and Locality Sensitive Hashing (LSH). We demonstrate, through an extensive set of experiments, how TSMBench provides a comprehensive evaluation of the performance of seven leading time series database systems while offering a detailed characterization of their capabilities and trade-offs.

“eXtremeDB leverages its sequence storage format to be best-in-class in filtering, aggregations, and window operations.”

Read the TSM-Bench: Benchmarking Time Series Database Systems for Monitoring Applications.

eXtremeDB Industry-standard, Audited STAC-M3 Benchmark Tests

The Securities Technology Analysis Center (STAC) provides the STAC-M3 benchmarks to evaluate time-series data management (tick database) solutions involving complex queries designed to reflect real-world capital markets computing demands. STAC oversees the tests and audits the results, to enable comparisons between vendors’ technologies for analytics of time series data.  eXtremeDB has repeatedly proven it is the fastest tick database.  

In multiple audited STAC-M3 benchmark tests, McObject’s eXtremeDB database management system has broken earlier records for the best (lowest) mean response times and for lowest standard deviation (highest predictability/lowest “jitter”) of test results. eXtremeDB set STAC-M3 records using the SQL database programming language. For a discussion of why this matters, see McObject CEO Steve Graves’ TabbFORUM article, “Is SQL Fast Enough for Tick Data?”

McObject & E8 Storage STAC-M3

All the queries used eXtremeDB v.8.0 for HPC and were programmed using Python and SQL, running on a Supermicro SYS-2028TP-HTTR, connected to an E8 Storage E8-X24 high-performance shared NVMe (Intel Optane™) storage appliance.

The results include:

• Faster mean response times than all previously published best results for 5 of the 17 operations.
• More consistent response times (lower standard deviation) than all previous best results for 7 of the 17 operations.
• 17.0x the performance of the previously published best result for the write benchmark (STAC-M3.β1.1T.WRITE.TIME) using a non-McObject database.
• 9.1x the performance of the previously published best result for the NBBO benchmark (STAC-M3.β1.1T.NBBO.TIME) using a non-McObject database.

eXtremeDB’s efficient row- and column-based data layout ensures the maximum amount of relevant market data is brought into CPU cache and avoids flooding the cache with unneeded data.  Click here to review the full STAC Report™.  Learn more about the fastest tick database with a free 60 day trial.

McObject & IBM POWER8 STAC-M3 Kanaga

McObject worked closely with IBM on three STAC-M3 implementations on the POWER8 processor architecture, with the latest utilizing a 2-socket IBM Power System S824L server (12 cores per socket, 24 POWER8 cores total) running Redhat Linux, eXtremeDB version 7.0 and 57TB IBM FlashSystem 900. Results included new speed and jitter records for all 21 of the benchmark queries that were more than:

• 21,139% and 19,875% faster than the best previously published result for the 10 thread Years 3 & 4 Market Snapshots, respectively (10T.YR3.MKTSNAP and 10T.YR4.MKTSNAP)
• 2,742% faster for the 10 thread Year 1 Market Snapshot (10T.YR1.MKTSNAP)
• 887% and 936% faster for the 100 thread Years 1 & 3 Volume Weighted Bid over 12 days, respectively (100T.YR1VWAB-12D-HO and 100T.YR3VWAB-12D-HO)
• Better than 500% faster than the best previously recorded time to complete all 21 benchmark tests

Get the audited benchmark report.  Learn more about the fastest time series database with a free 60 day trial.

Results of STAC-M3 benchmark using eXtremeDB on IBM POWER8 hardware

The combination of McObject’s SQL-based eXtremeDB database management system (DBMS), running on an IBM POWER8 S824L Linux server, achieved new records in a set of 17 STAC-M3 tests, including the best-ever results for jitter.  Adding together the mean response times for these 17 tests, eXtremeDB on POWER8 took 44% less time than the nearest competitive solution while using just one-third the number of cores. This solution provides a significant cost/performance benefit to clients in the financial industry, or any company needing analytics of time series data.

These new STAC-M3 results:

• Bested standing performance records in 6 of the 17 tests
• Set records for lowest standard deviation of results (lowest jitter) in 5 of 17 tests
• The results build on previous record-setting STAC-M3 implementations by McObject and IBM that, when combined, hold 16 of the 17 test performance records when compared to all other solutions using non-clustered 2-socket servers. These results demonstrate the value of eXtremeDB and the IBM POWER8 system for analytics and big data workloads in capital markets, where low-latency and low-jitter are key attributes.

Read McObject’s announcement of the STAC-M3 results. Review the STAC report, STAC-M3 Kanaga Audited Report eXtremeDB on IBM Power8.  Learn more about the fastest time series database with a free 60 day trial.

Results of cloud-based STAC-M3 using eXtremeDB on Lucera Compute

McObject worked with Lucera  to deliver a STAC-M3 benchmark using eXtremeDB on the Lucera Compute high-performance, collocated cloud computing infrastructure for financial institutions.

Together, the DBMS and cloud-based infrastructure beat standing records in five of the 17 benchmark tests. Compared to previous STAC-M3 benchmarks using a different DBMS, the McObject-Lucera pairing delivered performance that on average is 

• More than three times (3x) as fast as the best previously published result for the Multi-User Intervalized Statistics – 50 Threads benchmark (STAC-M3.β1.50T.STATS-UI.TIME)
• More than twice as fast for the Multi-User Intervalized Statistics – 100 Threads benchmark (STAC-M3.β1.100T.STATS-UI.TIME)
• More than twice as fast for the National Best Bid and Offer benchmark (STAC-M3.β1.1T.NBBO.TIME)

Read the announcement of the STAC-M3 Benchmark test results. Download the report, STAC-M3 eXtremeDB Financial Edition 6.0 on Lucera. Learn more about the fastest time series database with a free 60 day trial.

STAC-M3 Results for eXtremeDB on IBM POWER8 Hardware

eXtremeDB paired with IBM POWER8 hardware achieved new speed records for nine of the 17 benchmark tests, including performance that on average is

• More than four times (4x) as fast as the best previously published result for the “Volume Curves” benchmark
• More than four times (4x) as fast for the “Multi-User Intervalized” Statistics algorithm
• More than twice as fast for the “Market Snapshot” algorithm
• More than twice as fast for the “National Best Bid and Offer” (NBBO) benchmark

In these tests, eXtremeDB running on POWER8 hardware achieved a sum of mean benchmark times 1.6x faster than the best previously published STAC-M3 results (66 seconds vs. 106 seconds).

The tests marked the first published STAC-M3 results using industry-standard SQL as the database interface language. They utilized several recently introduced eXtremeDB features, including distributed query processing (the database was partitioned horizontally across 72 shards) and the ability to pipeline built in vector-based statistical functions from within SQL.  Learn more about SQL for tick data management.

Download the STAC report, STAC-M3 eXtremeDB Financial Edition 6.0 on IBM.  Learn more about the fastest time series database a free 60 day trial.

STAC-M3 Results for eXtremeDB with Kove, Dell and Mellanox Hardware

Highlights of these STAC-M3 benchmark tests– which examined eXtremeDB with hardware from Dell, Kove and Mellanox – include:

• 9x the previously published best result on tests using the “Theoretical P&L” algorithm
• 7x the best previously published result for the “Year High Bid” benchmark
• Double the previously published best results for both “National Best Bid and Offer” (NBBO) and “Market Snapshot” algorithms
• Lower standard deviations of latency than the previously best results for 13 of the 17 operations

The last result, record-setting low standard deviation of latency, means that financial calculations will be predictably fast. Capital markets developers can expect tighter bounds on application performance (performance will not swing wildly) and have confidence that processing will not bog down at critical times.

Read McObject’s announcement of the  STAC M3 Benchmark test results. Download the STAC report.  Learn more about this record setting database with a free 60 day trial.

McObject’s eXtremeDB DBMS Performance Comparison

In its own testing, McObject benchmarked eXtremeDB against a competing database management system used in real-time financial applications. Results are presented in the table below.

Details of McObject’s internal benchmark testing:

• The tests measured performance on financial calculations executing over tick streams
• Data volume: 20 million ticks for a single symbol
• Operations are expressed in quasi-SQL in the chart above
• The eXtremeDB implementation used its native API to its vector-based statistical function library
• The alternative DBMS used its proprietary development and querying language

Why is eXtremeDB faster?

What explains eXtremeDB’s faster performance? Differentiators include the following:

• eXtremeDB’s efficient row- and column-based data layout ensures the maximum amount of relevant market data is brought into CPU cache and avoids flooding the cache with unneeded data.
• Pipelining vector-based statistical functions keeps needed data near CPU cores, eliminating latency-inducing hand-offs of interim results between main memory and CPU cache.  Watch a video about pipelining.
• With distributed query processing, eXtremeDB partitions a database and distributes query processing across multiple servers, CPUs and/or CPU cores. Performance is accelerated via parallel execution of database operations and by harnessing the capabilities of many host computers rather than just one.
• With a code size of as little as 200K, eXtremeDB’s code path is very short. Calculations are performed using fewer CPU instructions, shaving milliseconds from execution time.  Vector-based statistical functions are created with a short code path to keep the code in CPU cache.

Evaluate a free trial of eXtremeDB, the fastest tick database.

Learn why others use SQL for tick data management and financial systems.