Time Series and Columnar Data

The sequence data type

Some data lends itself to columnar storage, such as time series. eXtremeDB includes support for ordered data (time series) as well as a library of vector-based analytics functions and SQL extensions to support SQL-style operations over time series data. eXtremeDB combines conventional row-oriented storage with column-oriented storage for data declared as a sequence.

A sequence is an unbounded array of eXtremeDB-supported scalar data elements (i.e. [u]int[1234], float and double). In the database schema, you name and define the sequence type and possibly specify whether it is ordered (descending or ascending). Multiple sequences are grouped together as elements of a class, which effectively forms a time series. Sequences are a columnar organization of data.

Furthermore, sequences are stored in tiles which, in turn, are sized to also take advantage of L1 cache. In other words, an entire sequence will often be loaded into the L1 data cache, eliminating multiple fetches from RAM across the much slower front-side bus.

(Please see page Using Sequences for details on usage and implementation of sequences.)

Design goals

Representative application domains include (but are not limited to) ticker plants, algorithmic trading, quantitative analysis, matching engines, and risk management. So the eXtremeDB design goals are driven by the requirements of these target applications, which are typically high transaction rate programs with:

To these goals was added the need to provide an easy-to-use approach to database application development and to give the developer the maximum possible configurability and control over the performance and impact of the database runtime within their applications. So, with these overriding requirements, the guiding design principles include: