Technical Questions

Like that of the eXtremeDB core runtime, the SQL module’s performance varies from one environment to another. All things being equal, though, eXtremeSQL gains a performance advantage through its use of a highly efficient and predictable rule-based SQL query optimizer. In contrast, most SQL DBMSs rely on a cost-based optimizer that must collect sample data, generate statistics and analyze thousands — sometimes hundreds of thousands — of possible execution plans; these requirements make cost-based optimizers’ operation CPU-intensive and unpredictable.

eXtremeSQL scales exceptionally, often showing near linear scalability with distributed databases (horizontally partitioned/sharded databases.

64-bit eXtremeDB on a 64-bit OS overcomes the 2- or 3-GB in-memory database size limitation.  More details can be found on our Technical Specifications page.

Yes. eXtremeDB transactions support the ACID properties (ACID refers to the principles of Atomicity, Consistency, Isolation and Durability).  eXtremeDB in-memory database transactions are durable in the sense that transactions are durable so long as the database itself is durable. Durability can be strengthened by periodically saving the in-memory database and, optionally, through the introduction of transaction logging. It should be noted that all databases are durable only as long as the media that they are stored in is durable, i.e. if a SSD or HDD fails, any database stored on it is no longer durable.

Yes. eXtremeDB is fully re-entrant and thread-safe. The default multiple reader, single writer (MURSIW) transaction manager coordinates concurrent access, and concurrency requires no explicit action on the part of the developer other than to use eXtremeDB’s programming interface for starting, committing and aborting transactions. eXtremeDB’s optional multi-version concurrency control (MVCC) transaction manager eliminates pessimistic database locking and can dramatically improve scalability and performance when multiple tasks or processes are concurrently modifying the database (versus read-only), especially on multi-core hardware. A more detailed discussion of the two transaction managers, with benchmark results comparing their performance, is available here.

The most popular interface for working with eXtremeDB in embedded/IoT systems is its fast, native navigational application programming interface (API) consisting of C/C++ functions. This is provided as both a type-safe and intuitive application-specific API (functions are generated based on the data design), and as a uniform data access (UDA) API for a consistent interface across all projects.

McObject also provides eXtremeSQL, a SQL programming interface that is more suitable for highly scalable IoT Big Data use cases.

eXtremeSQL includes a Java Database Connectivity (JDBC) driver and also supports ODBC, both of which expand eXtremeDB’s ability to share data with systems ranging from mainframes to servers and desktop applications, and enable developers to create new eXtremeDB-based applications using an open and familiar database API.

In addition, eXtremeDB’s Java Native Interface (JNI), C# native interface, and Python interface offer the fastest possible DBMS solutions in these languages, and the ability to access eXtremeDB while working entirely with “plain old” Java, C# and Python objects.

With its native C/C++ API, eXtremeDB creates a unique programming interface for each database, derived from the data definition language (DDL), or database schema. This approach offers many advantages in areas including type safety, programming ease, application maintainability, determinism (predictability) and even learning curve. These benefits are described in a white paper titled “SQL or Navigational Database APIs: Which Best Fits Embedded Systems?” available for download.

An eXtremeDB tree index is a tree-like structure in the application’s memory that contains a collection of keys. A key can be a simple one (based on a single field) or a compound one that contains more than one field. The tree-index stores the keys in sorted order. In order to support sorting, the tree-index must have a compare function. Regardless of the key type (simple or compound), the compare function must be able to compare two keys and determine whether the first key is less (compare returns -1), equal (0) or greater (1) than the second key. The tree-index algorithm sorts the keys based on this comparison. It is important to note that the tree-index algorithm sorts the keys based on their relative weight, determined from the compare function results, rather than on the key’s value. For more details, see the “Index Search and Navigation” section of the eXtremeDB User’s Guide.

Yes. eXtremeDB has a data definition language (DDL) and a database dictionary. The native DDL looks very much like C++. It is created with a text editor and processed by our schema processor—which generates .h and .c files that contain the programming interface for the database—and is compiled and linked with the rest of the application code. eXtremeDB also supports databases defined using the SQL DDL (e.g. CREATE TABLE, CREATE INDEX, etc), and databases defined using Java or C# class definitions. A database defined in any language can be used with any other language.

Yes. There is no theoretical limit to the complexity of the data you can define with eXtremeDB’s DDL. eXtremeDB supports C/C++ structs, nested structs, fixed and variable length arrays of any supported data type (including structs), and BLOBs.

The overhead imposed by the eXtremeDB core runtime varies based on the application data layout, the number of indexes used and other factors. eXtremeDB can add as little as 20% to the size of the raw data for its own pointers and other meta data required to maintain indexes and other references to the data.

eXtremeDB offers a number of persistence options. First of all, the core eXtremeDB has functions to write a database image to a stream such as a socket, a file or a pipe, and to load a database from a previously saved image. These operations run within the context of a transaction, assuring that a transaction-consistent image of the database is written (or read). In simple terms, you can capture a ‘snapshot’ of an in-memory database, on demand.

Second, the eXtremeDB Transaction Logging runtime captures database changes and writes them to a file known as a transaction log. In the event of a hardware or software failure, the eXtremeDB runtime can recover the database using the last snapshot and the subsequent log. McObject’s benchmark tests confirm that the eXtremeDB database system with transaction logging provides a significant speed advantage over traditional “on-disk” database storage.

Third, eXtremeDB High Availability maintains a master database and one or more synchronized replica databases within separate address spaces, with failover (i.e. an active/passive model). Typical hardware configurations include:

– Multiple processes or threads within the same hardware instance

– Two or more boards in a chassis with a high-speed bus for communication

– Separate address spaces on boards connected via industry standard communication media and protocol such as RS-485/RS-232, Control Area Network (CAN) or Ethernet

– Two or more controllers connected via a proprietary communication media and/or protocol

– Separate computers on a LAN

Fourth, the eXtremeDB Cluster feature manages databases across multiple hardware nodes, enabling two or more servers to share the workload. In an eXtremeDB Cluster deployment, every database instance serves as a “master” (i.e., and active/active model). Any process on any node can update its local database, and McObject’s clustering software replicates the changes to other nodes.

Fifth, eXtremeDB supports in-memory databases created in non-volatile memory (NVRAM). The runtime recovers the in-memory database after failure, provided that the in-memory image is in battery-backed RAM which retains its contents in the absence of system power (this includes newer generation of NVRAM that employ super capacitors in lieu of batteries).

Finally, eXtremeDB offers on-disk data storage alongside in-memory storage in a single database system. By specifying on a table-by-table basis which of a database’s records will be stored in memory and which will be cached in memory but ultimately saved to persistent storage, developers can take advantage of the most cost-effective and physical space-conserving approaches to data storage (byte-for-byte, disk storage can be cheaper than memory, and can also take less physical space).

The eXtremeDB core runtime does not call malloc. Instead, eXtremeDB implements its own, highly optimized memory managers. Use of these custom memory managers improves performance, reliability and even portability. For an in-depth look at eXtremeDB’s approach to memory management, see this Dr. Dobb’s Journal article by McObject’s co-founder/CTO Andrei Gorine.

eXtremeDB is designed and implemented to be highly portable. Currently, eXtremeDB runs on Win32 and Win64 platforms, Linux, many Unix platforms, and various embedded operating systems including VxWorks, INTEGRITY and QNX, and target CPUs including x86, PowerPC, ARM, MIPS, and others. The list of supported platforms is constantly expanding.  Learn more.

The eXtremeDB runtime can be built with the Microsoft Visual C/C++ compiler, gcc and all ANSI C compliant compilers. The eXtremeDB Data Definition Language compiler is currently built with Microsoft Visual Studio for Windows and gcc for Unix/Linux platforms.

Yes, in single-threaded mode. In this scenario, the eXtremeDB runtime does not use any operating systems services, or the C runtime library.

eXtremeDB’s core in-memory database system design does not require any file system. If a file system is present, to allow data persistence through a system crash, eXtremeDB provides backup functionality via its stream-oriented “save” and “load” interfaces. eXtremeDB can backup databases not only to a file, but also to any stream that is available in the operating environment, such as a pipe or a socket.

eXtremeDB is distributed in two versions: “debug” and “release.” In debug mode, eXtremeDB provides excellent debugging capabilities, with multiple traps throughout the eXtremeDB code that facilitate detection of application errors. For example, if an incorrect pointer is passed into eXtremeDB, chances are that a fatal exception will be raised by the eXtremeDB library before the pointer is used, which makes it possible to examine the call stack and find the source of the corruption.

In addition, eXtremeDB’s native, application-specific C/C++ API is type-safe. Methods that are generated to provide access to a certain object type always take a reference to that data type as a parameter. If a mistake is made, a compiler error will be generated. In release mode, all the above traps are removed from the eXtremeDB runtime code, and many functions inside the runtime are in-lined. That can significantly increase the performance of the eXtremeDB -based application. (The software quality benefits of a type-safe API are discussed in this Linux Journal article by McObject’s co-founder/CEO).

eXtremeDB High Availability (HA) implements database replication. The HA module offers master-slave (active/passive) replication, with one “main” node and one or more synchronized replicas, with fail-over. eXtremeDB HA is a popular choice when the principal goal of database replication is to maximize system up time.

In eXtremeDB Cluster deployments, every node serves as a master (active/active). The feature Cluster also has a “local tables” option that exempts specified eXtremeDB tables from cluster-wide replication, to accelerate processing at the local node. Local tables are shareable, when desired, through a scatter/gather mechanism. eXtremeDB Cluster provides fault-tolerance; it also dramatically increases available net processing power, reduces costs by enabling system expansion via low-cost (i.e. “commodity”) servers, enables more evenly balanced workloads, and delivers more scalable data management.

A third option is the eXtremeDB Data Relay technology included with eXtremeDB Transaction Logging. Data Relay enables selective data sharing between real-time systems based on eXtremeDB, and external systems such as enterprise DBMSs. It “opens up” eXtremeDB’s transaction buffer; for every object affected by a transaction, the buffer provides a coded value that indicates whether each operation was an insert, update, or delete. Data Relay provides a database cursor that iterates over objects in the buffer and extracts the coded value. Based on this reading, the application can take the next step of drilling down into the object, analyzing changes and determining whether to replicate the change to an enterprise system