March 9, 2009 In what is likely the first-ever database benchmark conducted on an off-the-shelf mobile phone, McObject® has successfully deployed its TestIndex application on T-Mobile’s Android-based G1smartphone, to measure performance of the SQLite embedded database alongside McObject’s open source Perst™ embedded database as the two perform typical data management functions.
The comparison showed that gaps in database performance can be measured as easily in this pocket-sized test bed as in more traditional benchmark environments with rack-mounted hardware. On three of the four TestIndex metrics, Perst performed database tasks significantly faster—more than 16 times faster, in the case of database inserts.
McObject has gathered benchmark results and analysis into a report, available for free download from McObject’s Web site. McObject also offers the TestIndex application, with complete source code, to run in the Android emulator as well as on the T-Mobile G1 and any future phones based on Google’s Android platform.
Mobile Devices’ Database Challenge
To support sophisticated features, applications running on mobile devices manage increasing volumes of complex data. Examples include location-based services (LBS) that rely on GIS data, MP3 players that index songs and playlists, and field sales and service applications that store enterprise data.
Incorporating an off-the-shelf embedded database for these tasks reduces coding and QA, and improves reliability and performance. Given mobile phones’ typically limited processing resources, though, the chosen database must be highly efficient.
Two quite different embedded database systems that have emerged for use on Google’s Android are SQLite, a relational embedded database that supports the SQL application programming interface (API), and Perst, an object-oriented embedded database that works directly with Java objects.
To compare Android databases in an “applets to applets” test, McObject developed the TestIndex benchmark application, which measures performance along the following parameters:
- Insert – loading 10,000 simple records into the database (with string and integer keys, using two B-tree indexes)
- Search – seeking these records via the indexes (there are 10,000 searches for random values in each index, so searching is performed 20,000 times)
- Scan – performing two traverses (sequential scans) through all records using both indexes, so that records are sorted by integer key and string key
- Delete – locate and remove all 10,000 records
SQLite vs. Perst: Benchmark Results
For the first TestIndex benchmark test in a “real” Android phone, McObject obtained T-Mobile’s G1 phone, equipped with all standard features.
The TestIndex application with both embedded databases was downloaded via USB from a PC to the G1 phone, where it received its own “Benchmark” icon. Clicking this launched TestIndex, which returned the following results (in milliseconds):
|Insert||299,166||18,214||Perst is 16.43 times faster|
|Search||104,796||15,743||Perst is 6.65 times faster|
|Scan||6,457||9,747||SQLite is 1.51 times faster|
|Delete||203,253||37,811||Perst is 5.38 times faster|
Examining the Results
According to McObject CEO Steve Graves, the performance disparity in insert and delete operations likely results, at least partially, from the lack of explicit transaction support in the SQLite Android API. Each update must be performed as a separate transaction, in autocommit mode, resulting in significant transaction processing overhead.
Perst’s edge in searching for records probably stems from latency in SQLite caused by using Android’s Java interface to access the native C language database, and SQLite’s overhead of parsing, optimizing and executing the interpreted SQL, Graves said. In contrast, Perst is all Java, and its interface works directly with database objects – no interpretation of an intermediate language is needed.
SQLite’s advantage in scan operations probably stems from the test’s simple tabular data layout, Graves said. As a relational database, SQLite organizes data in rows, and these rows are physically close to one another in storage, like rows of a spreadsheet. This proximity lends an edge in sequentially fetching the rows. In contrast, in Perst, everything is an object, including index pages, and objects are interleaved in the storage.
More details, including database definitions used in the test and an expanded performance analysis, are included in McObject’s freely downloadable benchmark report.
Perst is part of McObject’s family of powerful small footprint, high performance embedded database software products. The eXtremeDB™ in-memory embedded database from McObject is used widely in devices including MP3 players, WiMAX base stations, digital TVs, telecom/network communications equipment and military/aerospace technology. Perst is available for Java and .NET, including Java ME and .NET Compact Framework. CA’s Wily Technology credits Perst with delivering a shortened development cycle and a ten-fold performance improvement within its real-time Java application.
Founded by embedded database and real-time systems experts, McObject offers proven data management technology that makes applications and devices smarter, more reliable and more cost-effective to develop and maintain. McObject counts among its customers industry leaders such as Chrysler, Maximizer Software, Siemens, Phillips, EADS, JVC, Tyco Thermal Controls, F5 Networks, DIRECTV, CA, Motorola and Boeing. McObject, based in Issaquah, WA, is committed to providing innovative technology and first-rate services to customers and partners. The company can be reached at +1-425-888-8505, or visit www.mcobject.com.
McObject and eXtremeDB are trademarks or registered trademarks of McObject LLC. All other company or product names mentioned herein are trademarks or registered trademarks of their respective owners.