A Quick Look at Open Source Databases for Mobile App Development

Data is the life blood of the IT world. It is generated in great volumes in the course of our everyday lives. Efficient data management calls for the use of database management systems (DBMSs). This article presents readers with a selection of popular open source DBMSs.

Organisations today have a wide choice when it comes to choosing efficient database management systems (DBMSs). Various IT giants like Oracle, Microsoft, IBM and SAP offer high-end solutions that address all sorts of database requirements. Even newer IT companies like Amazon, Google and Rackspace have entered the fray, offering cloud-related features with regard to databases. According to Gartner analyst, Merv Adrian, 92.1 per cent of DBMS revenue comes from commercial software. Yet, new data is finding its way to open source databases as developers are opting for more convenience.

Open source cloud computing and database adoption is today widespread, as these databases provide huge benefits and are becoming a strong solution for every data management need within organisations. The DBMS space has been considerably impacted by the open source software (OSS) movement. The adoption of MySQL, MariaDB and other databases is saving organisations a lot of money, which would have otherwise been spent on licences and maintenance. Over recent years, various open source database projects have attained the maturity stage. In addition to cost savings, open source databases offer very easy tweaking, scaling and, overall, are feature-rich.

Apart from providing a solid backbone to enterprises in storing, retrieving and manipulating regular data during typical day-to-day operations, open source databases are also becoming a strong choice for mobile app developers.

Mobile operating systems have gained much popularity. Android and iOS have surpassed Windows OS as the most used operating systems in the world. As mobile OSs and mobile hardware are becoming more powerful, with mobiles becoming fully functional smart computing devices, data computation from mobiles is also increasing.

Databases are regarded as the most obvious way of storing and manipulating data. In the beginning, databases on mobile apps were handled on the server side or on the cloud. Mobile devices only communicated with them via the network. However, with the passage of time, to make the applications more responsive and less network dependent, offline databases have been gaining popularity. Nowadays, mobile apps store entire databases on the phone or have the option of keeping the data on the cloud, with synchronisation once a day, depending on the network connectivity. This leads to faster and more responsive applications.

Mobile databases

A mobile database is one that a mobile computing device can connect to over a wireless network. Mobile databases can further be classified as those that are physically separate from the central database server, those that reside on mobile devices, and those capable of communicating with a central database server or other mobile clients from remote sites and handling local queries without connectivity.

Mobile DBMSs should satisfy the following requirements:

• Small memory footprint
• Flash-optimised storage system
• Data synchronisation
• Security
• Low power consumption
• Self-management
• Strong coordination via embeddable features in applications

There are many mobile databases available in the market with many new ones also emerging, but it is the developer who has to choose the right database that satisfies all the requirements of the particular mobile app being created.

A few popular open source databases designed especially for mobile apps are discussed here.

SQLite

SQLite database is a small, highly reliable, embedded, fully-featured, compact and self-contained relational DBMS available as a public domain software package. As compared to other database management systems, SQLite is not a client-server database engine. It is regarded as a popular database, especially for application software like Web browsers and even for mobile app developers to store data from front-end mobile apps. SQLite offers an amazing set of tools to handle all sorts of data with ease and fewer constraints, compared to hosted and process based server relational databases.

Features

• Serverless: It doesn’t require a separate server process or system to operate. Its library has direct access to storage files.
• Zero configuration: No server is required, which means ‘No setup’. Creating a SQLite database instance is as easy as opening a simple file.
• Cross-platform: It is available for various platforms like Android, BSD, iOS, MAC, Solaris, Windows and VxWorks.
• Self-contained: The entire database is contained in a single library and is integrated into the host application.
• Transactional: SQLite transactions are fully ACID-compliant, which allows safe access from multiple processes.
• Reliability: SQLite development is under consistent development and proper testing is done before launching new versions.
• Fully-featured: It contains all the query language features found in the SQL2 standard.
• SQL extensions: SQLite provides a number of enhancements to SQL, which are not normally found in other database engines. These enhancements include the EXPLAIN keyword and manifest typing. It also provides statements like REPLACE and the ON CONFLICT clause that allows for adding control over the resolution of the constraint conflict. It also supports the ATTACH and DETACH commands.

Official website: https://www.sqlite.org/

Latest version: 3.22.0

ORMLite

Object Relational Mapping Lite (ORMLite) facilitates simple, lightweight Object Relational Mapping (ORM) functionality for persisting Java objects to SQL databases by avoiding the overhead of other ORMs. It simplifies complicated SQL operations via a flexible query builder and provides powerful abstract Database Access Object (DAO) classes.

ORMLite supports JDBC connections to MySQL, Postgres, H2, SQLite, Derby, HSQLDB and Microsoft SQL Server, and can also be extended to various other databases. It supports native database calls on Android OS. ORMLite is bulkier than SQLite or Realm, but slower than both of them. It is regarded as faster than other ORM based databases.

As there is no JDBC support in Android OS, ORMLite makes direct calls to Android database APIs to access SQLite databases.

Features

• Provides a set of lightweight C# extension methods around .NET System.Data.* interfaces.
• High performance database with support for indexes, text blobs, etc.
• Supports cross-platform multiple databases like SQL Server, SQLite, MySQL, PostgreSQL and Firebird running on both .NET and Mono platforms.
• Auto generates SQL to create and drop database tables.
• Has provisional support for DB2, Oracle, ODBC and Netezza.
• Has powerful abstract Database Access Object (DAO) classes.

Official website: http://ormlite.com/

Latest version: 5.0

Berkeley DB

Berkeley DB is an open source, high performance, industry leading embeddable storage engine that provides developers a fast, reliable, local database with almost zero administration. It was primarily developed by Sleepycat Software but was acquired by Oracle in 2006. Berkeley DB is written in C language with API bindings for C++, C#, Java, Perl, PHP, Python, Ruby, Smalltalk, TCL and many other programming languages. It provides powerful APIs for many languages including Android and iOS.

Berkeley DB provides powerful robust data storage features like other traditional RDBMSs such as ACID transactions, and recovery, locking, multi-process and multi-threading for high concurrency, replication and hot and cold backup.

Oracle Berkeley DB: Berkeley DB has powerful architecture, which is very simple compared to other RDBMS software. Oracle added support for SQL in the 11g R2 release based on the popular SQLite API by including a version of SQLite in Berkeley DB. All the powerful features of SQLite are included with Berkeley DB. As it includes the fully-featured SQL database engine, you can use JDBC, ODBC or any other compatible access layer as well.

Berkeley DB is highly flexible, and you can use it across a plethora of applications and as a replacement for custom, home-grown solutions. It provides fast, reliable and scalable persistence for applications that need to store data locally, run without interruption and access data in a predictable fashion.

Editions

The following are the three editions of Oracle Berkeley DB:

• Oracle Berkeley DB: Written in C. Current version: 6.2
• Berkeley DB Java Edition: Written in Java. Current version: 7.5.11
• Berkeley DB XML: Written in C++. Current version: 6.1.4

Features

• Data management: Fine grained and configurable locking for concurrent systems; sequential and indexed retrieval; stores data in-memory or on disk or a combination of both; strong encryption; library for data management to link embedded processes; and highly portable.
• Transactions: ACID compliant, nested transactions, disaster recovery options, configurable graph, deadlock detection and log file archival.
• Replication: Single and multiple replica models; delayed client synchronisation; scalable to thousands of replica nodes; non-stop upgrades; and PAXOS-compliant election algorithm.
• Deployment: Cross-platform, binary installer, open source and has a powerful administration.

Official website: http://www.oracle.com/technetwork/database/database-technologies/berkeleydb/overview/index.html

Latest version: Berkeley DB 12Cr1 (12.1.6.2.32)

ObjectBox DB

ObjectBox DB, developed by Greenrobot, is an object-oriented embedded database and is regarded as the best alternative to SQLite for mobile databases. It is specially designed for mobile database management and brings NoSQL Server technology to mobiles.

The guiding principle behind the ObjectBox DB design is, ‘Less code is better code’, which means ObjectBox DB is easy to work with even for newbies.

ObjectBox is a fully transactional database and consists of all the ACID features. It is a super-fast mobile database that persists objects and enables users to avoid repetitive tasks, providing a simple interface to data.

The ObjectBox database extensions combine advanced enterprise application capabilities — self-adjusting connection pooling, transaction handling and RDB-to-XML mappings — with a really straightforward o:XML interface.

ObjectBox DB is a fast, object-oriented mobile database for Android and Linux, and soon will support iOS. It supports JavaRx and Kotlin.

Features

• Fast: It is ten times faster than SQLite and delivers the best performance.
• Object API: It builds objects from the ground up, i.e., there are no ORM, no SQLite, no rows and no columns.
• Powerful query builder: Simple queries for objects with consistent checks at the time of compilation.
• Cross-platform
• Simple threading: Objects returned by ObjectBox work in all threads with no strings attached.
• DaoCompat library: This is an add-on library for greenDAO APIs for ObjectBox.
• Support: It has an active community with more than 150,000 active users, and a strong team for active beta, unit and final testing.

Official website: http://objectbox.io

Latest version: 1.4.1

Realm DB

Realm DB (Real Mobile Database) is an open source database specifically designed for mobile devices from scratch, and is available for various platforms including Xamarin and React Native.

It is serverless and cross-platform like SQLite. Data can be stored on the disk as well as in memory.

The key difference between Realm and ORMs is that the former is not an abstraction built on top of SQLite, but a whole new database engine. Rather than a relational model, it is based on an object store. Its core consists of a self-contained C++ library. It currently supports Android, iOS (Objective-C and Swift), Xamarin and React Native.

Realm is under active development and has been releasing updates pretty often. The development team is very responsive on the database’s issue tracker and StackOverflow.

Features

• Easy-to-use: Realm is very easy to use compared to CoreData and SQLite. Realm data models are defined using traditional NSObject-style classes with @properties. You can simply subclass RLMObject to create your Realm data model objects.
• Faster response: Realm is faster than other databases like SQLite, FMDB, CoreData and Couchbase Lite in app data storage. Even complex queries take nano seconds, and stay up-to-date with new data.
• Realm browser: This allows users to explore and administer realm databases.
• Cross-platform: Available for Android, iOS and other popular mobile operating system platforms.
• Data security: Provides enhanced security of data via transparent encryption and decryption.
• Concurrency: Realm handles concurrency with MVCC architecture, and these views ‘automatically update’ when a transaction is committed from any thread.
• Support: Realm is highly supported via proper documentation, the community and is always updating to the latest features. It supports Objective-C as well as Swift, which suggests that this could be a long-term solution for iOS developers.

Official website: https://realm.io/

Latest version: 4.3.1

Couchbase Lite

Couchbase Lite is a fully-featured open source NoSQL embedded JSON database designed for mobile devices. Data in Couchbase Lite is stored as JSON documents. Each document may have one or more attachments, which are essentially uninterpreted binary data that is stored and loaded separately from the document itself. Couchbase Lite supports persistent indices (called views), and uses MapReduce to manage and query them.

Couchbase Mobile is the solution provided by Couchbase Lite for mobile applications. It comprises three different components: Couchbase Lite, an embedded NoSQL database and Sync Gateway. Couchbase is an offline-first database, and syncs with the cloud when needed or when the network is available. Couchbase Lite runs locally on the device and persists data as JSON and binary format. All CRUD operations are performed on the local database. Developers do not need to write sync code (if needed) to sync the local database with the cloud, as this is handled by Sync Gateway.

Couchbase Lite provides native APIs for Android and iOS with plugins for Xamarin and PhoneGap.

Features

• It is lightweight — just 500KB to 1MB.
• Data is secured on the mobile device using 256-bit AES encryption.
• Synchronisation of data is from device-to-device via a peer-to-peer network.
• Contains both REST APIs and native APIs for data management.
• Cross-platform support available for Android, Mac, Windows and Linux.

Official website: https://www.couchbase.com/products/lite

Latest version: 5.0

LevelDB

LevelDB is an open source database for mobile and Web apps, and was designed and developed by Jeffrey Dean and Sanjay Ghemawat of Google Inc. It is a simple, key-value store database built by Google. It is currently used in Google Chrome and other software. It supports arbitrary byte arrays as both values and keys; singular put, get and delete operations; batched put and delete; and bi-directional iterators. Simple compression is performed using the fast Snappy algorithm.

LevelDB is not an SQL database and doesn’t have a relational data model. It doesn’t support SQL queries or indices. Applications make use of LevelDB as a library, and it doesn’t provide a server or command-line interface.

LevelDB is much better than SQLite and Kyoto Cabinet in terms of read and write operations, and also the best in batch writes. It also performs better in terms of read, write and synchronisation-based operations when compared to Berkeley DB and OpenLDAP Lightning DB.

LevelDB is written in C++. iOS developers can use it directly in their applications or through one of the several Objective-C wrappers that are available for it. Android developers can use LevelDB via JNI and NDK.

Features

• Makes multiple operations atomic.
• Creates consistent snapshots.
• Is iterative over key ranges.
• Offers automatic data compression using the Snappy Compression library.
• Basic operations: Put (key, value), get (key) and delete (key)
• Facilitates transient snapshots for a consistent view of the data

Official website: http://leveldb.org/

Latest version: 1.14

UnQLite

UnQLite is an open source embedded NoSQL database engine designed in 2012 by Mrad Chems Eddine and his colleagues while working on distributed P2P and VoIP solutions like Skype. Its key-value storage operations are similar to Berkeley DB and MongoDB, and it has an inbuilt scripting language called Jx9, which is like JavaScript.

The main design objective of UnQLite is to store information of all nodes like the IP-address, blobs, etc. UnQLite was created by considering SQLite3 as the backend, i.e., the VFS layer, locking mechanism and Transaction Manager with Jx9.

UnQLite architecture: UnQLite is suitable for embedded devices and is written in ANSI C. It is thread safe, fully re-entrant and compiles unmodified. It is available for various platforms like Windows, FreeBSD, Solaris, Mac and for mobile platforms.

Features

• JSON document store via Jx9
• Fully ACID-compliant
• Pluggable run-time interchangeable storage engine
• Simple, clean and easy-to-use API
• BSD licensed
• Key-value store
• Supports terabyte-sized databases

Official website: https://unqlite.org/

Latest version: 1.18

Firebase

The Firebase real-time database is cloud hosted. Data is stored as JSON and synchronised in real-time to every connected client. On building cross-platform apps with iOS, Android and JavaScript, all clients share one real-time database instance and automatically receive updates of new data.

Features

• Real-time, fast synchronisation: Firebase uses data synchronisation—every time data changes, any connected device receives that update within milliseconds.
• Offline access: Firebase apps remain responsive even when offline because the Firebase SDK persists your data to disk. Once connectivity is re-established, the client device receives any changes it missed, synchronising it with the current server state.
• Accessible via client devices: The Firebase real-time database can be accessed directly from a mobile device or Web browser; there’s no need for an application server. Security and data validation are available through its security rules, which are expression-based rules that are executed when data is read or written.

Official website: https://firebase.google.com/