Storage is a key requirement these days and has to be thoroughly planned, keeping in mind performance, future enhancements, scalability and availability, as all these factors directly impact the functioning of the system. With the recent advances in cloud computing, the need for storage has been broadly classified into three major typesobject store, block store and file system store. Ceph is an open source, reliable and easy-to-manage, next-generation distributed object store based storage platform that provides different interfaces for the storage of unstructured data from applications.
Ceph, being an object store, provides an interface for all three types of data storage, i.e., object store, block store and file based store, hence giving a unified storage platform. Ceph is the best example of software-defined storage (SDS). Analogous to the principles of cloud computing, Ceph scales out using commodity hardware, considering the challenges of scaling up. This helps us define different hardware for different workloads, and manage multiple nodes as a single entity. With distributed storage, Ceph allows the distribution of the load, which delivers the best possible performance from low cost devices. All this contributes to low cost hardware, high data efficiency, broader storage use cases and greater performance. These are some of the major benefits of Ceph, because of which it has been widely adopted recently by enterprises and cloud service providers. Major storage vendors like SanDisk and other enterprises like Red Hat have contributed significantly to Cephs development. So let us take a closer look at it.
Architecture and design
Ceph is a scalable storage cluster based on RADOS (reliable, autonomic distributed object store), which is a self-healing and self-managing distributed object storage system. It has two types of nodes, Ceph Monitor and Ceph OSD. A Ceph Monitor node maintains the state of the cluster map to check for node availability, and provides the cluster map to the cluster clients. A Ceph OSD Daemon is an intelligent peer, which stores the data, checks its own state and other OSDs and reports back to monitors. In case an OSD goes down, the Ceph OSD Daemon automatically triggers data rebalancing based on the new cluster map from Ceph Monitor. Data is always transferred from the OSD nodes and never from the monitors to the clients.
This architecture of Ceph might seem familiar to those who know about Google File System (GFS) and Hadoop Distributed File System (HDFS) but it is also very different from them in multiple ways. Ceph uses an algorithm known as CRUSH (controlled, scalable, decentralised placement of replicated data) for random and distributed data storage among the OSDs. Ceph doesnt need two round-trips for data retrieval like HDFS or GFS, in which one trip is to the central lookup table to find the data location and the second trip is to the located data node. Every bit of data stored in Ceph OSDs is self-calculated using the CRUSH algorithm and stored independent of any other attribute. When a client requests data from Ceph, this CRUSH algorithm is used to find the exact location of all the requested blocks, and the data is transferred by the responsible OSD nodes.
As and when any OSD goes down, a new cluster map is generated and the duplicate data of the crashed OSD is transferred to a new node based on results from the CRUSH algorithm.
Ceph as object storage
Object stores have certain advantages over traditional block storage. Every stored object is given an object ID, which makes it faster to search/access and easier to manage. An object store is highly scalable for a large number of objects but has limitations on the size of a single object.
Ceph provides the LibRADOS native library, which allows direct access to RADOS using apps with support for C, C++, Java, Python, Ruby and PHP. Ceph also features RADOSGW, a RESTful HTTP API as the object gateway, which is compatible with Amazons S3 and Openstacks Swift services. Ceph can be used to make a cloud object storage solution like Dropbox or Google Drive using the Openstack cloud platform.
HDFS, GFS, Gluster-Swift, EMC Atmos and NetApp StorageGRID Webscale are a few of the leading proprietary object store systems.
Ceph as block storage
Block store is the traditional form of disk data storage where the data is divided into blocks and stored using a file system. Block store is best suited for VM disk volume storage needs, where we store large singular files with higher read and write frequencies. Ceph block storage can be well served as a SAN (storage area network) solution.
One added advantage of Ceph is that it has thin-provisioning enabled, which helps in faster data duplication and efficient storage space utilisation. When a disk stored in Ceph is replicated, it doesnt occupy any memory for the replicated copy. Ceph uses the COW (copy-on-write) method where the block is replicated and updated only on write, else the block is not replicated.
Amazons EBS (Elastic Block Storage) is a popular block storage service used generally for volumes of VM instances.
Ceph as file system storage
A file system based storage is like any NAS (network attached storage) system, where the file system is managed by a remote storage device. Ceph uses the Ceph FS (Ceph file system), which provides a POSIX-compliant file system as an interface. A client system can mount this file system and access the file storage.
Ceph FS uses Ceph MDS (Ceph metadata server) in the storage cluster to store the metadata about the file system. This metadata consists of the file system tree structure, timestamps, permissions and other POSIX-compliant data. Ceph MDS can also be distributed by dividing the tree structure and storing it in different Ceph MDSs. This metadata is stored like any other object in the Ceph OSDs, and is replicated for backup and recovery purposes. FreeNAS, NAS4Free are some of the NAS solutions available. The Lustre file system is a famous open source parallel file storage solution. Oracle ZFS Storage and Dell Storage SCv2000 are a few of the popular NAS solutions from proprietary vendors.
Ceph in cloud platforms
Storage is an important component in the cloud service providers (CSP) list of services. Major CSPs provide multiple forms of cloud storage services, which include rapid storage, archival storage, object store, block store, NAS, SAN, VM disk storage, etc. All these solutions generally require different dedicated hardware and software in order to maintain QoS and performance.
Ceph as a single storage cluster allows multiple data storage interfaces, which can utilise different hardware for different storage needs, managed by a single SDS system.
Ceph divides the OSDs into placement groups for the CRUSH algorithm. These placement groups can be combined together to form a pool, which is like a logical partition for storing the objects in Ceph. Pools can help differentiate between the storage hardware based on performance. Ceph also has cache-tiering, which helps in creating a pool of faster storage devices as cache storage for expensive read/write operations. This helps in improved performance and efficient utilisation of the storage hardware.
With OpenStack as the cloud platform, Ceph can be used as a Swift object store and Cinder block store utilising the same storage hardware for multiple needs. Ceph can be used with other cloud platforms like CloudStack, Eucalyptus and OpenNebula.
Ceph as a backup solution
We have been using RAID systems for redundancy and data backup purposes. With Ceph as the storage platform, data redundancy and recovery is managed using the RADOS storage system. RADOS has features like self-healing and self-managing, which help in recovering the data from lost OSD nodes using instant replication and rebalancing. Ceph also allows taking an incremental snapshot of block storage using RBD.
OpenStack has a Ceph backup driver, which is an intelligent solution of VM volumes. It backs up the volumes to Cephs backend storage, and regularly performs an incremental backup on the volumes to maintain consistency.
CloudBerry Backup for Ceph is a popular tool, which helps in versatile control over Cephs backup and recovery mechanism.
According to several research organisations like Gartner Inc., Ceph has made a strategic entry into the enterprise IT space and will prove to be the next big evolution in storage technology. With the current adoption rate, Ceph will soon surpass the existing storage solutions at enterprises. There is a lot of development expected to happen with Ceph, which will bring about significant performance improvements to match the current proprietary solutions. Even if you dont take future enhancements into consideration, Ceph is a storage platform that definitely needs to be looked at for your next big deployment.
