Digital storage needs are increasing rapidly. According to IDC, there will be 40 zetta bytes of data in the digital universe by 2020. The open source community has developed a number of products to assist users to store, secure and manage all that data. Open source software defined storage provides the benefits of open source tools and the benefits of massive economies of scale by running the products on commodity hardware. The end result is scalable, agile, easily manageable, and loosely coupled environments for unstructured data storage. Software defined storage brings the power of virtualisation to data storage. It allows organisations to abstract and pool storage capacity, as well as compute and networking resources, across on-premise and cloud environments.
Here is an overview of nine open source storage solutions that are considered top-of-the-line today.
Gluster: A NAS/SAN solution
Gluster is an open source and general-purpose distributed file system that aggregates storage exports over the network to provide a single unified namespace. It is written in C, with a GPLv3 licence and access APIs – libglusterfs and FUSE. It pools storage servers over TCP/IP or InfiniBand Remote Direct Memory Access, which allows rapid provisioning of additional storage based on storage requirements. It can be used with low-cost commodity computers. Some of the popular uses of GlusterFS are in: unstructured data storage, virtual machine image storage, archiving, disaster recovery, cloud storage for service providers, semi-structured and structured data, Big Data, content delivery and streaming media. It is very suitable for unstructured data such as audio and video files, images, documents and log files. Glusters open source community develops GlusterFS and related products.
GlusterFS is based on the following concepts:
- Trusted Storage Pool is a collection of storage servers that work as a logical partition for all data and management operations
- Brick is the mixture of a node and an export directory
- Volume is a mountable entity and logical collection of bricks
Features
- Automatic failover without a centralised metadata server
- Layered approach to the file system, where features are added or removed on demand
- Disk file systems such as ext3, ext4, xfs, etc, are supported to store the data
- Can scale up to petabytes of storage
- Distributed geo-replication
- File snapshots
- Compression translator
FreeNAS: A NAS/SAN solution
FreeNAS is a FreeBSD-based operating system for network attached storage. It comes with an easy-to-use Web UI. NAS-oriented open source storage distributions of Linux and FreeBSD include FreeNAS, Gluster, Openfiler, etc, which can be connected and configured using a Web browser. It is released under a BSD licence. The FreeNAS Project was initially founded by Olivier Cochard-Labbe in 2005. It was PHP-based, easy to use and was based on an embedded firewall (also based on FreeBSD). FreeNAS can run from a USB disk or virtual machine. It is distributed as an ISO image. FreeNAS is available for both 32-bit and 64-bit architectures (a 32-bit system can only address up to 4 GB of RAM). The FreeNAS stable version is 9.2.1.5.
Features
- Supports protocols such as NFS, SSH, FTP, TFTP, BitTorrent, iTunes, CIFS (via Samba), etc
- Plug-in support for SlimServer and Xbox Media Stream Protocol
- iSCSI target feature is used to create virtual disks. ZFS, UFS, ext2, and ext3 are also supported.
- Read and write for FAT32 and NTFS
- Hard drives supported are (P/S)-ATA, SCSI, iSCSI, USB and FireWire; and booting is possible from a HDD, CD-ROM, floppy disk or USB flash drive.
- It provides snapshot support, deduplication, a removable log device, etc.
Openfiler: A NAS/SAN solution
Openfiler is a free network storage OS created by Xinit Systems, and is based on the rPath Linux distribution. It supports file-based Network Attached Storage (NAS) and block-based Storage Area Networking (SAN) functionality in a single unified framework. Openfiler uses the Linux 2.6 kernel base to deliver a wide-ranging storage management solution that fulfils the needs of enterprise applications. Openfiler supports on-disk file systems such as XFS, ext3, ReiserFS v3 and JFS. XFS and ext3 are journalled file systems that enhance data security and reduce the need for regular file system checks. Openfiler has the following hardware requirements: an x86- or x64-based computer with at least a 64-bit processor with 1.6 GHz or more, 512 MB of RAM, a CD-ROM or DVD-ROM drive if you are performing a local install, 10 GB of hard disk space (8 GB for OS installation and 2 GB for swap space). It is licensed under the GNU General Public License version 2. It supports the i386/AMD64 platforms. Installable images are available for x86_64 architectures. The download options are available at http://www.openfiler.com/community/download
Features
Full industry-standard protocol suite: NFSv3 support for all UNIX-based clients with additional support for ACL protocol extensions; CIFS or SMB support for Microsoft Windows-based clients; FTP support, WebDAV and HTTP 1.1 support; and NFSv4 support (testing)
Good block storage virtualisation: Point-in-time snapshots support with scheduling; synchronous/asynchronous volume replication and migration; optimal division of storage by full iSCSI target support. It also supports virtual iSCSI targets.
Extensive share management: Support for multiple shares per volume; multi-level share directory tree; multi-group-based access control on a per-share basis; multi-host/network- based access control on a per-share basis
Accounts management: Authentication by utilising pluggable authentication modules; Web interface-enabled configuration; it supports network directories such as LDAP, Hesiod, NIS, NT4 domain controller, and Active Directory.
Amanda: A backup and synchronisation solution
Advanced Maryland Automatic Network Disk Archiver, or Amanda, is an open source backup solution that can be used to take backups of data stored on multiple computers over a network. It is written in C and Perl. Windows, Mac OS X, Linux and Solaris are supported operating systems. It is distributed under the GPL, LGPL, Apache and Amanda licences. Amanda is available in two different flavoursthe community edition and a fully supported enterprise edition.
Features
- It supports tape-based and disk-based backup
- It supports tape spanning if a backup set does not fit in one tape
- It uses utilities and formats such as dump and/or GNU tar
- It supports multiple simultaneous writes to storage devices
Clonezilla: A backup and synchronisation solution
Clonezilla is a partition and disk imaging/cloning program distributed under the GNU General Public License version 2. It is utilised for bare metal backup and recovery. It comes in two flavoursClonezilla SE is used for group machine backups and restores, while Clonezilla Live is used for single machine backups and restores. It is written in Perl and UNIX shell. It supports POSIX GNU/Linux operating systems. The minimum system requirements for Clonezilla Live are 196 MB of RAM, a CD/DVD drive, a USB port, PXE or hard drive, and an x86 or x86-64 processor.
Features
- File systems supported for Microsoft Windows are: FAT12, FAT16, FAT32 and NTFS
- File systems supported for GNU/Linux are: ext2, ext3, ext4, reiserfs, reiser4, xfs, jfs and btrfs
- File system supported for FreeBSD, NetBSD, and OpenBSD is: UFS
- File systems supported for ESXi are: VMFS3 and VMFS5
- File system supported for Mac OS is: HFS+
- Cloning supported for 32-bit or 64-bit GNU/Linux, MS Windows, Intel-based Mac OS, FreeBSD, NetBSD, OpenBSD, Minix and VMware ESX
Pydio: An online data storage solution
Pydio is an open source tool that is used to utilises a server available on premise or in a cloud environment.
Features
- It provides more control, privacy, a better total cost of ownership and security
- Easy to install
- Easy integration with available employee directory
Libvirt: A storage management solution
Libvirt is an open source management tool for platform virtualisation. It also provides storage management on the physical host in different ways such as storage pools and volumes. A storage pool is a quantity of storage that is used by virtual machines (VMs). Storage pools are divided into storage volumes and assigned to VMs as block devices. Pools and volumes ensure that storage will be available for a VM. Management applications using libvirt enable users to allocate storage resources, run the virtual machines, shut them down, de-allocate the resources, etc.
Features
Libvirt supports storage pool types such as directory backend, logical backend, disk backend, local file system backend, network file system backend, iSCSI backend, SCSI backend, multipath backend, RBD (RADOS Block Device) backend, Sheepdog backend and Gluster backend.
7-Zip: A file compression solution
7-Zip is open source software written in C++ that is used to compress files in the 7z archive format. It was developed by Igor Pavlov and first released on July 18, 1999. A stable release has been available since November 18, 2010. It supports various operating systems such as GNU/Linux, Microsoft Windows and Mac OS X. Table 1 gives the file compression ratio of this storage solution.
It has a graphical user interface and command line interface.
Features
- Confidentiality increases due to AES-256 encryption in 7z and ZIP formats
- Supports localisation for 79 languages
- It has a powerful command line version and file manager
- Provides integration with Windows Shell
TrueCrypt: An encryption solution
TrueCrypt is an open source application used for data encryption before it is saved and data decryption before it is consumed. It supports operating systems such as Linux, OS X, Microsoft Windows, etc. TrueCrypt currently uses the XTS mode [XEX-based Tweaked CodeBook mode (TCB) with CipherText Stealing (CTS)]. Password, keyfile(s) or correct encryption keys are only ways to access data stored on an encrypted volume. They are used to protect data confidentiality even if the OS is not active.
Supported algorithms: Twofish, Serpent and AES
Supported hybrid combinations of algorithms: Cascades, Serpent-Twofish-AES, Serpent-AES, AES-Twofish-Serpent, Twofish-Serpent and AES-Twofish
Supported cryptographic hash functions: TrueCrypt supports the SHA-512, RIPEMD-160 and Whirlpool hash algorithms. Roadmap for future versions includes features such as support for Windows 8 to encrypt Windows system partitions/drives on UEFI-based computers, raw CD/DVD volumes, command line options for volume creation on the Windows operating system, etc. You can download these from: http://www.truecrypt.org/downloads
Features
- Two-factor authentication
- Individual disk partitions and swap space can be encrypted
- Encrypted container can be persisted in a file
- Hidden containers – one per outer container is supported; hidden containers are similar to nested containers
- Pre-boot authentication on Windows is required before booting the computer to encrypt the boot disk
- More than one active key
- Passphrase strengthening – key strengthening is used with pure text passwords
- Hardware acceleration – cryptographic accelerator expansion cards can be used
- Windows MBR volumes can be encrypted and not UEFI GPT drives
- Complete physical disk or logical volume, including the partition tables and master boot record can be encrypted
References
[1] http://www.truecrypt.org
[2] http://en.wikipedia.org/wiki/FreeNAS
[3] http://doc.freenas.org/index.php/Quick_Start_Guide
