Cloud computing has become central to many enterprise IT models, and architects are designing cloud systems to be as effective and beneficial as possible. Open source cloud platforms are part of the design philosophy that drives innovation in cloud services. Let’s explore how organisations can benefit from them.
Transformation in technology happens every five years. Yesterday’s invention is obsolete tomorrow. Enterprises are always looking at gaining competitive advantage through collaboration, process agility and innovative business models, at low costs. Cloud solutions provide much needed flexibility to enterprises to develop capabilities necessary to innovate and seize new business opportunities. However, businesses look for the following before migrating an application to the cloud:
- Better customer experience
- Mobile access
- Business agility and flexibility
- Reduced hardware and IT staffing expenditure
- Improved security
- Improved responsiveness
- Better analytics on application usage
- Reduced and/or re-allocated costs
Open source cloud is any cloud service or solution that is developed using open source technologies and software. This covers any public, private or hybrid cloud model providing SaaS, IaaS and PaaS built and operated entirely on open source technologies.
Cloud adoption: Major trends
According to Gartner, in 2023, worldwide public cloud spending will grow 20.7 per cent to a total of US$ 591.8 billion, up from US$ 490.3 billion in 2022. And by 2025, enterprises will spend more on public cloud services than traditional IT solutions. By the same year, 50 per cent of enterprise workloads will be on IaaS/PaaS of a hyperscale provider, either in public cloud, on-premises, or at the edge.
As per an IDC report, by 2025, 85 per cent of organisations will see a 35 per cent increase in sustainable efficiencies by using software and cloud based infrastructures.
MarketsandMarkets says: “The hybrid cloud market is expected to grow from US$ 44.6 billion in 2018 to US$ 97.6 billion by 2023, at a CAGR of 17 per cent during the forecast period.”
According to Forrester, 40 per cent of global firms will take a cloud-native-first strategy in 2023, as they look to increase agility and efficiency while reducing costs.
- An expanding application ecosystem, which drives market responsiveness
- Changing business model – enterprises soon become integrators of best-of-breed services through collaboration
- New regulatory requirements driven by a global collaborative economy and the need to address open markets
- Better customer experience
- Transformation and optimisation across different process stacks: sales, front office, middle office and back office
The top tech trends as a direct result of cloud adoption are:
- Data centre rationalisation, as hybrid clouds replace data centres.
- Edge computing — data is stored and processed at the edge of the network, and analysed geographically closer to its source.
- Increased automation and self-learning capabilities, greater data security and privacy, and more personalised cloud experiences, all due to AI and ML.
- Disaster recovery — cloud computing is effective in disaster recovery and offers businesses the ability to quickly restore critical systems in the event of a natural or man-made catastrophe.
- Multi- and hybrid clouds offer a combination of public and private clouds dedicated to a specific company, the data of which is the key business driver, such as insurance companies, banks, etc.
- Kubernetes enables large scale deployments that are highly scalable and efficient. It is an extensible, open source platform that runs applications from a single source while centrally managing services and workloads.
- Movement of IT development and testing to the cloud.
- Maximised productivity with scalability and high availability. Core functions will increasingly move to private cloud and non-core to public cloud.
- Emergence of cloud service brokerages – move towards a hybrid model.
- Enterprises moving beyond traditional roles by offering new digitised products like cloud based storage for customer files.
Let’s see how different industries are making use of the cloud.
- Communications (telecom services providing point-to-point contact): Converged billing, multi-party settlement, financials, HR, procurement
- Governments (federal, state and local government agencies as well as non-profits): Public safety software, grants and performance management, computer assisted dispatch, jail management, courts management, tax collections, records management, financials, HR, procurement
- Healthcare (healthcare providers, hospitals, physicians, clinics, hospices): Electronic health records, practice management, revenue cycle management, financials, HR, procurement
- Chemicals (fuel extraction, petroleum refining, primary derivatives from oil, chemicals and allied products): Oil and gas operations management, reserves management, capital planning and budgeting, well production software, financials, HR, procurement
- Utilities (electricity, water and gas utilities): Customer care, billing, smart meter infrastructure, energy trading risk management, SCADA, financials, HR, procurement
Drivers for open source cloud adoption
Generally, cloud computing must satisfy five essential principles. These are: on-demand service, access network, resource pooling, elasticity, and measured services. To do this, cloud computing provides three kinds of service models: Software as a Service (SaaS), Platform as a Service (PaaS) and Infrastructure as a Service.
By interacting with various customers, I have arrived at the following key reasons for which a business moves to the cloud:
- Reduction of capex and opex to deliver business services
- Minimisation of IT costs by improving delivery time and quality of the application
- Businesses can move applications to the cloud at their own speed owing to different factors like complexity of the application, data requirements, regulatory and compliance needs, modernisation prerequisites, cost implications, real-time requirements, etc
- Best-of-breed environments can be selected for the applications, using special hardware to run and deliver the functionality at the most optimal cost and effort
- Security and regulatory requirements — applications requiring jurisdictional provisioning and regulatory compliance require deeper due diligence to select the right candidates for cloud and the right hosting option for the application
- Complex integrations — applications with high integration may need to be co-located to deliver the best experience. Point of delivery applications can be deployed on specialised hardware systems or integrated with machinery that requires these applications to be hosted on-premises
- Seasonal requirements — applications that have seasonal spikes can employ cloud services to handle the additional load for a short period and avoid capital investments
- High availability — applications can be redundantly deployed for high availability across many cloud types to provide business process continuity and disaster recovery
- Single pane and single view of all the applications across clouds are a must to ensure effective governance, security and cost management. A common process across all environments is mandated to ensure this homogeneity
Benefits of cloud adoption
Cloud adoption helps businesses to increase their flexibility to deliver IT resources, improves disaster recovery capabilities and lowers IT capital expenses. There are other benefits too.
- Business acceleration: The cloud helps to speed up business processes, supports collaboration, and provides cost-effective solutions to free up IT budgets for innovative, revenue-generating projects.
- Cost reduction: It helps in reducing operating and capital costs and improves performance, productivity and business agility via a flexible, scalable solution. Organisations can choose the applications to move across the clouds and on-premises based on their requirements.
- Reliability: If one cloud service goes down, some functionality is still available to users from the other deployed clouds. Generally, one public cloud can be used as backup to another cloud.
- Risk management: The cloud helps in mitigating risks with a single, unified, cyber security solution.
- Managing legacy systems: Hybrid cloud can bridge the gap between legacy and new systems, leading to major cost savings.
- Scalability: Applications can scale infinitely by adopting a hybrid multi-cloud strategy while keeping the core of the business secure through on-premises hosting.
Cloud computing layers and the open source products they use
Figure 1 shows the cloud computing layers and the respective open source products/tools/software that map to each layer of the cloud.
The layers cover the cloud market, cloud broker platform, cloud management, SaaS, PaaS, IaaS, virtualisation software/management, security, analytics, monitoring tool and hardware.
Cloud based open source technology enables enterprises to create and deploy applications with speed and consistency.
Application platforms handle complex connectivity issues behind the scenes (including automatic documentation), letting users concentrate on creating the right workflows and business logic, and also making it easier and more cost effective for organisations to manage frequent change. It is imperative to make sure that the integration solution is optimised for connecting the new applications in the cloud environments with older applications, systems, services and databases, no matter where they are located.
Open application programming interfaces (APIs): The application architecture uses open APIs to enable quick and transparent integration with other enterprise applications and systems implemented by various organisations, thereby providing access to data and services, and promoting citizen/developer participation for the benefit of the community.
API management includes the full API management life cycle, integration with diverse applications and appliances to support management, and monitoring and tracking of all your API activities.
Large complex systems in an enterprise follow the API-centric approach for exposing services for integration to multiple third-party solutions. The APIs are RESTful, XML based, and stateless services. All APIs are accessible via the HTTPS protocol.
Service discoverability: This is the process of a client application querying the central registry to learn the location of services. Service discoverability ensures that metadata attached to a service describes its overall purpose and functionality, which makes the services easily discoverable. A repository of re-usable business logic components is maintained and made available as part of service discoverability.
Consul, Apache Zookeeper, Etcd, Netflix Eureka, etc, are the largely used open source service discovery tools.
Platform and database-agnostic: Applications need to be forward and backward compatible, and be deployable on any technology platform. They should be able to communicate with any data store.
For example, Jenkins is a continuous delivery tool that builds and tests software after every change. Jenkins can deploy new code to production, which helps in no downtime in making the upgraded product available to constituents.
PostGreSQL, MySQL, MongoDB, CouchBase are the popular open source databases.
Microservices: The design of the business and information services layer of the enterprise application is developed using ‘microservices’ architecture principles and cross-channel capabilities. Microservice architecture (MSA) allows the creation of services that are loosely coupled and have different programming language bases, are scalable, have a quicker delivery time, etc. Businesses can develop new enterprise applications as reusable microservices based on open standards. This will ensure independent scaling of individual services within an application with better resource utilisation.
Application life cycle automation tools like Ansible help in tracking, deploying and measuring the changes/enhancements of microservices.
Mule, Jboss, AngularJS, Bootstrap are the frequently used open source dev tools today. Other popular open source tools are Joomla, Jetty, OpenSSO, SOLR, Chef, Spring, and WordPress.
Integration: The value of a strong integration system lies in its ability to connect to a wide range of backend systems. It provides mediation, transformation, protocol and routing capabilities, and acts as a gateway to integrate with core backend systems. It also provides aggregation and broker communications. Therefore, it is important to consider an integration solution vendor that has the ability to connect in a predictable manner to other databases, frameworks, applications and endpoints.
It is important to go with open standards and open interfaces while choosing the integration backend, as it should be free from vendor-specific modifications and proprietary hooks.
Mule, JBoSS, and WS02 are popular open source integration products.
Security: Effective enterprise security is the need of the hour. It ensures that all the components of the cloud infrastructure are guarded against the fast growing security threats. Applications and endpoints need to be protected and users granted access to applications as per the eligibility configured as part of the RBAC (role based access control).
Oauth2 and OpenID are open source tools that are used very often for this.
Customer experience: A good customer experience means that the individual’s experience during all points of contact with an application matches his or her expectations.
HTML5 and Joomla are a few open source tools used for enhancing the customer experience.
Analytics: Enterprises may apply analytics to better their customer services. Predictive analytics helps business services to be data driven and take decisions appropriately.
scikit-learn and TensorFlow are important open source tools in this domain.
PaaS: This is a category of cloud computing services that provides a platform allowing customers to develop, run, and manage applications without the complexity of building and maintaining the infrastructure typically associated with developing and launching an application. This is the core platform on which cloud native applications and supporting systems are based.
Cloud Foundry, Cloudify and OpenShift are a few popular open source PaaS platforms.
Key open source cloud platforms
When choosing an open source cloud platform, businesses must look at:
- Deployment flexibility
- Ease of operations
- Choice of application stacks
- Language, database and framework support
- Scaling capabilities
- Quality of service
- Tooling for development and operations
- Business fitment
A few important and popular open source cloud platforms are listed below.
OpenStack: OpenStack is open source software for creating private and public clouds, built and disseminated by a large and democratic community of developers, in collaboration with users. It is mostly deployed as Infrastructure-as-a-Service (IaaS), whereby virtual servers and other resources are made available to customers. The software platform consists of interrelated components that control diverse, multi-vendor hardware pools of processing, storage, and networking resources throughout a data centre.
Cloud Foundry: Cloud Foundry is an open Platform-as-a-Service (PaaS) which provides a choice of clouds, developer frameworks and application services. Cloud Foundry makes it faster and easier to build, test, deploy and scale applications.
It has different distributions, and the popular ones are Pivotal and IBM. It contains application runtime and container runtime. It also has Pivotal application service and Pivotal container service.
OpenShift: This is Red Hat’s cloud computing PaaS offering. It is an application platform in the cloud, where application developers and teams can build, test, deploy and run their applications.
Cloudify: Cloudify was developed and designed on the principles of openness to power the IT transformation revolution. It enables organisations to design, build and deliver various business applications and network services. The latest version of Cloudify is 4.3, which incorporates enhanced features like advanced security, control and true self-service. It has introduced a totally new concept for container orchestration with Kubernetes (Kubernetes cloud native orchestration)
WSO2: This Platform-as-a-Service (PaaS) framework from the Apache free source community provides elastic scalability features for any type of service, using the underlying infra cloud. WSO2 has a microservices based architecture that fosters agility and flexibility.
Apache Cloud Stack: This open source software platform is designed to position and manage the cloud computing environment. It is an Infrastructure as a Service (IaaS) cloud computing platform. Various service providers use Cloud Stack to offer public, private, and hybrid cloud services.
Eucalyptus: This is an open source software platform for implementing Infrastructure as a Service (IaaS) in a private or hybrid cloud computing environment. It allows users to deliver their own data centres into a private cloud, thus extending the services to other organisations. Eucalyptus is designed to provide services compatible with Amazon’s EC2 cloud and Simple Storage Service (S3).
To sum up, open source cloud helps to drive digital transformation and realise its benefits like better business agility, an improved customer experience and enhanced quality of service. It helps enterprises to move from transaction-based services to relationship-based services. It also paves the way for new digital services to be rapidly built, changed, enhanced and deployed.
Open source cloud makes enterprise data available openly, so that external parties can build services with adequate visibility of the enterprise’s functioning and availability of interfaces.
Based on the various customer engagements I have had in the cloud migration space, I would recommend the following:
- Choose the right cloud provider; each provider has specific strengths and weaknesses.
- Evaluate these properly so that they meet customer needs.
- Use the right migration tools.
- No vendor lock-in of the application.
- Move to an agile operating model.
- Replace data centres of customers with private and hybrid clouds.
- Maximise productivity with scalability and high availability.
- Always migrate development and testing environments to the cloud.
- Move core functions to the private cloud and non-core to the public cloud.
- Wherever possible, adopt a hybrid model – it’s a safe bet.
- Transform architecture to a scale out capability, enabled for dynamic API based interaction.
- Bring cloud native capabilities to the application.
- Reduce costs by using a higher level of automation.
- Have skilled staff that knows all the new technologies and processes, e.g., microservices, DevOps, API management, AI, etc.
The author would like to thank Santosh Shinde of the BTIS Enterprise Architecture division of HCL Technologies Ltd for giving the required time and support while this article was being written as part of architecture practice efforts.
