By embracing open source frameworks and tools guided by a robust enterprise architecture strategy, organisations can build intelligent, adaptable, cost-effective AI solutions tailored to their needs.
Generative AI is inspiring enterprises to enhance workflows, automate tasks, and unlock new levels of efficiency. While large language models (LLMs) form the core of this intelligence, the real power lies in building AI agents that can leverage these models within specific business contexts. For enterprise architecture (EA), this means the development, deployment, and operationalisation of generative AI agents, especially by harnessing open source offerings.
The enterprise architect’s blueprint for AI agents
Enterprise architecture is vital in guiding an organisation’s strategic adoption and implementation of AI agents. These agents must deliver real business value and align with the enterprise strategy. EA ensures that generative AI agents are not just technological novelties but strategic assets aligned with business objectives and integrated seamlessly into the existing IT landscape. Here are the different ways in which EA can contribute.
Strategic alignment
EA defines how AI agents support business capabilities and strategic goals. This includes identifying high-impact use cases, prioritising agent development, and measuring business value.
Architectural governance
EA establishes standards and guidelines for agent development, deployment, and management. This ensures the agent ecosystem’s consistency, security, scalability, and compliance.
Integration and interoperability
EA designs the agent architecture to ensure seamless integration with existing enterprise systems, data sources, and applications. This involves defining APIs, data flows, and communication protocols.
Technology selection and standardisation
EA evaluates and recommends open source frameworks, tools, and platforms for building AI agents, promoting standardisation, and reducing technology sprawl.
Risk management and security
EA addresses security, privacy, ethical, and legal considerations related to AI agents. This includes defining security protocols, data governance policies, and ethical guidelines.
Enterprise architecture is the central orchestrator for AI agent initiatives within an enterprise. It ensures that AI agents are deployed effectively, securely, and in a manner that genuinely benefits the organisation’s strategic objectives. This is essential for realising the full potential of AI agents in the enterprise landscape.
Table 1: A comparison of open source generative AI agents
| Feature | LangChain | AutoGen | Haystack |
| Agent focus | General-purpose, flexible workflows | Multi-agent collaboration, complex tasks | Conversational AI, search systems |
| Strengths | Extensive integrations, chaining, mature | Multi-agent conversations, task decomposition | Document retrieval, QA, knowledge agents |
| Weaknesses | Can be complex for simple agents | Newer framework, rapidly evolving | More focused on search/QA use cases |
| Model support | Broad via integrations | Broad via integrations | Broad via integrations |
| Enterprise fit | Versatile, wide applicability | Ideal for complex, collaborative tasks | Strong for knowledge-centric applications |
Top open source offerings for building generative AI agents
The open source ecosystem is brimming with powerful tools and frameworks that empower enterprises to build their own generative AI agents. Navigating this landscape requires understanding the different categories of tools and identifying the best fit for specific needs. The following list categorises and highlights some of the most prominent open source offerings, providing a starting point for enterprises exploring this space.
Agent frameworks (orchestration and management)
Agent frameworks are essential for structuring and managing the complexities of AI agents. These frameworks provide the scaffolding for building intelligent workflows, handling agent interactions, and managing the overall agent lifecycle. Here are some leading open source agent frameworks.
LangChain
A widely adopted framework designed to simplify the development of LLM-powered applications, it provides modules for model integration, prompt management, memory, agent creation, and chains of operations. LangChain excels in building complex agent workflows through its flexible chaining mechanisms and extensive integrations.
AutoGen
Microsoft focuses on enabling the development of AI agents that can collaborate with humans to solve complex tasks. AutoGen stands out for its multi-agent conversation capabilities, which allow for sophisticated task decomposition and collaborative problem-solving.
Haystack (Deepset)
An open source framework for building search systems and conversational AI applications. Haystack provides robust components for document retrieval, question answering, and agentic interactions, making it suitable for knowledge-intensive agent applications.
These agent frameworks offer distinct strengths and cater to different agent development needs. LangChain provides broad flexibility and a mature ecosystem, AutoGen excels in collaborative multi-agent scenarios, and Haystack is particularly well-suited for knowledge-driven applications. Choosing the proper framework depends on the enterprise’s requirements and the type of AI agents being developed.
LLM model libraries and APIs
At the heart of any generative AI agent lies the large language model (LLM). Open source offers diverse LLMs, each with their own strengths and characteristics. Accessing and leveraging these models is facilitated by model libraries and APIs. The following are key open source resources for LLMs.
Hugging Face Transformers
This is a central hub for pre-trained models, datasets, and tools for natural language processing. It offers access to thousands of open source LLMs, including models like Llama 2, Falcon, and more, providing a vast selection for agent development.
vLLM
An open source library for fast LLM inference and deployment, vLLM optimises performance and efficiency, making it ideal for deploying agents requiring real-time responsiveness and scalability.
OpenAI compatible APIs (e.g., Together AI, Anyscale endpoints)
While OpenAI itself is not open source, several platforms offer open source models accessible through OpenAI-compatible APIs. These balance ease of use with access to powerful open source LLMs.
These open source LLM resources provide enterprises with a broad spectrum of choices. Hugging Face Transformers offers an extensive catalogue of models, vLLM prioritises performance, and OpenAI-compatible APIs provide accessibility and ease of integration. The selection of the LLM resource should be guided by the agent’s specific performance requirements, desired model characteristics, and integration preferences.
Tools for development and deployment
Building and deploying generative AI agents for enterprise use requires robust development and deployment infrastructure. Open source tools provide scalable, reliable, and manageable environments for agent lifecycles. Here are some essential open source tools for development and deployment.
Ray
A unified framework for scaling AI and Python applications, Ray provides the infrastructure for distributed agent execution, parallel processing, and resource management, which is essential for enterprise-grade agent deployments.
Kubernetes
This container orchestration platform allows for the scalable and resilient deployment of AI agents. It simplifies the management of agent deployments, ensuring high availability and efficient resource utilisation.
MLflow
This is an open source platform for managing the machine learning lifecycle, including agent development. MLflow provides tools for tracking experiments and packaging agents, and deploying them in production environments.
These tools are fundamental for creating enterprise-grade AI agent solutions. Ray enables scalable agent execution, Kubernetes ensures robust deployment and management, and MLflow streamlines the entire agent lifecycle from development to production. Leveraging these open source tools is crucial for building AI agent solutions that are not only intelligent but also scalable, reliable, and maintainable within an enterprise setting.
Open source generative AI agents: A comparative glance
A comparative overview is helpful to provide a clearer picture of the agent frameworks we have discussed. Table 1 compares the key features and characteristics of LangChain, AutoGen, and Haystack.
This comparison highlights the distinct focuses and strengths of each framework. LangChain’s versatility suits a wide range of agent-based use cases. AutoGen excels in multi-agent collaboration scenarios, and Haystack shines in knowledge-intensive applications like search and question-answering. Enterprises can use this comparison to select their framework based on their specific agent development goals.
Benefits of open source for enterprise AI agents
Adopting open source software for generative AI agents offers enterprises many advantages.
Customisation and flexibility
Open source allows enterprises to tailor agents precisely to their unique needs, data, and workflows. Models can be fine-tuned, architectures modified, and functionalities extended without vendor lock-in.
Data privacy and security
Open source agents can be deployed on-premises or within private clouds, providing greater control over sensitive data and ensuring compliance with data governance policies.
Cost-effectiveness
Leveraging open source models and frameworks can significantly reduce licensing costs associated with proprietary AI solutions.
Community and innovation
Open source projects benefit from vibrant communities, fostering rapid innovation, knowledge sharing, and collaborative problem-solving.
Transparency and auditability
Open source provides full transparency into the agent’s code and workings, enabling better auditability, explainability, and trust in AI systems.
Open source empowers enterprises with greater control, flexibility, and transparency in their AI agent initiatives. These benefits are particularly compelling for organisations seeking to build customised solutions, maintain data security/privacy, optimise costs, and leverage the collective intelligence of the open source community.
The challenges
While open source offers numerous advantages, enterprises must also know the challenges and considerations of adopting open source generative AI agents.
Complexity and expertise
Building and managing open source AI agents requires in-house AI, software development, and infrastructure expertise. Enterprises may need to invest in training or hire specialised talent.
Integration challenges
Integrating open source agents with existing enterprise systems can be complex and requires careful architectural planning and development effort.
Support and maintenance
While community support is available, enterprises are responsible for the ongoing maintenance, updates, and security of their open source agent deployments.
Maturity and stability
Some open source projects may be less mature or stable than commercial offerings. EA must carefully evaluate the maturity and reliability of chosen frameworks and tools.
Licensing and compliance
It is crucial to navigate the various open source licences and ensure compliance with legal and regulatory requirements.
While the benefits of open source generative AI agent solutions are significant, enterprises must be prepared to address these inherent challenges.
Open source generative AI agents offer a paradigm shift for enterprises seeking to leverage AI for workflow automation and business transformation. However, the key to successful AI solutions lies in strategic planning, careful technology selection, investment in expertise, and a proactive approach to addressing the unique challenges and opportunities presented by open source AI.
