Last month, we learnt what agentic AI is and how it works. In this second article on agentic AI, we will learn how to develop and design an AI-driven multi-agent service.
The growth of agentic artificial intelligence (AI) is being fuelled by the complexity of the tasks that machines do. AI agents are becoming an essential part of our technological landscape, be it for self-driving cars or conversational robots. At the heart of agentic AI is the architecture that determines how agents sense, learn, decide, and act. The tools and frameworks used to construct such agents are manifold, each bringing the capability to work on specific problems. Navigating through these platforms is overwhelming but is crucial for designing effective, smart agents.
In essence, agentic AI refers to systems that can make choices and act on their own without the need for constant human interference. The most typical use of this kind of AI is found in autonomous agents, which do the work of moving around spaces, solving problems, and engaging with humans or other machines. To construct such agents one needs to have a firm grasp of both theoretical principles and working tools.
Developers use specialised frameworks that provide tools for creating and managing AI agents efficiently. Here are some of the most important agentic AI frameworks used today.
Microsoft AutoGen
AutoGen is an open source programming framework for building AI agents and facilitating cooperation among multiple agents to solve tasks. It is designed to handle multi-agent systems where different AI agents collaborate to solve complex problems. It enables AI agents to work independently or with human input, allowing for flexible and scalable AI solutions.
Key features:
- Supports multi-agent workflows and real-time collaboration.
- Integrates human-in-the-loop interactions for oversight and decision-making.
- Provides memory and context management for AI conversations.
- Includes error-handling mechanisms to ensure reliability.
Use case
A cloud management AI system where different agents work together to detect and fix server issues autonomously.
LangChain
LangChain is a popular framework that helps developers create AI applications by chaining prompts, memory, and various tools together. It simplifies the process of integrating AI into different workflows.
Key features
- Modular design for creating and managing AI workflows.
- Built-in memory to enable stateful conversations.
- Compatibility with multiple AI models and APIs.
Use case
A chatbot for a bank that retrieves customer details and provides insights based on transaction history.
LangGraph
LangGraph extends LangChain by introducing a graph-based approach to managing workflows. This framework is useful for applications that require step-by-step task execution.
Key features
- Graph-based workflow management for AI agents.
- Handles dependencies between different tasks efficiently.
- Supports complex multi-step processes.
Use case
A medical diagnosis AI that retrieves patient history, analyses symptoms, and suggests treatments in a structured workflow.
Microsoft Semantic Kernel
Microsoft Semantic Kernel focuses on making AI systems understand user intent and context better. It is particularly useful for AI applications that require deep semantic reasoning.
Key features
- Context-aware decision-making for AI applications.
- Pre-built integrations with enterprise tools and systems.
- Enhanced semantic understanding for improved accuracy.
Use case
An AI-powered IT helpdesk that understands user queries and suggests the best solutions based on past interactions.
CrewAI
CrewAI is designed for multi-agent collaboration. It allows AI agents to take on different roles and work as a team to complete tasks efficiently.
Key features
- Role-based AI agents with specialised tasks.
- Customisable tools and workflows for different use cases.
- Task execution can be sequential or hierarchical.
Use case
A logistics management AI system where agents plan delivery routes and optimise warehouse operations collaboratively.
PhiData Agentic Framework
The PhiData Agentic Framework enhances AI assistants by integrating memory, knowledge, and tools to handle complex tasks.
Key features
- AI memory to retain past interactions.
- Access to multiple data sources, including databases and PDFs.
- Built-in tools for web search, document summarisation, and query execution.
Use case
A business AI assistant that retrieves reports, answers queries, and automates document processing.
Building agentic AI: A hands-on guide for developers
The goal of this system is to develop an AI-driven multi-agent service that automatically classifies user queries into different categories and routes them to specialised AI agents for appropriate responses. Specifically, the system categorises queries into:
- Weather-related questions (handled by the Weather Service Agent)
- News-related enquiries (handled by the News Service Agent)
- Joke requests (handled by the Joke Service Agent)
Importing dependencies
We begin by importing dependencies as follows.
- langgraph: Used for managing workflows and state transitions.
- time: Used for measuring performance.
- HuggingFaceHub and SystemMessage from langchain: To interact with the Hugging Face model hub and work with prompts.
- StateGraph from langgraph: To define a state machine that orchestrates the flow of different agents.
- BaseModel from pydantic: For defining structured data models (e.g., ExpertSystemState).
- os: For handling environment variables, specifically to set the Hugging Face API key.
Importing Dependencies import warnings warnings.filterwarnings(“ignore”, category=FutureWarning) import langgraph import time from langchain.llms import HuggingFaceHub from langchain.schema import SystemMessage from langgraph.graph import StateGraph from pydantic import BaseModel import os
Now, we set up the Hugging Face API key. This sets the environment variable for your Hugging Face API token so that you can interact with the model. Replace YOUR KEY HERE with actual API KEY.
os.environ[“HUGGINGFACEHUB_API_TOKEN”] = “YOUR KEY HERE”
Next, a specific Hugging Face model (Mistral-7B-Instruct-v0.1) is loaded. temperature controls randomness, and max_length controls the length of the generated responses.
llm = HuggingFaceHub(
repo_id=”mistralai/Mistral-7B-Instruct-v0.1”, # Public model
model_kwargs={“temperature”: 0.7, “max_length”: 256}
)
The state schema defines a structured model for the state of the system, which includes:
input_data: The user query. category: The category of the query (e.g., weather, news, joke). response: The response generated by the agent (e.g., weather forecast, news, joke). class ExpertSystemState(BaseModel): input_data: str # User query category: str = None # Categorized question type response: str = None # AI-generated response
The ‘helper’ function logs events during the workflow (like analysing a query, classifying a query, etc) for debugging or tracking purposes.
def log_event(step: str, message: str):
print(f”[{step}] {message}”)
Defining the service agents
Each agent corresponds to a different category of query. Here’s how each agent works.
- Super Agent (super_agent function):
- Analyses the user’s query and categorises it (weather, news, or joke).
- Uses the Hugging Face model to classify the query based on the prompt.
- Weather Service (weather_service function):
- Provides weather-related information based on the query.
- News Service (news_service function):
- Provides the latest news based on the query.
- Joke Service (joke_service function):
-
- Provides a joke based on the query.
Each of these functions generates a response using the Hugging Face model, logs it, and returns the ExpertSystemState with the appropriate response.
def super_agent(state: ExpertSystemState) -> ExpertSystemState:
log_event(“Super Agent”, f”Analyzing query: {state.input_data}”)
prompt = f”””
You are a Super Agent tasked with categorizing the query into one of the following categories:
- weather_service
- news_service- joke_service
Examples:Query: What is the weather forecast for today?
Category: weather_service
Query: What’s the latest news?
Category: news_service
Query: Tell me a joke.
Category: joke_service
Only return the category name.
Query: {state.input_data}
“””
classification = llm.invoke([SystemMessage(content=prompt)]).strip()
category = classification.split()[-1]
log_event(“Super Agent”, f”Classified as: {category}”)
return ExpertSystemState(input_data=state.input_data, category=category)
def weather_service(state: ExpertSystemState) -> ExpertSystemState:
log_event(“Weather Service Agent”, f”Providing weather forecast for: {state.input_data}”)
prompt = f”Provide the weather forecast for: {state.input_data}”
response = llm.invoke([SystemMessage(content=prompt)]).strip()
log_event(“Weather Service Agent”, f”Generated Response: {response}”)
return ExpertSystemState(input_data=state.input_data, category=state.category, response=response)
def news_service(state: ExpertSystemState) -> ExpertSystemState:
log_event(“News Service Agent”, f”Providing latest news for: {state.input_data}”)
prompt = f”Provide the latest news for: {state.input_data}”
response = llm.invoke([SystemMessage(content=prompt)]).strip()
log_event(“News Service Agent”, f”Generated Response: {response}”)
return ExpertSystemState(input_data=state.input_data, category=state.category, response=response)
def joke_service(state: ExpertSystemState) -> ExpertSystemState:
log_event(“Joke Service Agent”, f”Telling a joke for: {state.input_data}”)
prompt = f”Tell a joke for: {state.input_data}”
response = llm.invoke([SystemMessage(content=prompt)]).strip()
log_event(“Joke Service Agent”, f”Generated Response: {response}”)
return ExpertSystemState(input_data=state.input_data, category=state.category, response=response)
The StateGraph is created to build the workflow, which defines the flow of execution. Each service agent is added as a ‘node’ in the graph.
workflow = StateGraph(state_schema=ExpertSystemState) workflow.add_node(“super_agent”, super_agent) workflow.add_node(“weather_service”, weather_service) workflow.add_node(“news_service”, news_service) workflow.add_node(“joke_service”, joke_service)
The decision_function decides which service to route the query to, based on the category (weather_service, news_service, joke_service). If the category is not found, it defaults to the weather service.
def decision_function(state: ExpertSystemState):
routing = {
“weather_service”: “weather_service”,
“news_service”: “news_service”,
“joke_service”: “joke_service”
}
return routing.get(state.category, “weather_service”)
workflow.add_conditional_edges(“super_agent”, decision_function)
To set workflow entry and exit points, use the following code:
workflow.set_entry_point(“super_agent”) # Start point workflow.set_finish_point(“weather_service”) # Possible endpoint workflow.set_finish_point(“news_service”) # Possible endpoint workflow.set_finish_point(“joke_service”) # Possible endpoint app = workflow.compile()
Processing user queries
A list of user queries is defined. These are processed one by one as follows.
- The workflow is invoked with the query as input.
- The final response is obtained and processed.
- Time taken for each query is logged.
# List of user queries (Very definite questions)
user_queries = [
“What’s the weather like today?”,
“Tell me the latest news.”,
“Can you tell me a joke?”,
“What is today breaking news in Kolkata?”
]
for idx, query in enumerate(user_queries, start=1):
print(f”\n--- Processing Query {idx}/{len(user_queries)} ---”)
start_time = time.time()
final_state = app.invoke(ExpertSystemState(input_data=query))
final_state = ExpertSystemState(**final_state
end_time = time.time()
log_event(“Response Time”, f”{round(end_time - start_time, 2)} seconds”)
print(“\nFinal Response:”)
print(f”- User Query: {final_state.input_data}”)
print(f”- Category: {final_state.category}”)
print(f”- Final Response: {final_state.response}”)
print(“-” * 50)
The output is:
--- Processing Query 1/4 --- [Super Agent] Analyzing query: What’s the weather like today? [Super Agent] Classified as: weather_service [Weather Service Agent] Providing weather forecast for: What’s the weather like today? [Weather Service Agent] Generated Response: The weather forecast for today is sunny with a high of 25°C and a low of 15°C. [Response Time] 0.42 seconds Final Response: - User Query: What’s the weather like today? - Category: weather_service - Final Response: The weather forecast for today is sunny with a high of 25°C and a low of 15°C. -------------------------------------------------- --- Processing Query 2/4 --- [Super Agent] Analyzing query: Tell me the latest news. [Super Agent] Classified as: news_service [News Service Agent] Providing latest news for: Tell me the latest news. [News Service Agent] Generated Response: The latest news is that the stock market has reached an all-time high today. [Response Time] 0.48 seconds Final Response: - User Query: Tell me the latest news. - Category: news_service - Final Response: The latest news is that the stock market has reached an all-time high today. -------------------------------------------------- --- Processing Query 3/4 --- [Super Agent] Analyzing query: Can you tell me a joke? [Super Agent] Classified as: joke_service [Joke Service Agent] Telling a joke for: Can you tell me a joke? [Joke Service Agent] Generated Response: Why don’t scientists trust atoms? Because they make up everything! [Response Time] 0.43 seconds Final Response: - User Query: Can you tell me a joke? - Category: joke_service - Final Response: Why don’t scientists trust atoms? Because they make up everything! -------------------------------------------------- --- Processing Query 4/4 --- [Super Agent] Analyzing query: What is today breaking news in Kolkata? [Super Agent] Classified as: news_service [News Service Agent] Providing latest news for: What is today breaking news in Kolkata? [News Service Agent] Generated Response: Breaking news in Kolkata: A new metro line has been inaugurated today, enhancing the city’s public transport system. [Response Time] 0.45 seconds Final Response: - User Query: What is today breaking news in Kolkata? - Category: news_service - Final Response: Breaking news in Kolkata: A new metro line has been inaugurated today, enhancing the city’s public transport system. --------------------------------------------------
- The Super Agent categorises each query (weather, news, or joke).
- Based on the classification, the query is routed to the appropriate service agent (weather, news, or joke).
- Each service agent generates a response, such as a weather forecast, news update, or joke.
- The system logs the time taken to process each query and the final response.
- The output shows the final response (query, category, and generated response) for each user query.
A framework for future AI systems
Multi-agent architectures that enable adaptive learning, dynamic decision-making, and frictionless integration into varied ecosystems are the future of AI. Next-generation agentic AI architecture is composed of five integrated layers that enable the system’s overall function.
The Input Layer collects data from user interactions and real-time feeds to enable the system to be pre-empted with actionable intelligence. The Agent Orchestration Layer enables adaptive task management, collaborative actions, and performance monitoring. At each of these layers, agents have developed distinctive functions, such as planning and execution, self-scoring and optimising performance, employment of tools and domain-specific work, or continually learning and improving knowledge models. The Data Storage and Retrieval Layer receives centralised and distributed stores, optimised information access vector stores, and context-enriched knowledge graphs supporting context-based reasoning. In the Output Layer, AI insights are translated into individualised, context-aware outputs and continuously update system knowledge bases. The Service Layer provides enhanced AI capability across all platforms with cognitive recommendations and governance-compliance assurance.
This design also includes safeguards for safe, compliant, and responsible use of AI.
AI models designed in this way can keep a check on any bias, enable fairness and safety, and ensure compliance with regulations, ensuring the system remains strong in the long run. Also, partnership AI models enhance overall functionality by enabling interaction with external systems.
