AI/ML Infra Meetup | LLM Agents and Implementation Challenges
0 likes0 views
AI/ML Infra Meetup Aug. 14, 2025 Organized by Alluxio For more Alluxio Events: https://www.alluxio.io/events/ Speaker: - Pritish Udgata (Director - AI & MarTech Products @ Adobe) In this talk, Pritish Udgata from Adobe provides a comprehensive overview of implementation challenges and solutions for LLM agents. Topic include: - CoT vs RAG vs Agentic AI - Anatomy of an agent - Single Agent with MCP - Multi Agents with A2A - Implementation Challenges and Solutions
AI/ML Infra Meetup | LLM Agents and Implementation Challenges
- 1. LLM Agents AND Implementation Challenges Pritish Udgata Director of Products @ Adobe
- 2. Topics for today • Agentic AI • CoT vs RAG vs Agentic • Anatomy of an agent • Single Agent with MCP • Multi Agents with A2A • Implementation Challenges and Solutions
- 3. What is Agentic AI ? Agentic AI is an AI system that autonomously makes decisions and takes actions to achieve a goal without being told exactly what to do at every step.
- 4. Non-agentic vs Agentic AI START END Step 1 Step 2 Non-agentic AI “Developer” defined control path (static) All LLM applications follow some control path Step 1 Step 2 LLM Agentic AI “LLM” defined control path (dynamic) START END
- 5. CoT vs RAG vs Agentic Type Reason Plan Retrieve Proactive PROMPT : “Find me the best Italian restaurant nearby and book a table for 2 at 7 PM today” CoT ❌ ❌ ❌ “Bellagio is rated highly and is 2 miles away” RAG ❌ ❌ ❌ “Bellagio, rated 4.8, 2 miles away. Open until 10 PM.” Agentic “I booked a table for 2 at Bellagio at 7 PM. You’ll get a confirmation email shortly.” ● CoT - Chain of Thoughts ● RAG - Retrieval Augmented Generation
- 6. Anatomy of an agent In Agent / LLM Out Data Tools/APIs Memory Prompt Define persona, intent, constraint and goals Provide facts, contexts, signals and trends Connect to search, DB, and trigger action Store state, progress, history and patterns
- 7. Anatomy of an agent In Agent / LLM Out Data Tools/APIs Memory Prompt Define persona, intent, constraint and goals Provide facts, contexts, signals and trends Connect to search, DB, and trigger action Store state, progress, history and patterns Reason Plan Retrieve Proactive
- 8. Agent in an Enterprise Set-up Agent / LLM Data 1 Tools 1 Data 3 Tools 3 Data 2 Tools 2 Prompt Memory Multiple agents (N) must interact with multiple Data Sources/Tools (M). The classic “NxM problem” Example : N = 1, M = 6
- 9. MCP solves this problem Agent / LLM Data 1 Tools 1 Data 3 Tools 3 Data 2 Tools 2 Prompt Memory Model Context Protocol Standardizes how LLM applications communicate with data sources, tools
- 10. Single Agent with MCP Agent / LLM (MCP Host) Data 1 Tools 1 Data 3 Tools 3 Data 2 Tools 2 <MCP Protocol> MCP Server 1 MCP Server 2 MCP Server 3 MCP Client 1 MCP Client 2 MCP Client 3 Prompt Memory Maintain 1:1 connections with servers Expose specific capabilities through the standardized MCP The main application that uses LLM
- 11. Multi Agents with A2A Agent A (MCP Host) Data 1 Tools 1 Data 3 Tools 3 Data 2 Tools 2 <MCP Protocol> MCP Server 1 MCP Server 2 MCP Server 3 MCP Client 1 MCP Client 2 MCP Client 3 Agent B (MCP Host) Data 1 Tools 1 Data 2 Tools 2 <MCP Protocol> MCP Server 1 MCP Server 2 MCP Client 1 MCP Client 2 <A2A Protocol> Prompt Memory Prompt Memory A2A enables agent collaboration across frameworks and vendors Capability Discovery (Agent Cards) Secure Collaboration Task & State Management UX Negotiation A B C D
- 12. Implementation Challenges and Solutions Area Key Challenge(s) Solution(s) Tools / API Orchestration ● Picking the right tool ● Mapping LLM output to a tool ❏ Describe tools effectively ❏ Tool retrieval and omission ❏ Provide LLM right context (prompt structure) ❏ Schema/format matching (Tool 1 o/p > Too2 i/p) Memory Management ● Contextual memory (short-term) ● Persistent knowledge (long-term) ● Hallucination ❏ Context : In-memory storage (session context) ❏ Persistent : Vector (semantic), Graph (complex relationship), RDBMS (structured), Document (unstructured) ❏ Relevance filtering (READ what’s needed) ❏ Refresh and prune to avoid drift Security and Governance ● Unauthorized actions ● Compliance with org policies ❏ Role based access controls for tools/APIs ❏ Allow/deny list for sensitive actions ❏ Audit logging for decisions and actions Evaluation ● Agent performance ● Improve reason, retrieve, proactive ● Edge case failures ❏ Simulate before prod ❏ KPIs - task completion rate, accuracy, latency ❏ Implement human-in-the-loop (HITL)
- 13. Thank you for your time !