AI Agents at Work: Behind the Scenes of a Smarter Workflow

As AI systems evolve from being passive responders to autonomous agents that can plan, reason, and act, a new frontier has emerged: Agentic AI.

Agentic AI goes beyond automation. It refers to AI systems that operate with a level of autonomy and proactivity, capable of executing multi-step tasks, making decisions in dynamic environments, and collaborating with humans. For organizations, this shift presents both a massive opportunity and a complex implementation challenge.

To explore how enterprises are adapting to this paradigm, we spoke with Kaushiki Choubey, Engineering Lead, Generative AI at Lloyds Technology Centre. Kaushiki leads cutting-edge work in deploying agentic systems across financial services, from intelligent finance management to developer automation.

1. How have you replaced or enhanced workflows with AI agents in your organization, and what drove those decisions?

Answer: In our organization, we have increasingly integrated AI agents to enhance various workflows, driven by the need for more personalized, efficient, and real-time service delivery. Specifically, we are exploring the use of Agentic AI for customer spending management cases. This allows us to help customers better understand their spending patterns and manage their finances more effectively through automated insights and proactive recommendations.

Additionally, we see substantial potential in deploying AI agents as developers, which can accelerate software development cycles and improve code quality. In the financial services sector, AI agents are being utilized to provide more personalized advice tailored to individual client needs, support real-time savings opportunities, and strengthen fraud prevention through adaptive verification processes and transaction monitoring.

These decisions are motivated by the desire to deliver more empowering, secure, and efficient customer experiences while optimizing operational workflows and staying ahead in a competitive market.

2. How do you coordinate AI agents working across departments, and who is responsible for managing them?

Answer: In our organization, AI agents are coordinated through a structured approach that involves specialized teams within each department. These segregated teams are responsible for managing their respective AI agents, ensuring they meet departmental needs, stay updated, and operate effectively.

To maintain consistency, oversee integration, and prevent duplication of efforts, we have established a central unit that monitors the overall deployment of AI agents across departments. This central team manages issues related to duplicity of AI use cases, ensures adherence to organizational standards, and facilitates best practices and knowledge sharing among departments.

Overall, responsibility is shared: departmental teams handle day-to-day management and tuning of their AI agents, while the central unit provides oversight and strategic alignment to optimize AI deployment across the organization.

3. How do you ensure AI agents are safe, compliant, and unbiased in real-time operations?

Answer: We ensure AI agents are safe, compliant, and unbiased in real-time operations by focusing on three key areas through our developed packages and frameworks:

1. Robust Evaluation & Monitoring We use continuous monitoring and anomaly detection to identify deviations from expected behaviour, potential biases, or safety risks. Our tools facilitate rapid adversarial testing and provide explainability to trace decisions. We also integrate Human-in-the-Loop (HITL) for critical oversight.

2. Enhanced Decision Reliability Our agents are designed with confidence scoring and uncertainty quantification, allowing for human review when confidence is low. We also explore redundancy in decision-making and build frameworks for contextual awareness to improve reliability in dynamic environments.

3. Clear Task Completion & Reduced Ambiguity We emphasize unambiguous task definitions and quantifiable success metrics to guide agent behaviour. Through iterative refinement and extensive scenario-based testing, we continuously improve performance and reduce ambiguity in evaluation, ensuring clear, data-driven decisions on agent safety, compliance, and fairness.

4. What key benefits have you seen from using AI agents to streamline or transform business processes, and what challenges remain?

Answer: We've seen significant benefits using agentic AI to streamline business processes, primarily through reducing manual efforts and enhancing process effectiveness, even with complex edge cases.

Key Benefits:

• Massive Reduction in Manual Effort: AI agents automate repetitive, high-volume tasks, drastically cutting down human workload, increasing throughput, and freeing employees for higher-value, strategic work.
• Enhanced Process Effectiveness & Accuracy: Agents boost accuracy and consistency, proactively handle complex edge cases through contextual understanding and adaptive logic, and offer 24/7 availability.
• Data-Driven Decisions: Agents analyze vast datasets, providing insights for better, more agile decision-making.

Remaining Challenges:

• Data Quality & Integration: Poor data quality and integrating with legacy systems remain hurdles.
• Bias & Explainability: Mitigating algorithmic bias and ensuring transparency in decision-making are ongoing challenges.
• Novelty & Context: Agents still struggle with truly novel situations and understanding complex human intent.
• Scalability & Cost: Scaling deployments and managing associated costs effectively is complex.
• Change Management & Regulation: Adapting human workflows and navigating evolving AI ethics and regulations require continuous effort.

5. How do you measure the success of AI agents in your workflows, and what metrics matter most to your organization?

Answer: We measure AI agent success by focusing on Task Adherence, Tool Call Accuracy, Multi-turn Dialogue Handling, and Human-in-the-Loop (HITL) Feedback.

How We Measure:

• Response Quality: This is measured in terms of hallucinations, retrieval relevance, toxicity, code correctness and readability
• Efficiency and Optimization: We measure trajectory efficiency and convergence for efficient response.

Key Metrics:

• Task Adherence: We measure the agent's accuracy and completion rate of assigned tasks, including sub-task success and tool call accuracy. This ensures the agent is fulfilling its core purpose efficiently.
• Multi-turn Dialogue Handling: This assesses the agent's ability to maintain context and coherence across conversations. Metrics include contextual recall, coherence scores, and human escalation rates, vital for complex interactions.
• Human-in-the-Loop (HITL) Feedback: We quantify the effectiveness of human intervention. Metrics like intervention rate, correction rate, and the quality improvement from feedback help us continuously refine agents and build trust through human-AI collaboration.

As organizations transition into AI-first ecosystems, Agentic AI marks a fundamental shift in how work is done. It enables machines not just to respond, but to plan, act, and adapt working alongside humans in real time.

Kaushiki Choubey’s experience reflects how this shift is already underway in enterprise environments. From developer acceleration to financial decision support, the impact is tangible, if implemented thoughtfully, monitored rigorously, and measured meaningfully.

The future isn’t just automated. It’s agentic.