Building AI agents: 5% AI, 95% software engineering

View profile for Alex Wang
Alex Wang Alex Wang is an Influencer

Learn AI Together - I share my learning journey into AI & Data Science here, 90% buzzword-free. Follow me and let's grow together!

Building AI agents = 5% AI + 95% software engineering. Not exact ratio, but you get the point. We don’t just “build an agent”; we architect a system and let the AI sit inside it. Enterprise-ready agent platforms are classic software + reliability engineering work: ✅ Identity and access management ✅ Document filtering and governance (ACLs, redaction, PII masking) ✅ Schema mapping and data contracts ✅ Human-in-the-loop escalation ✅ Infra that scales across vector and SQL ✅ Observability, evals, guardrails, cost controls, and audit logs Think of agents like APIs that can reason, not magic. They still need: - Fine-grained access control - Storage that separates structured and unstructured knowledge - Tracing, fallback routing, and lineage - Flows that connect document pipelines with model orchestration (MCP, tools, integrations) Before you tune prompts, build the foundations. If you’re exploring agent workflows, check out our open-source platform, a lightweight framework we use to run multi-agent systems: GitHub: https://bit.ly/4kzE1Mt Explore our public investment: 🔗https://bit.ly/41JeDNO __________ For more on AI and learning materials, plz check my previous posts. I share my journey here. Join me and let's grow together. Alex Wang #AIagents #AgenticAI #EnterpriseAI #Technology

  • graphical user interface, diagram
Mohamed Chergui

Vorstand bei think tank Business Solutions AG | Diplom-Ingenieur

4w

This post nails a crucial truth that often gets lost in the hype: Building AI agents isn’t just about clever prompting, It’s software engineering at its core. The most robust systems are not those with the smartest models, but the ones with the best scaffolding: identity management, access control, observability, fallback logic, and governance. It reminds me of a simple but powerful idea: "AI agents are more like APIs that can reason – not magic". They need everything traditional software needs to scale safely. Before we talk about AGI, we need to talk about architecture. Thanks for bringing clarity Alex Wang.

Tushar Parmar

Talks About AI | AI/ML Development | Leadership | Cloud | Software | Automation

4w

Alex Wang Really like the framing here. AI agents aren’t magic, they’re engineered systems. The emphasis on governance, access control, and observability shows how much this is about enterprise-grade reliability rather than just prompt design. From your experience, what tends to be the bigger roadblock in scaling agent platforms: governance of data or reliability of the underlying systems?

Like
Reply
Ravi Kiran

AI Engineering Leader | Building AI Agents for Enterprises | LangChain Ambassador | Open-Source Contributor

4w

real AI impact comes when solid engineering meets smart models, not from models alone.

Hans Kuijs

Sr Project Manager bij Red Hat

1mo

Creating robust AI systems requires careful planning and engineering. These foundations truly make a difference. 🔧 #AIFrameworks

Great breakdown, Alex. It’s easy to get caught in the AI hype, but success with agentic systems depends on sound architecture, observability, and trust. Appreciate you shining a light on what it takes.

Like
Reply
Nikolas R.

Product Strategy / Product Design / Product Manager / Product Owner / Design Thinking / Agile Methodologies / Big Data ML & AI

4w

Alex Wang I was studying your diagram and Im a bit confused, so I can see you have Azure Foundry set up you give it compute and storage... where is MCP server sits? What is Users interface ? I presume if you expose endpoints (backend like a bit confused do you use node.js django ? -- in other words are all components in azure ? 100 lock on AZ cloud ? --- is it internal tool set up ... thanks !

Like
Reply
Abbas Ali Aloc

Solution Architect | IT Leader | People-Process-Technology Organizer for Success in Critical Architectures | TOGAF, PMP and AZURE Certified

2w

Great point about the 5% AI / 95% engineering split, Alex! So true that robust architecture is the unsung hero of effective AI agent deployments.

Like
Reply
Zabih Buda

Machine Learning Engineer & Data Scientist | Freelancer | Generative AI • NLP • RAG • Predictive Analytics | Python • SQL | Azure • AWS

3w

Great post, Alex Wang, I really like how you highlight that building AI agents is mostly software engineering, not just model tuning. The focus on things like access control, data governance, and human-in-the-loop workflows is so important and often overlooked. In my experience, having these foundations early makes scaling and maintaining agents much easier.

Like
Reply
Sandipan Bhaumik 🌱

Building a Community of AI Agent Builders | Data & AI Tech Leader | Speaker | Insights via talks & podcasts 🎙️

1mo

Exactly Alex Wang The engineering behind the scenes is what actually makes AI agents work in real-world systems

Vijayan (VJ) Seenisamy

Enterprise AI Role Strategist | Org & Capability Shift Lead @ Woolworths Group | Architecting Agent-Ready Teams | Amazon #1 Author

1mo

100% Alex Wang. Prompts build prototypes. Infrastructure builds trust. Without governance and observability, agents stay experiments.

See more comments

To view or add a comment, sign in

Explore content categories