Kuldeep Singh Sidhu

Want to ace your next AI/ML interview? Most engineers know about embeddings, but not the full vocabulary of vector databases that power RAG and search systems. This guide breaks it down clearly: ▪️Vector embeddings & similarity search ▪️ANN vs. exact search (speed vs. accuracy) ▪️Indexing methods: Flat, HNSW, IVF, PQ, IVFPQ ▪️Key performance metrics: recall, latency, throughput, QPS ▪️Scaling concepts: filtering, sharding, reindexing ▪️Popular DBs: FAISS, Pinecone, Milvus, Weaviate, Qdrant, Chroma 👉If you’re serious about RAG, search, or AI infra, you need to know these terms. ➕ Follow Naresh Edagotti for more content that makes complex AI topics feel simple.

153

36 Comments

𝗠𝗲𝘁𝗮 𝗷𝘂𝘀𝘁 𝗰𝗿𝘂𝘀𝗵𝗲𝗱 𝗲𝗮𝗿𝗻𝗶𝗻𝗴𝘀—𝗮𝗴𝗮𝗶𝗻. But here’s the twist: 🐳 𝗗𝗲𝗲𝗽𝗦𝗲𝗲𝗸 might help Meta cut costs. DeepSeek just built a cutting-edge AI model that runs inference at a fraction of the usual cost—and if history is any guide, this could be good news for Meta. Zuck has a track record of scaling competitors’ best ideas—from Snapchat’s 𝗦𝘁𝗼𝗿𝗶𝗲𝘀 to TikTok’s 𝗥𝗲𝗲𝗹𝘀. Could DeepSeek be his next blueprint? Meanwhile, Meta’s stock is on fire, up nearly +20% in the past month as the company doubles down on AI infrastructure with $𝟲𝟬 𝘁𝗼 $𝟲𝟱 𝗯𝗶𝗹𝗹𝗶𝗼𝗻 𝗶𝗻 𝗖𝗮𝗽𝗲𝘅 planned for FY25. 📊 𝗟𝗲𝘁'𝘀 𝗯𝗿𝗲𝗮𝗸 𝗱𝗼𝘄𝗻 𝘁𝗵𝗲 𝗸𝗲𝘆 𝘁𝗮𝗸𝗲𝗮𝘄𝗮𝘆𝘀. Read it here 👇 https://lnkd.in/gtFuH5sN 💡 Where do you stand? Drop your thoughts: • Is Meta’s AI spending justified? • Will margins expand even more through new AI breakthroughs?

93

2 Comments

Function Calling vs. MCP! (clearly explained with visuals and code) Before we talk about Model Context Protocol (MCP), here's how typical tool calling with LLMs work: 1. Prompt goes to the LLM. 2. LLM decides which tool to use. 3. Custom glue code builds the function call. 4. Backend runs the tool. Everything is hand-wired, which is great for demos but painful at scale. Enter MCP 🚀 ↳ A lightweight protocol that standardizes how tools are described, discovered, and executed. ↳ No glue code: Drop in an MCP server; the client + LLM handle the rest. ↳ Self-describing: Schemas are auto-published, so agents know the inputs/outputs. ↳ Permissioned: Tools can require explicit approval before they run. ↳ Decoupled: Implementation lives on the server; consumers just send a spec. Result: you focus on building tools once, not re-integrating them for every agent. How They Fit Together 🤝 Function Calling lets the LLM say what it wants, but you still wire up and run the code. MCP picks up from that intent: it finds the right tool, feeds it the inputs, runs it on the server, and returns the result. In one line—Function Calling captures intent; MCP handles everything else. For example, an agent might say, “I need to search the web,” using function calling. That request can be routed through MCP to select from available web search tools, invoke the correct one, and return the result. If you want to understand this better with code, I have linked my article in the comments. 👉 Interested in ML/AI Engineering? Sign up for our newsletter for in-depth lessons and get a FREE eBook with 150+ core DS/ML lessons: (link in the comments) _____ Share this with your network if you found this insightful ♻️ Follow me (Akshay Pachaar ) for more insights and tutorials on AI and Machine Learning!

1,076

23 Comments

Mythology meets machine learning: India's first case study in AI-powered storytelling at scale. There's been legitimate skepticism about AI in entertainment—can technology truly capture the emotional weight of something as sacred as the Mahabharat? What Collective Artist Network & JioHotstar have executed with Mahabharat: Ek Dharmayudh offers a compelling answer. This is production infrastructure, not just production assistance: ✅ 100 episodes with cinema-grade VFX ✅AI-driven character modeling for expressive consistency across the series ✅Workflows that compress multi-year timelines into months ✅Visual ambition that would've been cost-prohibitive a decade ago Today's AI pipelines enable higher visual standards (as compared to the ones we have grown up with) within scalable budgets—turning what was once a financial barrier into competitive advantage. This marks India's evolution from AI as a tool → AI as core infrastructure for entertainment. 💡The real question: If AI can industrialize the Mahabharat while maintaining cultural depth, what's the next frontier? Original Indian epics designed for global audiences? I would suggest to embrace rather than fighting it, change can be scary but it the only constant #AIInnovation #CulturalTech #ThoughtLeadership #StreamingStrategy #Mahabharat #AI

95

20 Comments

What things in Gen AI a Data Engineer should know! LLM - Large Language Model (act's like a brain) This LLM has 2 main stages - Pre training - Fine tuning although there are more.. In Pre training Big Companies will take a big chunk of data (100's of TB) available on internet, blogs, wikipedia etc and get a gist of it in a few GB's. Lets consider 70 Billion parameter model, each parameter here will be a float of 2 bytes and will take 140 GB. This will take 10's of days and 1000's of heavy machines, and 10's or 100's of million dollars to do this and we get a base/foundation model. This base model cannot answer question, rather if you give it a question it might give more questions. Now, we can leverage the foundation model and develop something. As a Data Engineer, => Prompt Engineering is the easiest thing to do, but when creating applications you need to write it in a programatic way. writing better prompts with more context can help us get better results. => Retrieval Augmented Generation (RAG) - As a Data Engineer you can index your company documents, here you would require to clean the data and index it in a vector database. Later when a prompt is given, it will try to do a simarity search and will search in your vector database. whatever results come will be augmented to your prompt and it's like more context is passed as part of the prompt for better results. So, setting up a vector DB, cleaning documents, and storing in vector DB can be directly in a Data Engineers portfolio. => Model Fine Tuning - This is where the world is moving towards. Instead of striving for bigger and bigger base model, it has been seen that a decent size base model with a good fine tuning done for specific tasks can perform better. So, we will take a foundation mode, and we might have 10's or 100's of thousands of quality questions and answers (high quality training data) on which we fine tune the model. There can be different fine tuned models specific to each task. Do not try using a single LLM for everything. A better way is to use specialised LLM's fine tuned differently to do different stuff's. A model might be good at summarisation, other might be good at judging which summaries are better. Also the LLM's can be given access to various tools like google search, libraries to plot a graph, calculator etc in order to achieve a task. => Now when you develop complex application, it might happen you use a LLM for summarisation, and the output of this LLM goes to other LLM for judging. So there can be a big pipeline, this can be developed using langchain. So I feel - A basic understanding of terminologies around Pre training - Vector Database - RAG (Augmenting the prompt) - Model Fine tuning - Lang Chain - Agentic AI are the things which at the minimum a Data Engineer should know. But there are many more things within this, which I will cover in extreme depth in my "AI for Data Engineers Program" I hope you learnt something new!

50

3 Comments