Dipanjan Sarkar

Dipanjan Sarkar

Head of Artificial Intelligence & Community

About

Dipanjan Sarkar is currently the Head of Artificial Intelligence & Community, Analytics Vidhya. He is also a published Author, and Consultant, boasting over 13 years of extensive expertise in Machine Learning, Deep Learning, Generative AI, Agentic AI, Computer Vision, and Natural Language Processing. His leadership spans Fortune 100 enterprises to startups, crafting end-to-end AI products and pioneering AI upskilling programs. A seasoned advisor, Dipanjan advises a diverse clientele, from Engineers and Architects to C-suite executives and PhDs, across Advanced Analytics, AI Strategy & Development. Recognitions include 'Top 10 Data Scientists in India, 2020', '40 under 40 Data Scientists, 2021', 'Google Developer Expert - Machine Learning, 2019', 'Top 50 AI Thought Leaders, 2022', 'Google Champion Innovator Cloud AIML, 2022', 'Top 10 AI Leaders in India, 2024', '40 Under 40, 2025', alongside global accolades including Top 100 Influential AI Voices in LinkedIn.

Everyone is building and shipping GenAI and Agentic features, but how do you prove they are truly reliable in the real world?

GenAI and Agentic AI systems can retrieve knowledge, plan tasks, call tools, reflect, and even collaborate, but knowing whether they are actually correct is surprisingly hard, and the popular shortcuts quietly mislead you: 

  • Off-the-shelf libraries and metrics that return confident but wrong scores 
  • LLM-as-judge that behaves like a biased black box 
  • Agents that reach the right answer through the wrong process 
  • No clear way to know whether a judge can even be trusted 

This session draws from my hands-on experience building & evaluating RAG and Agentic AI systems over the past few years. We'll focus on three pillars:  

  • Evaluating GenAI/RAG systems 
  • Evaluating Agentic AI systems  
  • Building judges you can actually trust 

Overall there will be one constant running theme: surface metrics lie, so you must evaluate what is happening under the hood, not just the final text. 

Agenda 

  1. Introduction
  • Why evaluation is the hardest (and most skipped) part of shipping GenAI 
  • The core trap: surface metrics lie - "relevant ≠ correct" and "right answer ≠ right process" 
  • An evaluation taxonomy - deterministic vs LLM-as-judge, component vs trajectory 
  1. Evaluating GenAI & RAG Systems
  • Deterministic retrieval metrics — Precision@K / Recall@K and where they shine 
  • Off-the-shelf LLM-judge libraries — RAGAS, DeepEval etc. why you should not trust them blindly 
  • Building your own judges - deterministic, chain-of-thought (CoT) and hybrid (deterministic + LLM-judge) decision-graph (DAG) approaches 
  1. Evaluating Agentic AI Systems
  • Capturing the full agent trajectory, tool calls (not just final responses) in LangGraph 
  • The four evaluation layers - final decision, tool calls, trajectory, and reasoning 
  • Deterministic-first - why most agent checks are a simple ==, not a fancy judge 
  • Reasoning quality - where an LLM judge (CoT / DAG) genuinely earns its place 
  • Newer methods worth knowing 
  1. Bonus: Coding Agents for Evals (if timepermits)
  • Eval-driven development - core methodology and workflow 
  • Packaging evals as reusable skills - an "eval-runner" you and your team can invoke on demand 
  • Key limitations and future scope  

Throughout the Session 

  • Best practices and caveats for real-world readiness 
  • Hands-on code demos on real examples 
Read More →

New to the world of Agentic AI and want to quickly get proficient in the key aspects of learning, building, deploying, evaluating and monitoring Agentic AI Systems? This is the workshop for you!

In this workshop you will get a comprehensive coverage of the breadth as well as a deep dive into the depth of the vast world of Agentic AI Systems.

Over the course of five core modules plus bonus content, you will spend the entire day focusing on the following key areas:

• Learn essential concepts of Generative AI, Agentic AI, Multimodal Agents and Agentic RAG Systems

• Deep dive into industry-standard design patterns for architecting Agentic AI Systems - Tool-Use, Reflection, Planning, Multi-Agent

• Leverage industry-standard frameworks including LangChain and LangGraph to build simple and advanced Agentic AI & Agentic RAG Systems

• Learn essentials of how to deploy Agentic AI Systems as APIs as well as evaluate and monitor them

While we want to keep the discussions as framework- and tool-agnostic as possible, since 90% of the workshop will be hands-on focused, we will be using LangChain and LangGraph (currently the leading framework used in the industry) for most of the hands-on demos for building Agents. While the focus of the workshop is more on building Agentic AI Systems, we will also showcase how you can build a basic web service or API on top of an Agent using FastAPI and deploy with a basic frontend UI, monitor and evaluate it using frameworks like LangSmith and building our own custom evaluation frameworks.

Overall, expect to learn from 10+ end-to-end hands-on demos which you can take home with you after the workshop! Every demo, including the bonus content, is a complete self-contained notebook - so anything we do not get to demo live is still yours to work through afterwards.

Additional Points 

  • Prerequisites: Solid understanding of Python. You need to know how to code. Knowledge of other areas like NLP and Generative AI will be useful.
  • Content Provided: Slides, complete code notebooks, datasets.
  • Infrastructure: All the hands-on demos run on Google Colab - including the deployment and monitoring section, where we deploy the agent as an API on the same Google Cloud machine that Colab is running on. You only need a Google account and a browser; nothing to install locally. (Make sure your laptop can access Google Colab and Google Docs)
  • Accounts you need to create beforehand: Alongside Colab, a few of the demos use external services. All of them have a free tier and none of them need billing or a credit card: 
    • Google AI Studio - Gemini API Key to read PDFs and images
    • LangSmith - for tracing, monitoring agent runs
    • Tavily - for web search in the long-term memory bonus demo

Important Note: We will provide you with an OpenAI API key during the workshop, so you do not need your own-if you have your own key, you are welcome to use it. The three accounts above are the ones you should sign up for in advance, and we will send step-by-step instructions ahead of time. Getting these ready before the session is essential for running the hands-on demos live along with the instructor.

Read More →
View All Speakers

We use cookies essential for this site to function well. Please click to help us improve its usefulness with additional cookies. Learn about our use of cookies in our Privacy Policy & Cookies Policy.

Show details