Large Language Models (LLMs) are redefining NLP with their remarkable reasoning capabilities, but they still hallucinate, making up facts that can derail decision-critical tasks like clinical trial matching or medical entity extraction. In this session, we’ll explore how understanding and quantifying uncertainty can help tackle this reliability gap.

We’ll demystify uncertainty vs. confidence, break down aleatoric vs. epistemic uncertainty, and walk through estimation techniques for white-box (e.g., LLaMA), grey-box (e.g., GPT-3), and black-box (e.g., GPT-4) models. Expect hands-on demonstrations using open-source LLMs and tools, with a reality check on why SoftMax scores alone can be misleading.

We’ll also shine a spotlight on Small Language Models (SLMs) on why they’re not just cheaper, but potentially more predictable and controllable, offering a compelling alternative for hallucination-sensitive use cases.

Whether you're deploying LLMs in production or experimenting with SLMs, this talk will equip you with tools to make your models more trustworthy.