Ashish Tripathy

This session offers a comprehensive guide to building a scalable Voice AI Contact Center using PipeCat. Pipecat is an open-source Python framework for building real-time voice and multimodal conversational agents. You'll learn how to design, implement, and deploy a voice-powered system capable of handling patient appointment scheduling, answering common medical queries, and intelligently escalating complex issues to a supervisor (either a secondary voice-agent or a human). The session will begin with an introduction to PipeCat and Voice AI Fundamentals, explaining how PipeCat orchestrates speech-to-speech pipelines by layering LLM-driven logic on top of telephony transports like Twilio or WebRTC. We will demonstrate how PipeCat handles latency, interruption management, and context tracking effectively.

The workshop will then delve into building a healthcare booking and support workflow, showing how to capture patient speech, transcribe it, invoke LLM function calls to backend appointment-booking APIs, and synthesize audio replies. You'll also learn how to embed domain-specific knowledge (e.g., clinic hours, insurance policies) into prompt templates for efficient FAQ answering. We will then cover designing multiple voice personalities and supervision logic, including configuring distinct TTS voices (e.g., a friendly "Receptionist" and a formal "Supervisor" voice for escalations). You'll discover how PipeCat simplifies switching personalities based on sentiment or intent detection, and how to route calls to a live human agent when needed. Finally, we will discuss scaling and deploying your contact center, outlining best practices for horizontal scaling through containerizing PipeCat workers, configuring autoscaling groups, monitoring per-minute costs of STT/TTS/LLM calls, and implementing caching or context summarization to reduce expensive long-session inference, ensuring sub-second voice-to-voice latency even under heavy load.

The practical applications covered will include automated appointment booking, where we'll build a PipeCat handler that transcribes patient speech, extracts entities via an LLM function call, and interacts with a REST API to check slot availability, confirming appointments with synthesized TTS responses. We will also demonstrate how to answer insurance and billing queries by embedding a small knowledge base of insurance coverage rules and matching queries to preloaded FAQ text or LLM prompts to synthesize confident audio replies based on clinic policy tables. Furthermore, we will configure dynamic personality switching and escalation, setting up two TTS voices ("Front Desk" and "Supervisor") and illustrating how PipeCat triggers a personality switch based on sentiment analysis, flagging emergencies or complex issues and either engaging another PipeCat instance or bridging the call to a live human operator.