From Chunks to Connections: Graph RAG with Neo4j for Hierarchical Intelligence

Vector-based RAG has transformed how AI applications are built, but it often flattens data into chunks, losing the relationships, hierarchies, and dependencies that give information its true meaning. When working with structured data such as org charts, codebases, policy trees, or product catalogs , this limitation can significantly impact the quality of retrieval and reasoning.

In this session, we move beyond embeddings and explore how Graph RAG with Neo4j brings relationships into the retrieval process. You’ll learn how combining semantic vector search with graph traversal enables deeper insights and multi-hop reasoning. We’ll also demonstrate how LLMs can dynamically generate Cypher queries to navigate complex data structures and unlock intelligence that traditional chunk-based systems cannot achieve.

This is a practically focused and architecture-driven session designed to help practitioners build more intelligent, context-rich RAG systems.

Key Takeaways:

• Limitations of Vector RAG – understanding how chunk-based retrieval loses structure and context
• Introduction to Graph RAG – leveraging relationships and hierarchies for better retrieval
• Hybrid Retrieval Approach – combining vector search with graph traversal
• Dynamic Querying – using LLMs to generate Cypher queries for complex reasoning
• Multi-Hop Reasoning – enabling deeper insights across connected data
• System Design Perspective – building scalable, graph-powered GenAI pipelines

Farook is a Senior Software Engineering Lead at Optum, building enterprise-grade GenAI systems that transform organizational knowledge into intelligent, automated workflows.

With over a decade of experience in full-stack and cloud engineering, he specializes in designing scalable AI platforms, combining agentic workflows, knowledge graphs, and production-ready RAG systems to deliver impactful solutions at scale.