DataHour: DIY Semantic Code Search

Imagine a new contributor is trying to solve a simple beginner task but is overwhelmed by the complexity of the repository. Knowing which file to change and where to make the change can be time-consuming.

Popular code search tools are based on keyword matches and are not based on natural language. Existing Code search tools do not semantically understand the codebase. This talk will explore building a DIY Semantic Code Search tool to simplify the process of navigating complex codebases, which is particularly beneficial for new contributors onboard to open-source GitHub projects.

We will demonstrate how to build a locally deployable Natural Language-based Code Search tool using open-source LLMs that can help beginners understand any repository and start contributing. Leveraging the RAG paradigm (Retrieval Augmented Generation), we show a two-step process of retrieving relevant files for the task using Vector Search and an open-source LLM to generate answers to the natural language questions to accomplish the task at hand.

The solution would be a DIY solution that anyone can build, test, and deploy locally for their local repositories using open-source technologies only.

Raghavan Muthuregunathan is a distinguished Senior Engineering Manager renowned for his leadership in the domain of artificial intelligence, particularly within the realm of search technologies. Currently stationed in San Francisco, California, he serves as the driving force behind the LinkedIn Search AI team, steering its endeavors towards innovation and excellence.

Raghavan Muthuregunathan is a distinguished Senior Engineering Manager renowned for his leadership in the domain of artificial intelligence, particularly within the realm of search technologies. Currently stationed in San Francisco, California, he serves as the driving force behind the LinkedIn Search AI team, steering its endeavors towards innovation and excellence.

Prior to his tenure at LinkedIn, Raghavan honed his expertise at Microsoft Bing, where he played a pivotal role in shaping the search landscape. With a rich background in engineering and a passion for leveraging technology to solve complex challenges, he has left an indelible mark on the industry.