{"id":1823,"date":"2023-06-20T15:15:23","date_gmt":"2023-06-20T09:45:23","guid":{"rendered":"https:\/\/www.analyticsvidhya.com\/datahack-summit-2023\/?page_id=1823"},"modified":"2023-07-19T19:07:36","modified_gmt":"2023-07-19T13:37:36","slug":"how-to-leverage-llms-to-build-a-scalable-search-engine","status":"publish","type":"page","link":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/session\/how-to-leverage-llms-to-build-a-scalable-search-engine\/","title":{"rendered":"How to Leverage LLMs to build a Scalable Search Engine"},"content":{"rendered":"

“In this session, we will explore the fascinating world of leveraging Language Models (LLMs) to build a scalable search engine that delivers high-quality results while maintaining production readiness and controlling latency. We will dive into the challenges associated with deploying LLMs in real-world production environments and examine the tradeoffs involved.<\/p>\n

The evolution of search over the years has revolutionized the way we access information, making it an indispensable tool in our daily lives. Businesses across various industries have harnessed the power of search to create innovative solutions and enhance user experiences. Understanding the business cases powered by search is crucial in order to leverage LLMs effectively and efficiently.<\/p>\n

We will delve into the process of leveraging LLMs to build a search engine tailored to your specific needs. This includes understanding how to retrain the model for a custom use case, ensuring that it delivers accurate and relevant results. Additionally, we will explore the engineering stack required to support a low-latency, real-time search engine powered by LLMs.<\/p>\n

Key Takeaways:<\/strong><\/p>\n

    \n
  1. Gain insights into the evolution and importance of search in today’s digital landscape.<\/li>\n
  2. Understand how LLMs can be effectively utilized to build a scalable search engine.<\/li>\n
  3. Learn the techniques for retraining LLMs to meet custom use cases.<\/li>\n
  4. Explore the engineering stack necessary for supporting a low-latency, real-time search engine powered by<\/li>\n
  5. LLMs.<\/li>\n
  6. Recognize the challenges and tradeoffs involved in deploying LLMs in production environments.”<\/li>\n<\/ol>\n","protected":false},"excerpt":{"rendered":"

    “In this session, we will explore the fascinating world of leveraging Language Models (LLMs) to build a scalable search engine that delivers high-quality results while maintaining production readiness and controlling latency. We will dive into the challenges associated with deploying LLMs in real-world production environments and examine the tradeoffs involved. The evolution of search over […]<\/p>\n","protected":false},"author":1,"featured_media":1824,"parent":1126,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"session-details.php","meta":[],"acf":[],"yoast_head":"\nHow to Leverage LLMs to build a Scalable Search Engine - DataHack Summit 2023<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.analyticsvidhya.com\/dhs-2023\/session\/how-to-leverage-llms-to-build-a-scalable-search-engine\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"How to Leverage LLMs to build a Scalable Search Engine - DataHack Summit 2023\" \/>\n<meta property=\"og:description\" content=\"“In this session, we will explore the fascinating world of leveraging Language Models (LLMs) to build a scalable search engine that delivers high-quality results while maintaining production readiness and controlling latency. We will dive into the challenges associated with deploying LLMs in real-world production environments and examine the tradeoffs involved. The evolution of search over […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.analyticsvidhya.com\/dhs-2023\/session\/how-to-leverage-llms-to-build-a-scalable-search-engine\/\" \/>\n<meta property=\"og:site_name\" content=\"DataHack Summit 2023\" \/>\n<meta property=\"article:modified_time\" content=\"2023-07-19T13:37:36+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.analyticsvidhya.com\/dhs-2023\/wp-content\/uploads\/2023\/06\/s-scalable-search.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"500\" \/>\n\t<meta property=\"og:image:height\" content=\"250\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.analyticsvidhya.com\/dhs-2023\/session\/how-to-leverage-llms-to-build-a-scalable-search-engine\/\",\"url\":\"https:\/\/www.analyticsvidhya.com\/dhs-2023\/session\/how-to-leverage-llms-to-build-a-scalable-search-engine\/\",\"name\":\"How to Leverage LLMs to build a Scalable Search Engine - DataHack Summit 2023\",\"isPartOf\":{\"@id\":\"https:\/\/www.analyticsvidhya.com\/dhs-2023\/#website\"},\"datePublished\":\"2023-06-20T09:45:23+00:00\",\"dateModified\":\"2023-07-19T13:37:36+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/www.analyticsvidhya.com\/dhs-2023\/session\/how-to-leverage-llms-to-build-a-scalable-search-engine\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.analyticsvidhya.com\/dhs-2023\/session\/how-to-leverage-llms-to-build-a-scalable-search-engine\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.analyticsvidhya.com\/dhs-2023\/session\/how-to-leverage-llms-to-build-a-scalable-search-engine\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.analyticsvidhya.com\/dhs-2023\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Session\",\"item\":\"https:\/\/www.analyticsvidhya.com\/dhs-2023\/session\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"How to Leverage LLMs to build a Scalable Search Engine\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.analyticsvidhya.com\/dhs-2023\/#website\",\"url\":\"https:\/\/www.analyticsvidhya.com\/dhs-2023\/\",\"name\":\"DataHack Summit 2023\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.analyticsvidhya.com\/dhs-2023\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"How to Leverage LLMs to build a Scalable Search Engine - DataHack Summit 2023","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/session\/how-to-leverage-llms-to-build-a-scalable-search-engine\/","og_locale":"en_US","og_type":"article","og_title":"How to Leverage LLMs to build a Scalable Search Engine - DataHack Summit 2023","og_description":"“In this session, we will explore the fascinating world of leveraging Language Models (LLMs) to build a scalable search engine that delivers high-quality results while maintaining production readiness and controlling latency. We will dive into the challenges associated with deploying LLMs in real-world production environments and examine the tradeoffs involved. The evolution of search over […]","og_url":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/session\/how-to-leverage-llms-to-build-a-scalable-search-engine\/","og_site_name":"DataHack Summit 2023","article_modified_time":"2023-07-19T13:37:36+00:00","og_image":[{"width":500,"height":250,"url":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/wp-content\/uploads\/2023\/06\/s-scalable-search.jpg","type":"image\/jpeg"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/session\/how-to-leverage-llms-to-build-a-scalable-search-engine\/","url":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/session\/how-to-leverage-llms-to-build-a-scalable-search-engine\/","name":"How to Leverage LLMs to build a Scalable Search Engine - DataHack Summit 2023","isPartOf":{"@id":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/#website"},"datePublished":"2023-06-20T09:45:23+00:00","dateModified":"2023-07-19T13:37:36+00:00","breadcrumb":{"@id":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/session\/how-to-leverage-llms-to-build-a-scalable-search-engine\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.analyticsvidhya.com\/dhs-2023\/session\/how-to-leverage-llms-to-build-a-scalable-search-engine\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/session\/how-to-leverage-llms-to-build-a-scalable-search-engine\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/"},{"@type":"ListItem","position":2,"name":"Session","item":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/session\/"},{"@type":"ListItem","position":3,"name":"How to Leverage LLMs to build a Scalable Search Engine"}]},{"@type":"WebSite","@id":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/#website","url":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/","name":"DataHack Summit 2023","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/wp-json\/wp\/v2\/pages\/1823"}],"collection":[{"href":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/wp-json\/wp\/v2\/comments?post=1823"}],"version-history":[{"count":4,"href":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/wp-json\/wp\/v2\/pages\/1823\/revisions"}],"predecessor-version":[{"id":2105,"href":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/wp-json\/wp\/v2\/pages\/1823\/revisions\/2105"}],"up":[{"embeddable":true,"href":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/wp-json\/wp\/v2\/pages\/1126"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/wp-json\/wp\/v2\/media\/1824"}],"wp:attachment":[{"href":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/wp-json\/wp\/v2\/media?parent=1823"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}