{"id":1094,"date":"2023-04-27T13:16:22","date_gmt":"2023-04-27T13:16:22","guid":{"rendered":"https:\/\/www.analyticsvidhya.com\/datahack-summit-2023\/?page_id=1094"},"modified":"2023-08-04T10:42:54","modified_gmt":"2023-08-04T05:12:54","slug":"natural-language-processing-using-generative-models","status":"publish","type":"page","link":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/workshop\/natural-language-processing-using-generative-models\/","title":{"rendered":"Natural Language Processing using Generative Models"},"content":{"rendered":"<p><span style=\"font-weight: 400;\">Natural Language Processing (NLP) is a fascinating field of artificial intelligence that aims to bridge the gap between humans and machines by enabling computers to understand, interpret and generate human-like language. In recent years, the development of generative models has revolutionized the way we approach NLP tasks because of its innovative and generative capabilities.<\/span><\/p>\n<ul>\n<li aria-level=\"1\">\n<h4><span style=\"font-weight: 400;\">Module 1: Introduction to Generative AI<\/span><\/h4>\n<\/li>\n<\/ul>\n<ul>\n<li aria-level=\"2\"><span style=\"font-weight: 400;\">Introduction to Generative AI<\/span><\/li>\n<\/ul>\n<ul>\n<li style=\"list-style-type: none;\">\n<ul>\n<li style=\"list-style-type: none;\">\n<ul>\n<li style=\"font-weight: 400;\" aria-level=\"3\"><span style=\"font-weight: 400;\">NLP (GPT)<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"3\"><span style=\"font-weight: 400;\">Code Generation (Co-Pilot)<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"3\"><span style=\"font-weight: 400;\">Computer Vision (Mid-Journey)<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"3\"><span style=\"font-weight: 400;\">Audio (OpenAI Whisper)<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"3\"><span style=\"font-weight: 400;\">Video (Make-a-video)<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"3\"><span style=\"font-weight: 400;\">Multimodal (GPT4, MUM)<\/span><\/li>\n<\/ul>\n<\/li>\n<li style=\"font-weight: 400;\" aria-level=\"2\"><span style=\"font-weight: 400;\">The NLP Journey (TF-IDF to Sequence Modeling to Transformers)<\/span><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<ul>\n<li style=\"font-weight: 400;\" aria-level=\"1\">\n<h4><span style=\"font-weight: 400;\">Module 2: LM Fundamentals<\/span><\/h4>\n<ul>\n<li style=\"font-weight: 400;\" aria-level=\"2\"><span style=\"font-weight: 400;\">Embeddings (word2Vec, GloVe, etc)<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"2\"><span style=\"font-weight: 400;\">The OG Language Models (LSTMs, BERT, XLNET, Distill-BERT,GPT)<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"2\"><span style=\"font-weight: 400;\">LM Fundamentals (Pretraining, Discriminative Fine Tuning)<\/span>\n<ul>\n<li style=\"font-weight: 400;\" aria-level=\"3\"><span style=\"font-weight: 400;\">Attention, Positional Embeddings<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"3\"><span style=\"font-weight: 400;\">Foundation Models<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"3\"><span style=\"font-weight: 400;\">Transfer Learning<\/span><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<ul>\n<li style=\"font-weight: 400;\" aria-level=\"1\">\n<h4><span style=\"font-weight: 400;\">Module 3: Diving into ChatGPT<\/span><\/h4>\n<ul>\n<li style=\"font-weight: 400;\" aria-level=\"2\"><span style=\"font-weight: 400;\">The Awakening of GPT-3 and LLMs<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"2\"><span style=\"font-weight: 400;\">Understanding ChatGPT\u00a0<\/span>\n<ul>\n<li style=\"font-weight: 400;\" aria-level=\"3\"><span style=\"font-weight: 400;\">Instruction Tuning (InstructGPT, ControlNet)<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"3\"><span style=\"font-weight: 400;\">SFT<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"3\"><span style=\"font-weight: 400;\">RLHF<\/span><\/li>\n<\/ul>\n<\/li>\n<li style=\"font-weight: 400;\" aria-level=\"2\"><span style=\"font-weight: 400;\">Prompt Engineering<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"2\"><span style=\"font-weight: 400;\">Evaluation of LLMs\/Benchmarks<\/span><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<ul>\n<li style=\"font-weight: 400;\" aria-level=\"1\">\n<h4><span style=\"font-weight: 400;\">Module 4: <\/span><span style=\"font-weight: 400;\">Mastering the Pretraining and Fine Tuning of LLMs<\/span><\/h4>\n<ul>\n<li style=\"font-weight: 400;\" aria-level=\"2\"><span style=\"font-weight: 400;\">Setting up an LLM on local\u00a0<\/span>\n<ul>\n<li style=\"font-weight: 400;\" aria-level=\"3\"><span style=\"font-weight: 400;\">HuggingFace<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"3\"><span style=\"font-weight: 400;\">openLLama<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"3\"><span style=\"font-weight: 400;\">Falcon<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"3\"><span style=\"font-weight: 400;\">GPT4All\/ PrivateGPT<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"3\"><span style=\"font-weight: 400;\">RedPajama<\/span><\/li>\n<\/ul>\n<\/li>\n<li style=\"font-weight: 400;\" aria-level=\"2\"><span style=\"font-weight: 400;\">Best Practices to train LLMs<\/span><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<ul>\n<li style=\"font-weight: 400;\" aria-level=\"1\">\n<h4><span style=\"font-weight: 400;\">Module 5: Master LLMs Part-2<\/span><\/h4>\n<ul>\n<li style=\"font-weight: 400;\" aria-level=\"2\"><span style=\"font-weight: 400;\">Challenges associated with LLMs and their pre-training<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"2\"><span style=\"font-weight: 400;\">Fine Tuning LLMs (PEFT techniques: Prefix-tuning, LoRa, QLoRa etc)<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"2\"><span style=\"font-weight: 400;\">Prompt Tuning\/Optimization<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"2\"><span style=\"font-weight: 400;\">AI Tooling (AI Agents, LangChain, VectorDB etc)<\/span><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<ul>\n<li style=\"font-weight: 400;\" aria-level=\"1\">\n<h4><span style=\"font-weight: 400;\">Module 6: Next Frontier<\/span><\/h4>\n<ul>\n<li style=\"font-weight: 400;\" aria-level=\"2\"><span style=\"font-weight: 400;\">New AI Tools (chatGPT Plugins, BARD, PALM, Co-Pilot)<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"2\"><span style=\"font-weight: 400;\">What Next? (AutoGPT, GPT-4 and beyond)<\/span><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p><strong>Pre-requisites:<\/strong><\/p>\n<ul>\n<li>System Requirement and Setup\n<ul>\n<li>Laptop with at least 4-8 GB of RAM<\/li>\n<li>We will be using a cloud jupyter notebook powered by GPU for the workshop<\/li>\n<\/ul>\n<\/li>\n<li>Offline Setup [Optional]\n<ul>\n<li>GPU good to have!<\/li>\n<li>Install Python3.9 or higher version(<a href=\"https:\/\/www.python.org\/downloads\/\" target=\"_blank\" rel=\"noopener\">Resource<\/a>)<\/li>\n<li>Install jupyter notebook (<a href=\"https:\/\/jupyter.org\/install\" target=\"_blank\" rel=\"noopener\">Resource<\/a>)<\/li>\n<\/ul>\n<\/li>\n<li>Pre-reads\n<ul>\n<li>Programming knowledge in Python (<a href=\"https:\/\/www.analyticsvidhya.com\/blog\/2016\/01\/complete-tutorial-learn-data-science-python-scratch-2\/\" target=\"_blank\" rel=\"noopener\">Resource<\/a>)<\/li>\n<li>Jupyter Notebook Environment familiarity (<a href=\"https:\/\/www.analyticsvidhya.com\/blog\/2018\/05\/starters-guide-jupyter-notebook\/\" target=\"_blank\" rel=\"noopener\">Resource<\/a>)<\/li>\n<li>Basics of Machine learning and Deep Learning (<a href=\"https:\/\/www.analyticsvidhya.com\/blog\/2017\/09\/common-machine-learning-algorithms\/\" target=\"_blank\" rel=\"noopener\">Resource<\/a>,<a href=\"https:\/\/www.analyticsvidhya.com\/blog\/2020\/07\/neural-networks-from-scratch-in-python-and-r\/\" target=\"_blank\" rel=\"noopener\">Resource<\/a>)<\/li>\n<li>Fundamentals of NLP(<a href=\"https:\/\/www.analyticsvidhya.com\/blog\/2017\/01\/ultimate-guide-to-understand-implement-natural-language-processing-codes-in-python\/\" target=\"_blank\" rel=\"noopener\">Resource1<\/a>,<a href=\"https:\/\/www.analyticsvidhya.com\/blog\/2017\/06\/word-embeddings-count-word2veec\/\" target=\"_blank\" rel=\"noopener\">Resource2<\/a>)<\/li>\n<li>Familiarity with Pytorch(<a href=\"https:\/\/www.analyticsvidhya.com\/blog\/2020\/07\/how-to-train-an-image-classification-model-in-pytorch-and-tensorflow\/\" target=\"_blank\" rel=\"noopener\">Resource<\/a>)<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>Note: These are tentative details and are subject to change.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Natural Language Processing (NLP) is a fascinating field of artificial intelligence that aims to bridge the gap between humans and machines by enabling computers to understand, interpret and generate human-like language. In recent years, the development of generative models has revolutionized the way we approach NLP tasks because of its innovative and generative capabilities. Module [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":1691,"parent":890,"menu_order":7,"comment_status":"closed","ping_status":"closed","template":"workshop-detail.php","meta":[],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v20.7 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Natural Language Processing using Generative Models - DataHack Summit 2023<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.analyticsvidhya.com\/dhs-2023\/workshop\/natural-language-processing-using-generative-models\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Natural Language Processing using Generative Models - DataHack Summit 2023\" \/>\n<meta property=\"og:description\" content=\"Natural Language Processing (NLP) is a fascinating field of artificial intelligence that aims to bridge the gap between humans and machines by enabling computers to understand, interpret and generate human-like language. In recent years, the development of generative models has revolutionized the way we approach NLP tasks because of its innovative and generative capabilities. Module [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.analyticsvidhya.com\/dhs-2023\/workshop\/natural-language-processing-using-generative-models\/\" \/>\n<meta property=\"og:site_name\" content=\"DataHack Summit 2023\" \/>\n<meta property=\"article:modified_time\" content=\"2023-08-04T05:12:54+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.analyticsvidhya.com\/dhs-2023\/wp-content\/uploads\/2023\/06\/nlp_soldout.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"500\" \/>\n\t<meta property=\"og:image:height\" content=\"250\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.analyticsvidhya.com\/dhs-2023\/workshop\/natural-language-processing-using-generative-models\/\",\"url\":\"https:\/\/www.analyticsvidhya.com\/dhs-2023\/workshop\/natural-language-processing-using-generative-models\/\",\"name\":\"Natural Language Processing using Generative Models - DataHack Summit 2023\",\"isPartOf\":{\"@id\":\"https:\/\/www.analyticsvidhya.com\/dhs-2023\/#website\"},\"datePublished\":\"2023-04-27T13:16:22+00:00\",\"dateModified\":\"2023-08-04T05:12:54+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/www.analyticsvidhya.com\/dhs-2023\/workshop\/natural-language-processing-using-generative-models\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.analyticsvidhya.com\/dhs-2023\/workshop\/natural-language-processing-using-generative-models\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.analyticsvidhya.com\/dhs-2023\/workshop\/natural-language-processing-using-generative-models\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.analyticsvidhya.com\/dhs-2023\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Workshop\",\"item\":\"https:\/\/www.analyticsvidhya.com\/dhs-2023\/workshop\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Natural Language Processing using Generative Models\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.analyticsvidhya.com\/dhs-2023\/#website\",\"url\":\"https:\/\/www.analyticsvidhya.com\/dhs-2023\/\",\"name\":\"DataHack Summit 2023\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.analyticsvidhya.com\/dhs-2023\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Natural Language Processing using Generative Models - DataHack Summit 2023","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/workshop\/natural-language-processing-using-generative-models\/","og_locale":"en_US","og_type":"article","og_title":"Natural Language Processing using Generative Models - DataHack Summit 2023","og_description":"Natural Language Processing (NLP) is a fascinating field of artificial intelligence that aims to bridge the gap between humans and machines by enabling computers to understand, interpret and generate human-like language. In recent years, the development of generative models has revolutionized the way we approach NLP tasks because of its innovative and generative capabilities. Module [&hellip;]","og_url":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/workshop\/natural-language-processing-using-generative-models\/","og_site_name":"DataHack Summit 2023","article_modified_time":"2023-08-04T05:12:54+00:00","og_image":[{"width":500,"height":250,"url":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/wp-content\/uploads\/2023\/06\/nlp_soldout.jpg","type":"image\/jpeg"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/workshop\/natural-language-processing-using-generative-models\/","url":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/workshop\/natural-language-processing-using-generative-models\/","name":"Natural Language Processing using Generative Models - DataHack Summit 2023","isPartOf":{"@id":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/#website"},"datePublished":"2023-04-27T13:16:22+00:00","dateModified":"2023-08-04T05:12:54+00:00","breadcrumb":{"@id":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/workshop\/natural-language-processing-using-generative-models\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.analyticsvidhya.com\/dhs-2023\/workshop\/natural-language-processing-using-generative-models\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/workshop\/natural-language-processing-using-generative-models\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/"},{"@type":"ListItem","position":2,"name":"Workshop","item":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/workshop\/"},{"@type":"ListItem","position":3,"name":"Natural Language Processing using Generative Models"}]},{"@type":"WebSite","@id":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/#website","url":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/","name":"DataHack Summit 2023","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/wp-json\/wp\/v2\/pages\/1094"}],"collection":[{"href":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/wp-json\/wp\/v2\/comments?post=1094"}],"version-history":[{"count":27,"href":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/wp-json\/wp\/v2\/pages\/1094\/revisions"}],"predecessor-version":[{"id":3319,"href":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/wp-json\/wp\/v2\/pages\/1094\/revisions\/3319"}],"up":[{"embeddable":true,"href":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/wp-json\/wp\/v2\/pages\/890"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/wp-json\/wp\/v2\/media\/1691"}],"wp:attachment":[{"href":"https:\/\/www.analyticsvidhya.com\/dhs-2023\/wp-json\/wp\/v2\/media?parent=1094"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}