Natural Language Processing using Generative Models

Calendar 5th August, 2023 clock 9:30 am - 5:30 pm location RENAISSANCE :- Race Course Rd, Madhava Nagar Extension

Natural Language Processing (NLP) is a fascinating field of artificial intelligence that aims to bridge the gap between humans and machines by enabling computers to understand, interpret and generate human-like language. In recent years, the development of generative models has revolutionized the way we approach NLP tasks because of its innovative and generative capabilities.

  • Module 1: Introduction to Generative AI

  • Introduction to Generative AI
      • NLP (GPT)
      • Code Generation (Co-Pilot)
      • Computer Vision (Mid-Journey)
      • Audio (OpenAI Whisper)
      • Video (Make-a-video)
      • Multimodal (GPT4, MUM)
    • The NLP Journey (TF-IDF to Sequence Modeling to Transformers)
  • Module 2: LM Fundamentals

    • Embeddings (word2Vec, GloVe, etc)
    • The OG Language Models (LSTMs, BERT, XLNET, Distill-BERT,GPT)
    • LM Fundamentals (Pretraining, Discriminative Fine Tuning)
      • Attention, Positional Embeddings
      • Foundation Models
      • Transfer Learning
  • Module 3: Diving into ChatGPT

    • The Awakening of GPT-3 and LLMs
    • Understanding ChatGPT 
      • Instruction Tuning (InstructGPT, ControlNet)
      • SFT
      • RLHF
    • Prompt Engineering
    • Evaluation of LLMs/Benchmarks
  • Module 4: Mastering the Pretraining and Fine Tuning of LLMs

    • Setting up an LLM on local 
      • HuggingFace
      • openLLama
      • Falcon
      • GPT4All/ PrivateGPT
      • RedPajama
    • Best Practices to train LLMs
  • Module 5: Master LLMs Part-2

    • Challenges associated with LLMs and their pre-training
    • Fine Tuning LLMs (PEFT techniques: Prefix-tuning, LoRa, QLoRa etc)
    • Prompt Tuning/Optimization
    • AI Tooling (AI Agents, LangChain, VectorDB etc)
  • Module 6: Next Frontier

    • New AI Tools (chatGPT Plugins, BARD, PALM, Co-Pilot)
    • What Next? (AutoGPT, GPT-4 and beyond)

Pre-requisites:

  • System Requirement and Setup
    • Laptop with at least 4-8 GB of RAM
    • We will be using a cloud jupyter notebook powered by GPU for the workshop
  • Offline Setup [Optional]
    • GPU good to have!
    • Install Python3.9 or higher version(Resource)
    • Install jupyter notebook (Resource)
  • Pre-reads

Note: These are tentative details and are subject to change.

Download Full Agenda