Natural Language Processing using Generative Models

Natural Language Processing (NLP) is a fascinating field of artificial intelligence that aims to bridge the gap between humans and machines by enabling computers to understand, interpret and generate human-like language. In recent years, the development of generative models has revolutionized the way we approach NLP tasks because of its innovative and generative capabilities.

• Module 1: Introduction to Generative AI

• Introduction to Generative AI

• • • NLP (GPT)
    • Code Generation (Co-Pilot)
    • Computer Vision (Mid-Journey)
    • Audio (OpenAI Whisper)
    • Video (Make-a-video)
    • Multimodal (GPT4, MUM)
  • The NLP Journey (TF-IDF to Sequence Modeling to Transformers)

• Module 2: LM Fundamentals

  • Embeddings (word2Vec, GloVe, etc)
  • The OG Language Models (LSTMs, BERT, XLNET, Distill-BERT,GPT)
  • LM Fundamentals (Pretraining, Discriminative Fine Tuning)
    • Attention, Positional Embeddings
    • Foundation Models
    • Transfer Learning

• Module 3: Diving into ChatGPT

  • The Awakening of GPT-3 and LLMs
  • Understanding ChatGPT 
    • Instruction Tuning (InstructGPT, ControlNet)
    • SFT
    • RLHF
  • Prompt Engineering
  • Evaluation of LLMs/Benchmarks

• Module 4: Mastering the Pretraining and Fine Tuning of LLMs

  • Setting up an LLM on local 
    • HuggingFace
    • openLLama
    • Falcon
    • GPT4All/ PrivateGPT
    • RedPajama
  • Best Practices to train LLMs

• Module 5: Master LLMs Part-2

  • Challenges associated with LLMs and their pre-training
  • Fine Tuning LLMs (PEFT techniques: Prefix-tuning, LoRa, QLoRa etc)
  • Prompt Tuning/Optimization
  • AI Tooling (AI Agents, LangChain, VectorDB etc)

• Module 6: Next Frontier

  • New AI Tools (chatGPT Plugins, BARD, PALM, Co-Pilot)
  • What Next? (AutoGPT, GPT-4 and beyond)

Pre-requisites:

• System Requirement and Setup
  • Laptop with at least 4-8 GB of RAM
  • We will be using a cloud jupyter notebook powered by GPU for the workshop
• Offline Setup [Optional]
  • GPU good to have!
  • Install Python3.9 or higher version(Resource)
  • Install jupyter notebook (Resource)
• Pre-reads
  • Programming knowledge in Python (Resource)
  • Jupyter Notebook Environment familiarity (Resource)
  • Basics of Machine learning and Deep Learning (Resource,Resource)
  • Fundamentals of NLP(Resource1,Resource2)
  • Familiarity with Pytorch(Resource)

Note: These are tentative details and are subject to change.