In this module, we will take a rapid tour of Generative AI and dive into the specifics of Large Language Models. You will learn about different model architectures, such as encoders, decoders, and encoder-decoder in the context of transformers, along with state-space and multi-modal models. We will also cover various types of LLMs and their applications including prompting, summarization, and question answering. A hands-on introduction to the HuggingFace ecosystem will allow participants to experiment with transformer-based LLMs for various NLP tasks.

• Whirlwind Tour of Generative AI
• Introduction to Large Language Models
• What are LLMs?
• Key architectures (e.g., Encoder, Decoder, Encoder-Decoder, State-space Models, Multi-modal models)
• Types and Applications of LLMs
• Foundational and Fine-tuned LLMs
• Applications - Prompting, Summarization, Extraction, Q&A, RAG
• Hands-on: Introduction to the HuggingFace ecosystem. NLP applications with foundational and fine-tuned LLMs

This module focuses on the nuts and bolts of training LLMs from scratch and the fine-tuning process. You will understand the stages of data preparation, tokenization, pre-training, and fine-tuning. We will also explore the differences between training a model from scratch and fine-tuning a pre-trained model, learning about the benefits and challenges of each. The module includes hands-on sessions on training an LLM from scratch for text \ code generation and fine-tuning for popular NLP tasks such as classification \ translation.

• Understanding the LLM Training Process
• Data preparation, tokenization, pre-training and fine-tuning language models
• Understanding pre-training vs. fine-tuning
• Training an LLM from Scratch
• Understand the key steps to train an LLM from scratch
• Introduction to HuggingFace datasets, tokenizers, Trainer constructs
• Hands-on: Training an LLM from scratch - Text and code generation
• Full Fine-tuning a pre-trained LLM
• Understand the key-steps to fine-tune all layers and parameters of an LLM
• Key advantages and limitations of full-fine tuning
• Methods to improve and accelerate fine-tuning performance
• Hands-on: Fine-tune an LLM for a popular NLP task - Classification \ Translation etc.

In this module, we will explore parameter-efficient fine-tuning techniques that help in fine-tuning LLMs without expensive computational infrastructure as well as compute time. Techniques such as LoRA, QLoRA, and prompt-based methods like prompt tuning and prefix tuning will be covered. The session will introduce HuggingFace's parameter-efficient fine-tuning (PEFT) library and include hands-on sessions in fine-tuning LLMs for tasks such as classification and question-answering using these advanced methods.

• Understand what is parameter-efficient fine-tuning, advantages and disadvantages
• Key parameter-efficient fine-tuning techniques:
• Decomposition methods: LoRA, QLoRA, DoRA and more
• Prompt-base methods: prompt tuning, prefix tuning, p-tuning
• Activation-based methods: IA3
• Introduction to HuggingFace PEFT and key constructs
• Hands-on: Fine-tune an LLM using LoRa for a simple NLP task e.g. Classification
• Hands-on: Fine-tune an LLM using QLoRA for a simple NLP task e.g. Classification
• Hands-on: Fine-tune an LLM using Prompt-tuning for a simple NLP task e.g. Classification
• Hands-on: Fine-tune LLMs using above techniques on more complex tasks like Question-Answering
• Hands-on: Learn how to switch and merge different adapters using PEFT

This module  will deep-dive into instruction-based fine-tuning using approaches like Supervised Fine-tuning (SFT), Reward Modeling (RM), and Reinforcement Learning with Human Feedback (RLHF). Participants will learn about different reinforcement learning policies and their application in fine-tuning LLMs. Hands-on sessions will include fine-tuning models like LLaMA 3, Mistral using HuggingFace TRL and Unsloth AI.

• Understand what is instruction-based fine-tuning of LLMs and key components and steps
• Supervised fine-tuning (SFT)
• Reward modeling (RM)
• Reinforcement learning with human feedback (RLHF)
• Understand different RL policies like Proximal Policy Optimization (PPO), Direct preference optimization (DPO) etc.
• Introduction to HuggingFace TRL and key constructs
• Introduction to Unsloth AI for even more efficient PEFT
• Hands-on: Fine-tune LLaMA 3 on prompt-based instructions using Supervised Fine-tuning with HuggingFace TRL and Unsloth AI
• Hands-on: Go through the complete process of SFT + RM + RLHF on an instruction-based task for fine-tuning LLMs like LLaMA, Mistral etc.

The final module will summarize key topics covered in the workshop and discuss best practices for fine-tuning LLMs. It will address the fine-tuning of both open-source and commercial LLMs like ChatGPT. The session aims to provide a roadmap for future learning and application of the skills gained during the workshop.

• Understand the best-practices w.r.t fine-tuning LLMs
• How to fine-tune closed commercial LLMs like ChatGPT or Gemini
• Future scope, where to go from here?