Build a Simple Chatbot Using NLTK Library in Python

Raghav Agrawal 16 Feb, 2024 • 10 min read

Introduction

In today’s digital age, where communication is increasingly driven by artificial intelligence (AI) technologies, building your own chatbot has never been more accessible. With the rise of platforms like ChatGPT from OpenAI and powerful libraries such as NLTK (Natural Language Toolkit) in Python, creating a basic Python chatbot has become a straightforward endeavor for aspiring data scientists and developers.

In this article, we’ll embark on a journey to understand the fundamentals of NLTK chatbot development, exploring the synergy between AI, natural language processing (NLP), and the versatility of NLTK, all while taking our first steps into the exciting world of conversational AI. Let’s bring your conversational AI dreams to life with, one line of code at a time!

 Chatbot using Python and NLTK

Learning Objectives:

  • Understand what chatbots are and why we need them.
  • Learn about the different types of AI chatbots.
  • Learn how to build your own rule-based chatbot and self-learning chatbot using the Python NLTK library.

This article was published as a part of the Data Science Blogathon

What are Chatbots?

Chatbots are AI-powered software applications designed to simulate human-like conversations with users through text or speech interfaces. They leverage natural language processing (NLP) and machine learning algorithms to understand and respond to user queries or commands in a conversational manner.

Chatbots can be deployed across various platforms, including websites, messaging apps, and virtual assistants, to provide a wide range of services such as customer support, information retrieval, task automation, and entertainment. They play a crucial role in improving efficiency, enhancing user experience, and scaling customer service operations for businesses across different industries.

Why Do We Need Chatbots?

  • Enhanced Customer Service: Chatbots provide instant responses to customer queries, ensuring round-the-clock support without the need for human intervention. This results in faster resolution times and improved customer satisfaction.
  • Scalability: With chatbots, businesses can handle multiple customer interactions simultaneously, scaling their support operations to accommodate growing demand without significantly increasing costs.
  • Cost Efficiency: Implementing chatbots reduces the need for hiring and training additional customer service representatives, resulting in cost savings for businesses over time.
  • 24/7 Availability: Chatbots operate continuously, offering support to users regardless of the time of day or geographical location. This ensures that customers can receive assistance whenever they need it, leading to higher engagement and retention rates.
  • Data Collection and Analysis: Chatbots can gather valuable customer data during interactions, such as preferences, frequently asked questions, and pain points. This data can be analyzed to identify trends, improve products or services, and tailor marketing strategies, driving business growth and innovation.
Build a simple Chatbot using NLTK Library in Python

Types of Chatbots

There are mainly 2 types of AI chatbots.

1) Rule-based Chatbots: As the name suggests, there are certain rules by which chatbot operates. Like a machine learning model, we train the chatbots on user intents and relevant responses, and based on these intents chatbot identifies the new user’s intent and response to him.

2) Self-learning chatbots: Self-learning bots are highly efficient because they are capable to grab and identify the user’s intent on their own. they are built using advanced tools and techniques of machine learning, deep learning, and NLP.

Self-learning bots are further divided into 2 subcategories.

  • Retrieval-based chatbots: Retrieval-based is somewhat the same as rule-based where predefined input patterns and responses are embedded.
  • Generative-based chatbots: It is based on the same phenomenon as Machine Translation built using sequence 2 sequences neural network.

Most organizations use self-learning chatbots along with embedding some rules like the hybrid version of both methods which makes chatbots powerful enough to handle each situation during a conversation with a customer.

Building A Chatbot Using Python

Now we have an immense understanding of the theory of chatbots and their advancement in the future. Let’s make our hands dirty by building one simple rule-based chatbot using Python for ourselves.

We will design a simple GUI using the Python Tkinter module using which we will create a text box and button to submit user intent and on the action, we will build a function where we will match the user intent and respond to him on his intent. If you do not have the Tkinter module installed, then first install it using the pip command.

pip install tkinter
from tkinter import *
root = Tk()
root.title("Chatbot")
def send():
    send = "You -> "+e.get()
    txt.insert(END, "n"+send)
    user = e.get().lower()
    if(user == "hello"):
        txt.insert(END, "n" + "Bot -> Hi")
    elif(user == "hi" or user == "hii" or user == "hiiii"):
        txt.insert(END, "n" + "Bot -> Hello")
    elif(e.get() == "how are you"):
        txt.insert(END, "n" + "Bot -> fine! and you")
    elif(user == "fine" or user == "i am good" or user == "i am doing good"):
        txt.insert(END, "n" + "Bot -> Great! how can I help you.")
    else:
        txt.insert(END, "n" + "Bot -> Sorry! I dind't got you")
    e.delete(0, END)
txt = Text(root)
txt.grid(row=0, column=0, columnspan=2)
e = Entry(root, width=100)
e.grid(row=1, column=0)
send = Button(root, text="Send", command=send).grid(row=1, column=1)
root.mainloop()

Explanation – First we created a blank window, After that, we created a text field using the entry method and a Button widget which on triggering calls the function send, and in return, it gets the chatbot response. We have used a basic If-else control statement to build a simple rule-based chatbot. And you can interact with the chatbot by running the application from the interface and you can see the output as below figure.

Chatbot using Python and NLTK minloop

Building Rule-Based Chatbot Using Python NLTK Library

NLTK stands for Natural language toolkit used to deal with NLP applications and chatbot is one among them. Now we will advance our Rule-based chatbots using the NLTK library. Please install the NLTK library first before working using the pip command.

pip instal nltk

The first thing is to import the necessary library and classes we need to use.

import nltk
from nltk.chat.util import Chat, reflections
  • Chat – Chat is a class that contains complete logic for processing the text data that the chatbot receives and finding useful information out of it.
  • reflections – Another import we have done is reflections which is a dictionary containing basic input and corresponding outputs. You can also create your own dictionary with more responses you want. if you print reflections it will be something like this.
reflections = {
  "i am"       : "you are",
  "i was"      : "you were",
  "i"          : "you",
  "i'm"        : "you are",
  "i'd"        : "you would",
  "i've"       : "you have",
  "i'll"       : "you will",
  "my"         : "your",
  "you are"    : "I am",
  "you were"   : "I was",
  "you've"     : "I have",
  "you'll"     : "I will",
  "your"       : "my",
  "yours"      : "mine",
  "you"        : "me",
  "me"         : "you"
}

let’s start building logic for the NLTK chatbot.

After importing the libraries, First, we have to create rules. The lines of code given below create a simple set of rules. the first line describes the user input which we have taken as raw string input and the next line is our chatbot response. You can modify these pairs as per the questions and answers you want.

pairs = [
    [
        r"my name is (.*)",
        ["Hello %1, How are you today ?",]
    ],
    [
        r"hi|hey|hello",
        ["Hello", "Hey there",]
    ], 
    [
        r"what is your name ?",
        ["I am a bot created by Analytics Vidhya. you can call me crazy!",]
    ],
    [
        r"how are you ?",
        ["I'm doing goodnHow about You ?",]
    ],
    [
        r"sorry (.*)",
        ["Its alright","Its OK, never mind",]
    ],
    [
        r"I am fine",
        ["Great to hear that, How can I help you?",]
    ],
    [
        r"i'm (.*) doing good",
        ["Nice to hear that","How can I help you?:)",]
    ],
    [
        r"(.*) age?",
        ["I'm a computer program dudenSeriously you are asking me this?",]
    ],
    [
        r"what (.*) want ?",
        ["Make me an offer I can't refuse",]
    ],
    [
        r"(.*) created ?",
        ["Raghav created me using Python's NLTK library ","top secret ;)",]
    ],
    [
        r"(.*) (location|city) ?",
        ['Indore, Madhya Pradesh',]
    ],
    [
        r"how is weather in (.*)?",
        ["Weather in %1 is awesome like always","Too hot man here in %1","Too cold man here in %1","Never even heard about %1"]
    ],
    [
        r"i work in (.*)?",
        ["%1 is an Amazing company, I have heard about it. But they are in huge loss these days.",]
    ],
    [
        r"(.*)raining in (.*)",
        ["No rain since last week here in %2","Damn its raining too much here in %2"]
    ],
    [
        r"how (.*) health(.*)",
        ["I'm a computer program, so I'm always healthy ",]
    ],
    [
        r"(.*) (sports|game) ?",
        ["I'm a very big fan of Football",]
    ],
    [
        r"who (.*) sportsperson ?",
        ["Messy","Ronaldo","Roony"]
    ],
    [
        r"who (.*) (moviestar|actor)?",
        ["Brad Pitt"]
    ],
    [
        r"i am looking for online guides and courses to learn data science, can you suggest?",
        ["Crazy_Tech has many great articles with each step explanation along with code, you can explore"]
    ],
    [
        r"quit",
        ["BBye take care. See you soon :) ","It was nice talking to you. See you soon :)"]
    ],
]

After creating pairs of rules, we will define a function to initiate the chat process. The function is very simple which first greets the user and asks for any help. The conversation starts from here by calling a Chat class and passing pairs and reflections to it.

def chat():
    print("Hi! I am a chatbot created by Analytics Vidhya for your service")
    chat = Chat(pairs, reflections)
    chat.converse()
#initiate the conversation
if __name__ == "__main__":
    chat()

We have created an amazing Rule-based chatbot just by using Python and NLTK library. The nltk.chat works on various regex patterns present in user Intent and corresponding to it, presents the output to a user. Let’s run the application and chat with your created chatbot.

nltk.chat

Building Self Learning Chatbots Using Python NLTK Library

To create a self-learning chatbot using the NLTK library in Python, you’ll need a solid understanding of Python, Keras, and natural language processing (NLP).

Here are the 6 steps to create a chatbot in Python from scratch:

  1. Import required libraries
  2. Import and load the data file
  3. Preprocess data
  4. Create training and testing data
  5. Build the model
  6. Predict the response
  7. Run the chatbot

Below is the step-by-step guide for building a simple chatbot:

Step 1: Install Required Modules

Begin by installing the necessary modules using the pip command:

pip install tensorflow keras pickle nltk

Step 2: Import and Load Data File

Import the required packages and load the data file (`intents.json` in this case) containing intents for the chatbot.

import nltk

from nltk.stem import WordNetLemmatizer

lemmatizer = WordNetLemmatizer()

import json

import pickle

import numpy as np

from keras.models import Sequential

from keras.layers import Dense, Activation, Dropout

from keras.optimizers import SGD

import random

# Load data from intents.json

data_file = open('intents.json').read()

intents = json.loads(data_file)

Step 3: Preprocess Data

The “preprocess data” step involves tokenizing, lemmatizing, removing stop words, and removing duplicate words to prepare the text data for further analysis or modeling.

import nltk
from nltk.stem import WordNetLemmatizer
from nltk.corpus import stopwords

# Download NLTK resources
nltk.download('punkt')
nltk.download('wordnet')
nltk.download('stopwords')

# Initialize lemmatizer and stopwords
lemmatizer = WordNetLemmatizer()
stop_words = set(stopwords.words('english'))

# Sample data
data = [
    "The quick brown fox jumps over the lazy dog",
    "A bird in the hand is worth two in the bush",
    "Actions speak louder than words"
]

# Tokenize, lemmatize, and remove stopwords
tokenized_data = []
for sentence in data:
    tokens = nltk.word_tokenize(sentence.lower())  # Tokenize and convert to lowercase
    lemmatized_tokens = [lemmatizer.lemmatize(token) for token in tokens]  # Lemmatize tokens
    filtered_tokens = [token for token in lemmatized_tokens if token not in stop_words]  # Remove stop words
    tokenized_data.append(filtered_tokens)

# Remove duplicate words
for i in range(len(tokenized_data)):
    tokenized_data[i] = list(set(tokenized_data[i]))

print(tokenized_data)

Step 4: Create Training and Testing Data

Prepare the training data by converting text into numerical form.

# Create training data

training = []

output_empty = [0] * len(classes)

for doc in documents:

    bag = []

    pattern_words = doc[0]

    pattern_words = [lemmatizer.lemmatize(word.lower()) for word in pattern_words]

    

    for w in words:

        bag.append(1) if w in pattern_words else bag.append(0)

    

    output_row = list(output_empty)

    output_row[classes.index(doc[1])] = 1

    training.append([bag, output_row])

# Shuffle and convert to numpy array

random.shuffle(training)

training = np.array(training)

# Separate features and labels

train_x = list(training[:,0])

train_y = list(training[:,1])

Step 5: Build the Model

Create a neural network model using Keras.

model = Sequential()

model.add(Dense(128, input_shape=(len(train_x[0]),), activation='relu'))

model.add(Dropout(0.5))

model.add(Dense(64, activation='relu'))

model.add(Dropout(0.5))

model.add(Dense(len(train_y[0]), activation='softmax'))

# Compile the model

sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)

model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])

# Train the model

model.fit(np.array(train_x), np.array(train_y), epochs=200, batch_size=5, verbose=1)

# Save the model

model.save('chatbot_model.h5')

Step 6: Predict the Response

Implement a function to predict responses based on user input.

def predict_class(sentence, model):

    p = bow(sentence, words, show_details=False)

    res = model.predict(np.array([p]))[0]

    results = [[i,r] for i,r in enumerate(res) if r>ERROR_THRESHOLD]

    results.sort(key=lambda x: x[1], reverse=True)

    return_list = []

    for r in results:

        return_list.append({"intent": classes[r[0]], "probability": str(r[1])})

    return return_list

Step 7: Run the Chatbot

Develop a graphical user interface to interact with the chatbot.

# GUI with Tkinter

import tkinter

from tkinter import *

# Function to send message and get response

def send():

    msg = EntryBox.get("1.0",'end-1c').strip()

    EntryBox.delete("0.0",END)

    if msg != '':

        ChatLog.config(state=NORMAL)

        ChatLog.insert(END, "You: " + msg + '\n\n')

        ChatLog.config(foreground="#442265", font=("Verdana", 12))

        res = chatbot_response(msg)

        ChatLog.insert(END, "Bot: " + res + '\n\n')

        ChatLog.config(state=DISABLED)

        ChatLog.yview(END)

# GUI setup

base = Tk()

base.title("Chatbot")

base.geometry("400x500")

base.resizable(width=FALSE, height=FALSE)

ChatLog = Text(base, bd=0, bg="white", height="8", width="50", font="Arial")

ChatLog.config(state=DISABLED)

scrollbar = Scrollbar(base, command=ChatLog.yview, cursor="heart")

ChatLog['yscrollcommand'] = scrollbar.set

SendButton = Button(base, font=("Verdana",12,'bold'), text="Send", width="12", height=5, bd=0, bg="#32de97", activebackground="#3c9d9b",fg='#ffffff', command= send )

EntryBox = Text(base, bd=0, bg="white",width="29", height="5", font="Arial")

scrollbar.place(x=376,y=6, height=386)

ChatLog.place(x=6,y=6, height=386, width=370)

EntryBox.place(x=128, y=401, height=90, width=265)

SendButton.place(x=6, y=401, height=90)

base.mainloop()

By following these steps and running the appropriate files, you can create a self-learning chatbot using the NLTK library in Python.

Conclusion

Chatbots are the top application of Natural Language processing and today it is simple to create and integrate with various social media handles and websites. Today most Chatbots are created using tools like Dialogflow, RASA, etc. This was a quick introduction to chatbots to present an understanding of how businesses are transforming using Data science and artificial Intelligence.

Key Takeaways:

  • Chatbots are AI-powered software applications designed to simulate human-like conversations with users through text or speech interfaces.
  • The two main types of AI chatbots are rule-based and self-learning.
  • Self-learning chatbots are of two types: retrieval-based and generative-based chatbots.

Frequently Asked Questions

Q1. What is the fullform of NLTK?

A. The full form of NLTK is the “Natural Language Toolkit.”

Q2. How do I create a simple chatbot in Python using NLTK?

A. To create a chatbot in Python using NLTK:
1. Install NLTK: Use pip install nltk to install the NLTK library.
2. Import NLTK: Import necessary modules from NLTK, such as nltk.chat.util.Chat.
3. Define Patterns and Responses: Create pairs of patterns and corresponding responses.
4. Create a Chatbot: Initialize a Chat object with the pairs of patterns and responses.
5. Start Chatting: Use the converse() method to interact with the chatbot.
With these steps, you can quickly build a simple chatbot in Python using NLTK.

Q3. How do I create a Self-learning chatbot in Python using NLTK?

A. To create a self-learning chatbot in Python using NLTK, you can follow these steps:
1. Install NLTK and Necessary Libraries: Use pip install nltk to install NLTK.
2. Load and Preprocess Data: Load a dataset containing intents and preprocess the data.
3. Create Training Data: Prepare the training data by converting text into numerical form.
4. Build a Model: Create a neural network model using libraries like Keras or TensorFlow.
5. Train the Model: Train the model using the training data.
6. Predict Responses: Implement a function to predict responses based on user input.
7. Develop a User Interface and Run it: Create a graphical user interface (GUI) for users to interact with the chatbot.
By following these steps, you can develop a self-learning chatbot in Python using NLTK. This chatbot will be capable of learning from user interactions and improving its responses over time.

The media shown in this article are not owned by Analytics Vidhya and are used at the Author’s discretion.

Raghav Agrawal 16 Feb 2024

I am a final year undergraduate who loves to learn and write about technology. I am a passionate learner, and a data science enthusiast. I am learning and working in data science field from past 2 years, and aspire to grow as Big data architect.

Frequently Asked Questions

Lorem ipsum dolor sit amet, consectetur adipiscing elit,

Responses From Readers

Clear

thulasi
thulasi 19 Oct, 2022

how to use if,else or set keywords in python chatbot lased on lists

Natural Language Processing
Become a full stack data scientist