Generative AI, especially the Generative Large Language Models, have taken over the world since their birth. This was only possible because they could integrate with different applications, from generating working programmable codes to creating fully GenerativeAI-managed Chat Support Systems. But most of the Large Language Models in the Generative AI space have been closed to the public; most were not open-sourced. While there do exist a few Open Source models, but are nowhere near the closed-source Large Language Models. But recently, FalconAI, an LLM, was released, which topped the OpenLLM leaderboard and was made Open Sourced. With this model in this guide, we will create a chat application with Falcon AI, LangChain, and Chainlit.
In this article you will get to know about the chainlit vs streamlit whts the differences and about the benefits of it.
This article was published as a part of the Data Science Blogathon.
In the Generative AI field, Falcon AI is one of the recently introduced Large Language Models known for taking first place in the OpenLLM Leaderboard. Falcon AI was introduced by UAE’s Technology Innovation Institute (TII). Falcon AI’s architecture is designed in a way that is optimized for Inference. When it was first introduced, Falcon AI topped the OpenLLM Leaderboard by moving ahead of state-of-the-art models like Llama, Anthropic, DeepMind, etc. The model was trained on AWS Cloud with 384 GPUs attached continuously for two months.
Currently, it consists of two models, Falcon 40B(40 Billion Parameters) and Falcon 7B(7 Billion Parameters). The main part is that the Falcon AI makers have mentioned that the model will be Open Sourced, thus allowing developers to work with it for commercial use without restrictions. Falcon AI even provides the Instruct models, the Falcon-7B-Instruct and Falcon-40B-Instruct, with which we can quickly get started to build chat applications. In this guide, we will work with the Falcon-7B-Instruct model.
Chainlit library is similar to Python’s Streamlit Library. But the intended purpose of this Chainlit library is to build chat applications with Large Language Models quickly, i.e., to create a UI similar to ChatGPT. Developing conversational chat applications within minutes with the Chainlit package is possible. This library is seamlessly integrated with LangFlow and LangChain(the library to build applications with Large Language Models), which we will do later in this guide.
Chainlit even allows for visualizing multi-step reasoning; it lets us see the intermediate results to know how the Large Language Model reached the output to a question. So you can clearly see the chain of thoughts of the model through the UI itself to understand how the LLM concluded the given question. Chainlit is restricted to a text conversation and allows for sending and receiving Images to and from the respective Generative AI models. It even lets us update the Prompt Template in the UI instead of returning to the code and changing it.
There are two ways to work with the Falcon-7B-Instruct model. One is the traditional way, where we download the model to the local machine and then use it directly. But because this is a Large Language Model, it will need high GPU memory to make it work. Hence we go with the other option, calling the model directly through the Inference API. Inference API is a HuggingFace API token with which we can access all the transformer models in the HuggingFace.
To access this token, we need to create an Account in HuggingFace, which we can do by going to the official HuggingFace website. After logging in/signing in with your details, go to your profile and click on the Settings section. The process from there will be
So in Settings, go to Access Tokens. You will create a new token, which we must work with the Falcon-7B-Instruct model. Click on the New Token to create the new token. Enter a name for the token and set the Role option to Write. Now click on Generate to generate our new Token. With this token, we can access the Falcon-7B-Instruct model and build applications.
Before we dive into our application, we will create an ideal environment for the code to work. For this, we need to install the necessary Python libraries needed. Firstly, we will start by installing the libraries that support the model. For this, we will do a pip install of the below libraries.
$ pip install huggingface_hub
$ pip install transformers
These commands will install the HuggingFace Hub and the Transformers libraries. These libraries call the Falcon-7B-Instruct model, which resides in the HuggingFace. Next, we will be installing the LangChain library for Python.
$ pip install langchain
This will install the LangChain Package for Python, which we will work with to create our chat application with the Falcon Large Language Model. Finally, without the UI, the conversational application is not done. So for this, we will be downloading the chainlit library.
$ pip install chainlit
This will install the Chainlit library for Python. With the help of this library, we will be building the UI for our conversational chat application. After installing chainlit, we need to test the package. For this, use the below command in the terminal.
chainlit hello
After entering this command, a new window with the address localhost and PORT 8000 will appear. The UI will then be visible. This tells that the chainlit library is installed properly and ready to work with other libraries in Python.
In this section, we will start building our application. We have all the necessary libraries to go forward to build our very own conversational chat application. The first thing we will be doing is importing the libraries and storing the HuggingFace Inference API in an environmental object.
import os
import chainlit as cl
from langchain import HuggingFaceHub, PromptTemplate, LLMChain
os.environ['API_KEY'] = 'Your API Key'
Now we will be inferring the Falcon Instruct model through the HuggingFaceHub module. For this, first, we must provide the path to the model in the Hugging Face. The code for this will be
model_id = 'tiiuae/falcon-7b-instruct'
falcon_llm = HuggingFaceHub(huggingfacehub_api_token=os.environ['API_KEY'],
repo_id=model_id,
model_kwargs={"temperature":0.8,"max_new_tokens":2000})
Now we have clearly defined what model we will be working with. And the HuggingFace API will let us connect to this model and run our queries to start building our application.
After the model selection, the next is defining the Prompt Template. The Prompt Template tells how the model should behave. It tells how the model should interpret the question provided by the user. It even tells how the model should conclude to give the output to the user’s query. The code for defining our Prompt Template would be
template = """
You are an AI assistant that provides helpful answers to user queries.
{question}
"""
prompt = PromptTemplate(template=template, input_variables=['question'])
The above template variable defines and sets the context of the Prompt Template for the Falcon model. The context here is simple, the AI needs to provide helpful answers to user queries, followed by the input variable {question}. Then this template, along with the variables defined in it, is given to the PromptTemplate function, which is then assigned to a variable. This variable is now the Prompt Template, which will later be chained together with the model.
Now we have both the Falcon LLM and the Prompt Template ready. The final part will be chaining both these models together. We will work with the LLMChain object from the LangChain library for this. The code for this will be
falcon_chain = LLMChain(llm=falcon_llm,
prompt=prompt,
verbose=True)
With the help of LLMChain, we have chained the Falcon-7B-Instruct model with our very own PromptTemplate that we have created. We have even set the verbose = True, which is helpful to know what happens when the code is being run. Now let’s test the model by giving a query to it
print(falcon_chain.run("What are the colors in the Rainbow?"))
Here, we have asked the model what the rainbow colors are. The rainbow contains VIBGYOR (Violet, Indigo, Blue, Green, Yellow, Orange, and Red) colors. The output generated by the Falcon 7B Instruct model is spot on to the question asked. Setting the verbose option lets us see the Prompt after formatting and tells us where the chain starts and ends. Finally, we are ready to create a UI for our conversational chat application.
Chainlit: Built specifically for chat rooms. Makes conversations flow smoothly and remembers past talks. Has cool features like showing the “thought process” behind answers and handling pictures. Might be a bit new and have less help available, but perfect for creating chatbots or talking to big language models.
Streamlit: is Like a toolbox with lots of pieces (charts, buttons) for any kind of room (data dashboards, reports). Easy to use, even for beginners. Rebuilds the whole house every time you change a piece, which can be slow for big houses.
In this section, we will work with Chainlit Package to create the UI for our application. Chainlit is a Python library that lets us build Chat Interfaces for Large Language Models in minutes. It is integrated with LangFlow and even LangChain, the library we previously worked on. Creating the Chat Interface with Chainlit is simple. We have to write the following code:
@cl.langchain_factory(use_async=False)
def factory():
prompt = PromptTemplate(template=template, input_variables=['question'])
falcon_chain = LLMChain(llm=falcon_llm,
prompt=prompt,
verbose=True)
return falcon_chain
That’s it. Now when we run the code, a Chat Interface will be visible. But how is this possible The thing is, Chainlit takes care of everything. Behind the scenes, it manages the webhook connections, it is responsible for creating a separate LangChain Instance(Chain, Agent, etc) for each user that visits the site. To run our application, we type the following in the terminal.
$ chainlit run app.py -w
The -w indicates auto-reload whenever we make changes live in our application code. After entering this, a new tab gets opened with localhost:8000
This is the opening page, i.e., the welcome screen of Chainlit. We see that Chainlit builds an entire Chat Interface for us just with a single decorator. Let’s try interacting with the Falcon Model through this UI
We see that the UI and the Falcon Instruct model are working perfectly fine. The model can provide swift answers to the questions asked. It really tried to explain the second question based on the user’s context (explain to a 5-year-old). This is the beginning of what we can achieve with these Open Sourced Generative AI models. With little to few modifications, we can be able to create much more problem-oriented, real scenario-based applications.
As the Chat Interface is a website, it is completely possible to host it on any of the cloud platforms. We can containerize the application, then try to deploy it in any container-based services in Google Cloud, AWS, Azure, or other cloud services. With that, we can share our applications with the outside world.
In this walkthrough, we have seen how to build a simple Chat Application with the new Open Source Falcon Large Language Model, LangChain, and Chainlit. We have leveraged these three packages and have interconnected them to create a full-fledged solution from Code to Working Application. We have even seen how to obtain the HuggingFace Inference API Key to access thousands of pre-trained models from the HuggingFace library. With the help of LangChain, we chained the LLM with custom Prompt Templates. Finally, with Chainlit, we could create a Chat Application Interface around our LangChain Falcon model within minutes. Hope you like this article, and get the difference about the chainlit vs streamlit and whts the differences and whts the uses in it.
Some of the key takeaways from this guide include:
A. The Inference API is created by HuggingFace, allowing you to access thousands of pre-trained models in the HuggingFace library. With this API, you can access a variety of models, including Generative AI models, Natural Language Processing Models, Audio Classification, and Computer Vision models.
A. They are. Especially the Falcon 40B(40 Billion Parameters) model. This model has surpassed other state-of-the-art models like Llama and DeepMind and acquired the top position in the OpenLLM Leaderboard.
A. Chainlit is a Python Library that is developed for creating UI. With Chainlit, creating ready-to-work Chat Interfaces for Large Language Models within minutes is possible. The Chainlit Package seamlessly integrates with LangFlow and LangChain, other packages that are worked with to create applications with Large Language Models.
A. Yes. The Falcon 40B(40 Billion Parameters) and the Falcon 7B(7 Billion Parameters) are Open Sourced. This states that anyone can work with these models to create commercial applications without restrictions.
The media shown in this article is not owned by Analytics Vidhya and is used at the Author’s discretion.
Lorem ipsum dolor sit amet, consectetur adipiscing elit,