Shivani Sharma — August 10, 2021
Intermediate Programming Project Python

This article was published as a part of the Data Science Blogathon

What is a Voice Assistant?

A voice assistant is additionally called an intelligent personal assistant. it’s a form of software agent which could be accustomed to assist its users on their genuine computer-based tasks by taking their voice as a command. The intuition behind the creation of the bot is to open and manage basic computer tasks like opening apps, playing music, changing the wallpaper, open youTube, etc. just by asking orally.

History of Voice Assistants

How Voice Assistants Are Changing Our Lives | Smartsheet

Figure: A modern history of Voice Assistants

In recent times, Voice assistants got the key platform after Apple integrated the foremost astonishing Virtual Assistant — Siri which is officially a part of Apple Inc. During the quantity of the 1970s, researchers at Carnegie Mellon University in Pittsburgh, Pennsylvania — with the considerable help of the U.S Department of Defence and its Defence Advanced Research Projects Agency (DARPA) — made Harpy.

Almost 1,000 words are often remembered by this assistant, we will easily compare it with the vocabulary of a three-year-old child. Biggest organizations like Apple, Google, Amazon, and IBM have created so many products that support the voice assistant and the work is still on.

In 1993, Macintosh began to assembling speech recognition with its Macintosh PCs with PlainTalk. There are so many live examples that you are using on your mobile and laptop/desktop. So, let’s just begin with the implementation of voice assistants using the NLP approach.

Prerequisites to build a voice assistant in python

System requirements:

1.Python 2.7

2. Spyder IDE

3. macOS Mojave(version 10.14)

Install all these python libraries :

pip install SpeechRecognition
pip install beautifulsoup4
pip install vlc
pip install youtube-dl
pip install pyowm
pip install wikipedia

Let’s begin with building our desktop voice assistant using python

Start by importing all the required libraries:

import speech_recognition as sr
import os
import sys
import re
import webbrowser
import smtplib
import requests
import subprocess
from pyowm import OWM
import youtube_dl
import vlc
import urllib
import urllib2
import json
from bs4 import BeautifulSoup as soup
from urllib2 import urlopen
import Wikipedia
import random
from time import strftime

To perform all the features of a voice assistant the most important and the initial thing is to give years to your assistant so that it can hear your voice and save your command to give you respective output. So our initiative is to create a way that may interpret user voice responses.

def myCommand():
    r = sr.Recognizer()
    with sr.Microphone() as source:
        print('Say something...')
        r.pause_threshold = 1
        r.adjust_for_ambient_noise(source, duration=1)
        audio = r.listen(source)
    try:
        command = r.recognize_google(audio).lower()
        print('You asked: ' + command + 'n')
    #if the speech received is not recognizable the loop will back to continue to listen for commands if unrecognizable speech is received
    except sr.UnknownValueError:
        print('....')
        command = myCommand();
    return command
#Now the task is to convert the recognizable speech to text
def sofiaResponse(audio):
    print(audio)
    for line in audio.splitlines():
        os.system("say " + audio)

To execute multiple commands again and again we had to form a loop. you would like to pass the myCommand() function as a parameter to the assistant function.

while True:
    assistant(myCommand())

Our next step is to make multiple if statements like each of the features. So allow us to work out the thanks for creating these small modules inside if statement for each command.

1. Open the subreddit Reddit within the browser.

Any command is given by the user to open any subreddit using Reddit and also the command must be like “Hey Sofia! are you ready to please open Reddit subreddit_name”. only the italic bold phrase should be used because it’s. you will be ready to use any quiet prefix, just make certain of the italic bold phrase.

How it works: If you have got said the phrase open Reddit in your command then it’ll explore for the subreddit name within the user command using re. search(). Using the Python Webbrowser module you open the subreddit by searching it on www.reddit.com. A high-level user interface is provided by the Webbrowser module for the purpose of displaying Web-based documents to users.

if 'open reddit' in command:
        reg_ex = re.search('open reddit (.*)', command)
        url = 'https://www.reddit.com/'
        if reg_ex:
            subreddit = reg_ex.group(1)
            urls = urls + 'r/' + subreddit
        webbrowser.open(urls)
        sofiaResponse('Sure Sir,I am displaying the available Reddit content on the screen.')

So, the above code will open your required Reddit in your default browser.

2. Open any website within the browser.

You can open any website just by saying “open website.com” or “open website.org”. For example: “Please open facebook.com” or “Hey, are you ready to open linkedin.com” like this you’ll ask Sofia to open any website for you.

How it works: If you have got said the word open in your command then it’ll inquiry for website name within the user command using re. search(). Next, it’ll append the website name to https://www. and employing an applications program module the entire URL gets opened within the browser.

elif 'open' in command:
        reg_ex = re.search('open (.+)', command)
        if reg_ex:
            domain = reg_ex.group(1)
            print(domain)
            url = 'https://www.' + domain
            webbrowser.open(url)
            sofiaResponse('Sir, The website you are requesting to open is available in your screen now')
        else:
            pass

3. Send Email.

You can give the command to your Assistant for sending an Email.

How it works: If you have said the word email in your command then the bot will fire the recipient, If my response is Rajat, the bot will use pythons smtplib library. Sending mail is finished with Python’s smtplib using an SMTP server. 

First, it’ll initiate Gmail SMTP using smtplib.SMTP(), then identify the server using ehlo() function, then encrypting the session starts(), then login to your mailbox using login(), then sending the message using sendmail(). So, the above code will open your required Reddit in your default browser.

elif 'email' in command:
        sofiaResponse('Who is the recipient?')
        recipient = myCommand()if 'rajat' in recipient:
            sofiaResponse('What should I say to him?')
            content = myCommand()
            mail = smtplib.SMTP('smtp.gmail.com', 587)
            mail.ehlo()
            mail.starttls()
            mail.login('your_email_address', 'your_password')
            mail.sendmail('sender_email', 'receiver_email', content)
            mail.close()
            sofiaResponse('Email has been sent successfuly. You can check your inbox.')else:
            sofiaResponse('I don't know what you mean!')

4. Launch any system application.

All you would like to mention is ‘’Sofia launch calendar” or “Sofia could you please launch the skype” etc. and Sofia are going to be prepared to launch that system application for you.

How it works: If you have got said the word launch in your command then it’ll rummage around for the applying name(if it’s present in your system) within the user command using re. search(). it’ll then append the suffix “.app” to the appliance name. Now your application name is as an example say calendar.

elif 'launch' in command:
        reg_ex = re.search('launch (.*)', command)
        if reg_ex:
            appname = reg_ex.group(1)
            appname1 = appname+".app"
            subprocess.Popen(["open", "-n", "/Applications/" + appname1], stdout=subprocess.PIPE)sofiaResponse('I have launched the desired application')

5. You’ll be able to get the present temperature and weather of just about any city from this chatbot.

Sofia can even tell you the weather, maximum and minimum temperature of any city around the world. The user just must say something like “what is that this weather in London” or “tell me this weather in Delhi”.

How it works: If you’ve said the phrase current weather in your command then it’ll hunt for city name using re. search(). I’ve got used pythons pyowm library to urge the weather of any city. get_status() will provide you with all the atmospheric conditions like haze, cloudy, rainy, etc and get_temperature() will provide you with the max and min temperature of the town/city.

elif 'current weather' in command:
     reg_exp = re.search('current weather forecast in (.*)', command)
     if reg_ex[:
         city = reg_ex.group(1)
         owm = OWM(API_key='ab0d5e80e8dafb2cb81fa9e82431c1fa')
         obs = owm.weather_at_place(city)
         w = obs.get_weather()
         k = w.get_status()
         x = w.get_temperature(unit='celsius')
         sofiaResponse('Current weather in %s is %s. The max temperature is %0.2f and the min temperature is %0.2f degree celcius' % (city, k, x['temp_max'], x['temp_min']))

6. Tells you the current time.

Sofia could you please help me in letting me know the current time ?” or “what is that the time now ?” is asked for getting the time.

How it works: Its quite simple

elif 'time' in command:
     import datetime
     now = datetime.datetime.now()
     sofiaResponse('Acctual time is %d hours %d minutes' % (now.hour, now.minute))

7. Greetings/ leave

To greet your voice assistant, you’ll be able to simply say “hello Sofia” or once you realized that your work is finished and you’re now not visiting use that simply says “shutdown Sofia”.

How it works: If you have said the word hello in your command, then looking at the time of the day, the bot will greet the user. If the time is over noon, the bot will respond “Hello Sir. Good afternoon”. And once you give the command as shutdown, sys. exit() are going to be called to terminate the program.

#Greet Sofia
    elif 'hello' in command:
        day_time = int(strftime('%H'))
        if day_time < 12:
            sofiaResponse('Hello Sir. Good morning')
        elif 12 <= day_time < 18:
            sofiaResponse('Hello Sir. Good afternoon')
        else:
            sofiaResponse('Hello Sir. Good evening')#to terminate the program
elif 'shutdown' in command:
     sofiaResponse('Bye bye Sir. Have a nice day')
     sys.exit()

8. Play you a song on the VLC media player

This feature allows your voice bot to play your required song on the VLC media player. The user will say “Sofia are you able to play a song for me”, the assistant will ask “What song shall I play Sir?”. As soon as you take the name of the song that you wanna hear the Sofia will download it by searching from Youtube to your local drive and with the help of your favorite VLC media player, she plays that. Thus the previously downloaded song will automatically get deleted whenever you request a replacement song.

How it works: If you have said the phrase play me a song in your command, then it’ll ask you what video song to play. The song you’ll evoke is searched on youtube.com, If found then the song goes to be downloaded in your local directory using pythons youtube_dl library.

 Now if the next time you enkindle the opposite song, the local directory goes to be flushed and a novel song is downloaded during this directory.

elif 'play me a song' in command:
        path = '/Users/shivani/Documents/videos/'
        folder = path
        for the_file in os.listdir(folder):
            file_path = os.path.join(folder, the_file)
            try:
                if os.path.isfile(file_path):
                    os.unlink(file_path)
            except Exception as e:
                print(e)sofiaResponse('What song shall I play Sir?')mysong = myCommand()
        if mysong:
            flag = 0
            url = "https://www.youtube.com/results?search_query=" + mysong.replace(' ', '+')
            response = urllib2.urlopen(url)
            html = response.read()
            soup1 = soup(html,"lxml")
            url_list = []
            for vid in soup1.findAll(attrs={'class':'yt-uix-tile-link'}):
                if ('https://www.youtube.com' + vid['href']).startswith("https://www.youtube.com/watch?v="):
                    flag = 1
                    final_url = 'https://www.youtube.com' + vid['href']
                    url_list.append(final_url)url = url_list[0]
            ydl_opts = {}os.chdir(path)
            with youtube_dl.YoutubeDL(ydl_opts) as ydl:
                ydl.download([url])
            vlc.play(path)if flag == 0:
                sofiaResponse('I have not found anything in Youtube ')

9. Change desktop wallpaper.

You guys can even change your desktop wallpaper using this feature. after you say something like “change wallpaper” or “Sofia please change wallpaper” the bot will download random wallpaper from Unsplash. the command sets it as your desktop background.

How it works: If the command asked by you is for changing the wallpaper what the bot did is it downloads any random picture from unsplash.com, saves it within the local directory, and using subprocess. call() it will set that image as your desktop wallpaper. I’ve got used Unsplash API to induce access to its content. Now if the subsequent time you ask to alter the wallpaper again, your local directory is flushed and a replacement wallpaper is going to be downloaded there in the directory.

elif 'change wallpaper' in command:
        folder = '/Users/shivani/Documents/wallpaper/'
        for the_file in os.listdir(folder):
            file_path = os.path.join(folder, the_file)
            try:
                if os.path.isfile(file_path):
                    os.unlink(file_path)
            except Exception as e:
                print(e)
        api_key = 'fd66364c0ad9e0f8aabe54ec3cfbed0a947f3f4014ce3b841bf2ff6e20948795'
        url = 'https://api.unsplash.com/photos/random?client_id=' + api_key #pic from unspalsh.com
        f = urllib2.urlopen(url)
        json_string = f.read()
        f.close()
        parsed_json = json.loads(json_string)
        pics = parsed_json['urls']['full']
        urllib.urlretrieve(pics, "/Users/shivani/Documents/wallpaper/a") # Location where we download the image to.
        subprocess.call(["killall Dock"], shell=True)
        sofiaResponse('wallpaper changed successfully')

10. Tells you the latest news feeds.

Sofia can even tell you the most recent news update. The user just must say “Sofia what is the highest news for today ?” or “hey dm me the current affairs”.

How it works: If you have got said the phrase news for today in your command then it’ll scrape data using Beautiful Soup from Google News RSS() and skim it for you. For convenience, I’ve got a set number of stories limit to fifteen.

elif 'news for today' in command:
        try:
            news_url="https://news.google.com/news/rss"
            Client=urlopen(news_url)
            xml_page=Client.read()
            Client.close()
            soup_page=soup(xml_page,"xml")
            news_list=soup_page.findAll("item")
            for news in news_list[:15]:
                sofiaResponse(news.title.text.encode('utf-8'))
        except Exception as e:
                print(e)

11. Replies for almost everything you ask.

Your bot can fetch details of just about anything you ask her. Like “Sofi I want to know about different Constitutional laws in one go”. So as you’ll be able to see you’ll be able to ask about almost anything.

How it works: If you’ve got said the phrase tell me about in your command then it’ll seek for the keyword within the user command using re. search(). Using pythons Wikipedia library it’ll explore that topic and extract the primary 500 characters(if you don’t specify the limit the bot will read the full page for you). Wikipedia may be a Python library that creates it easy to access and parse data from Wikipedia.

elif 'tell me about' in command:
        reg_exp = re.search('just search and tell me about (.*)', command)
        try:
            if reg_exp:
                topic = reg_ex.group(1)
                ny = wikipedia.page(topic)
                sofiaResponse(ny.content[:500].encode('utf-8'))
        except Exception as e:
                sofiaResponse(e)

Let’s put everything together and the final code will be:

import speech_recognition as sr
import os
import sys
import re
import webbrowser
import smtplib
import requests
import subprocess
from pyowm import OWM
import youtube_dl
import vlc
import urllib
import urllib2
import json
from bs4 import BeautifulSoup as soup
from urllib2 import urlopen
import wikipedia
import random
from time import strftimedef sofiaResponse(audio):
    "speaks audio passed as argument"
    print(audio)
    for line in audio.splitlines():
        os.system("say " + audio)def myCommand():
    "listens for commands"
    r = sr.Recognizer()
    with sr.Microphone() as source:
        print('Say something...')
        r.pause_threshold = 1
        r.adjust_for_ambient_noise(source, duration=1)
        audio = r.listen(source)
    try:
        command = r.recognize_google(audio).lower()
        print('You asked: ' + command + 'n')
    except sr.UnknownValueError:
        print('....')
        command = myCommand();
    return commanddef assistant(command):
    if 'open reddit' in command:
        reg_ex = re.search('open reddit (.*)', command)
        url = 'https://www.reddit.com/'
        if reg_ex:
            subreddit = reg_ex.group(1)
            urls = urls + 'r/' + subreddit
        webbrowser.open(urls)
        sofiaResponse('Sure Sir,I am displaying the available Reddit content on the screen..')elif 'shutdown' in command:
        sofiaResponse('Bye bye Sir. Have a nice day')
        sys.exit()#open website
    elif 'open' in command:
        reg_ex = re.search('open (.+)', command)
        if reg_ex:
            domain = reg_ex.group(1)
            print(domain)
            url = 'https://www.' + domain
            webbrowser.open(url)
            sofiaResponse('Sir, The website you are requesting to open is available in your screen now.')
        else:
            pass#greetings
    elif 'hello' in command:
        day_time = int(strftime('%H'))
        if day_time < 12:
            sofiaResponse('Hello Sir. Good morning')
        elif 12 <= day_time < 18:
            sofiaResponse('Hello Sir. Good afternoon')
        else:
            sofiaResponse('Hello Sir. Good evening')elif 'help me' in command:
        sofiaResponse("""
        You can use these commands and I'll help you out:1. Open Reddit subreddit: Opens the subreddit in the default browser.
        2. Open xyz.com: replace xyz with any website name
        3. Send email/email: Follow-up questions such as recipient name, the content will be asked in order.
        4. Current weather in {cityname}: Tells you the current condition and temperature
        5. Hello
        6. play me a video: Plays song on your VLC media player
        7. change the wallpaper: Change desktop wallpaper
        8. news for today: reads top news of today
        9. time: Current system time
        10. top stories from google news (RSS feeds)
        11. tell me about xyz : tells you about xyz
        """)#joke
    elif 'joke' in command:
        res = requests.get(
                'https://icanhazdadjoke.com/',
                headers={"Accept":"application/json"})
        if res.status_code == requests.codes.ok:
            sofiaResponse(str(res.json()['joke']))
        else:
            sofiaResponse('oops!I ran out of jokes')#top stories from google news
    elif 'news for today' in command:
        try:
            news_url="https://news.google.com/news/rss"
            Client=urlopen(news_url)
            xml_page=Client.read()
            Client.close()
            soup_page=soup(xml_page,"xml")
            news_list=soup_page.findAll("item")
            for news in news_list[:15]:
                sofiaResponse(news.title.text.encode('utf-8'))
        except Exception as e:
                print(e)#current weather
    elif 'current weather' in command:
        reg_ex = re.search('current weather in (.*)', command)
        if reg_ex:
            city = reg_ex.group(1)
            owm = OWM(API_key='ab0d5e80e8dafb2cb81fa9e82431c1fa')
            obs = owm.weather_at_place(city)
            w = obs.get_weather()
            k = w.get_status()
            x = w.get_temperature(unit='celsius')
            sofiaResponse('Current weather in %s is %s. The maximum temperature is %0.2f and the minimum temperature is %0.2f degree celcius' % (city, k, x['temp_max'], x['temp_min']))#time
    elif 'time' in command:
        import datetime
        now = datetime.datetime.now()
        sofiaResponse('Current time is %d hours %d minutes' % (now.hour, now.minute))elif 'email' in command:
        sofiaResponse('Who is the recipient?')
        recipient = myCommand()
        if 'shivi' in recipient:
            sofiaResponse('What should I say to him?')
            content = myCommand()
            mail = smtplib.SMTP('smtp.gmail.com', 587)
            mail.ehlo()
            mail.starttls()
            mail.login('your_email_address', 'your_password')
            mail.sendmail('sender_email', 'receiver_email', content)
            mail.close()
            sofiaResponse('Email has been sent successfuly. You can check your inbox.')
        else:
            sofiaResponse('I don't know what you mean!')#launch any application
    elif 'launch' in command:
        reg_ex = re.search('launch (.*)', command)
        if reg_ex:
            appname = reg_ex.group(1)
            appname1 = appname+".app"
            subprocess.Popen(["open", "-n", "/Applications/" + appname1], stdout=subprocess.PIPE)sofiaResponse('I have launched the desired application')#play youtube song
    elif 'play me a song' in command:
        path = '/Users/shivani/Documents/videos/'
        folder = path
        for the_file in os.listdir(folder):
            file_path = os.path.join(folder, the_file)
            try:
                if os.path.isfile(file_path):
                    os.unlink(file_path)
            except Exception as e:
                print(e)sofiaResponse('What song shall I play Sir?')
        mysong = myCommand()
        if mysong:
            flag = 0
            url = "https://www.youtube.com/results?search_query=" + mysong.replace(' ', '+')
            response = urllib2.urlopen(url)
            html = response.read()
            soup1 = soup(html,"lxml")
            url_list = []
            for vid in soup1.findAll(attrs={'class':'yt-uix-tile-link'}):
                if ('https://www.youtube.com' + vid['href']).startswith("https://www.youtube.com/watch?v="):
                    flag = 1
                    final_url = 'https://www.youtube.com' + vid['href']
                    url_list.append(final_url)url = url_list[0]
            ydl_opts = {}os.chdir(path)
            with youtube_dl.YoutubeDL(ydl_opts) as ydl:
                ydl.download([url])
            vlc.play(path)if flag == 0:
                sofiaResponse('I have not found anything in Youtube ')#change wallpaper
    elif 'change wallpaper' in command:
        folder = '/Users/shivani/Documents/wallpaper/'
        for the_file in os.listdir(folder):
            file_path = os.path.join(folder, the_file)
            try:
                if os.path.isfile(file_path):
                    os.unlink(file_path)
            except Exception as e:
                print(e)
        api_key = 'fd66364c0ad9e0f8aabe54ec3cfbed0a947f3f4014ce3b841bf2ff6e20948795'
        url = 'https://api.unsplash.com/photos/random?client_id=' + api_key #pic from unspalsh.com
        f = urllib2.urlopen(url)
        json_string = f.read()
        f.close()
        parsed_json = json.loads(json_string)
        photo = parsed_json['urls']['full']
        urllib.urlretrieve(photo, "/Users/shivani/Documents/wallpaper/a") # Location where we download the image to.
        subprocess.call(["killall Dock"], shell=True)
        sofiaResponse('wallpaper changed successfully')#askme anything
    elif 'tell me about' in command:
        reg_exp = re.search('just tell me about (.*)', command)
        try:
            if reg_exp:
                topic = reg_ex.group(1)
                ny = wikipedia.page(topic)
                sofiaResponse(ny.content[:500].encode('utf-8'))
        except Exception as e:
                print(e)
                sofiaResponse(e)sofiaResponse('Hi User, I am Sofia and I am your personal voice assistant based on deep learning and nlp, Please give a command or say "help me" and I will tell you what all I can do for you.')#loop to continue executing multiple commands
while True:
    assistant(myCommand())

Conclusion

By writing this simple and coolest code you will create a voice-based chatting assistant. Except for these features, you’ll also include many alternative features in your voice assistant. Please note that after you begin executing your program, be loud and clear while you’re interacting with the voice assistant because it’s going to happen that if your voice isn’t clear your voice assistant might not be ready to interpret you properly. It will increase its efficiency with the amount of usage. Then, this data may be utilized to see user preferences and tastes, which may be a long-term point for creating a home smarter. I hope you guys have found it enjoyable and meaningful. Share your thoughts/comments/doubts related to the chatbot creation within the comment section.

The media shown in this article are not owned by Analytics Vidhya and are used at the Author’s discretion.

About the Author

Our Top Authors

  • Analytics Vidhya
  • Guest Blog
  • Tavish Srivastava
  • Aishwarya Singh
  • Aniruddha Bhandari
  • Abhishek Sharma
  • Aarshay Jain

Download Analytics Vidhya App for the Latest blog/Article

Leave a Reply Your email address will not be published. Required fields are marked *