Voice and Text Chatbot designed for WhatsApp and Telegram

Instructions for Running Locally WhatsApp Bot Application

To execute the WhatsApp Bot application on your local machine, kindly follow the subsections below in the specified order.

Necessary Components

To locally run the app successfully, please ensure that you have these components ready:

• WhatsApp account


• Free Twilio account (no credit card needed; comes with 15 USD credit)
  • For more instructions about creating a Twilio account please refer to this page.
• IBM Cloud account (credit card required; no payment - all used services are free)
  Create an instance for the free versions of the following services (you can select Dallas as the location):
  • Speech-to-Text
  • Text-to-Speech
  • Watson Assistant (please see the section Watson Services)
  • Object Storage
  • Cloudant
• ngrok downloaded and installed
  • Also, a free ngrok account

ngrok

ngrok is a third-party software that can be used to expose "local servers to the public internet over secure tunnels" (ngrok documentation). In this specific context of our chatbot, we use Flask to create the web app that handles all of the messaging between Watson Assistant and WhatsApp users, but by default, Flask starts a local host to deploy the application. ngrok provides a random public url that can forward requests to a local host.

Environment Set-up

After acquiring the necessary components and cloning our GitHub repository, you would need to create a Python 3 virtual environment (using virtualenv for example), and install the dependencies. Below you can see how to do these steps:

In the project local folder in your computer:
• Creating the virtual environment:
  • Mac/Linux/Windows $ python3 -m venv <name_of_virtualenv>
• Activating the virtual environment:
  • Mac/Linux (bash/zsh) - $ source <venv>/bin/activate
  • Windows (cmd.exe) - C:\> <venv>\Scripts\activate.bat
  • Windows (PowerShell) - PS C:\> <venv>\Scripts\Activate.ps1
• Installing the dependencies:
  • Should work on Windows - $ pip install -r requirements_windows_local.txt
  • Should work on Mac - $ python3 -m pip install -r requirements.txt

In the src directory, you should also create a .env file. This will store the API keys and the code will search for them.

• Mac/Linux - $ touch .env
• Windows - $ type nul > your_file.txt

In the .env file, you can store API keys with these exact names. We do this to protect sensitive information, namely, API keys and other IDs. STT, TTS, Watson Assistant, COS and Cloudant API keys can be found on the IBM Cloud website (by going to Resource List and then opening each service). For the Watson Assistant ID, you first have to launch Watson Assistant (this can be done from the resource list) and create an assistant. After that, under Assistant Settings the assistant ID will be there.

These are the API keys that you will need to run the app:

• TWILIO_ACCOUNT_SID ¹
• TWILIO_AUTH_TOKEN ¹
• TWILIO_SANDBOX_NUMBER ²
• WA_API_KEY ³
• WA_ID ³
• WA_SERVICE_URL ³
• COS_API_KEY_ID ⁴
• COS_BUCKET ⁴
• COS_BUCKET_LINK ⁴
• COS_ENDPOINT ⁴
• COS_INSTANCE_CRN ⁴
• IBM_CLOUDANT_URL ⁴
• IBM_CLOUDANT_APIKEY ⁴
• IBM_CLOUDANT_DATABASE ⁴
• STT_API_KEY ³
• STT_SERVICE_URL ³
• STT_MODEL ³
• TTS_API_KEY ³
• TTS_DEFAULT_VOICE ³
• TTS_SERVICE_URL ³
• DEFAULT_ERROR_MESSAGE ⁵

¹Avaliable on Twilio Console
²Avaliable on Twilio Sandbox Settings
³Refer to Watson Services page
⁴Refer to Data Storage page
⁵A default message for reply to media unsupported. Spaces must be replaced by an underscore.

We have curated a page which contains detailed instructions on the process of creating a Twilio Sandbox account and obtaining the essential API keys. To access the page, kindly follow this link.

The following image is an example of how the .env file should be formatted:

Running the App

First, run the command for your respective OS on the command line (in the folder ngrok were installed):

• Mac/Linux - $ ./ngrok http 8080
• Windows - $ ngrok http 8080

After this, an ngrok interface, similar to this, should appear:

There would be two "Forwarding" sections of the interface - you can copy the https://____.ngrok.io url.

Configuring Twilio

After completing the previous steps, we are ready to configure Twilio. We have prepared a special page with more information on how to create a Twilio Sandbox account. You can access it by clicking here.

Now, you would open up the Twilio console:

Then navigate to your WhatsApp Sandbox settings:

1. Messaging
2. Settings
3. WhatsApp sandbox settings

With the to see a screen just like this one:

Then you should copy and paste your ngrok url in the box "WHEN A MESSAGE COMES IN". Append the path "/chatbot-message" to your ngrok url too - so the final url that goes in that box should be similar to https://____.ngrok.io/chatbot-message. All other settings can be left as the default.

Now, in the src/ folder:

1. Mac/Linux - $ gunicorn -b :8080 whatsapp:app
   Windows - $ waitress-serve --listen=*:8080 whatsapp:app


2. Send the "join ..." message from your WhatsApp account to the Twilio number that you see on the sandbox page - you should see a sentence that looks like "Invite your friends to your Sandbox. Ask them to send a WhatsApp message to..."


3. If you refresh the sandbox page, you should see your number under "Sandbox Participants" on the sandbox page

After this is done, you now should be able to interact with the bot on WhatsApp. Yeeey!

You can monitor bot activity through Cloudant database and access the media files sent and received by the bot in the Cloud Object Storage bucket you create. You can also view a reduced version of the conversation history in the Twilio Console, going to the Overview page (Messaging -> Overview), under "Recent Messages".

Of course, there should also be a working Watson Assistant dialog skill attached to your Watson Assistant assistant (that correlates with WATSON_ASSISTANT_ID) in order for you to receive sensible responses from the app.

Important Note: If you terminate the ngrok process and run it again, you will have to update the ngrok url in the box "WHEN A MESSAGE COMES IN" on the Twilio Sandbox page - this is because ngrok creates a random url every time you run $ ngrok http 8080.

Handling Errors

The code has try-except blocks in many places, so as to catch Twilio and Flask specific errors. Most errors should produce a debug output within the terminal. Some errors might be visible from the Twilio console, and Twilio might email you when errors occur.

When errors occur, a good first step is to identify if it has do with the path that Twilio is forwarding messages too. If it is, the ngrok interface should show error codes of 4XX. If it is not, it is likely that the error occurred within the Flask app.