TJBot - Build a Talking Robot With Watson Conversation

45,734

113

138

About: TJBot is an open source paper robot that I designed to help you learn programming Artificial Intelligence in a fun way! You can laser cut or 3D print him, then use one of his recipes to bring him to life! I ...

This instructable guides you through connecting a Raspberry Pi to Watson conversation services and making a talking robot. You will use (1) Watson Speech to Text to convert your voice to text, (2) Watson Assistant to process the text and calculate a response, and (3) Watson Text to Speech to talk the response back.

Step 1: Parts

Step 2: Prepare Your Pi

If you have used Raspberry Pi before, install Node.js and go to the next step. Otherwise, follow the instructions below to set up your Pi:


Getting Started with Your Pi

Raspberry Pi is similar to a full computer, which means you need a monitor, mouse, and keyboard for it. If you have a TV around, you can connect your Pi to your TV via a HDMI cable. In most of the Pi kits, the SD card is already preloaded with an image of the Raspberry Pi Operating System. You need to put the SD card in the Pi, turn the Pi ON and follow the instructions on screen to complete the installation of the operating system. If you have problems setting up your Pi, you can troubleshoot here.


Install Packages

Open a terminal application on the Pi and execute the following commands to install the latest version of Node.js and npm (Node Package Manager). You need these packages later to run your code.

curl -sL http://ibm.biz/tjbot-bootstrap | sudo sh -

Step 3: Assemble TJBot

Here is a quick video of how to fold the kit. The detailed instructions are available on another instructable: Build TJBot out of Cardboard

Once your TJBot is ready, plug in your USB microphone and the speaker.

Depending on which audio output source you are using with your Pi (HDMI, 3.5mm audio jack, Bluetooth, USB speaker), you may need to set the audio config.

HDMI/ 3.5mm Audio Jack

If you use HDMI or 3.5mm audio jack, you may need to set the audio config. To do this, go to the terminal and open up raspi-config.

 sudo raspi-config  

This will open up the Raspberry Pi configuration screen:

Select "Advanced Options" and press Enter, then select "Audio" and press Enter. Choose the correct channel for the output audio. If you have connected an external speaker to the audio jack, you should select 3.5mm jack.

USB Speaker

If you have a USB audio, you need to update your /usr/share/alsa/alsa.config to set the USB audio as the default device. Begin with running the following command to make sure your USB is connected and listed there.

 lsusb 

Next is to detect the card number of your USB audio.

aplay -l 

Take a note of the card number associated with your USB Audio. Then go to the alsa.config file to set it as default.

sudo nano /usr/share/alsa/alsa.conf

Look for

defaults.ctl.card 0
defaults.pcm.card 0

and update the card number (0 here) to the card number of your USB audio.

Different versions of Raspberry Pi OS may need a different setup. If you have problem with your USB setup, check out this guide to troubleshoot.

Bluetooth Speaker

If you plan to use a Bluetooth speaker, you may need to install the Raspberry Pi Bluetooth software. Here is a good tutorial to set it up: Play audio on a bluetooth speaker with Raspberry Pi 3

Step 4: Download the Sample Code

The source code is available at github. Download or clone the code and execute the following commands from a terminal to install its dependencies..

Here are the instructions for how to clone a repository from github if you have not done that before.


git clone https://github.com/ibmtjbot/tjbot.git

cd tjbot/recipes/conversation
npm install

Pro tip: if you get an error for npm install that says npm not found, you should first install npm on your machine.

This is the command line to install npm

sudo apt-get install npm

Step 5: Update Your Credentials

In this step, we help you get API access to three Watson conversation services.

You will use three types of Watson services in this recipe: (1) Watson Speech to Text, (2) Watson Assistant, (3) Watson Text to Speech. You need to copy your credentials for all these services.

Create instances of the Watson Assistant, Watson Speech to Text, and Watson Text to Speech services and note the authentication credentials.

Import the workspace-sample.json file into the Watson Assistant service and note the workspace ID.

Make a copy the default configuration file and update it with the Watson service credentials and the conversation workspace ID.

$ cp config.default.js config.js
$ nano config.js
<enter your credentials and the conversation workspace ID in the specified places>

Step 6: Run the Code

Now, you are ready to talk to your TJBot!

Open a terminal and execute the following command:

sudo node conversation.js

Watson conversation uses intents to label the purpose of a sentence. For example when you ask TJBot "Please introduce yourself", the intent is to make an introduction. You can add your own new intents in the Conversation editor, but for now, we have started you off with a few intents:

  • Introduction. You can say phrases such as "Watson, please introduce yourself", "Watson, who are you", and "Watson, can you introduce yourself"
  • Joke. You can ask "Watson, please tell me a joke" or "Watson, I would like to hear a joke".

For a complete list, check the content of workspace-sample.json

An attention word is used so TJBot knows you are talking to him. The default attention word is 'Watson', but you can change it in config.js as follows. Update the configuration file to change the robot name in tjConfig section:

// set up TJBot's configuration

exports.tjConfig = {

log: {   level: 'verbose'    },

robot: {   name: 'tee jay bot'  }

};

You can change the 'name' to whatever you would like to call your TJBot. In addition, if you change the gender to 'female', TJBot will use a female voice to speak to you!

Enjoy! Don't forget to share a picture/video of your setup #TJBot! :-)

Troubleshoot

If you see TJBot's response on the terminal but don't hear TJBot talking, there is a good chance that one of these two things has happened: (1) The audio output is directed to a wrong channel (you can fix it from raspi-config), (2) your sound modules are blocked. In that case, go to /etc/modprobe.d/ and remove blacklist-rgb-led.conf

Then run the following command:

sudo update-initramfs -u

Reboot and confirm the "snd" modules are running by executing the command "lsmod". This should solve the problem.

lsmod

5 People Made This Project!

Recommendations

  • Arduino Contest 2019

    Arduino Contest 2019
  • Classroom Science Contest

    Classroom Science Contest
  • Gardening Contest

    Gardening Contest

138 Discussions

0
None
CristianC172

Question 2 months ago

I can't talk with tjbot, I only have the TextToSpeach.
pi@raspberrypi:~/tjbot/recipes/conversation $ sudo node conversation.js
verbose: TJBot initializing microphone
verbose: TJBot initializing LED
verbose: TJBot initializing servo motor on PIN 7
verbose: TJBot initializing assistant service
verbose: TJBot initializing speech_to_text service
verbose: TJBot initializing text_to_speech service
verbose: TJBot initializing visual_recognition service
info: Hello from TJBot! My name is Richi.
verbose: TJBot library version v1.5.1
You can ask me to introduce myself or tell you a joke.
Try saying, "Richi, please introduce yourself" or "Richi, what can you do?"
You can also say, "Richi, tell me a joke!"
verbose: TJBot initializing microphone
pi@raspberrypi:~/tjbot/recipes/conversation $

1 answer
0
None
zacaintmynameCristianC172

Answer 5 weeks ago

is the problem that after verbose: TJBot initializing microphone, the command ends? I am facing the exact same problem!

0
None
CristianC172

Question 8 weeks ago

Hello, can I use other microphone? Or maybe I Can not use the USB microphone? if maybe exist a form that i can skip the microphone
Thanks

0
None
Ricardo Zaponi

Question 2 months ago

Hello, I have the following problem.



Error.JPG
0
None
CristianC172

Question 2 months ago on Step 6

Hi, I have other problem:
I can't talk or write to Tjbot, when a execute te command "sudo node conversation.js"
"pi@raspberrypi:~/tjbot/recipes/conversation $ sudo node conversation.js
verbose: TJBot initializing microphone
verbose: TJBot initializing LED
verbose: TJBot initializing servo motor on PIN 7
verbose: TJBot initializing Camera
verbose: TJBot initializing assistant service
verbose: TJBot initializing speech_to_text service
verbose: TJBot initializing text_to_speech service
verbose: TJBot initializing visual_recognition service
info: Hello from TJBot! My name is Watson.
verbose: TJBot library version v1.5.1
You can ask me to introduce myself or tell you a joke.
Try saying, "Watson, please introduce yourself" or "Watson, what can you do?"
You can also say, "Watson, tell me a joke!"
verbose: TJBot initializing microphone"
pi@raspberrypi:~/tjbot/recipes/conversation $

he dies, so, in other words, it comes out

0
None
CristianC172

2 months ago on Step 5

Hello, my name's Cristian and i need you :c
maybe can you be more explicit with "Where're go the line of code"?
because i have a problem that say:
module.js:549
throw err;
^
Error: Cannot find module 'tjbot'
at Function.Module._resolveFilename (module.js:547:15)
at Function.Module._load (module.js:474:25)
at Module.require (module.js:596:17)
at require (internal/module.js:11:18)
at Object.<anonymous> (/home/pi/tjbot/recipes/conversation/conversation.js:17:13)
at Module._compile (module.js:652:30)
at Object.Module._extensions..js (module.js:663:10)
at Module.load (module.js:565:32)
at tryModuleLoad (module.js:505:12)
at Function.Module._load (module.js:497:3)

Thanks

4 replies
0
None
CristianC172LSPica

Reply 2 months ago

¡Hey! Thank you very much, really that, ¡Wow! thank you

"pi@raspberrypi:~/tjbot/recipes/conversation $ sudo npm install
> websocket@1.0.28 install /home/pi/tjbot/recipes/conversation/node_modules/websocket
> (node-gyp rebuild 2> builderror.log) || (exit 0)
sh: 1: cannot create builderror.log: Permission denied
npm notice created a lockfile as package-lock.json. You should commit this file.
added 75 packages from 130 contributors and audited 139 packages in 41.97s
found 0 vulnerabilities
pi@raspberrypi:~/tjbot/recipes/conversation $ sudo node conversation.js
verbose: TJBot initializing microphone
verbose: TJBot initializing LED
verbose: TJBot initializing servo motor on PIN 7
verbose: TJBot initializing Camera
verbose: TJBot initializing assistant service
verbose: TJBot initializing speech_to_text service
verbose: TJBot initializing text_to_speech service
verbose: TJBot initializing visual_recognition service
info: Hello from TJBot! My name is Watson.
verbose: TJBot library version v1.5.1
You can ask me to introduce myself or tell you a joke.
Try saying, "Watson, please introduce yourself" or "Watson, what can you do?"
You can also say, "Watson, tell me a joke!"
verbose: TJBot initializing microphone"

I need now fix the microphone but, not problem, thank you

0
None
TJBotCristianC172

Reply 2 months ago

Did you run npm install in step 4? That command should install all the modules that you need. Can you confirm that you are able to successfully run npm install in the conversation directory? I'd suggest you delete your current node_modules directory (removes the previous installation) and run npm install again.

0
None
CristianC172TJBot

Reply 2 months ago

HI, me again
I have the same problem...
"pi@raspberrypi:~/tjbot/recipes/conversation $ cp config.default.js config.js
pi@raspberrypi:~/tjbot/recipes/conversation $ npm install
(node:2701) [DEP0022] DeprecationWarning: os.tmpDir() is deprecated. Use os.tmpdir() instead.
pi@raspberrypi:~/tjbot/recipes/conversation $ cp config.default.js config.js
pi@raspberrypi:~/tjbot/recipes/conversation $ nano config.js
pi@raspberrypi:~/tjbot/recipes/conversation $ sudo node conversation.js

verbose: TJBot initializing microphone
verbose: TJBot initializing LED
verbose: TJBot initializing servo motor on PIN 7
verbose: TJBot initializing Camera
verbose: TJBot initializing assistant service
module.js:549
throw err;
^
Error: Cannot find module 'extend'
at Function.Module._resolveFilename (module.js:547:15)
at Function.Module._load (module.js:474:25)
at Module.require (module.js:596:17)
at require (internal/module.js:11:18)
at Object.<anonymous> (/home/pi/tjbot/recipes/conversation/node_modules/tjbot/node_modules/watson-developer-cloud/assistant/v1.js:27:14)
at Module._compile (module.js:652:30)
at Object.Module._extensions..js (module.js:663:10)
at Module.load (module.js:565:32)
at tryModuleLoad (module.js:505:12)
at Function.Module._load (module.js:497:3)
at Module.require (module.js:596:17)
at require (internal/module.js:11:18)
at TJBot._createServiceAPI (/home/pi/tjbot/recipes/conversation/node_modules/tjbot/lib/tjbot.js:352:27)
at new TJBot (/home/pi/tjbot/recipes/conversation/node_modules/tjbot/lib/tjbot.js:91:18)
at Object.<anonymous> (/home/pi/tjbot/recipes/conversation/conversation.js:36:10)
at Module._compile (module.js:652:30)"

and when I execute "npm install" this's the answer:

(node:2701) [DEP0022] DeprecationWarning: os.tmpDir() is deprecated. Use os.tmpdir() instead.

0
None
Aurovindhya

4 months ago

The sudo node conversation.js ran.. And the output msg was Hello from TJBot!....... Etc..
Then when I gave the voice input.. There was no response and after sometime..this error popped..
Unable to retrieve TTS voices message=socket hang up, stack=Error: socket hang up..
Please reply asap.. And help me..
Thanks in advance..

1 reply
0
None
TJBotAurovindhya

Reply 3 months ago

Maybe TJBot is hearing an intent that he doesn't understand? Did you say Watson and use one of the intents in the workspace-sample.json to talk to TJBot? I just updated the instructions for that section. Perhaps that solves the problem.

0
None
rkundra

Question 3 months ago on Step 3

what if I use headphones with an inbuilt microphone?

0
None
JLG20

6 months ago

Hi eveyone. I need help, I installed TJBOT into my Raspberry PI 3 (and node js 9), when execute sudo node conversation.js, I receive the following message:

verbose: TJBot initializing microphone
verbose: TJBot initializing LED
verbose: TJBot initializing servo motor on PIN 7
verbose: TJBot initializing assistant service
verbose: TJBot initializing speech_to_text service
verbose: TJBot initializing text_to_speech service
verbose: TJBot initializing visual_recognition service
/home/pi/tjbot/recipes/conversation/node_modules/tjbot/lib/tjbot.js:542
throw new Error(
^
Error: Missing authentication credentials for tone_analyzer service: username/password or apikey are required.
at TJBot._createServiceAPI (/home/pi/tjbot/recipes/conversation/node_modules/tjbot/lib/tjbot.js:542:19)
at new TJBot (/home/pi/tjbot/recipes/conversation/node_modules/tjbot/lib/tjbot.js:116:18)
at Object.<anonymous> (/home/pi/tjbot/recipes/conversation/conversation.js:40:10)
at Module._compile (internal/modules/cjs/loader.js:654:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:665:10)
at Module.load (internal/modules/cjs/loader.js:566:32)
at tryModuleLoad (internal/modules/cjs/loader.js:506:12)
at Function.Module._load (internal/modules/cjs/loader.js:498:3)
at Function.Module.runMain (internal/modules/cjs/loader.js:695:10)
at startup (internal/bootstrap/node.js:201:19)
at bootstrapNodeJSCore (internal/bootstrap/node.js:516:3)

2 replies
0
None
elvickJLG20

Reply 6 months ago

I have the same error

0
None
ElmarKleikampelvick

Reply 5 months ago

Hi JLG20

Edit with a text editor ...~/Desktop/tjbot/recipes/conversation/config.js
You have to enter the credential for Watson Visual Recognition (apikey, and not the url)
Check in Dashboard of IBM Bluemix website what skills you already have, if it´s not there activate Watson Visual Recognition and look for the apikey there
In this last line change api_key to apikey (api_key is faulty here)
If yo´re done it should work.

0
None
thilom1

6 months ago

Hello,
I build my TJBot like instructed .
It seems to work, he did not answear.
I cn tell him something he reconizes the words, but he dont answear at the USB Speaker.
My Config is:
USB - Speaker, USB Mikrophone,
can someone help me?
Thanks Menne
here is the Console-Log:

erbose: TJBot initializing microphone
verbose: TJBot initializing LED
verbose: TJBot initializing servo motor on PIN 7
verbose: TJBot initializing Camera
verbose: TJBot initializing assistant service
verbose: TJBot initializing speech_to_text service
verbose: TJBot initializing text_to_speech service
info: Hello from TJBot! My name is Watson.
verbose: TJBot library version v1.4.0
You can ask me to introduce myself or tell you a joke.
Try saying, "Watson, please introduce yourself" or "Watson, what can you do?"
You can also say, "Watson, tell me a joke!"
verbose: TJBot initializing microphone
info: TJBot heard: what %HESITATION telling a joke
info: TJBot heard: what do you tell me a joke
info: TJBot heard: what can you do
info: TJBot heard: what please introduce your
info: TJBot heard: what
info: TJBot heard: please interviews you also
info: TJBot heard: and what he did leave the joke

1 reply
0
None
ElmarKleikampthilom1

Reply 5 months ago

Hi thilom1,

check with aplay -l the parameters of your soundcard (generally it is plughw:1,0)
edit ~/Desktop/tjbot/recipes/conversation/node_modules/tjbot/lib/tjbot.js

look for microphone and edit the parameter there to what aplay -l showed
look for speaker and edit the parameter there to what aplay -l showed

In my settings for the microphone it was 1,0 and the mic worked fine.
But the speaker was set with 0,0 and it didn´t work.
I changed it to 1,0 and everything worked fine.

Good luck

0
None
YuxingW

Question 5 months ago

Hi, can someone help me with the credentials for the TJBot. I can't find the 36 digit username and 12 character password. Can someone tell me where it is. I have an attached photo of what my credential page look like on IBM cloud. Thanks!!!!

CREDENTIAL.jpg
1 answer
0
None
ElmarKleikampYuxingW

Answer 5 months ago

Hi XuxingW,
you need the apikey and the url.
-The apikey is the first line in your credentials
-The url is the last line in your credentials
Edit with a text editor ...~/Desktop/tjbot/recipes/conversation/config.js
If there is only a config.default.js type
cp config.default.js config.js
Normally you see username and password fields in the file
However something has changed with the parameters, not yet implemented in the config.default.js file:
-Replace password with apikey
-Replace username with url
Remember that password is listed on top, then comes username
So after having changed them, url is on the bootom line
Insert the string from line 1 of your credentials to apikey
Insert the string from the last line of your credentials to url
Use the different credentials for conversation, stt, tts

At the end you also have to enter the credential for Watson Visual Recognition (apikey, and not the url)
Check in Dashboard of IBM Bluemix website what skills you already have, if it´s not there activate Watson Visual Recognition and look for the apikey there
In this last line change api_key to apikey (api_key is faulty here)
If yo´re done it should work.