TJBot - Use Your Voice to Control a Light

28,336

65

51

About: TJBot is an open source paper robot that I designed to help you learn programming Artificial Intelligence in a fun way! You can laser cut or 3D print him, then use one of his recipes to bring him to life! I ...

This instructable guides you through connecting a Raspberry Pi to Watson Speech to Text and change the color of a LED through speech. For example, you may say "Turn the light green" to change the color of the LED to green.

Supplies:

Step 1: Parts

Step 2: Prepare Your Pi

If you have used Raspberry Pi before, install Node.js and the Alsa packages and go to the next step. Otherwise, follow the instructions below to set up your Pi:


Getting Started with Your Pi

Raspberry Pi is similar to a full computer, which means you need a monitor, mouse, and keyboard for it. If you have a TV around, you can connect your Pi to your TV via a HDMI cable. In most of the Pi kits, the SD card is already preloaded with an image of the Raspberry Pi Operating System. You need to put the SD card in the Pi, turn the Pi ON and follow the instructions on screen to complete the installation of the operating system. If you have problems setting up your Pi, you can troubleshoot here.

Here is an easy way to setting WiFi up on your Raspberry Pi via the command line


Install Packages

Open a terminal application on the Pi and execute the following commands to install the latest version of Node.js and npm (Node Package Manager). You need these packages later to run your code.

curl -sL http://ibm.biz/tjbot-bootstrap | sudo sh -

Step 3: Assemble TJBot

Here is a quick video of how to fold the kit. The detailed instructions are available on another instructable: Build TJBot out of Cardboard

Once your TJBot is ready, get ready to connect your LED to the Pi Pins.

Attention

It's very important to make sure you connect the LED to the right pins, otherwise you may damage your LED or the Pi.
ALWAYS start with the GND pin. GND is the longest leg of your LED. The LED has a flat side. The GND is the second leg from the flat side. Again if you are unsure, DO NOT connect the LED to the Pi.




Connect the GND to GND pin on the pi (see the picture). Then connect 'Data in', the 4th leg from the flat side, to the Pi's GPIO pin. Lastly, connect the third leg from the flat side to the 3.3V on the Pi.

Step 4: Download the Sample Code

The sample code is available at github. Download or clone the sample code and execute the following commands from a terminal to install its dependencies.

Here are the instructions for how to clone a repository from github if you have not done that before.

git clone <a href="https://github.com/ibmtjbot/tjbot.git"> https://github.com/ibmtjbot/tjbot.git
</a>
cd tjbot/recipes/speech_to_text
npm install

Pro tip: if you get an error for npm install that says npm not found, you should first install npm on your machine.

This is the command line to install npm
sudo apt-get install npm

Step 5: Update Your Credentials

Create an instance of the Watson Speech to Text service and note the authentication credentials.
Make a copy of the default configuration file and update it with the Watson service credentials.

$ cp config.default.js config.js

$ nano config.js

enter your credentials in the specified places>

Step 6: Run the Code

Once done with testing the LED, you can run the main module:

sudo node stt.js

Now talk to your microphone to change the color of the LED.
Say "Turn the light blue" to change the light to blue. You can try other colors: yellow, green, orange, purple, magenta, red, blue, aqua, white). You can either say "Turn the light on" or "Turn the light off"!


Troubleshoot

Is the LED not lighting up or showing the wrong color? No worries, we can fix it.

Follow this link for troubleshooting.

Enjoy! Don't forget to share a picture/video of your setup #TJBot! :-)

Step 7: What's Next?

You can add new colors to your color palette in stt.js. TJBot uses a NeoPixel RGB LED, which means it can show any combination of red, green, and blue.

We have hidden a disco party for you. Find the code for disco party in stt.js and uncomment the code. Now you can ask TJBot to show you the disco lights by saying "Let's have a disco party"!

Try implementing your own TJBot party and share it with us #TJBot!

Once ready to move on, try the next recipe to make TJBot respond to emotions using Watson.

4 People Made This Project!

Recommendations

  • Arduino Contest 2019

    Arduino Contest 2019
  • IoT Challenge

    IoT Challenge
  • Colors of the Rainbow Contest

    Colors of the Rainbow Contest

51 Discussions

0
None
TJBotPatrickStingley

Reply 4 months ago

Correct, if you don't have npm, you should install it first. I should add a tip to the instruction with your suggestion. Thanks for sharing!

0
None
Aarushi Mittal

1 year ago

i do not understand whats wrong. i get the entire code on the console after error: the speech to text returned an error.

also , i have snd modules displayed after i run snd ...

1 reply
0
None
agstew

10 months ago

Hi, IBM Cloud has moved some services (and eventually all) to token-based Identity and Access Management (IAM) authentication. This replaces the user id and password authentication currently used/required by this recipe. Can someone tell me what to do to get this recipe to accept this new form of authentication? Thanks

1 reply
0
None
JacobG105

6 months ago

I don’t know how or where to copy the speech to text file. Also, when I try the cp config.default.js config.js, it say that no file or directory exists. Can you help me?

1 reply
0
None
TJBotJacobG105

Reply 4 months ago

Hi Jacob, you need to download the repository from github: https://github.com/ibmtjbot/tjbot (look for the big green button on the right of the page that says "clone or download", then click or tap and choose "Download Zip". The speech to text file is in tjbot/recipes/speech_to_text Now your config file shouldn't complain! Here is more details: tjbot/recipes/speech_to_text at master ibmtjbot/tjbot

0
None
JacobG105

1 year ago

Hello there! I have a few problems, can you help me?

First off, after you upload your credentials and paste them on to the nano config.js, what do you do?

Second, when I tried to run: sudo node stt.js, it says Syntax error unexpected toke n. I dont know how to fix that.

Thanks!

0
None
Gustavofelix

1 year ago

Hi, everybody!

When I run sudo node stt.js, I receive the info that "TJBot is listening, you may speak now". However, right after the Console gives me the message, the program is closed.

I'm using a USB mic and no Error Message is showed for me.

I already read MauricioM82 comment, but that didn't help either. Is something related to audio configuration too?

Thank You!

0
None
MauricioM82

2 years ago

Hi.

When I run stt.js program it shows the message:
"TJBot is listening, you may speak now."
but don't give me time to speak and close the program right after showing the message. No errors, nothing, just show the message and close the program.

I am using an USB sound card with both input and output. Both arecord and aplay commands works just fine, I am able to record and listen to the audio I've recorded. I am also able to use the voice control to control bb-8.

Any suggestions?

Thank you.

8 replies
0
None
victordibiaMauricioM82

Reply 2 years ago

Hi MauricioM82,

You are correct, the issue is definitely related to the app having difficulty setting up audio. For arecord on command line, do you pass any parameters for your app to work correctly ? For example if your sound input might be listed with an unexpected hardware ID or might need several parameters to work.

To find out your hardware device ID run arecord -L ,and then specify that as a parameter when you instantiate the mic in stt.js.

In the sample apps we provide, we use the mic npm library (

https://www.npmjs.com/package/mic) to record audio. You can provide hardware options when you instantiate your mic to record audio e.g

var micInstance = mic({ 'rate': '44100', 'channels': '2', 'debug': false, 'exitO

nSilence': 6, 'device': 'plughw:1,0' });


A related issue (solved) on recording from a bluetooth and usb mic can be found here

https://github.com/ibmtjbot/tjbot/issues/12

0
None
geneLovictordibia

Reply 2 years ago

Hi Victordibia

First off, thanks for this fun and easy to follow recipe!

I've followed all the steps on the tutorial however i'm getting an error after running the command "sudo node stt.js".

I believe the trouble is derived from raspbian jessie not seeing my usb mic.

The error reads "the speech_to_text service returned an error" and then prints lines of code. It appears right after the console logs "TjBot initializing microphone"

I've ran asplay -l and a usb sound card does not appear. However when i go to the raspberry pi settings > preferences > audio devices> the dropdown shows my usb mic, but when I attempt at making it the default audio, it never saves the change.

I've been stuck on this problem some time, so any help would be greatly appreciated.

I'm running Raspbian 8.0 Jessie.

Thanks for reading and I hope to hear from you soon!

0
None
Charan1591geneLo

Reply 1 year ago

hi ,

I'm facing the same issue

Did you solve it .?

0
None
MauricioM82victordibia

Reply 2 years ago

Great!

Thank you for your help. So, I specified the device when instantiating the mic and it worked, it now recognizes my voice.

The problem now is that the led don't change colors (it's green, and always green).

I did the suggestions on the Troubleshooting part of the article but after reboot I see several "snd" modules loaded, like

snd_usb_audio 118800 3
snd_hwdep 5727 1 snd_usb_audio
snd_usbmidi_lib 19801 1 snd_usb_audio
snd_rawmidi 18516 1 snd_usbmidi_lib
snd_seq_device 3621 1 snd_rawmidi
snd_pcm 75762 1 snd_usb_audio
snd_timer 19288 1 snd_pcm

Any suggestions?

Thank you in advance.

0
None
MauricioM82MauricioM82

Reply 2 years ago

Maybe I found the answer.. I may be using a different type of led.

We have the ground pin and one of each other is for one color.

Does the led used on the article has a special name, or some way I can identify it to purchase in a store?

Thank you.

0
None
MauricioM82victordibia

Reply 2 years ago

I live in Brazil and it's really hard to find the adafruit version or anything similar around here. And shipping from adafruit site to Brazil is way way way to expensive.

Maybe someone could ship a kit from the site to me via regular mail? :D

Thank you for all your help.

0
None
TJBotMauricioM82

Reply 2 years ago

We used a NeoPixel LED. Here is a link to the specs: https://www.adafruit.com/products/1734

0
None
duyhat

2 years ago

Hi,

Thanks for the interesting tutorial and nice recipe!

I have a Common Anode + LED instead of the one used in tutorial so I went ahead and made a quick change to the code to make it work with my LED, if someone wants to try, here is the separated branch in my fork:

https://github.com/dnguyenv/tjbot/tree/common-anod...

Changes:

- Removed rpi-ws281x-native library, use rpio npm lib
- Changed the main tss.js file to work with my LED
- Added my wire diagram (https://github.com/dnguyenv/tjbot/blob/common-anode/recipes/speech_to_text/tts.png)
- Changed README to be relevant to the change

Best - Duy