FindyBot3000 - a Voice Controlled Organizer

49,573

702

128

About: Software engineer and long time hacker, but new to Instructables! Started tinkering with Arduino in highschool building cool little robots.

As any maker can attest, over the years, one collects a lot of parts. Perhaps it all started with a few resistors and LEDs, a breadboard, and an Arduino. But soon there were motors and motor drivers, IR rangefinders and tactile switches. Accelerometers, microphones, heatshrink tubes, potentiometers, relays, hookup wire, all those things salvaged from that printer you took apart... and the list goes on.

Now, organizing such a collection of goodies is no easy task. Some may have turned to baggies, for others, boxes. And then there were those who filled shelves with boxes, filled with baggies, filled with parts. All intricately labelled, yet still so cumbersome to manage.

This Instructable takes a stab at solving the age-old problem of organizing a bunch of things. So without further ado, FindyBot3000.

Supplies:

Step 1: The Problem

'nuff said.

Step 2: Table of Contents

This project is an Internet-of-Things organizer, designed for hands-free searching and intuitive voice-controlled management of parts.

This Instructable is organized as follows:

Overview - Describing how the tech is all linked

Prerequisites - Required Tools, Software, Electronics, and Parts Lists

Construction

  • Mechanical - Building the organizer
  • Electrical - Wiring it all up

Basic Testing

Software

  • Download and setup necessary accounts
  • Voice-Control - Linking Google Home to Particle Photon
  • The Cloud - Deploying an Azure Function and SQL Database on Microsoft Azure

Future Plans - New features, animations, debugging techniques, etc.

Step 3: Overview

A broad tech stack is used, ranging from lower-level C++ programming (Arduino-style!) to higher-level C# programming for the (Microsoft) Cloud!

Technologies and Languages

While many makers will be familiar with Google Home/Google Assistant, the other technologies may be unknown. Here's a quick summary:

Human - That's you!

Google Home- This is what you talk to, and turns voice into text. Behind the scenes, Google Home uses Google Assistant, which is really doing all the work. This project can actually work with any device that supports Google Assistant, like an Android phone or tablet.

IFTTT - If-this-then-that, a free online service which allows different apps to communicate. In this project, IFTTT is used to connect Google Home with Particle Photon.

Particle Photon - A cheap WiFi capable microcontroller, programmable Arduino-style! This chip drives the LED display on the organizer. Development with this microcontroller has us working with:

  • C++ (Arduino style)
  • Webhooks - Send/receive real-time data between applications (like Particle Photon to an Azure Function)
  • Particle Dev - The integrated development environment (IDE) which is used to code for the Photon.

Azure Function- A piece of code that runs in the cloud. No virtual machines or web apps to worry about. This does all the heavy lifting in terms of processing text, building SQL queries, and any other work that would be much more difficult or (computationally) expensive to do on a microcontroller like the Photon.

SQL Database - Keeps track of all the parts: names, quantities, locations, search tags, dates, etc. The database is queried to insert, remove, find, or update parts.

In the next step, a sequence diagram shows how all these pieces fit together.

Step 4: The Ebb and Flow

In the UML sequence diagram shown, the flow of each component is laid out. There's a ton of technology behind it all, so it's pretty incredible that FindyBot3000 can find parts within a second. This Instructable won't go into the nitty-gritty technical details of how each step is implemented. Instead, we will focus on building the physical assembly, deploying the necessary software components, and configuring everything to work together.

Now this is a pretty complicated drawing, showing how all the software components talk to each other, and in which order. Follow the arrows, noting which lifelines (vertical lines) they are pointing to, to get a sense of what is happening and when.

You may be wondering -- why don't we send the data from Google Home directly to the Azure Function, instead of first to the Photon? The reason is to provide a single source of debugging information. By logging all commands which are incoming and outgoing from the Photon, we can easily identify at which step in the sequence data loss or a bug may have occurred.

We will come back to this diagram in later steps, but for now, on to construction!

Step 5: Parts Lists

Tools:

  • Jigsaw w/ fine wood blade
  • Drill with 1/16th bit, 1/4" bit, and countersink bit
  • Screwdriver
  • Dremel w/ mutipurpose cutting bit
  • Soldering Iron
  • Solder
  • Scissors
  • Retractable utility knife
  • Duct tape
  • Double-sided foam tape
  • Clamps
  • Super glue
  • Sand paper
  • Measuring tape
  • Carpenter/rafter/combination square

Hardware:

ItemQuantityCostTotal CostNotes
8'x7"x3/4" pine board3$8.00$24Hardware Store
AcroMils 64-compartment organizer2$28.65$57.30
AcroMils 24-compartment organizer2$34.95$69.90
1-3/4" Wood screws18$0.10$1.80Hardware Store
3/4" Pan-head sheet metal screws8$0.10$0.80Hardware Store
Square foam exercise pads1$20.99$20.990.5", grey
2'x2' 1/4" plywood1$15$15Hardware Store
Total Cost:$189.79

Electronics:

ItemQuantityCostTotal CostNote
LED strip7$15.76$110.325V, 2 meter, 60 pixels/meter, 4mm wide, black PCB
Heatshrink tubing1$6.99$6.99An assortment of tubes will last you a while
Servo extention cable, 10cm7$7.99$7.99Pack of 10. Only 7 needed.
16ft 2-wire cable1$8.77$8.77Check hardware store for by-the-foot pricing
Particle Photon1$19$19The microcontroller. Can be bought on Amazon here.
Half-size breadboard1$7.99$7.99Because I didn't want to design a PCB
Jumper wires1$5.99$5.99Hook everything up
25A Solid state relay1$13.99$13.99Turn on/off the power supply to the LEDs. 5V input. Prices fluctuate.
14 Gauge Hookup wire3$0.25$0.75Your local hardware store will sell this by the foot
5V 60A Power supply1$27.77$27.77Powers all 840 RGB LEDs
5V 2A Power supply1$4.77$4.77Powers the Particle Photon
Power cord1$10$10Price fluctuates a lot, any grounded 3-wire cord will do
3.3V - 5V Logic level converter1$2.95$2.95Particle Photon (3.3V) > Converter > LED strip (5V)
Total:$227.28


Accounts needed: You'll need to create an account with each of these services


Software:

  • (Necessary)Particle Dev - Write Arduino-style code to run on the Particle Photon
  • (Optional) Visual studio, community edition - Modify the Azure Function and manually manipulate the SQL database
    • Extensions: Azure Development, .NET desktop development, Desktop development with C++, Data storage and processing
  • (Optional) Postman - For unit testing. Sends HTTP Post requests to the Azure Function.

Step 6: Construction - Design Files and Animation

The short animation demonstrates how FindyBot3000 was assembled.

Fusion 360 was used to design the CAD model, animation, and drawings for FindyBot3000.

By the way, you can use Fusion 360 for FREE as a hobbyist, like I am doing for this project!

I couldn't find an easy way to upload the parts, so the table below provides links to them in the Fusion 360 cloud:

ImagePart
Assembly
Foam Pad
Vertical Board
Horizontal Board
Middle Board
Lower Cabinet
Upper Cabinet

Drawings:

Horizontal board drawing

Vertical board drawing

Middle board drawing

Foam pad drawing

Step 7: Construction - Foam Pads

FindyBot3000 bundles the four individual organizer boxes in a slim wooden frame, sandwiched between slices of foam pad. The foam pads offer an aesthetically pleasing way of concealing all the wiring for the LED strips, as you'll see in later steps.

The first order of business is cutting the foam pads to size, from which the board dimensions are measured. Just lay the foam on a markable surface, then the board on top of the foam, and use a utility knife to cut the foam to the width of the board. With the foam cut out, measure the height needed by propping it up against the organizer box. Use a carpenter square or end of a board to make a square cut.

Great, first step done!

Step 8: Construction - Cutting the Boards

Cut the bottom, middle and top boards to length. Layering the cabinets as shown in the first picture produces the results. Stack the horizontal boards and the cabinets to find the total height of the assembly, and use that to mark the length the vertical boards need to be.

Alternatively, you can use the technical drawings. I've put images of them in this step for convenience.

Step 9: Construction - Frame Assembly

Begin by assembling the horizontal boards with the right vertical board. Remember to drill pilot and countersink holes to relieve strain on the end of the boards.

Step 10: Construction - Prepare the Cabinets

The horizontal supports for all the containers in the cabinet are recessed about a half inch behind the bezel. To route the LED strips along these horizontal supports, a drill is used to make holes in the bezel to thread the LED strips through. I used sandpaper and a dremel to tidy up the holes.

Step 11: Construction - Install the LED Strips

1. Take each of the 7 2-meter long LED strips, and find the middle. The contact terminals in the middle will have solder on them, making them different from all the rest. (There is solder on the terminals every 50cm on these LED strips). Use a pair of scissors to cut the solder pads in half. Do this for all LED strips, leaving you with 14 1-meter long lengths. Leave the paper backing on for now.

2. Critical Step: Find a LED strip length with a male-end connector, as shown. Note: the direction of the little arrows on the LED strip should point away from the connector.


3. From the top-left of the four-cabinet assembly, feed the led strips in as shown. This will produce a zig-zag pattern when connecting all the LEDs. Note: starting from the top, for every odd row the arrows should point to the right. For every even row, the arrows should point to the left.

Step 12: Quick Testing

Before we finish up the assembly, it's a good idea to make sure all the wiring is correct and the LEDs are working.

1. Setup the circuit as shown in the Fritzing diagram.

2. Open up Particle Dev, or use the Particle Web IDE, to upload HelloWorld.ino to your photon.

This test lights up four LEDs along the length of some of the strip, to test that it's working.

WARNING: Do NOT turn on more than 4 LEDs at once. You risk consuming more power than the USB connection can supply. We will address this in the next step.

Step 13: Construction - Install the LED Strip Power Wires

You may be wondering, why have all those power wires? Do I really need them? Yes, for two reasons:

1. To support turning on several LEDs at once, in case the search result turns up multiple items

2. To provide consistent 5V to all LEDs, as there is a small voltage drop across each LED.

To see what happens if you don't apply power at regular intervals, watch this video:

Why have such a large power supply? Do we really need 60A?

Technically, no. But if you want to run the LEDs at full brightness, then yes. So why limit your options?

Power Calculations

How do we arrive at 60A?

The manufacturer mentions 18W/meter @ 60 pixels/meter in the datasheet, and by datasheet, I mean a bunch of text hastily scribbled down on the product page:

Each row on the organizer is exactly 1 meter, and there are 14 rows, so we have 14 meters of LED goodness.

14 meters at 100% brightness, means the total wattage of the strip is 18W/m*14m = 252W.

We can use the power equation: P = IV, to give us a rough estimate of current (Amps) needed. Rearranging:

Power, P, is measured in Watts; I = Amperes; V = Volts

I = P/V = 252Watts/5V = 50.4A.

The next smallest power supply down from 60A was 40A - too small to power all LEDs at full brightness. So 60A it is. It's a good idea to get a power supply that exceeds the power requirements of your system by some margin, a ~12% safety margin (60/50.4 - 1) in this case is sufficient.

But what happens if you don't know how many watts are used per meter of your LED strip? Well, you can estimate it using rough estimates for other LEDs you may have. Most standard 5mm LEDs, like the red ones shown in the demo video, draw around 20mA of current when running off of 5V. Using that information, we can extrapolate power requirements for an LED strip:


1. 20mA (milliamperes) is drawn per LED, that's 0.020A (1000mA = 1A)

2. RGB LEDs have three LEDs each: one red, blue, and green

3. Find the number of pixels per meter, in this case, there are 60 pixels per meter.

At 14 rows, with each row being 1 meter long, that leaves us with:

14 meters * 60 pixels/meter * 3 LEDs/pixel * 0.020A / LED = 50.4A

Hey great, that's the same Amp requirements as calculated in our previous equation!

Step 14: Construction - Foam Pads V2

We're not done with those foam pads just yet. In the first step, we cut them to size. Now we'll route cutouts and channels for all the wiring to run through. Align the pad to the wooden frame, and use a marker to mark the routes the wires take along the foam pads, then use a dremel to cut holes and channels for said wire to run through. Since we haven't fastened the cabinets to the wood yet, you can just slide them length wise along the frame to access both sides.

As you can see, I did a pretty quick and dirty job of this. But no matter, it never sees the light of day once everything is put together.

Step 15: Construction - Final Touches to the Assembly

To aid with wiring, the cabinets were never fastened in place.

With the wiring done, and the foam pads in place, finally install the vertical board on the left.

Nudge the cabinets so they are flush with the front of the frame assembly, and then drill two holes on the bottom of each cabinet into the supporting wood plank below. Fasten the cabinets to the assembly with the 3/4" machine pan-head screws.

Step 16: Construction - Adjusting the Boards

I found that over time, the foam pads applied a little too much lateral pressure to the cabinets. This resulted in the LED strips buckling off of the cabinet rows. To fix that, I used some wooden wedges to push the cabinets apart and pull the LED strips taught.

Step 17: Electrical - Hooking Up the Power Supply and Testing

WARNING: HIGH VOLTAGE - In this step, we attach the beefy power supply up to all the LED strip power lines.

Do not, under any circumstances, wire up the power supply with the power cable plugged in.

1. Begin with testing that the power supply turns on. See the first photo. Then unplug the power supply cable.

2. Wire up only the first row of LEDs to the power supply

3. Disconnect the second and third row of LEDs

4. Repeat the test on Step 12: Quick Testing. If the LEDs work as expected, great! If not, make sure the 5V and GND power supply lines are wired correctly.

5. When the first row of LEDs is tested and working with the power supply, go on and wire up the remaining LED strip power lines. Repeat Step 12: Quick Testing, but make the following modification:

BeforeAfter

This time, the four-led pewpew that zig-zags across the display should maintain the same brightness the entire way through. Like so:

6. Once all the LEDs are working as intended, go on and cleanup the wiring. Use some double-sided foam tape to secure the power supply.

Step 18: Electrical - Wiring Up the Particle Photon

The particle photon drives two important pieces of FindyBot3000:

1. The Relay - this allows FindyBot3000 to operate in low-power mode, by turning on the LED power supply only when needed.

2. The LEDs - the Photon drives the LEDs through the voltage step-up converter.

Step 19: Software - Download and Install

In this step, we link together the Google Home and Particle Photon.

Prerequisites:

- Log into your Github account

- Download and install Particle Dev

- Download and install Github Desktop

Steps:

1. Navigate to the FindyBot3000 repo on Github: https://github.com/Inventor22/FindyBot3000

2. Clone the repo to your computer

3. Open Particle Dev, and load the project folder: FindyBot3000\Photon\FindyBot

3. Plug in the Photon to your computer via USB cable, and select your photon from the devices menu in Particle Dev

4. Flash the particle photon with the firmware

Before we can test to see if the photon is working with the flashed firmware, we need to setup Google Home to talk to the Particle Photon, through IFTTT.

Step 20: Software - Linking in IFTTT

To test the firmware we just uploaded to the Particle Photon, there are a few simple commands we can trigger on the Google Home.

First, we make an IFTTT applet that is triggered when we say "Ok Google, Welcome $", where $ is the name of the individual you would like to welcome.

As soon as the applet has deployed, you can begin testing your FindyBot3000!

Step 21: Software - Remaining IFTTT Commands

Use the table above to create the remaining IFTTT applets.

Currently FindyBot3000 supports 10 commands which require a SQL database, and another 4 commands which run locally on the Photon.

Requires SQL Database:

FindItem - Find an item

FindTags - Find items matching provided tags, colored on a gradient according to number of tags matched

InsertItem - Insert a new item into the organizer

RemoveItem - Remove an existing item from the organizer

AddTags - Add tags to an existing object. Ex: "Add tags diode light emitting to green LEDs"

SetQuantity - Set the quantity of an existing item

UpdateQuantity - Update the quantity of an existing item

ShowAllBoxes - Light up all the boxes which have items in them with random colors

BundleWith - Bundle a new item with an existing item. Ex: "Bundle 8 blue LEDs with red LEDs"

HowMany - Show, in ASCII text on the display, the remaining quantity of an item.

Local Commands:

Welcome - Scroll text across the display, welcoming the specified individual

SetBrightness - Change the brightness of the LEDs

SetDisplay - Turn on and off the display

ChangeColors - Change the color gradient used by the FindTags command

Step 22: Software - Deploy the Azure Function

A picture speaks a thousand words...

Navigate to the Microsoft Azure Portal and log in.

Here we deploy the Azure Function, and set up continuous integration through Github. In this way, when new changes are pushed to the forked repository you created, they will automatically update the Azure Function with the new bits.

Step 23: Software - Link Particle Photon to Azure Function

In order for the Particle Photon to successfully send data to the Azure Function, it needs to know where to send the message. In this step, we setup a Webhook between the Photon and our Azure Function.

Step 24: Software - Deploy the SQL Database

In this step we deploy the SQL Database, and provide access to the Azure Function via the sql_connection string.

Step 25: Software - Create the SQL Tables

So we've got the Photon talking to the Azure Function, and the Azure Function talking to the SQL Database. But We're not out of the woods just yet. In order for the SQL queries in the Azure Function to work correctly, they need to query SQL Tables. In this step, we create the necessary tables in the SQL Database.

Step 26: Final Thoughts

There's a lot more that can be done!

In the future, there will be room for an LCD to display debug information, or animating the lights, beyond just text scrolling.

Step 27: Thank You!

Just a quick update -- Looks like this Instructable has won the grand prize in the organization contest! Thanks for the votes everyone :)

Organization Contest

Grand Prize in the
Organization Contest

Share

    Recommendations

    • Fandom Contest

      Fandom Contest
    • Gardening Contest

      Gardening Contest
    • Party Challenge

      Party Challenge

    128 Discussions

    0
    None
    prnet

    11 days ago

    Massive project! Absolutely brillant idea! Congratulation, I'm really impressed.

    0
    None
    Inventor22TaranA4

    Answer 11 days ago

    Hi Taran, I use a database to store the location all the items which have been inserted using the "Insert Item" command. When the user asks for an item, "Ok Google, find yellow LEDs", the program extracts the words "yellow leds", then uses that as a 'key' to lookup the entry in the database. If an entry for the item is found, the coordinates of the box that the item is in are sent to the organizer, which then lights up the LEDs for that box.

    It works similarly to how a dictionary works, but instead of finding the definition for a word or words, we get the coordinates of the box the item is in.

    0
    None
    Strickce

    Question 18 days ago

    I have all the software connected, and am sending requests to Azure, but I have a syntax error in the command sent. I'm pretty sure that my IFTTT output is wrong, but I can't seem to get the correct stuff entered there. Azure tells me that there is a problem with the command input, and it seems to be associated with quotes. Could someone show me what a good "Insert Item" IFTTT event contains?
    This is what I see on the serial monitor:

    googleAssistantEventHandler event: Google_InsertItem, data: {“Info”:”red LEDs”,”Quantity”:50}
    {“Info”:”red LEDs”,”Quantity”:50}
    {"command":"InsertItem", "data":{“Info”:”red LEDs”,”Quantity”:50}}

    The screen shots are the Azure error I get and my IFTTT screen


    Update: I found the problem to be smart quotes in the data stream. I had to turn off Smart Punctuation in my iPad, and then re-enter the command to IFTTT. All is good now. Cannot wait to get the LED strips and actually build this. I am building a unit with 6-boxes instead of 4, and expect that to be a pretty easy change. I believe modifying the array dimensions should take care of most of the work.

    Screen Shot 2019-05-09 at 9.22.17 AM.pngIMG_0015.JPG
    1 answer
    0
    None
    Inventor22Strickce

    Answer 11 days ago

    Glad to hear you solved the problem Strickce. In my experience 90% of bugs with this project have been random unaccounted or inserted characters here and there which break the Json format one way or another. A good resource for analyzing and testing the Azure Function for any bugs / quality of output is to run the Azure Function locally with Visual Studio and send it requests using Postman (https://www.getpostman.com/). I've added the requests I used for testing to Github: (https://github.com/Inventor22/FindyBot3000/tree/master/Testing/Postman). Just import that file to Postman and you should be good to go.

    As for 6 boxes (nice!), you're right you'll need to just update the array dimensions (In both the Particle Photon and the Azure Function code). There's also LED offset and LED width arrays (boxLedOffsetByColumnTop, boxLedWidthByColumnTop, etc.) which will need to be updated to account for the couple extra cabinets. The code is pretty janky around that so let me know if you need clarification on any of it.

    0
    None
    Sebastian-Si

    25 days ago

    I'v just tested it, works like a charm, but why MS Azure, it's a bit to expensive for project like this.
    Have you wondered about local webservice?

    1 reply
    0
    None
    Inventor22Sebastian-Si

    Reply 11 days ago

    Great to hear. I chose Azure for a couple reasons: I was already familiar with the Azure ecosystem, and I wanted to learn about Serverless Functions. I thought about running a local SQL (or other) database solution, or even rolling my own via EEPROM or an SD card, but I specifically wanted this device to be an IoT thing, so I quickly threw away those ideas.

    In a future revision I may opt for on-chip storage and voice recognition, most likely running off a Raspberry Pi.

    0
    None
    Hey Jude

    4 weeks ago

    I love the tagging feature - as not only is it easy to forget what you called something at the first instance of assignment, but also for others to use the system it's essential!

    I've watched a lot of Voice enabled tech, but this is actually one of the best I've seen.

    Have you considered creating an App for this, with Google Home or Alexa, as it would simplify the code/SW part, and then the assembly could be easily done.

    If for example the App allowed you to input the size/numbers/reference of the compartments, it would even allow easy updating. Granted this is a lot of effort on your part, but I'd part for the UI/UX of the App! I think it could be applied to many things, from food to pharma!

    Well done on the win. First class!

    1 reply
    0
    None
    Inventor22Hey Jude

    Reply 11 days ago

    Yeah, I had the same problem originally -- I'd forget what I called a part and end up wasting time manually looking for it. The tagging feature is key for letting others search for parts, as you mentioned.

    I looked into creating a custom Google Assistant app, but the learning curve was a bit too steep for my timeline, and the integration with IFTTT was dead simple. So I made the trade off -- ease of use, for a little less functionality. A UI may come later down the road, including a little LCD to show part inventories and other debug info without the need to debug through a computer.

    Thanks for the comment!

    0
    None
    jatin7233

    4 weeks ago

    Can I use my android phone Google assistant instead of Google home??
    And if yes, then how?

    1 reply
    0
    None
    PortalGamesMais.jatin7233

    Reply 25 days ago

    yes you can! When setting up the google assitant command on ifttt it will ask you to login to your google account, as long as google assistant on your phone is connected to the same account it will work no extra steps needed.

    0
    None
    zpaeric

    4 weeks ago

    This was an excellent project! I plan to make one as well, but I have many more parts than a set of single drawers can hold. If I built my shelves 5 deep, I can use flashing leds in the groups of 4 to identify which bin in that row contains the parts. No flash could be the front drawer and each flashing led would indicate drawer 2,3,4 and 5!

    Thank you for the inspiration! Waiting for parts to arrive to get this started. I will post progress, and may have some programming questions to handle the 3D array...

    1 reply
    0
    None
    Inventor22zpaeric

    Reply 4 weeks ago

    Thanks. That sounds great, a 3D setup would definitely be more compact. A bunch of different ideas could be used to represent the Z-axis, I like the flashing LEDs idea you mentioned. For sure, feel free to PM me or post any questions here.

    1
    None
    matti04_electronics

    5 weeks ago

    unbelivable! It is FANTASTIC! thank you so much for sharing, it is so cool! Have a great day :)

    0
    None
    KitP6

    5 weeks ago

    The linked solid state relay is not one that can work on 5v DC switching input (They may have changed the product listing). :(

    1 reply
    0
    None
    Inventor22KitP6

    Reply 5 weeks ago

    Thanks, I've updated the Instructable with a new link and price.

    0
    None
    KonradO6

    6 weeks ago

    Will it work with other languages supported by Google Assistant?

    1 reply
    0
    None
    PortalGamesMais.KonradO6

    Reply 6 weeks ago

    I’m from Brazil and I tried to test ifttt with google assistant in português and it wouldn’t work because ifttt was set to English. After changing my google assistant to English it worked , so I guess that if ifttt supports your language than you can use it