Voice control is the next big thing in (home) automation. Just look at all the companies wanting a piece of the cake: Amazon with Alexa, Apple with Siri and Google with Home Assistant. And indeed when you start playing with voice recognition and synthesizes you realise how much fun it is. Do I believe in it ?? I am not quite sure yet. There are some pitfalls, however I do believe that the large companies will be pushing it into our homes sooner or later.
I already wrote a major story on Google's Home Assistant running on a Raspberry Pi and controlling an ESP-8266.
And there are two stories on this weblog that also use voice control but take a different approach. They do not use the Home Assistant but do use voice recognition through a dedicated Android APP. The first project used Bluetooth to communicate with an Arduino and was written in 2016. The second project used Wifi to communicate with an ESP8266
In these previous entries I showed you how to control an ESP8266 by an Android App instead of the usual web/page control.
For those who did not read these stories here is a brief explanation.
Normally you control an ESP8266 by a webpage with buttons, sliders etc etc. This can rise a problem as anyone in your network can control the ESP through that webpage. You can build in some safety measures but that involves a lot of work.
When you control an ESP/8266 with an Android dedicated App only those who have the App can control the ESP.
Now this project is not exactly a worthy opponent of Google Assist but it does the job it was designed for remarkably well.
The version I am going to present you today is even more remarkable as it uses a general purpose Voice App which I specailly devellopped and a very easy to understand program in ESP-Basic.
The difference in approach.
In the previous version where I commanded the ESP by voice I showed you how to make an app that reacted to several fixed commands. You can re-read that story here: https://lucstechblog.blogspot.nl/2018/02/wifi-voice-commander.html This works but is however not very flexible. To alter the command you must alter the App in Mit's App Inventor which is fairly easy done however time consuming. Next to that you will need to alter the ESP program also if you add or remove commands. And that involves re-writing the C++ program, compiling it again and transferring it to the ESP-8266.
In this story I am going to show you how to make a general purpose App for Android that only translates the speech to text which is then send to the ESP8266 to be analysed.
This way you only need to alter the ESP-Basic program if you want to alter commands and that is easily done.
Mit App Inventor and ESP-Basic.
This is not going to be a tutorial on how to use App-Inventor. There is ample doncumentation around on that. Mit's own website is full of tutorials and sample programs. So please check that first if you are not familiar with this easy to use Android Devellopment platform.
This is also no tutorial on how to work with ESP-Basic. That has been covered in a previous story on this web-log that you can read here.
https://lucstechblog.blogspot.nl/2017/03/back-to-basic-basic-language-on-esp8266.html
Working with Mit's App Inventor really is very easy and you will be designing and programming Android App's in notime. Communication with the ESP using Wifi or an Arduino over Bluetooth is a piece of cake and therefoer I urge you to delve into this as I will publish some more projects using this tool.
Off course there is a link to both the App and the source code at the end of this story.
The Designer screen
Start with downloading the Wifirelay.aia source code to your computer. Then point your browser to http://appinventor.mit.edu/explore/
On the right side of the screen click on the orange square that says "Create Apps"
Now you can log in with your Google credentials. If you do not have a Google account make sure you get one first.
In the projects menu on the top of your screen choose -- import project(.aia) from my computer.
When done your screen will look the same as above.
On the right side of the screen you will find a column with all the components that are used in your App. Clicking a component will highlight it on the example screen and in the last column on the right side of the screen you will be able to alter the settings for that component.
As you can see there are a few labels that do nothing more as putting text on your screen. Then there is a textbox that allows you, when running the App, to fill in the IP adress of the ESP8266 that is going to be connected.
Next there is a button called "Voice_input" whick I gave an image of a microphone.
And below that there is an invisible label. That label will get visible after you spoke your command and will be filled with the text you spoke. This makes sure you have a visible check if the command was pronounced well and recognised well.
The last two components are the Web1, Speechrecogniser1 and TextToSpeech1 component. They are very important and working in the back-ground. The Web component takes care of the communiaction between the ESP8266 and the App. The Speechrecogniser1 will translate your spoken commands into mwritten text and the TextToSpeech1 component is going to be used to have the App speak out loud when the function has completed.
There is one thing I really want to Emphasize.
Look at the TextToSpeech1 component carefully. If you leave it at its default values the App will speak out any text in the language you have set as your default language in your phone. If that is English all is well. However my phone is set default to Dutch. Therefore English texts will not be recognised well by the Speechrecogniser and the confirmation text will be spoken out awkward. So if you are going to use another language as English make sure you set this to your preferred language.
When all is set switch on the right (top) side of the screen to the blocks section wher the actual program will be written.
The Blocks
Well the program almost speaks for itself except for one small detail which I will get into in detail.
Then the program just waits till you press the button called "Voice_input"
Next comes the main part.
First the result from the Speechrecogniser is put in the global variable "global_input1"
The next 2 lines in that block replace any spaces in your spoken text by a "."
The reason for this is that the web communication, for some reason or another, will only send the first word in a sentence.
Now we do not want to send one word commands. We want to send complete sentenses like "living room ceiling light on".
If you speak out this command the web component will only send the word "living".
So the program lines change this in "living.room.ceiling.light.on"
In the next step, which is the ESP-Basic program, we will filter the dots out again.
The following sequence of lines actually sends the command over the net.
The command is build up by the word "http://" followed by the IP_Adress the word "/msg?saywhat=" and the command you spoke in all lower case letters.
This sums up to the following.
If your command would be "living room ceiling light on" the command send over the network would be: http://xxxx.xxxx.xxxx.xxxx/msg?saywhat=living.room.ceiling.light.on
You can alter "saywhat" in another word, but then you will have to alter this later on in the ESP program to in exact the same word. This way you can build some security into your program.
And the last block gives an audible feedback that confirms that your command has been send. I used the phrase "yes master" but you can alter it to your own liking. I like a touch of slavery in my electronics.
In the video at the bottom of this story you can see that I altered the App a bit so it repeats the command I gave instaed of saying "yes master" I also edited the Basic program a bit to incorporate a command that puts all lights off.
In the previous entries in this series the program was written in C++ (Arduino code). Nothing wrong with that however this time I am going to write the program in ESP-Basic which gives us a lot of ease and flexibility.
The Basic program
msgbranch [mybranch]
textbox command
wait
[mybranch]
command = msgget("saywhat")
command = replace(command , "." , " ")
print command
wait
Let us start with a test.
This small program starts when the information is received over wifi. The -- msgbranch -- command is executed and at the program jumps to the mybranch subroutine.
In the mybranch subroutine the first thing that happens is a test if the received information starts with "saywhat" this is the trigger that actually starts the code. If "saywhat" is not received nothing will happen.
Next step is that any received dots are replaced by spaces again. This way the sentence in our example "living.room.ceiling.light.on" is replaced by "living room ceiling light on". Remember we have to do that because the Wifi communication does not treat spaces well.
The program prints the received command and also places it in the textbox.
This way we have a check if the spoken commands are recieved well and not misinterpreted.
For this project I have used the commands:
living room on / off
garage door open / close
garage light on /off
bathroom light on /off
living room ceiling light on / off
water pump on / off
Recognise these commands ??
Yep they are the same commands I used in my story about commanding the ESP8266 by Google Home Assistant which you can read here: https://lucstechblog.blogspot.nl/2017/05/google-home-and-esp8266.html
In fact is even exact the same Basic program.
So let us have a closer look at the complete program.
'-----------------------------
'init
'-----------------------------
io(po,d1,0)
io(po,d2,0)
io(po,d3,0)
io(po,d5,0)
io(po,d6,0)
io(po,d7,0)
msgbranch [mybranch]
textbox command
wait
[mybranch]
command = msgget("saywhat")
command = replace(command , "." , " ")
print command
if command = "living room on" then
io(po,d1,1)
endif
if command = "living room off" then
io(po,d1,0)
endif
if command = "garage door open" then
io(po,d2,1)
endif
if command = "garage door close" then
io(po,d2,0)
endif
if command = "garage light on" then
io(po,d3,1)
endif
if command = "garage light off" then
io(po,d3,0)
endif
if command = "bathroom light on" then
io(po,d5,1)
endif
if command = "bathroom light off" then
io(po,d5,0)
endif
if command = "living room ceiling light on" then
io(po,d6,1)
endif
if command = "living room ceiling light off" then
io(po,d6,0)
endif
if command = "water pump on" then
io(po,d7,1)
endif
if command = "water pump off" then
io(po,d7,0)
endif
wait
After the init part where all IO ports are set as output and put off the program starts a MESSAGE BRANCH.
It waits till it gets a command from the internet and displays that command in a textbox on your webpage. This makes it easy to determine if your spoken commands are in the right way interpreted by the App and well transmitted over wifi.
If certain commands are always faulty interpreted you can use the Basic test program (from above) and use a different spoken command.
[mybranch]
command = msgget("saywhat")
command = replace(command , "." , " ")
print command
if command = "living room on" then
io(po,d1,1)
endif
if command = "living room off" then
io(po,d1,0)
endif
When a command has been received from the App the program jumps to [mybranch]
The first thing that happens is to see if -- saywhat -- has been received. You can alter that in the program to any other word you like. However remember to change that phrase in the APP also to exact the same word. If -- saywhat -- (or your own choice) is not received the program will not do anything.
The next line is a bit tricky:
command = replace(command , "." , " ")
As said above web communication has a bit of a problem with spaces. Therefore we altered thye spaces in the APP in dots (".") this line changes them back in spaces.
To see if everything went well the received command is printed on your webpage by the print command.
The next lines decide which command has been received and accordingly put the accompaning I/O port on or off.
As you can see this far more easier as the C++ (Arduino code) version and much easier to edit. Better still: the Basic program can be altered over the Air. When working with C++ you will always have to attach the ESP to a computer. So all in all this is more convenient. And remember in this version you only need to alter the Basic program if you need to change commands. The APP stays the same. You just give other voice commands.
That is all folks !!
The hardware
This is as you can see very straightforward just a bunch of leds connected to a delimiter resistor of 220 Ohm and attached to the I/O ports of the ESP8266.
Getting the IP adress
We need to know the IP adress of the ESP8266 to be able to send commands to it.
So open your web-browser and poit it to the IP adress of your router. Most likely that will be 192.168.1.1 but consult your routers manual for this.
So delve into your router and look at your connected devices. Mine looks like above.
The lower icons are the wired apparatus and as you can see it is my computer (Nescio), printerserver (actually a Raspberry) called Epson as it is connected to my Epson printer, my Buffalo Nas, my Domoticz domotica system and a raspberry. The last one is my DIY Google Home.
The top line icons are my wireless apparatus. There you can see a Thermometer (an ESP with a DS-18b20) and a device called 165. That last one is the ESP this all is about. I named it 165 as this is its IP adress and easily remembered. So in my case the local IP adress is 192.168.1.165
I assume you are going to to use this from within your local network. If you are going to use this from other places as your home you will need to make your ESP8266 accessible from the outside world. You can do this by opening a gate (port) in your router and use port-forwarding.
There is a detailed description on this in my story about attaching an ESP8266 to Google Home which you can read here http://lucstechblog.blogspot.nl/2017/05/google-home-and-esp8266.html (about halfway down the story).
How to use it
First let us look at the Basic program.
If you are building a permanent setup first rename the program in default.bas
Next go to the settings page and check the box that makes sure that default.bas is run automatically at startup.
The ESP will need a longer startup time but it automatically starts the program.
On your phone start the ESP-Voice APP
In the box where it says fill in your IP and the figure 192.168.1.XXX is shown replace that number by the IP number you found in your router.
If you are going to use this globally, meaning also outside of your private network, replace the IP number by xxxx.xxxx.xxxx.xxxx:8085
The xxx.xxx.xxx.xxx being the global IP adress you found on http://www.whatsmyip.org/ and the 8085 being the port number you are using on your router. This way you can use this APP anywhere in the world where you have an internet connection.
Tap the microphone icon and wait till the voice recogniser has started and speak one of the commands you entered in your Basic program and the connected led will go on or off. Neat Huh ???
Now connect the ESP's I/O ports with relays to a ledstrip like I have shown here https://lucstechblog.blogspot.nl/2016/05/esp8266-controlled-ledstrip.html or to a relay like I have shown here https://lucstechblog.blogspot.nl/2016/09/esp-relayserver.html or a string of neopixels like I described at the bottom of this story https://lucstechblog.blogspot.nl/2017/03/back-to-basic-basic-language-on-esp8266.html and start automating your home by voice commands.
That's it. A really easy way to control any devices you like by voice commands be it on your local network or from any place on the world.
Bonus
As I explained the Basic program is the same program I used for controlling the ESP by The Google Home Assistant. The consequence is therefore that you can now control the ESP in two different ways. The first is by the Home Assistant and the second is by the App on your phone. So you can control the ESP now also from rooms where there is no Home Assistant present or even from a location far way from your home.
Expansion
There is room for expansion.
Lets assume you have multiple ESP's setup for your projects. Each ESP can be equipped by the same Basic program but can have altered commands programmed. The only thing you will have to do is to alter the IP adress in the APP's field and you can adress another ESP.
If you want you can even alter the APP in such a way that you make some buttons on the screen that automatically connect to several ESP's.
Source code:
As usual you can find the source code for the APP and the APK file on github:
https://github.com/Lucvolders/ESP-Voice
So experiment and have fun.
Luc Volders