Friday, February 16, 2018

ESP Voice Control with ESP-Basic

For an index to all of my stories click here.

Voice control is the next big thing in (home) automation. Just look at all the companies wanting a piece of the cake: Amazon with Alexa, Apple with Siri and Google with Home Assistant. And indeed when you start playing with voice recognition and synthesizes you realise how much fun it is. Do I believe in it ?? I am not quite sure yet. There are some pitfalls, however I do believe that the large companies will be pushing it into our homes sooner or later.

I already wrote a major story on Google's Home Assistant running on a Raspberry Pi and controlling an ESP-8266.

And there are two stories on this weblog that also use voice control but take a different approach. They do not use the Home Assistant but do use voice recognition through a dedicated Android APP. The first project used Bluetooth to communicate with an Arduino and was written in 2016. The second project used Wifi to communicate with an ESP8266

In these previous entries I showed you how to control an ESP8266 by an Android App instead of the usual web/page control.

For those who did not read these stories here is a brief explanation.
Normally you control an ESP8266 by a webpage with buttons, sliders etc etc. This can rise a problem as anyone in your network can control the ESP through that webpage. You can build in some safety measures but that involves a lot of work.
When you control an ESP/8266 with an Android dedicated App only those who have the App can control the ESP.

Now this project is not exactly a worthy opponent of Google Assist but it does the job it was designed for remarkably well.

The version I am going to present you today is even more remarkable as it uses a general purpose Voice App which I specailly devellopped and a very easy to understand program in ESP-Basic.

The difference in approach.

In the previous version where I commanded the ESP by voice I showed you how to make an app that reacted to several fixed commands. You can re-read that story here: https://lucstechblog.blogspot.nl/2018/02/wifi-voice-commander.html This works but is however not very flexible. To alter the command you must alter the App in Mit's App Inventor which is fairly easy done however time consuming. Next to that you will need to alter the ESP program also if you add or remove commands. And that involves re-writing the C++ program, compiling it again and transferring it to the ESP-8266.

In this story I am going to show you how to make a general purpose App for Android that only translates the speech to text which is then send to the ESP8266 to be analysed.
This way you only need to alter the ESP-Basic program if you want to alter commands and that is easily done.

Mit App Inventor and ESP-Basic.

This is not going to be a tutorial on how to use App-Inventor. There is ample doncumentation around on that. Mit's own website is full of tutorials and sample programs. So please check that first if you are not familiar with this easy to use Android Devellopment platform.
This is also no tutorial on how to work with ESP-Basic. That has been covered  in a previous story on this web-log that you can read here.

https://lucstechblog.blogspot.nl/2017/03/back-to-basic-basic-language-on-esp8266.html

Working with Mit's App Inventor really is very easy and you will be designing and programming Android App's in notime. Communication with the ESP using Wifi or an Arduino over Bluetooth is a piece of cake and therefoer I urge you to delve into this as I will publish some more projects using this tool.

Off course there is a link to both the App and the source code at the end of this story.

The Designer screen

Start with downloading the Wifirelay.aia source code to your computer. Then point your browser to http://appinventor.mit.edu/explore/
On the right side of the screen click on the orange square that says "Create Apps"
Now you can log in with your Google credentials. If you do not have a Google account make sure you get one first.
In the projects menu on the top of your screen choose -- import project(.aia) from my computer.



When done your screen will look the same as above.

On the right side of the screen you will find a column with all the components that are used in your App. Clicking a component will highlight it on the example screen and in the last column on the right side of the screen you will be able to alter the settings for that component.

As you can see there are a few labels that do nothing more as putting text on your screen. Then there is a textbox that allows you, when running the App, to fill in the IP adress of the ESP8266 that is going to be connected.
Next there is a button called "Voice_input" whick I gave an image of a microphone.
And below that there is an invisible label. That label will get visible after you spoke your command and will be filled with the text you spoke. This makes sure you have a visible check if the command was pronounced well and recognised well.
The last two components are the Web1, Speechrecogniser1 and TextToSpeech1 component. They are very important and working in the back-ground. The Web component takes care of the communiaction between the ESP8266 and the App. The Speechrecogniser1 will translate your spoken commands into mwritten text and the TextToSpeech1 component is going to be used to have the App speak out loud when the function has completed.



There is one thing I really want to Emphasize.
Look at the TextToSpeech1 component carefully. If you leave it at its default values the App will speak out any text in the language you have set as your default language in your phone. If that is English all is well. However my phone is set default to Dutch. Therefore English texts will not be recognised well by the Speechrecogniser and the confirmation text will be spoken out awkward. So if you are going to use another language as English make sure you set this to your preferred language.

When all is set switch on the right (top) side of the screen to the blocks section wher the actual program will be written.

The Blocks

Well the program almost speaks for itself except for one small detail which I will get into in detail.
 

First a global variable called input1 is initialised.
Then the program just waits till you press the button called "Voice_input"

Next comes the main part.

First the result from the Speechrecogniser is put in the global variable "global_input1"
The next 2 lines in that block replace any spaces in your spoken text by a "."
The reason for this is that the web communication, for some reason or another, will only send the first word in a sentence.
Now we do not want to send one word commands. We want to send complete sentenses like "living room ceiling light on".
If you speak out this command the web component will only send the word "living".
So the program lines change this in "living.room.ceiling.light.on"

In the next step, which is the ESP-Basic program, we will filter the dots out again.

The following sequence of lines actually sends the command over the net.
The command is build up by the word "http://" followed by the IP_Adress the word "/msg?saywhat=" and the command you spoke in all lower case letters.

This sums up to the following.
If your command would be "living room ceiling light on" the command send over the network would be: http://xxxx.xxxx.xxxx.xxxx/msg?saywhat=living.room.ceiling.light.on

You can alter "saywhat" in another word, but then you will have to alter this later on in the ESP program to in exact the same word. This way you can build some security into your program.

And the last block gives an audible feedback that confirms that your command has been send. I used the phrase "yes master" but you can alter it to your own liking. I like a touch of slavery in my electronics.


In the video at the bottom of this story you can see that I altered the App a bit so it repeats the command I gave instaed of saying "yes master" I also edited the Basic program a bit to incorporate a command that puts all lights off.

In the previous entries in this series the program was written in C++ (Arduino code). Nothing wrong with that however this time I am going to write the program in ESP-Basic which gives us a lot of ease and flexibility.

The Basic program



msgbranch [mybranch]

textbox command

wait

[mybranch]
command = msgget("saywhat") 
command = replace(command , "." , " ")
print command
wait

Let us start with a test.

This small program starts when the information is received over wifi. The -- msgbranch -- command is executed and at the program jumps to the mybranch subroutine.

In the mybranch subroutine the first thing that happens is a test if the received information starts with "saywhat" this is the trigger that actually starts the code. If  "saywhat" is not received nothing will happen.

Next step is that any received dots are replaced by spaces again. This way the sentence in our example "living.room.ceiling.light.on" is replaced by "living room ceiling light on". Remember we have to do that because the Wifi communication does not treat spaces well.

The program prints the received command and also places it in the textbox.

This way we have a check if the spoken commands are recieved well and not misinterpreted.

For this project I have used the commands:

living room on / off
garage door open / close
garage light on /off
bathroom light on /off
living room ceiling light on / off
water pump on / off


Recognise these commands ??
Yep they are the same commands I used in my story about commanding the ESP8266 by Google Home Assistant which you can read here: https://lucstechblog.blogspot.nl/2017/05/google-home-and-esp8266.html
In fact is even exact the same Basic program.

So let us have a closer look at the complete program.



'-----------------------------
'init
'-----------------------------
io(po,d1,0)
io(po,d2,0)
io(po,d3,0)
io(po,d5,0)
io(po,d6,0)
io(po,d7,0)

msgbranch [mybranch]

textbox command

wait

[mybranch]

command = msgget("saywhat") 
command = replace(command , "." , " ")
print command
if command = "living room on" then
io(po,d1,1)
endif
if command = "living room off" then
io(po,d1,0)
endif

if command = "garage door open" then
io(po,d2,1)
endif
if command = "garage door close" then
io(po,d2,0)
endif

if command = "garage light on" then
io(po,d3,1)
endif
if command = "garage light off" then
io(po,d3,0)
endif

if command = "bathroom light on" then
io(po,d5,1)
endif
if command = "bathroom light off" then
io(po,d5,0)
endif

if command = "living room ceiling light on" then
io(po,d6,1)
endif
if command = "living room ceiling light off" then
io(po,d6,0)
endif

if command = "water pump on" then
io(po,d7,1)
endif
if command = "water pump off" then
io(po,d7,0)
endif

wait

After the init part where all IO ports are set as output and put off the program starts a MESSAGE BRANCH.
It waits till it gets a command from the internet and displays that command in a textbox on your webpage. This makes it easy to determine if your spoken commands are in the right way interpreted by the App and well transmitted over wifi.
If certain commands are always faulty interpreted you can use the Basic test program (from above) and use a different spoken command.



[mybranch]

command = msgget("saywhat") 
command = replace(command , "." , " ")
print command
if command = "living room on" then
io(po,d1,1)
endif
if command = "living room off" then
io(po,d1,0)
endif

When a command has been received from the App the program jumps to [mybranch]

The first thing that happens is to see if  -- saywhat -- has been received. You can alter that in the program to any other word you like. However remember to change that phrase in the APP also to exact the same word. If -- saywhat -- (or your own choice) is not received the program will not do anything.

The next line is a bit tricky:

command = replace(command , "." , " ")

As said above web communication has a bit of a problem with spaces. Therefore we altered thye spaces in the APP in dots (".") this line changes them back in spaces.

To see if everything went well the received command is printed on your webpage by the print command.

The next lines decide which command has been received and accordingly put the accompaning I/O port on or off.

As you can see this far more easier as the C++ (Arduino code) version and much easier to edit. Better still: the Basic program can be altered over the Air. When working with C++ you will always have to attach the ESP to a computer. So all in all this is more convenient. And remember in this version you only need to alter the Basic program if you need to change commands. The APP stays the same. You just give other voice commands.

That is all folks !!

The hardware





This is as you can see very straightforward just a bunch of leds connected to a delimiter resistor of 220 Ohm and attached to the I/O ports of the ESP8266.


Getting the IP adress

We need to know the IP adress of the ESP8266 to be able to send commands to it.

So open your web-browser and poit it to the IP adress of your router. Most likely that will be 192.168.1.1 but consult your routers manual for this.



So delve into your router and look at your connected devices. Mine looks like above.

The lower icons are the wired apparatus and as you can see it is my computer (Nescio), printerserver (actually a Raspberry) called Epson as it is connected to my Epson printer, my Buffalo Nas, my Domoticz domotica system and a raspberry. The last one is my DIY Google Home.

The top line icons are my wireless apparatus. There you can see a Thermometer (an ESP with a DS-18b20) and a device called 165. That last one is the ESP this all is about. I named it 165 as this is its IP adress and easily remembered. So in my case the local IP adress is 192.168.1.165

I assume you are going to to use this from within your local network. If you are going to use this from other places as your home you will need to make your ESP8266 accessible from the outside world. You can do this by opening a gate (port) in your router and use port-forwarding.

There is a detailed description on this in my story about attaching an ESP8266 to Google Home which you can read here http://lucstechblog.blogspot.nl/2017/05/google-home-and-esp8266.html (about halfway down the story).

How to use it

First let us look at the Basic program.
If you are building a permanent setup first rename the program in default.bas




Next go to the settings page and check the box that makes sure that default.bas is run automatically at startup.
The ESP will need a longer startup time but it automatically starts the program.

On your phone start the ESP-Voice APP

 

In the box where it says fill in your IP and the figure 192.168.1.XXX is shown replace that number by the IP number you found in your router.

If you are going to use this globally, meaning also outside of your private network, replace the IP number by xxxx.xxxx.xxxx.xxxx:8085

The xxx.xxx.xxx.xxx being the global IP adress you found on http://www.whatsmyip.org/ and the 8085 being the port number you are using on your router. This way you can use this APP anywhere in the world where you have an internet connection.

Tap the microphone icon and wait till the voice recogniser has started and speak one of the commands you entered in your Basic program and the connected led will go on or off. Neat Huh ???

Now connect the ESP's I/O ports with relays to a ledstrip like I have shown here https://lucstechblog.blogspot.nl/2016/05/esp8266-controlled-ledstrip.html or to a relay like I have shown here https://lucstechblog.blogspot.nl/2016/09/esp-relayserver.html or a string of neopixels like I described at the bottom of this story https://lucstechblog.blogspot.nl/2017/03/back-to-basic-basic-language-on-esp8266.html and start automating your home by voice commands.

That's it. A really easy way to control any devices you like by voice commands be it on your local network or from any place on the world.

Bonus

As I explained the Basic program is the same program I used for controlling the ESP by The Google Home Assistant. The consequence is therefore that you can now control the ESP in two different ways. The first is by the Home Assistant and the second is by the App on your phone. So you can control the ESP now also from rooms where there is no Home Assistant present or even from a location far way from your home.

Expansion

There is room for expansion.
Lets assume you have multiple ESP's setup for your projects. Each ESP can be equipped by the same Basic program but can have altered commands programmed. The only thing you will have to do is to alter the IP adress in the APP's field and you can adress another ESP.

If you want you can even alter the APP in such a way that you make some buttons on the screen that automatically connect to several ESP's.


Source code:

As usual you can find the source code for the APP and the APK file on github:

https://github.com/Lucvolders/ESP-Voice

So experiment and have fun.

Luc Volders

Friday, February 2, 2018

Wifi Voice Commander

For an index to all of my stories click here.

In the previous entry in this series I demonstrated how to make an App in Mit's App Inventor to control several relays attached to an ESP8266. You can re-read that story by clicking here:
https://lucstechblog.blogspot.nl/2018/01/wifi-relay.html

The reason why I showed you this method is that you do not want to have everybody in your house accessing certain electrical appliances. By using a web-page which you normally do when controlling an ESP it is easily accessible by anyone. By using an App on your phone anybody wanting to access the ESP must have the App and use the right commands to control the ESP. That gives you some extra security which can be usefull for certain purposes.

Next to that we can give the App some extra functionality: you can use spoken commands. 
Maybe you remember the Voice Command project in which I controlled an Attiny85 over bluetooth by speaking commands to my Android Phone.

Well that is exactly what we are going to do. Only we are going to do it over Wifi now. This way you can control lights/relays/pumps or whatever by speaking to them over your local network or from anywhere on the internet. Now how cool is that !!!

To achieve this you will need a program (App) on your Android phone and a program on the ESP8266. As discussed in the previous entry in this series devellopping an App for Android is easyly done with App Inventor from the MIT.

Don't worry. If this is not your thing you can download the App from the link at the bottom of this story. However then you are stuck with the commands I pre-programmed in them. You can also download the source code and alter it to your own whishes.

What ? No Iphone APP

Well lets face it, they are shamessly overpriced. I use an Alcatel Touch Pop C7 prepaid Android phone which I paid 90 Euro's for.
No way I can get an Apple Iphone for such a low price.

Next to that I do not know of any Apple App-Devellopment environment which is as easy to use as Mit's App Inventor for Android.

So till one of the readers is going to donate me a 700 Euro Apple Iphone you're out there on your own.

Lets start with making the App.

This is not going to be a tutorial on how to use App Inventor. I urge you to play around with this program and definitely have a look at all the provided examples. Ans also have a look on all the resources that can be found on the internet about App Inventor. But basically the program is so easy to use that you will be developping your own App's in no-time.

The Designer screen

Start with downloading the WifirelayVoice.aia source code to your computer. You can download it from my Github repositry with a link at the end of this story. Then point your browser to http://appinventor.mit.edu/explore/

On the right side of the screen click on the orange square that says "Create Apps"
Now you can log in with your Google credentials. If you do not have a Google account make sure you get one first.
In the projects menu on the top of your screen choose -- import project(.aia) from my computer.

 

When done your screen will look the same as above.

On the right side of the screen you will find a column with all the components that are used in your App. Clicking a component will highlight it on the example screen and in the last column on the right side of the screen you will be able to alter the settings for that component.

As you can see there are three labels that do nothing more as putting text on your screen. Then there is a textbox that allows you, when running the App, to fill in the IP adress of the ESP8266 that is going to be connected.

Next there is a large green Microphone button labeled "Voice_Input" which will activate the speechrecognizer.
And lastly there are three hidden components which are shown at the bottom of the designerscreen. The Web1 component connects the App to the internet. The TextToSpeech1 component (which was used in another project you can read by clicking here) translates written commands to spoken words. This way the app can give an audible feedback. And the SpeechRecognizer does the opposite: it translates your spoken words to plain text.




There is one small label on the screen called "Monitor".
This will display the command received from the textrecognizer on your screen. So you can see if the command you spoke was interpreted right. It is an extra check, besides the audible control, to help you analyse why a command was not executed.

By clicking on each of the components you can alter its properties like setting different backgroundcolors, altering text and fonts etc etc etc.




Just like in the previous project there is one thing I really want to Emphasize.
Look at the TextToSpeech1 component carefully. If you leave it at its default values the App will speak out any text in the language you have set as your default language in your phone. If that is English all is well. However my phone is set default to Dutch and therfore the text that will be spoken will sound very awkward. So if English is not the default language on your phone you can change the setting for the TextToSpeech1 component to English or alter the text in the program so it will speak your language.

When all is set switch, on the right (top) side of the screen, to the blocks section where the actual program will be written.

The Blocks

The program starts with initialising a global variable and setting it to empty.



 



Then the program waits till the button "Voice_Input" is clicked. If that happens the Speechrecognizer is started.

When speech has been detected the program really starts its magic.
First the label "Monitor" gets the text that is spoken. This way you can check wether the Speechrecognizer indeed recognized what you spoke.
The variable "global_input 1" gets set to the words that have been recognized.

Next a set of if-then statements check wether it is a usable command that is recognized. The commands you can use are:

- living room on
- living room off
- garage light on
- garage light of
- all lights on
- all lights off

You can alter these to your own purposes or liking. You can even alter them in your native language as long as you keep in mind that you should set the language in your phone also to your native language. You can do that in the designer screen as discussed earlier.

Let's look at one of the blocks.



the IF section tests wether the command is valid.
If that is true the Then section is performed.

In the Then section a web-command is build starting with "http://" followed by getting the IP-Adress you filled in in the main screen, and then the actual command that is getting send to the ESP. In this case that is "/sw2on".

So the complete command send over the web is: http://xxxx.xxxx.xxxx.xxxx/sw2on

And last but not least there will be an audible feed back in this case being "Garage light on"

And that's all that is to it.
Like I told you: devellopping App's with App Inventor is really easy.

You can expand this App with as many commands as you like. Just make sure you will add these command also to the ESP program which is discussed next.

The ESP8266 program.

Now the App is ready we'll have a look at the ESP side of this project. This is the same as in the previous project in this series where we used buttons to control the ESP.

This is written in C++ (generally called Arduino code) and edited and uploaded with the Arduino IDE. So I presume that you have installed the ESP8266 prerequisites in the Arduino IDE. If not do so now. The internet is full of tutorials for this so that should be no problem.

Let's have a look at the program.


 /* =================================================  
 *  Controlling NodeMCU through an Android APP  
 *  
 * Devices  
 *  ==> Relay1 represented by led 1  
 *  ==> Relay2 represented by led 2  
 *    
 * Groups   
 *  ==> All Devices ON/OFF  
 *  
 * Program by Luc Volders  
 * http://lucstechblog.blogspot.nl/  
 *=================================================*/  
 #include <ESP8266WiFi.h>  
 WiFiClient client;  
 WiFiServer server(80);  
 const char* ssid = "XXXXXXXXXX";  
 const char* password = "YYYYYYYYYY";  
 String command =""; // Command received from Android device  
 // Set led/relay Pins  
 int switch1 = 13;  
 int switch2 = 15;  
 void setup()  
 {  
  Serial.begin(115200);  
  pinMode(switch1, OUTPUT);   
  pinMode(switch2, OUTPUT);    
  digitalWrite(switch1,LOW);  
  digitalWrite(switch2,LOW);  
  connectWiFi();  
  server.begin();  
 }  
 void loop()  
 {  
   client = server.available();  
   if (!client) return;   
   command = checkClient ();  
      if (command == "sw1on") digitalWrite(switch1,HIGH);  
   else if (command == "sw1off") digitalWrite(switch1,LOW);  
   else if (command == "sw2on") digitalWrite(switch2,HIGH);     
   else if (command == "sw2off") digitalWrite(switch2,LOW);  
   else if (command == "allon")   
   {  
    digitalWrite(switch1,HIGH);  
    digitalWrite(switch2,HIGH);  
   }  
   else if (command == "alloff")   
   {  
    digitalWrite(switch1,LOW);  
    digitalWrite(switch2,LOW);  
   }  
   sendBackEcho(command); // send command echo back to android device  
   command = "";  
 }  
 /* connecting WiFi */  
 void connectWiFi()  
 {  
  Serial.println("Connecting to WIFI");  
  WiFi.begin(ssid, password);  
  while ((!(WiFi.status() == WL_CONNECTED)))  
  {  
   delay(300);  
   Serial.print("..");  
  }  
  Serial.println("");  
  Serial.println("WiFi connection established");  
  Serial.println("IP number is : ");  
  Serial.print((WiFi.localIP()));  
 }  
 /* check command received from Android Device */  
 String checkClient (void)  
 {  
  while(!client.available()) delay(1);   
  String request = client.readStringUntil('\r');  
  request.remove(0, 5);  
  request.remove(request.length()-9,9);  
  return request;  
 }  
 /* send command echo back to android device */  
 void sendBackEcho(String echo)  
 {  
  client.println("HTTP/1.1 200 OK");  
  client.println("Content-Type: text/html");  
  client.println("");  
  client.println("<!DOCTYPE HTML>");  
  client.println("<html>");  
  client.println(echo);  
  client.println("</html>");  
  client.stop();  
  delay(1);  
 }  


First replace the XXXXXX and YYYYY in the program with the name and password for your router.

As you can see I use pin 13 and 15 as the pins where the leds will be attached (which equals D7 and D8 on the NodeMCU). You can alter them for your own needs or leave them as they are. If you added more voice commands in the App you also need to assign more pins for leds.

Halfway down the program you cann see the checks for the commands sw1on and sw1off etc.
If you have altered the names for the commands in your App you will need to alter them here to. And if you have added more buttons in the App you will need to add more commands here.

The program has some serial communication which can be helpfull when debugging it. If all works flawlessly you can comment the serial communication out or delete the relevant lines.

The rest of the program takes care of receiving the commands from the App.

Upload the program to your ESP8266 and you're set to go.

The Hardware




The hardware setup is straightforward as the above breadboard setup shows you. It is again the same setup as used in the previous project in this series. For those to lazy to click to it I will give the details here again.

I have used a red led and a blue led for demonstration purposes.




The schematics are pretty straightforward to and the photo shows you my setup on a breadboard.

You can change the leds for dedicated relays like I used in the realy commander project that you can find by clicking here.

Or use a general relay controlled by a BC547 as in the project you can find by clicking here.

You can also expand it to many more leds.

Where is it

Again this should look familiar if you have read the previous story in this line-up.

In the App you need to fill in the IP adress of the ESP8266. There are 2 ways to find that IP adress.

You can use the Arduino IDE and open the serial monitor at 115200 baud. The program will show you the IP adress.

The second way is to open the web-page of your router like I did in the following picture.




The four icons on the bottom of the screen are my wired stations: a raspberry network server attached to my Epson printer, My Computer called Nescio, my Domoticz home automation system and a Buffalo Nas.
The three top Icons represent my wireless stations. The first one is an ESP that is a permanent Thermometer that serves a web-page and send info to Thingspeak (all written in ESP-Basic), my Andoid Phone and an unknown station. That last one is the interesting one.



 These are the details for the unknown wireless device and there we have the IP number we are looking for.

Real life demo




Fill in the found IP-number in the App and push the microphone button. The screen changes to a Google voice input. Say your command and see the lights go on or off.


The video shows you how the app functions although that is very obvious. You can clearly hear how the speech component works.

So what's next ??

Well in the next stories I am going to show you how easy this is accomplished with ESP-Basic instead of using C++ as in this story. 




Source code
 
As always I herebye give you the complete source code as well as the Android APK file that can be direct installed on your phone.

 https://github.com/Lucvolders/Wifi-Voice-relay
 
 

So till next time
Have fun

Luc Volders

Friday, January 19, 2018

Wifi Relay

I have shown you how to control leds and relays over wifi before on this blog. The first time was in May 2016 when I showed you how to control a ledstrip with an ESP8266. You can re-read that story by clicking here.

The second story was about a relay-server that could be accessed over the internet but as a bonus also has manual control. Something I have not seen before and still have not seen in other stories (however the internet is a big place so I can be mistaking). You can re-read that story by clicking here.

These projects and most projects you can find on the internet have one thing in common and that is that they all work using a web-page to send commands to the ledstrips, relays etc.

There might be moments that you want to control things over wifi but not use a webpage for this. An example would be that you need the project being accessed from the internet or your local network, but not seen by everybody or controlled by everybody. I can imagine that you do not want your kids/visitors to control the lights or electronics in your mancave.

Therefore we are going to control an ESP8266 in a totally different way. We are going to make our own Android APP in Mit's App Inventor and write a special program for the ESP8266 that listens to the APP. There is no web-page, so the ESP8266 can only be controlled from the APP and even better: only if you know the exact commands to control it.
To be specific: the app will have buttons to put leds on and off and give an audible feedback in the form of spoken words like I demonstrated in the project that can be found by clicking on this link: http://lucstechblog.blogspot.nl/2017/10/phone-speak-to-me.html

Don't worry. Writing your own APP with App Inventor is very easy and what is more you will get all source code and even the Android APK file here so you can use it right away or alter things to your own needs.

Let's make the APP

This is not going to be a tutorial on how to use App Inventor. I urge you to play around with this program and definitely have a look at all the provided examples and tutorials. Also have a look at all the resources that can be found on the internet about App Inventor. But basically the program is so easy to use that you will be devellopping your own App's in no-time.

The Designer screen

Start with downloading the Wifirelay.aia source code to your computer. Then point your browser to http://appinventor.mit.edu/explore/
On the right side of the screen click on the orange square that says "Create Apps"
Now you can log in with your Google credentials. If you do not have a Google account make sure you get one first.
In the projects menu on the top of your screen choose -- import project(.aia) from my computer.



When done your screen will look the same as above.

On the right side of the screen you will find a column with all the components that are used in your App. Clicking a component will highlight it on the example screen and in the last column on the right side of the screen you will be able to alter the settings for that component.

As you can see there are a few labels that do nothing more as putting text on your screen. Then there is a textbox that allows you when running the App to fill in the IP adress of the ESP8266 that is going to be connected.
Next there is a tablearrangement which makes it possible to put components in rows and columns on the screen.
The tablearrangement contains labels with the name of the led and buttons for setting each led on or off.
The last two components are the Web1 and TextToSpeech1 component. They are very important and working in the back-ground. The Web component takes care of the communiaction between the ESP8266 and the App. The TextToSpeech1 component is going to be used to have the App speak out loud which function it has completed.



There is one thing I really want to Emphasize.
Look at the TextToSpeech1 component carefully. If you leave it at its default values the App will speak out any text in the language you have set as your default language in your phone. If that is English all is well. However my phone is set default to Dutch and therfore the text that will be spoken will sound very awkward. So if English is not the default language on your phone you can change the setting for the TextToSpeech1 component to English or alter the text in the program so it will speak your language.

When all is set switch on the right (top) side of the screen to the blocks section where the actual program will be written.

The Blocks




As you can see the program is divided in 6 blocks. Each block starts a function when one of the buttons on the screen of the phone is pressed. There are six buttons (Led 1 ON, Led 1 OFF, Led 2 ON, Led 2 OFF, ALL ON and ALL OFF) and as you can see each button has its own block of code.





Let's look at one of the blocks in detail.

This is the part of the program that will set Led 1 ON.
This block is executed when the button Led1_on is clicked.
If the button is clicked a web-url will be generated that consist of the IP adres which is found in the text-block IP_adress followed by "/sw1on".
With this information the Web1 function is called and when finished the message "Lamp 1 on" will be heard on your phone as an audible check.

That's all.
All blocks are similar exept for the command that is send through the web and the audible feedback.
Make sure that you alter the words that will be spoken in your own language or set the TextToSpeech1 component to English like discussed above. You can alter the "/sw1on" etc commands in the program, but then make sure that you also alter them in the C program for the ESP8266 that follows in the next steps.



The picture above shows you how the actual screen will look on your Android phone or tablet.

Expanding the APP

As you will have guessed by now it is very easy to expand this App for many more leds/relays. Use the designer screen to add more buttons and then switch back to the blocks screen. First click on a block and then choose Duplicate Block by clicking the right mouse button. Alter the name in the - when - section to the name of the new button. Do not forget that you will have to duplicate both the ON and OFF block and make sure you alter the "/sw1on" and "/sw1off" in the number you are going to add and alter also the speech component.

If you have altered the blocks then let Mit's App Inventor build a new APK for you. If you have not altered anything you can install the provided APK at the bottom of this entry.

The ESP8266 program.

Now the App is ready we'll have a look at the ESP side of this project.

This is written in C++ (generally called Arduino code) and edited and uploaded with the Arduino IDE. So I presume that you have installed the ESP8266 prerequisites in the Arduino IDE. If not do so now. The internet is full of tutorials for this so that should be no problem.

Let's have a look at the program.


 /* 

=================================================  
 *  Controlling NodeMCU through an Android APP  
 *  
 * Devices  
 *  ==> Relay1 represented by led 1  
 *  ==> Relay2 represented by led 2  
 *    
 * Groups   
 *  ==> All Devices ON/OFF  
 *  
 * Program by Luc Volders  
 * http://lucstechblog.blogspot.nl/  
 *=================================================*/  
 #include <ESP8266WiFi.h>  
 WiFiClient client;  
 WiFiServer server(80);  
 const char* ssid = "XXXXXXXXXX";  
 const char* password = "YYYYYYYYYY";  
 String command =""; // Command received from Android device  
 // Set led/relay Pins  
 int switch1 = 13;  
 int switch2 = 15;  
 void setup()  
 {  
  Serial.begin(115200);  
  pinMode(switch1, OUTPUT);   
  pinMode(switch2, OUTPUT);    
  digitalWrite(switch1,LOW);  
  digitalWrite(switch2,LOW);  
  connectWiFi();  
  server.begin();  
 }  
 void loop()  
 {  
   client = server.available();  
   if (!client) return;   
   command = checkClient ();  
      if (command == "sw1on") digitalWrite(switch1,HIGH);  
   else if (command == "sw1off") digitalWrite(switch1,LOW);  
   else if (command == "sw2on") digitalWrite(switch2,HIGH);     
   else if (command == "sw2off") digitalWrite(switch2,LOW);  
   else if (command == "allon")   
   {  
    digitalWrite(switch1,HIGH);  
    digitalWrite(switch2,HIGH);  
   }  
   else if (command == "alloff")   
   {  
    digitalWrite(switch1,LOW);  
    digitalWrite(switch2,LOW);  
   }  
   sendBackEcho(command); // send command echo back to android device  
   command = "";  
 }  
 /* connecting WiFi */  
 void connectWiFi()  
 {  
  Serial.println("Connecting to WIFI");  
  WiFi.begin(ssid, password);  
  while ((!(WiFi.status() == WL_CONNECTED)))  
  {  
   delay(300);  
   Serial.print("..");  
  }  
  Serial.println("");  
  Serial.println("WiFi connection established");  
  Serial.println("IP number is : ");  
  Serial.print((WiFi.localIP()));  
 }  
 /* check command received from Android Device */  
 String checkClient (void)  
 {  
  while(!client.available()) delay(1);   
  String request = client.readStringUntil('\r');  
  request.remove(0, 5);  
  request.remove(request.length()-9,9);  
  return request;  
 }  
 /* send command echo back to android device */  
 void sendBackEcho(String echo)  
 {  
  client.println("HTTP/1.1 200 OK");  
  client.println("Content-Type: text/html");  
  client.println("");  
  client.println("<!DOCTYPE HTML>");  
  client.println("<html>");  
  client.println(echo);  
  client.println("</html>");  
  client.stop();  
  delay(1);  
 }  

First replace the XXXXXX and YYYYY in the program with the name and password for your router.

As you can see I use pin 13 and 15 as the pins where the leds will be attached (which equals D7 and D8 on the NodeMCU). You can alter them for your own needs or leave them as they are. If you added more switches in the App you need to assign more pins for leds.

Halfway down the program you cann see the checks for the commands sw1on and sw1off etc.
If you have altered the names for the commands in your App you will need to alter them here to. And if you have added more buttons in the App you will need to add more commands here.

The program has some serial communication which can be helpfull when debugging it. If all works flawlessly you can comment the serial communication out or delete the relevant lines.

The rest of the program takes care of receiving the commands from the App.

Upload the program to your ESP8266 and you're set to go.

The Hardware



The hardware setup is straightforward as the above breadboard setup shows you. I have used a red led and a blue led for demonstration purposes.The delimiting resistor is 220 ohm, and attached to GPIO13 and GPIO15.






The schematics are pretty straightforward too and the photo shows you my setup on a breadboard.

You can change the leds for dedicated relays like I used in the relay commander project that you can find by clicking here.

Or use a general relay controlled by a BC547 as in the project you can find by clicking here.


Where is it

In the App you need to fill in the IP adress of the ESP8266. There are 2 ways to find that IP adress.

You can use the Arduino IDE and open the serial monitor at 115200 baud. The program will show you the IP adress.

The second way is to open the web-page of your router like I did in the following picture.


The four icons on the bottom of the screen are my wired stations: a raspberry network server attached to my Epson printer, My Computer called Nescio, my Domoticz home automation system and a Buffalo Nas.
The three top Icons represent my wireless stations. The first one is an ESP that is a permanent Thermometer that serves a web-page and send info to Thingspeak (all written in ESP-Basic), my Andoid Phone and an unknown station. That last one is the interesting one.


These are the details for the unknown wireless device and there we have the IP number we are looking for.

Real life demo

Fill in the found IP-number in the App and start pushing the buttons.

 

The video shows you how the app functions although that is very obvious. You can clearly hear how the speech component works.

Remember the great dictator series on this blog ???
Well then you will know what the next step will be. Commanding your lights with your voice.

Rests me only to give you the links to the source code and the APK file for direct install:

https://github.com/Lucvolders/Wifi-relay

Till then.
Have fun


Luc Volders

Friday, January 12, 2018

All stories organised

As this blog grows it becomes more and more difficult to find stories that might interest you.

So I made an index.
You can find it on the top-right side of this page right below the banner under the chapter pages. you can also click (or save) this link:

http://lucstechblog.blogspot.nl/p/index-of-my-stories.html

The index is divided in chapters that represent the point of intrest.
Under each chapter you will find the related stories and a short description.

Look at the sample below:

IOT (Internet of Things)

Back to Basic - introduction to ESP-Basic
Burglars invited - Be carefull what you publisch on public sites
Cheapskate - test wether your fridge light is on or off
ESP Controlled ledstrip - Control a single color ledstrip from a webpage
Google Home and the ESP8266 - Turn on your devices with your voice
IFTTT part 3 Alert over IFTTT - Tweet an alarm when your PIR has noticed motion
IFTTT part 4 - Send a notification to my phone
Rain Sensor - Get an alert on a webpage when it rains
Relay Server  - Control your lights over the internet AND manual


As you can see the point of intrest in this part is IOT and there are at the moment of this writing 9 stories related to IOT alphabetically arranged.

From now the index will be updated each time a new story is published and I will put a pointer to the index on each new story that is published.

Hope you find this usefull.

Luc

Friday, December 8, 2017

Lightmouse

While surfing over the internet I stumbled upon the Leap Motion Controller and I was flabbergasted. It was something that that really amazed me. Controlling your computer with your hands instead of a mouse. I wanted one.

Then I started thinking. Wouldn't it be possible to make something like this by myself. Well not actually tracking the movement of your fingers. That would be to complicated. But tracking the movement of a hand should be possible.

The general idea was that when your hand moves over a surface the place where the hand is, is in the shade and the rest would be in the light. And that is something you can easily measure by using LDR's (Light Dependend Resistors). So I made a simple setup.



I used two small breadboards next to eachother. This makes it possible to put the LDR's on the breadboard at a certain distance. I attached the LDR's to the analog ports of the Arduino Pro Micro.

Mind you: for this project you really need an Arduino Pro Micro as this has a real USB interface and not a USB-serial interface. We will be needing the USB interface for simulating a mouse.

The LDR's have a pull down resistor for more accurate measuring. To calculate which value the pull-down resistor needs in your particular case use the formula described in this article.
If you do not use the right pull-down resistors the setup will not function  as intended and react poorly at your movements.
In my setup I am using 10K resistors because my setup is intended to work in full light. However if you are using this in more ambient circumstances make sure you alter the values of the resistors using the described formula in the formentioned article.

The first tests were done by just reading the values the LDR's would give when I moved my hand above it. And the results were very satisfying.

Now I had to find the right formula for making something usefull out of this.

Let us look at the following simplified setup.





The square represents the total setup and the black dots represent the LDR's.

If I want to go left my hand has to cover LDR A and C.
For going right it has to cover B and D
Up means covering A and B
Going down can be achieved by covering C and D

By covering the individual leds I indicate that I want to go diagonal.

This can be easily represented in a formula.
Think about this.
If I cover LDR A and C they will get less light as LDR B and D.

X direction =  (A+C) - (B +D)

If the outcome is negative I am moving left. If the outcome is positive I am moving right. Remember that the LDR's give a higher value where there is more light.

Y direction = (A+B) - (C+D)

Same drill as the X direction.

The individual leds are even easier.

If the Value of LDR A drops then I am covering the LDR and want to move UP-LEFT.

And this is really all that is to it.

The only thing I need to test is wether I am actually moving my hand above the setup.
That can easily be done by the following test:

  x = ((A+C) -(B+D));
  y = ((A+B )- (C+D));
  axy = (abs(abs(x)-abs(y)));
 
  if ((abs(x)>25) && (abs(y)>25))

Look at this formula. It just tests if the difference between X and Y is bigger as 25 and that indicates that something is going on above the LDR's. I have substituted the formula in my program by using the variables s1 to s3 which represent the analog lines A0 to A3.

Now we only have to tell the Arduino Pro Micro that it has to act as a mouse by the following command:

void setup()
{
  Mouse.begin();
}

And here we get at a more difficult part of the program. Look at the following lines:

  if ((x<0) && ((abs(x))>(abs(y))) && (y>0) && (axy>100))
    {
      //going left
      Mouse.move(-10, 0, 0);
    }


Let's examine it in detail.

 if ((x<0) && ((abs(x))>(abs(y))) && (y>0)

The first part just tests if  X = smaller as 0. If that is true then this means you want to move left.
As a safety measure I also test if the X value is really bigger as the Y value. That indicates that there is indeed less light on the X LDR's.

&& (axy>100))

This last part tests wether there is a significant difference between the X and Y values. That is to make sure it is my hand moving above the LDR's and not just some shade of a cloud passing by.

Just one last thing to do:

Mouse.move(-10, 0, 0);

And that is why you need to use the Arduino Pro Micro.
This command actually moves the mousepointer of your PC (or Raspberry) to the left by 10 pixels. The 0 following the -10 makes sure that the mouse stays on the same line (Y-coordinate( and just the X moves. The second 0 indicates that the scroll wheel of the mouse should do nothing.

So here is the full listing:



/*Light Mouse
  Luc Volders

Decides on which way the mouse will go by checking
which LDR's get light on and which do not

using 4 LDR's put in a square form
top:    A1, A3
bottom: A0, A2
*/


int s1, s2, s3, s4, x ,y, axy;


void setup()
{
  Mouse.begin();
}

void loop() {
  
  s1 = analogRead(A0);
  s2 = analogRead(A1);
  s3 = analogRead(A2);
  s4 = analogRead(A3);

  x=((s3+s4) -(s1+s2));
  y=((s1+s3)-(s4+s2));
  axy = (abs(abs(x)-abs(y)));
  
  if ((abs(x)>25) && (abs(y)>25)) // test for action
  {
  if ((x<0) && ((abs(x))>(abs(y))) && (y>0) && (axy>100))
    {
      //going left
      Mouse.move(-10, 0, 0);
    }
  if (((x>0) && (y>0)) && (axy>100))
    {
      //going right
      Mouse.move(10, 0, 0);
    }
  if (x<0 && y<0 && axy>100)
    {
      //going up
      Mouse.move(0, -10, 0);
    }
  if (x<0 && (abs(x)<abs(y))&& y>0 && axy>100)
    {  
      //going down
       Mouse.move(0, 10, 0);
    }
  // -------------------------------------------------------
  // schuin
  // -------------------------------------------------------
    if (((x<0) && (y<0)) && (axy<100))
    {
     // moving left-up
      Mouse.move(-10, -10, 0);
    }
    if (((x>0) && (y<0)) && (axy<100))
    {
      //moving right-up
      Mouse.move(10, -10, 0);
    }
    if (((x<0) && (y>0)) && (axy<100))
    {
      //moving left-down
      Mouse.move(-10, 10, 0);
    }
    if (((x>0) && (y>0)) && (axy<100))
    {
      //moving right-down
      Mouse.move(10, 10, 0);
    }
  }
  
  delay(50);
}


After I was sure everything worked fine on the breadboard setup I made an experimental setup using a cardboard.


And after thorough testing I sat behind my computer and designed a casing. This was done in Tinkercad. This design proves that Tinkercad is quite capable.








And this is how it looks in real life.



And man does it work !!!!



I am sorry the video is of such a bad quality but it is the best I could do in my crammed man/cave. I had to make the video using one hand and using the other hand to operate the device.

It also works on my Raspberry Pi and using a USB/OTG adapter it also works on an Android device. I tried it on my Phone and on a tablet with great results.

A lot of possibilities come to my mind for this device. Using it as a joystick for games is obvious. How about adapting the hardware and making a remote for a model boat or car. Gesture controlling a Power-point presentation is another option. Use your imagination.

You can as always find all files on my Github repositry.
https://github.com/Lucvolders/Light-Mouse

So try this for yourselves and have fun.
Till next time

Luc Volders