There is so much to discover, and so little time.
This should be the headline of this weblog. Nevertheless this is something I discovered recently which I do not want to keep from you.
You may be aware that Google is heavily investing in AI. I already showed you how to use the Google Assistant, both the home build version and the smart-phone version, with your home automation projects in these stories:
https://lucstechblog.blogspot.nl/2017/05/google-home-and-esp8266.html
But now they have started another project which involves image recognition. You can find all details on AIY- projects: https://aiyprojects.withgoogle.com/
As with the Assistant they made this available for the Raspberry so a wide audience can start experimenting with this technology, which is really great. However the costs for this project rather high because it involves a dedicated image processing board next to a Raspberry.
So what if I tell you that you can start playing around with Image Recognition for virtually free ???
Read on !!
You know I have been coding Android App's with MIT's App Inventor. Well there is a derivate around (actually there are several) for some time. The name is Thunkable: https://thunkable.com/
Thunkable works the same as App Inventor but has some extra features which I am not going to discuss here. They will come up in a later blog-post. However there is one APP I have build with Thunkable which is pretty amazing. It is an APP that does image recognition. I am not telling you that I invented this APP myself. I just copied it from the Thunkable website and made some (very small) adjustements.
So get starting with the original App Inventor and also get an account with Thunkable, and learn to code your own Android APP's.
Making the APP.
The more experienced APP Inventor users can make this APP also in APP-Inventor. What you do need to do is to download the Artificial Intelligence extension from Pura Vida Extensions: https://puravidaapps.com/extensions.php and install that into APP-Inventor
So setup your Thunkable account on https://thunkable.com/ and download the Thunkable Live APP from the Google Playstore https://play.google.com/store/apps/details?id=com.thunkable.appinventor.aicompanion3&hl=en so you can test the app while building it. This is the same process as MIT's App-Inventor uses.
Start with creating a new APP and build your screen in the designer like the image shows you.
- Image 1 an image component found in the User Interface section
- Label 1 a label component also from the User Interface section
- Button 1 from the User Interface section
- Camera 1 from the Media section
- Image_recognizer1 from the Artifical Intelligence section
The last 2 components are invisible components that work in the background.
As usual I have used a wild color scheme. So make sure you use something to your own liking.
When the screen is designed to your liking switch over to the blocks part and make the following code:
And that is really all.
When the Button is clicked the camera of your phone will open the Camera application and you just take a picture like you normally would. When you think it is OK press the V for confirmation.
The text of the label will change into 'thinking' and the image is send into the cloud for image recognition.
As soon as is is r4eceived back the text of the label will change into what the image recognition software thinks it has found.
Getting the APP on your phone
Run it live or make an APK file (in the export menu) and download it to your computer and transfer it then to your phone. The easiest way to do that is to copy it to Google Drive and download it on your phone from your Google Drive.
When downloaded on your phone open the download folder and click the file. Give the Phone permission to install it and your done.
Running the APP
The App will work on modern phones and also on some pretty old ones. My faithfull Alcatel Pop C7 with Android 4.2.2 has no problems with it. And my new Nokia 5 certainly has no problems with it. So I am guessing it will work on yours to.
Pretty Amazing !!!
Well there is no 100% score with this as image recognition is still in it's infancy. However I had some pretty amazing results which I do not want to keep from you.
That's a miss however close !!
Future expansions.
I have just been playing with this and as a Proof of Concept it really is pretty amazing. But what can we do further with it.
First there is an extension that automates taking pictures. You could use that to trigger taking the picture automatically every few minutes.
Or you could have a sensor sending a signal over bluetooth that would trigger taking the picture. I have covered bluetooth communication between Android and an Arduino in these stories:
http://lucstechblog.blogspot.nl/2016/02/arduino-to-android-using-bluetooth.html
https://lucstechblog.blogspot.nl/2016/03/arduino-to-android-part-ii.html
Next you could test if the picture indeed contains what you expect like a person or a red item or whatever you come up with and have the APP send something back over wifi to an ESP or trigger an IFTTT action.
Use your imagination and have fun.
Till next time
Luc Volders