Story
„Alexa, who is at the door?“ – A face recognizing Arduino camera using AWS Rekognition for my grandmother
Ever wanted to know who rang your front door by asking Alexa?
Imagine it rings at your door and you could just ask Alexa who’s there. Without going to the window or opening the door, you know if it’s a friend or a stranger, because Alexa tells you the name or if it’s an unknown person.
We had the idea to build this solution, since our grandmother is nearly 80 years old and short-sighted. Due to the distance of her garden door, she can hardly see who rings the door when looking out of her window. To help her distinguish people ringing at the garden door, she should be able to ask Alexa, without even the need to go to the window or opening the front door.
To build the solution, we used:
- an Arduino Uno with ethernet shield (can easily be ported to an esp8266 for wifi connectivity)
- a camera (Arducam mini)
- a switch connected to one of the inputs of the Arduino
- Amazon Web Services: Alexa Skills Kit, Lambda, API Gateway, S3, Rekognition
We wrote an Arduino sketch that, when the switch is pressed, records an image with the arducam. Due to memory constraints, the image is not saved locally, but read from the camera buffer in chunks and directly posted using a http put request to Amazon Web Services, where a Lambda function (via AWS Api Gateway) processes the uploaded image, stores it in S3 and then passes it to AWS Rekognition. If the face is already known, the name of the person is returned. If an unknown person rings the door, the image is not deleted but stored to be tagged later on.
Using any of your Alexa devices, you can then use the smart cam skill we created.
There are two intents:
– „…who rang the door?“: If the person has been tagged before, Alexa will say e.g. „Peter rang the door“
– „…the person is Peter“: if a previously unknown person rings the door, this will associate the name to the face. Alexa will then confirm the association, and remember the face.
Example (known person rings the door)
- Someone rings the door
- Grandmother asks „Alexa, ask smart camera who rang the door“
- Alexa answers „Peter rang the door“
Example (unknown person rings the door)
- Someone rings the door
- Grandmother asks „Alexa, ask smart camera who rang the door“
- Alexa answers „I did not recognize the person, please tell me who rang the door. For example say: Alexa, tell smart camera that the person ist Peter“
- Grandma then says „Alexa, tell smart camera that the person is Thomas“
- Alexa answers „Thank you! I will remember that Thomas rang the door“
- Next time Alexa will recognize Thomas. (Saving my Grandma from evil people)
Example (camera did not capture a face)
- Someone rings the door
- Grandmother asks „Alexa, ask smart camera who rang the door“
- Alexa answers „The camera could not detect any face“.
Voice User Interface (VUI Diagram)
Hardware Assembly
- Connect the Arduino Uno and the Ethernet Shield
- Connect the Arducam as shown in the picture
- Connect the pushbutton, connect your ethernet cable
- Install the Arduino IDE and select “Arduino uno” as the board type.
- Adapt the IP Address to your address range, and hostname (where to send the JPEGs to), and upload the sketch
Software Stack
Software Setup
The project uses several AWS services, and setup guides for the whole project can be found on Github:
In addition, the projects code can be found on Github and is fully documented for your reading pleasure.
Alexa Skill
We have submitted our skill to Amazon however it is currently being certified. The Skill ID is amzn1.ask.skill.fa2a11c4-2a92-4094-978c-05744757c63b. Set up your own skill using the setup guide above!
COMMENTS