In last years, the algorithms complexity and video resolution has increased. The cost of a powerful computer which can run image recognition software is getting bigger as video cards prices go up. Alternatives must to be found, big companies started developing hardware components for specific tasks in order to hardware accelerate the algorithm. We developed a solution which accelerates image processing, like facial recognition, in hardware. The system is much cheaper and consumes less energy, than a regular PC and it is more powerful.
Objectives
The aim of the project was to develop a system which can run applications that are usually run on powerful machines, on much slower and cheaper ones using hardware acceleration.
Features-in-Brief
Hardware accelerated facial recognition
Voice assistance integration
Live streaming
Sends notifications on phone
Allows to open door using a pin
Temperature monitoring
Remote access to your system using a webserver
Project Summary
A hardware-accelerated security system that runs facial recognition in hardware, takes photos when motion is detected, allows you to unlock the door using facial recognition or a pin and even sends a notification on your phone.
It coordinates everything in the system, it doesn’t need a lot of resources.
Temperature readingReads the temperature from the custom IP and save it into a file to be used by web server.
Keyboard moduleCheck if there are any key presses. It also stores the last 4 keys to check the inserted pin. It the pin is correct, it unlocks the door.
Door ControlChecks if it receives any commands from server, a familiar face was detected or a correct pin was inserted. If one of the above conditions are met, it unlocks the door for few seconds.
Light ControlChecks if it receives any command from the web server to change the state of light.
Motion detectorGets a frame from camera and writes it into memory. Then, it sends it to the motion detector to check for any pixel changed. If the sum of all modified pixels is bigger than threshold, it starts the face detect part.
Face detectGet from hardware component the location of faces in image and sends them to facial recognition algorithm developed in python.
Live streamStream the swf live video stream from webcam to a remote location on port 8081.
Notification SystemThe system can send notifications to your phone. The phone checks periodically the web server for any events in the system. If the door was unlocked, the user receives a notification on the phone.
Web serverThe Zybo Z20 also hosts a web server using integrated 1Gbs ethernet port.
The Web server is created on Linux using apache, it also has a database for saving room logs and pictures when detecting motion.
Voice assistanceThe voice assistance is not running on Zybo, we are planning to hardware accelerate this one too, but for now, it is running on Amazon Alexa Dot with a custom skill wrote in C# for it.
Facial recognitionThe facial recognition is running the dlib library. It is the state of the art in facial recognition. The library uses deep metric learning tooling and the accuracy of algorithm is 99.38% based on Labeled Faces in the Wild benchmark.
Linux moduleA Linux module for Kernel was also developed to allocate the memory and convert the virtual address to physical. The DMA uses physical addresses and the Controller is dealing with virtual addresses. It allocate the size of a Full HD frame and sends to Controller App the virtual address to access it and the physical address to be able to transfer it to the DMA.
Beniamin Bia & Cosmin Nechifor

Comments