Our Data
For our project the main source of data is the image files we are filtering. So far we have used images in JPEG and PNG format. These formats represent images as a 3D matrix of pixels. At each individual row and column index there are three pixels ranging in value from 0 to 255 representing the red, green, and blue components of the color of the image at that location. Our filters are also in PNG or JPEG format, which allows us to superimpose the filter onto the given input images and output a filtered image.
What We've Done
Original Goals
The overall goal of our project is to create filters similar to those in Snapchat using digital signal processing tools. In order to use this, we need facial recognition software to detect faces that we can then add hats, ears, and such on the face.
Along with this, we want to ultimately improve the facial recognition algorithms we use to be even better for our filters.
​
The overall goal of our project is to create filters similar to those in Snapchat using digital signal processing tools. In order to use this, we need facial recognition software to detect faces that we can then add hats, ears, and such on the face.
Along with this, we want to ultimately improve the facial recognition algorithms we use to be even better for our filters.
​
Design Processes and DSP Tools
The first step was to find software that supported facial recognition and was easy enough to use for our goals. We explored different options, such as C++ with Visual Studio, open source cv software, but ultimately we decided on using the Computer Vision and Image Processing toolboxes in Matlab.
​
These toolboxes provide functions that detect faces, noses, mouths, and eyes using the Viola-Jones algorithm. The Viola-Jones algorithm is a method of facial detection for computer vision which performs facial detection by using a variety of feature objects and comparing them to different regions of the image. These feature objects borrow from the idea of Haar wavelets by creating shapes which represent areas of dark and light which are common to most faces. The algorithm is designed using cascade architecture, meaning that it begins by using the simplest features to classify the different regions of an image as either not a face or possibly a face. The regions which are deemed to definitely not be a face are ignored in subsequent iterations and more complex features are used to analyze the remaining image sections. The process of classifying different regions of the image as not a face or possibly a face can be repeated as many times as necessary to achieve the desired detection of false positive rate. Because the classifier ignores sections of the image determined to not be facial features, the entire algorithm can run in a very short time with a high level of accuracy.
Examples
We will add pictures to our gallery of images with just boxes and no filters showing both the correct and incorrect facial detection. Below are some of the successful images with filters we have generated. Our process for generating these images with filters is detecting boxes around faces and facial features, resizing the image, removing the background of the filter, and replacing the appropriate parts of the original image with the filter.
Challenges
There are two main challenges we have run into so far. The facial feature detection algorithms we are currently using are imperfect. For example, multiple noses have been detected on photos where there is only one nose. The other challenge is not using live video. Originally, the goal was to implement filters on live video feed using webcams from our laptops, but we do not have the Matlab toolboxes required on our laptops.
Upcoming Tasks
1. Implement a Different Algorithm
​
We chose the Viola-Jones algorithm to perform our initial testing for facial detection but we want to try to implement one or two other methods to see if there is a more efficient way to handle facial detection or if we can use the methods from other methods to improve the Viola-Jones algorithm in some way. We have identified two possible methods for this which we would like to look into more. The first method, called the Deep Dense Face Detector is a relatively new method of facial detection which uses neural networks to very accurately detect and recognize faces. One of the important benefits of this algorithm is that it is able to detect faces which have been rotated, one of the main weaknesses of the Viola-Jones algorithm.
​
Another method we could possibly try to implement is the method of eigenfaces. This method was developed by Sirovich and Kirby and uses a collection of faces to establish a basis which can then be used to evaluate other images. Wong, Lam, and Siu describe a method for facial detection which makes use of eigenfaces and genetic algorithms to analyze images and determine if a face is present. This method can detect faces which have been rotated, which again addresses a major concern with the Viola-Jones algorithm.
​
2. Improving Current Algorithm Using Training
​
One way to make the Viola-Jones algorithm more accurate is by training is to teach it to better recognize faces in different situations. This training can be performed through built in functions of the Matlab Computer Vision toolbox. The detector is given positive and negative images to learn which features to look for in images to find similar objects. By doing this we can see if it is possible to create a better training set than the default set built into the toolbox and thus improve our detection accuracy.
​
3. Attempt Implementation with Video Files
​
We would like to be able to apply our filters to video files in addition to still images, which will require us to use addition algorithms to track moving objects once the faces have been initially detected. In order to do this we can use the KLT algorithm in addition to the Viola-Jones algorithm we are currently implementing. The KLT algorithm can use the location of the face detected and track points on the face as it moves, keeping the filter in place even if the face is rotated, adding versatility not present in the normal Viola-Jones algorithm.
​
4. Create Blurring Filters Using 2D Convolution
​
We plan to use the facial detection to sort out matrices from the original image. Then to get a blurring effect we will convolve (2D convolution) those matrices with matrices that we vary the values until we get the desired blurred effect. We plan to try this on the entire face and then on different facial features.
What We've Learned
While working on this project, our team has learned how to use the image processing and computer vision toolboxes in Matlab to create filtered images. We've also learned the theory behind different facial recognition algorithms such as the Viola-Jones algorithm.