Friday, July 21, 2017

Snapchat like Image Overlays with Dlib, OpenCV, and Python

You're probably familiar with Snapchat, and it's filters feature where you can put some cool and funny image overlays on your face images. As computer vision enthusiasts, we typically look at applications like these, and try to understand how it's done, and whether we can build something similar.

It turns out, we can make our own application with Snapchat like image overlays using Python, OpenCV, and Dlib.

Snapchat like Image Overlays with Dlib, OpenCV, and Python
Snapchat like Image Overlays with Dlib, OpenCV, and Python

So, how do we build it?
  1. We'll first load the Webcam feed using OpenCV.
  2. We'll load an image (in our example, and image for the 'eye') to be used as the overlay.
  3. Use Dlib's face detection to localize the faces, and then use facial landmarks to find where the eyes are.
  4. Calculate the size and the position of the overlay for each eye.
  5. Finally, place the overlay image over each eye, resized to the correct size.

Let's start.

Tuesday, July 11, 2017

Codes of Interest Facebook Community is now Live!

We are on Facebook!

The Codes of Interest Page is now live on Facebook. I created the page so that our community can come together to share ideas, discuss about questions, quickly address issues you face with your Deep Learning / Machine Learning and Computer Vision experiments, and talk about what you would like to see from the Codes of Interest site.

The Codes of Interest Facebook Page

... and start discussing.

Encog-Node: Simple Machine Learning on Node.js

Before I got into serious Machine Learning and Computer Vision coding (which I mostly use Python for), I did a lot of my development on Node.js. Few years back (around 2012), I was trying to add a simple neural network to one of my Node.js applications. I looked around, but couldn't find a satisfactory node module which was lightweight and flexible. Around that time, I came across the Encog Machine Learning framework, which was created by Jeff Heaton, and was one of the most popular Machine Learning libraries for Java at the time. I noticed that there was a Javascript version of the Encog library, which worked surprisingly well, and set myself on to porting that to Node.js.

I released the first version of Encog-Node in early 2012, and the latest version v0.3.0 is now available from NPM -, and is recommended for anyone who wants to add lightweight, simple machine learning capabilities to their Node.js applications.

GitHub user Rui Cardoso contributed a lot for the latest release, with restructuring and cleaning up the codebase, and adding more examples.

You can install it by simply running,
 npm install encog-node  
in your node application.

Friday, July 7, 2017

Milestones of Deep Learning

Deep Learning has been around for about a decade now. We talked about how Deep Learning evolved through Artificial Intelligence, and Machine Learning (See "What is Deep Learning?"). Since its inception, Deep Learning has taken the world by storm due to its success. Here are some of the more significant achievements of Deep Learning throughout the years,

AlexNet - 2012

The AlexNet Architecture
The AlexNet Architecture (Image from the research paper)

  • Proved that Convolutional Neural Networks actually works. AlexNet - and its research paper "ImageNet Classification with Deep Convolutional Neural Networks" by Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton - is commonly considered as what brought Deep Learning in to the mainstream.
  • Won 2012 ILSVRC (ImageNet Large-Scale Visual Recognition Challenge) with 15.4% error rate. (For reference, the 2nd best entry at ILSVRC had 26.2% error rate).
  • 8 layers: 5 convolutional, 3 fully connected.
  • Used ReLU for the non-linearity function rather than the conventional tanh function used until then.
  • Introduced the use of Dropout Layers, and Data Augmentation to overcome overfitting.
Research Paper: ImageNet Classification with Deep Convolutional Neural Networks - Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton

Wednesday, June 28, 2017

Machine UI : An IDE for Machine Learning, currently in Alpha

Machine UI, or just "Machine" as it's commonly referred, is an IDE for Machine Learning, which is currently in its Alpha stage. It has been designed to work with TensorFlow, and aims at simplifying setting up machine Learning experiments so that you spend more time experimenting, and less time configuring.

The interface of Machine UI
The interface of Machine UI (Note: This is a screenshot from their announcement video)

As per their announcement video, the machine learning experiments are set up visually. The input data, convolutions, and the outputs are placed as nodes on a graph. You can think of it as a more interactive version of the Tensor Board which comes with TensorFlow.

Friday, June 9, 2017

Wink Detection using Dlib and OpenCV

A couple of weeks ago, I was going through a tutorial for eye blink detection by Adrian at PyImageSearch. It was an excellent tutorial, which explained the use of Eye Aspect Ratio (EAR) in order to detect when an eye gets closed. Then, few weeks back, I was having a chat with Shirish Ranade, a reader of this blog and a fellow computer vision and machine learning enthusiast, on whether we can perform an action by winking at the computer. So, I decided to try out a code to detect winking.

Wink Detection Running with Dlib and OpenCV
Wink Detection Running with Dlib and OpenCV
It's an interesting idea to perform an action or a task just by winking at your computer. It can be thought as a form of gesture detection or facial expression detection as well. So, here's how you can build your own 'wink' detector for it.

We start by importing all the necessary packages,
 import numpy as np  
 import cv2  
 import dlib  
 from scipy.spatial import distance as dist  

Tuesday, May 23, 2017

OpenCV 3.2 and Dlib 19.4 Packages Now Available from Conda-Forge

Anaconda is an asset for us Machine Learning enthusiasts. Not only does it have the ability to create fully isolated Python environments, its pre-built packages for many operating systems and architectures helps you to spend less time setting up, and more time doing actual machine learning stuff.

In some scenarios, to get some Python packages to work in certain environments, getting them from Anaconda was the only way that worked. If you tried installing Dlib or OpenCV 3 on Windows 64-Bit, then you know the effort it takes to get them setup, if it wasn't for Anaconda.

OpenCV and Dlib working perfectly together, thanks to Conda
OpenCV and Dlib working perfectly together, thanks to Conda

If you check my posts on Installing Dlib on Anaconda Python on Windows, and Installing OpenCV 3 on Anaconda Python 3.5 on Windows, you know how easy it is to use conda to install them on Python 3.5 64-Bit on Windows.

But there was a catch: The Anaconda registry only had OpenCV 3.1 and Dlib 19.0 - not the latest versions.