How to Build and Install the Latest Version of Dlib on Anaconda on Windows

Dlib is a toolkit for C++ and Python containing machine learning algorithms and tools for creating complex software to solve real world problems. Dlib provides algorithms for machine learning/deep learning, multi-class classification and clustering models, support vector machines, regression models, a large set of numerical algorithms for areas such as matrix manipulations and linear algebra, graphical model inference algorithms, and utility algorithms for computer vision and image processing. And due to C++ implementations backing most of these implementations, they’re optimized to the point that can be used in some real-time applications as well.

If you’re interested in facial recognition models or facial emotion processing, then Dlib is a library you should definitely try out.

Dlib v19.19 in action on conda on Windows

But with all the great features in Dlib, installing it has always been a little bit troublesome because of some specific dependency requirements it needs which had a habit of almost always conflicting with your other libraries. With the latest versions however, installing Dlib has become somewhat simple.

If you’re using Anaconda Python for your python experiments, like me, you’ll find that there is no native Dlib package in the native conda package list. In one of my earlier tutorials I showed how to install the Dlib conda package from the conda-forge channel in to your conda environment. The conda-force package works perfectly fine, and it’s still one of the quickest ways to install Dlib.

But if you really want the latest official package of Dlib installed (v19.19 as the latest at the time of this writing) then using the pip package is the way to go. In order to install the Dlib pip package you’ll first have to setup some dependencies.

Track any object in a video with Dlib Correlation Trackers

Training an object detector is bit of a complicated task. You need to have a proper training dataset with the relevant bounding boxes, and then use something like a HOG feature extractor with a SVM classifier for the detection - such as the Dlib Object Detection classes (link).

But that's a lot of work if you just need to track an object across a limited number of frames, or just need to detect motion or direction of movement. For that, we can easily use the Correlation Trackers feature in Dlib.

Object Tracking

See it in action,

Object Tracking in Action

Correlation Trackers - as their name suggests - works by correlating a set of pixels from one frame to the next.

Let's see how to build it.

Snapchat like Image Overlays with Dlib, OpenCV, and Python

You're probably familiar with Snapchat, and it's filters feature where you can put some cool and funny image overlays on your face images. As computer vision enthusiasts, we typically look at applications like these, and try to understand how it's done, and whether we can build something similar.

It turns out, we can make our own application with Snapchat like image overlays using Python, OpenCV, and Dlib.

Snapchat like Image Overlays with Dlib, OpenCV, and Python

So, how do we build it?

1. We'll first load the Webcam feed using OpenCV.
2. We'll load an image (in our example, and image for the 'eye') to be used as the overlay.
3. Use Dlib's face detection to localize the faces, and then use facial landmarks to find where the eyes are.
4. Calculate the size and the position of the overlay for each eye.
5. Finally, place the overlay image over each eye, resized to the correct size.

Let's start.

Wink Detection using Dlib and OpenCV

A couple of weeks ago, I was going through a tutorial for eye blink detection by Adrian at PyImageSearch. It was an excellent tutorial, which explained the use of Eye Aspect Ratio (EAR) in order to detect when an eye gets closed. Then, few weeks back, I was having a chat with Shirish Ranade, a reader of this blog and a fellow computer vision and machine learning enthusiast, on whether we can perform an action by winking at the computer. So, I decided to try out a code to detect winking.

Wink Detection Running with Dlib and OpenCV

It's an interesting idea to perform an action or a task just by winking at your computer. It can be thought as a form of gesture detection or facial expression detection as well. So, here's how you can build your own 'wink' detector for it.

We start by importing all the necessary packages,

 import numpy as np  
 import cv2  
 import dlib  
 from scipy.spatial import distance as dist  

OpenCV 3.2 and Dlib 19.4 Packages Now Available from Conda-Forge

Anaconda is an asset for us Machine Learning enthusiasts. Not only does it have the ability to create fully isolated Python environments, its pre-built packages for many operating systems and architectures helps you to spend less time setting up, and more time doing actual machine learning stuff.

In some scenarios, to get some Python packages to work in certain environments, getting them from Anaconda was the only way that worked. If you tried installing Dlib or OpenCV 3 on Windows 64-Bit, then you know the effort it takes to get them setup, if it wasn't for Anaconda.

OpenCV and Dlib working perfectly together, thanks to Conda

If you check my posts on Installing Dlib on Anaconda Python on Windows, and Installing OpenCV 3 on Anaconda Python 3.5 on Windows, you know how easy it is to use conda to install them on Python 3.5 64-Bit on Windows.

But there was a catch: The Anaconda registry only had OpenCV 3.1 and Dlib 19.0 - not the latest versions.

Extracting individual Facial Features from Dlib Face Landmarks

If you remember, in my last post on Dlib, I showed how to get the Face Landmark Detection feature of Dlib working with OpenCV. We saw how to use the pre-trained 68 facial landmark model that comes with Dlib with the shape predictor functionality of Dlib, and then to convert the output of into a numpy array to use it in an OpenCV context. We were able to get all 68 feature points on to our face image.

Dlib detecting the 68 Face Landmarks

The 68 feature points which the Dlib model detects include the Jawline of the face, left and right eyes, left and right eyebrows, the nose, and the mouth. So, what if you only want to detect few of those features on a face? E.g. you may only want to detect the positions of the eyes and the nose. Is there a way to extract only few of the features from the Dlib shape predictor?

There is actually a very simple way to do that. Here’s how.

Getting Dlib Face Landmark Detection working with OpenCV

Dlib has excellent Face Detection and Face Landmark Detection algorithms built-in. Its face detection is based on Histogram of Oriented Gradients (HOG) feature combined with a linear classifier, on a sliding window detection scheme (Ref. http://dlib.net/) and it provides pre-trained models for face landmark detection. It also provides handy utility functions like dlib.get_frontal_face_detector() to make our lives easier.

Dlib Face Landmark Detection in action

Note: Image used for testing is in the Public Domain - https://en.wikipedia.org/wiki/File:Arnold_Schwarzenegger_edit%28ws%29.jpg

To check out Dlib with it's native functions, you can try out the Dlib example from the official site: http://dlib.net/face_landmark_detection.py.html.It works well, but we can do better.

Although Dlib offers all the simplicity in implementing face landmark detection, it's still no match for the flexibility of OpenCV. (Simply put, Dlib is a library for Machine Learning, while OpenCV is for Computer Vision and Image Processing)

So, can we use Dlib face landmark detection functionality in an OpenCV context? Yes, here's how.

Installing Dlib on Anaconda Python on Windows

Update - 16/Dec/2019: There is now an easier way to install Dlib from the official pip package. Check out the new tutorial How to Build and Install the Latest Version of Dlib on Anaconda on Windows

Dlib is a Machine Learning library, primarily written in C++, but has a Python package also. It has many useful and optimized algorithms useful for machine learning, linear algebra, data structures, image processing, and much more available out-of-the-box.

"Dlib is a modern C++ toolkit containing machine learning algorithms and tools for creating complex software in C++ to solve real world problems. It is used in both industry and academia in a wide range of domains including robotics, embedded devices, mobile phones, and large high performance computing environments. Dlib's open source licensing allows you to use it in any application, free of charge." - dlib.net

One of the most popular features in Dlib is Facial Landmark Detection. Dlib installation ships with a pre-trained shape predictor model named shape_predictor_68_face_landmarks.dat, which as the name suggests, is trained to detect 68 facial keypoints including eyes, eyebrows, mouth, nose, face outline, etc.

Dlib's Facial Landmark Detection in action

You can view the sample code for face landmark detection here at the Dlib website, and download the pre-trained model from http://dlib.net/files/shape_predictor_68_face_landmarks.dat.bz2

(Make sure to unzip the .bz2 file once you download it)

Of course, Dlib is capable of much more than face landmark detection. I'm hoping to dig into some cool features of Dlib in later posts.

But first, we need to install it.