Installing OpenCV got easier

OpenCV is undoubtedly the unmatched de facto standard library for computer vision. Not only does it provide a near complete set of vision algorithms, the set of primitive graphics functions it provides to manipulate images makes it essential to many of our projects.

OpenCV being used when building a Keras CNN model

Installing the latest version of OpenCV used to be hard. The pre-built binaries available was never up to the tasks we wanted. Compiling from source was an option, but was tedious and time consuming.

Then, we got the anaconda versions of OpenCV from Conda-Forge, which we could simply install using,

conda install -c conda-forge opencv

Now, things are going to be simpler, as Anaconda native OpenCV packages are now available.

Using Multiple Cameras with OpenCV

As you know, OpenCV is capable of reading from any connected camera in your system, whether it's a built-in webcam (in a laptop) or a USB connected one.

But what if, you wanted to read from more than one cam at the same time?

Can OpenCV handle it?

OpenCV accessing 2 cameras at once

Yes, it can!

It's quite simple. Here's how to do it.

Setting up the OpenMV Cam

Few weeks back, we talked about OpenMV - the small embedded computer vision module with a built-in camera, that can be programmed to perform various vision tasks. It gives you the ability to bring computer vision in to your embedded projects.

After I first read about it, I was eager to get my hands on a kit. Their official site - openmv.io - offers international shipping, as well as links to local distributes in some countries. The shipping was quite fast.

The OpenMV Cam M7

First Impressions

The cam comes with a really nice clear plastic case. The headers (used to connect other shields on to the board) comes separate (pictured above). This gives you the option to solder them - or other types of headers - yourself if needed. Headers aren't essential for the basic usage of OpenMV. Both the case and the board itself has an excellent build quality.

The cam itself is smaller than I expected: about 2/3 the size of a credit card.

The size of the OpenMV Cam

Embedded Computer Vision with OpenMV

Have you ever wanted to put computer vision into an embedded device? But wasn't sure whether it's possible to cram in a computer vision algorithm into a small hardware device?

Well, that's what the OpenMV project is all about.

OpenMV is a programmable embedded device, with a built-in camera, that you can program with variety of vision tasks.

The OpenMV Cam M7 (Image source)

The OpenMV project has been started back in 2013 with a goal of becoming the "Arduino of Machine Vision". They now have the The OpenMV Cam M7 (pictured above), powered by a 216MHz ARM Cortex M7 processor and 512KB.

Track any object in a video with Dlib Correlation Trackers

Training an object detector is bit of a complicated task. You need to have a proper training dataset with the relevant bounding boxes, and then use something like a HOG feature extractor with a SVM classifier for the detection - such as the Dlib Object Detection classes (link).

But that's a lot of work if you just need to track an object across a limited number of frames, or just need to detect motion or direction of movement. For that, we can easily use the Correlation Trackers feature in Dlib.

Object Tracking

See it in action,

Object Tracking in Action

Correlation Trackers - as their name suggests - works by correlating a set of pixels from one frame to the next.

Let's see how to build it.

Using Data Augmentations in Keras

When I did the article on Using Bottleneck Features for Multi-Class Classification in Keras and TensorFlow, a few of you asked about using data augmentation in the model. So, I decided to do few articles experimenting various data augmentations on a bottleneck model. As a start, here's a quick tutorial explaining what data augmentation is, and how to do it in Keras.

The idea of augmenting the data is simple: we perform random transformations and normalization on the input data so that the model we’re training never sees the same input twice. With little data, this can greatly reduce the chance of the model overfitting.

But, trying to manually add transformations to the input data would be a tedious task.

Which is why Keras has built-in functions to do just that.

The Keras Preprocessing package has the ImageDataGeneraor function, which can be configured to perform the random transformations and the normalization of input images as needed. And, coupled with the flow() and flow_from_directory() functions, can be used to automatically load the data, apply the augmentations, and feed into the model.

Let’s write a small script to see the data augmentation capabilities of ImageDataGeneraor.

Extracting individual Facial Features from Dlib Face Landmarks

If you remember, in my last post on Dlib, I showed how to get the Face Landmark Detection feature of Dlib working with OpenCV. We saw how to use the pre-trained 68 facial landmark model that comes with Dlib with the shape predictor functionality of Dlib, and then to convert the output of into a numpy array to use it in an OpenCV context. We were able to get all 68 feature points on to our face image.

Dlib detecting the 68 Face Landmarks

The 68 feature points which the Dlib model detects include the Jawline of the face, left and right eyes, left and right eyebrows, the nose, and the mouth. So, what if you only want to detect few of those features on a face? E.g. you may only want to detect the positions of the eyes and the nose. Is there a way to extract only few of the features from the Dlib shape predictor?

There is actually a very simple way to do that. Here’s how.

Installing OpenCV from source on Anaconda Python on Ubuntu 16.10

I recently switched to Linux for my Machine Learning experiments, and I did a post on How to install Keras and Anaconda Python on Ubuntu 16.10.

Now, I wanted to install OpenCV on Ubuntu also. Since OpenCV does not have a pre-built package for Linux, it meant I had to compile OpenCV from source.

OpenCV 3.1 running on Lubuntu 16.10

Adrian of PyImageSearch has recently done a post about how to compile OpenCV on Ubuntu 16.04 using virtualenv. I followed his steps as a base, but had to make numerous adjustments to some of the packages which gets installed (e.g. libpng-dev, libhdf5-serial-dev) and the build commands due to the changes from Ubuntu 16.04 to 16.10, and because I'm using Anaconda environments rather than virtualenv.

I'll be installing OpenCV 3.1, and will be using the Lubuntu 16.10 virtual machine which I used in my earlier post. But the steps and commands will be exactly the same for any flavor of Ubuntu 16.10.

First, as a habit, get and install the latest updates for Ubuntu,

 sudo apt-get update   
 sudo apt-get upgrade   

Then (if you have not done already) install the necessary build tools,

 sudo apt-get install build-essential cmake git unzip pkg-config  

Then, we install the following packages which allows OpenCV to interact with various image and video formats,

 sudo apt-get install libjpeg8-dev libtiff5-dev libjasper-dev libpng-dev  
 sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev libv4l-dev  
 sudo apt-get install libxvidcore-dev libx264-dev  

Note: on Ubuntu 16.04, the package name for libpng was libpng12-dev. But on 16.10, it should be libpng-dev.

Getting Dlib Face Landmark Detection working with OpenCV

Dlib has excellent Face Detection and Face Landmark Detection algorithms built-in. Its face detection is based on Histogram of Oriented Gradients (HOG) feature combined with a linear classifier, on a sliding window detection scheme (Ref. http://dlib.net/) and it provides pre-trained models for face landmark detection. It also provides handy utility functions like dlib.get_frontal_face_detector() to make our lives easier.

Dlib Face Landmark Detection in action

Note: Image used for testing is in the Public Domain - https://en.wikipedia.org/wiki/File:Arnold_Schwarzenegger_edit%28ws%29.jpg

To check out Dlib with it's native functions, you can try out the Dlib example from the official site: http://dlib.net/face_landmark_detection.py.html.It works well, but we can do better.

Although Dlib offers all the simplicity in implementing face landmark detection, it's still no match for the flexibility of OpenCV. (Simply put, Dlib is a library for Machine Learning, while OpenCV is for Computer Vision and Image Processing)

So, can we use Dlib face landmark detection functionality in an OpenCV context? Yes, here's how.