Friday, December 8, 2017

What is AlphaGo, AlphaGo Zero, and AlphaZero

AlphaGo – developed by the DeepMind team of Google – is an AI program which plays the board game Go.

A Go board (By Donarreiskoffer - Self-photographed, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=43383)
A Go board (By Donarreiskoffer - Self-photographed, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=43383)


The Go board game is an abstract strategy game, which has been invented in China over 2500 years ago. Despite its simple set of rules, Go is considered to be much more complex than Chess, and is one of the most studied strategy game of all time.

The AlphaGo Logo
The AlphaGo Logo

The AlphaGo uses a Monte Carlo tree search algorithm to find moves using the trained deep neural network which works as its knowledge core. AlphaGo was initially trained on a training set of over 30 million moves data from human Go matches. It was then further trained by letting it compete against copies of itself using reinforcement learning.

Wednesday, November 15, 2017

TensorFlow Lite Developer Preview Announced

TensorFlow yesterday (14th Nov) announced the developer preview of TensorFlow Lite, a lightweight solution of TensorFlow for mobile and embedded devices, targeted for low-latency inference of on-device machine learning models.

TensorFlow Lite Logo
TensorFlow Lite Logo

TensorFlow Lite is an evolution of TensorFlow Mobile, and designed to be lightweight, cross-platform (Android and iOS for a start), and fast.

Through the Android Neural Networks API, TensorFlow Lite would be capable of utilizing purpose-built machine learning hardware in the devices as they become available.

A trained TensorFlow model can be converted to the TensorFlow Lite format (.tflite) using the provided converter, and deployed to the mobile app (Android or iOS), where the converted model gets executed using the TF Lite Interpreter.

TensorFlow Lite contains a C++ API with a Java API wrapper on Android.

It has out-of-the-box support for MobileNet, Inception V3, and Smart Reply Models.

Read more about TensorFlow Lite on the following links,

Related Links,
https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/lite
https://developer.android.com/ndk/guides/neuralnetworks/index.html



Build Deeper: Deep Learning Beginners' Guide is the ultimate guide for anyone taking their first step into Deep Learning.

Get your copy now!

Sunday, October 29, 2017

Codes of Interest becomes 1 Year Old!

It's been one year!



I began the Codes of Interest Blog a year ago with a simple thought - to share my enthusiasm of Machine Learning and Computer Vision with everyone.

It's been a fulfilling journey.

Now, after a year, Codes of Interest has become a thriving community of like-minded and engaging Machine Learning and Computer Vision enthusiasts. We've covered over 40 tutorials on Deep Learning on libraries like Keras, TensorFlow, and Theano, and playing around Computer Vision libraries such as OpenCV and Dlib.

The constant feedback and suggestions from you all has allowed me to launch the Codes of Interest Facebook Community.
And, I was able to release my first book on Deep Learning: Build Deeper: Deep Learning Beginners' Guide, which is now available on Amazon.

Build Deeper: Deep Learning Beginners' Guide

I'm getting more and more emails and comments daily. So much that I'm having trouble responding to all.

So, I'm taking this opportunity to thank all the readers and subscribers of Codes of Interest for your input and encouragement. I can assure you that more exciting content is coming.

Wednesday, September 27, 2017

Migrating a Model to Keras 2.0

Keras v2.0 has been released for a couple of months now - v2.0.0 released on 5th May, 2017, while the latest version is 2.0.8 at the time of this writing. It brought in a lot of new features and improvements, but also made some syntax changes. Trying to run a code with the old syntax may result in anything from a flood of deprecation warnings, to not being able to run the code at all. Since there are many code examples online which uses the older syntax - including some older posts in Codes of Interest - it's better to know how to get such older syntax model to work on the 2.0 API.

The complete list of changes in Keras v2.0 was extensive, but the following list would help you to narrow down majority of the changes.

The most prominent change is the changing of image_dim_ordering parameter to image_data_format, and its associated values from "tf", and "th" to "channels_last" and "channels_first". We talked about this change in detail in our earlier post "What is the image_data_format parameter in Keras, and why is it important".

Likewise, in all the places where "dim_ordering" argument/parameter was used, it has been changed to "data_format".

All of the Convolution* layers have now need renamed to Conv*.
E.g. Convolution2D is renamed to Conv2D

Saturday, September 9, 2017

What is the image_data_format parameter in Keras, and why is it important

We've talked about the image_dim_ordering parameter in Keras and why is it important. But since from Keras v2 changed the name of the parameter, I thought of bringing this up again.

As you know, Keras  is a higher-level neural networks library for Python, which is capable of running on top of TensorFlow, CNTK (Microsoft Cognitive Toolkit), or Theano, (and with limited support for MXNet and Deeplearning4j), which Keras refers to as 'Backends'.

Which backend Keras should use is defined in the keras.json file, which is located at ~/.keras/keras.json in Linux and Mac OS, and at %USERPROFILE%\.keras\keras.json on Windows.

The default keras.json file (default set to TensorFlow) would look like this,
 {  
   "epsilon": 1e-07,  
   "floatx": "float32",  
   "image_data_format": "channels_last",  
   "backend": "tensorflow"  
 }  
The "backend" parameter should either be "tensorflow", "cntk", or "theano". When switching the backend, make sure to switch the "image_data_format" parameter too. For "tensorflow "or "cntk" backends, it should be “channels_last”. For “theano”, it should be “channels_first”.

Wednesday, August 30, 2017

Build Deeper: Deep Learning Beginners' Guide

I've been away from writing a post for about three weeks. That's because I've been preparing something exciting.

Today, I'm happy to announce the first book release from Codes of Interest - Build Deeper: Deep Learning Beginners' Guide.

Build Deeper: Deep Learning Beginners' Guide
Build Deeper: Deep Learning Beginners' Guide

Deep Learning has become a household name. It’s the bleeding edge in AI, and already achieving some phenomenal feats. Breakthroughs are happening daily, and the tech giants are not only pursuing it, they’re leading it.

Build Deeper: Deep Learning Beginners' Guide is the ultimate guide for anyone taking their first step into Deep Learning. Learn what Deep Learning is, and how it came to be. See what it's capable of, and its milestones. And get hands-on with building your first Deep Learning model.

All you need to get started is a bit of enthusiasm, and some basic programming skills.

Build Deeper: Deep Learning Beginners' Guide is now available from Amazon.



Tuesday, August 8, 2017

Using Bottleneck Features for Multi-Class Classification in Keras and TensorFlow

Training an Image Classification model - even with Deep Learning - is not an easy task. In order to get sufficient accuracy, without overfitting requires a lot of training data. If you try to train a deep learning model from scratch, and hope build a classification system with similar level of capability of an ImageNet-level model, then you'll need a dataset of about a million training examples (plus, validation examples also). Needless to say, it's not easy to acquire, or build such a dataset practically.

So, is there any hope for us to build a good image classification system ourselves?

Yes, there is!

Luckily, Deep Learning supports an immensely useful feature called 'Transfer Learning'. Basically, you are able to take a pre-trained deep learning model - which is trained on a large-scale dataset such as ImageNet - and re-purpose it to handle an entirely different problem. The idea is that since the model has already learned certain features from a large dataset, it may be able to use those features as a base to learn the particular classification problem we present it with.

This task is further simplified since popular deep learning models such as VGG16 and their pre-trained ImageNet weights are readily available. The Keras framework even has them built-in in the keras.applications package.

An image classification system built with transfer learning
An image classification system built with transfer learning


The basic technique to get transfer learning working is to get a pre-trained model (with the weights loaded) and remove final fully-connected layers from that model. We then use the remaining portion of the model as a feature extractor for our smaller dataset. These extracted features are called "Bottleneck Features" (i.e. the last activation maps before the fully-connected layers in the original model). We then train a small fully-connected network on those extracted bottleneck features in order to get the classes we need as outputs for our problem.