Showing posts with label TensorFlow. Show all posts
Showing posts with label TensorFlow. Show all posts

Tuesday, April 18, 2023

Installing TensorFlow on Pop!_OS using Tensorman

Pop!_OS allows easy installation and management of Tensorflow using 'tensorman'.

Installing TensorFlow on Pop!_OS using Tensorman
Installing TensorFlow on Pop!_OS using Tensorman

First, make sure you have all the updates installed:

sudo apt update
sudo apt full-upgrade

Then, install the tensorman package:

sudo apt install tensorman

 In order to get Nvidia CUDA support, install the nvidia docker package:

sudo apt install nvidia-docker2

Tuesday, September 22, 2020

Using model.fit() instead of fit_generator() with Data Generators - TF.Keras

If you have been using data generators in Keras, such as ImageDataGenerator for augment and load the input data, then you would be familiar with the using the *_generator() methods (fit_generator(), evaluate_generator(), etc.) to pass the generators when trainning the model. 

But recently, if you have switched to TensorFlow 2.1 or later (and tf.keras), you might have been getting a warning message such as,

Model.fit_generator (from tensorflow.python.keras.engine.training) is deprecated and will be removed in a future version.
Instructions for updating:
Please use Model.fit, which supports generators.

Or,

Model.evaluate_generator (from tensorflow.python.keras.engine.training) is deprecated and will be removed in a future version.
Instructions for updating:
Please use Model.evaluate, which supports generators.


fit_generator() Deprecation Warning
fit_generator() Deprecation Warning

This is because in tf.keras, as well as the latest version of multi-backend Keras, the model.fit() function can take generators as well. 

Wednesday, January 1, 2020

Fixing the KeyError: 'acc' and KeyError: 'val_acc' Errors in Keras 2.3.x

Have you been using the 'History' object returned by the fit() functions of Keras to graph or visualize the training history of your models? And have you been getting a 'KeyError' type error such as the following since recent Keras upgrade and wondering why?


Traceback (most recent call last):
  File "lenet_mnist_keras.py", line 163, in <module>
    graph_training_history(history)
  File "lenet_mnist_keras.py", line 87, in graph_training_history
    plt.plot(history.history['acc'])
KeyError: 'acc'

The KeyError: 'acc' when attempting to read the history object
The KeyError: 'acc' when attempting to read the history object


Traceback (most recent call last):
  File "lenet_mnist_keras.py", line 163, in <module>
    graph_training_history(history)
  File "lenet_mnist_keras.py", line 88, in graph_training_history
    plt.plot(history.history['val_acc'])
KeyError: 'val_acc'

The KeyError: 'val_acc' when attempting to read the history object
The KeyError: 'val_acc' when attempting to read the history object

Well, this is due to a breaking change introduced in Keras release 2.3.0.

Sunday, December 29, 2019

How to Install TensorFlow 2.0 on Anaconda (with CUDA support)

TensorFlow 2.0 has been released for a few months now. This latest version comes with many new features and improvements, such as eager execution, multi-GPU support, tighter Keras integration, and new deployment options such as TensorFlow Serving.

So, it's time we all switched to TensorFlow 2.0.

TensorFlow 2.0


The Anaconda-native TensorFlow 2.0 packages are now available in the main conda repository. So, let's see how we can install TensorFlow 2.0 on Anaconda Python. This method will work on both Windows and Linux. And, if you have a CUDA capable NVIDIA GPU, you can enable GPU support as well.

Step 1.Creating a New Conda Environment


We'll start by creating a new conda environment. (I'll name it 'tensorflow2'. You can choose another name if you like):

conda create --name tensorflow2 python=3.7 anaconda


Create a new conda environment
Create a new conda environment

Tuesday, October 1, 2019

TensorFlow 2.0 Released!

After months in the Alpha state, Google has now released the final stable version of TensorFlow 2.0.
TensorFlow 2.0 aims at providing a easy to use yet flexible and powerful machine learning platform.

TensorFlow 2.0 Logo

The new version also hopes to simplify deployment of TF models to any platform by standardizing the model formats. You will be able to run TensorFlow models on a variety of runtimes, such as using TensorFlow Serving - a flexible, high-performance serving system for machine learning models, designed for production environments -, on browser or through Node.js using TensorFlow.js, and on mobile through TensorFlow Lite.

Thursday, November 22, 2018

Installing the New Anaconda Native TensorFlow Package

For a while now, the most reliable two ways to get TensorFlow installed is to either use the pip package, or compile from source.
Compiling TensorFlow from source takes hours, and still prone to errors (see "Failed Attempts at Building TensorFlow GPU from Source"). While the pip package is relatively easier, getting the GPU version of TensorFlow installed using pip was a hassle.

But not anymore. Because the conda native TensorFlow packages are here now.

Installing is quite easy.

Note: Don't install the pip and conda versions of TensorFlow on the same conda environment. If you already have the pip version installed uninstall it using,

pip uninstall tensorflow


To install the CPU version of TensorFlow, just run,

conda install tensorflow


To install the GPU version,

conda install tensorflow-gpu


Tuesday, July 10, 2018

Failed Attempts at Building TensorFlow GPU from Source

For the last 3 weeks, I've been trying to build TensorFlow from source. I wanted to get TensorFlow GPU version working on Windows with CUDA 9.2 and cuDNN 7.1. Since the pre-built wheels only work with CUDA 9.0, the only way we can get it working with 9.2 is to build it ourselves from source.

The Windows build of TensorFlow is done using CMake. The official instructions are here: https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/cmake

Unfortunately, as I found out after multiple attempts, the build process is not as simple as it sounds.
Every attempt I have made to build it failed so far.

But, I decided to post the steps I took - which didn't work - so that you all may be able use it as a reference if you decided to try it out yourselves. Again, note that these steps did not work.

First, I started with gathering all the dependencies to build on Windows 10:
  • Visual Studio 2015 Community Edition With Update 3 (14.0.25431.01) with C++
  • Anaconda Python 3.6.5
  • Git for Windows 2.18.0
  • Swigwin 3.0.12
  • CUDA Toolkit 9.2
  • cuDNN 7.1
  • CMake 3.11.3

Wednesday, November 15, 2017

TensorFlow Lite Developer Preview Announced

TensorFlow yesterday (14th Nov) announced the developer preview of TensorFlow Lite, a lightweight solution of TensorFlow for mobile and embedded devices, targeted for low-latency inference of on-device machine learning models.

TensorFlow Lite Logo
TensorFlow Lite Logo

TensorFlow Lite is an evolution of TensorFlow Mobile, and designed to be lightweight, cross-platform (Android and iOS for a start), and fast.

Through the Android Neural Networks API, TensorFlow Lite would be capable of utilizing purpose-built machine learning hardware in the devices as they become available.

A trained TensorFlow model can be converted to the TensorFlow Lite format (.tflite) using the provided converter, and deployed to the mobile app (Android or iOS), where the converted model gets executed using the TF Lite Interpreter.

TensorFlow Lite contains a C++ API with a Java API wrapper on Android.

It has out-of-the-box support for MobileNet, Inception V3, and Smart Reply Models.

Read more about TensorFlow Lite on the following links,

Related Links,




Build Deeper: The Path to Deep Learning

Learn the bleeding edge of AI in the most practical way: By getting hands-on with Python, TensorFlow, Keras, and OpenCV. Go a little deeper...

Get your copy now!

Saturday, September 9, 2017

What is the image_data_format parameter in Keras, and why is it important

We've talked about the image_dim_ordering parameter in Keras and why is it important. But since from Keras v2 changed the name of the parameter, I thought of bringing this up again.

As you know, Keras  is a higher-level neural networks library for Python, which is capable of running on top of TensorFlow, CNTK (Microsoft Cognitive Toolkit), or Theano, (and with limited support for MXNet and Deeplearning4j), which Keras refers to as 'Backends'.

The 'image_data_format' parameter in the keras.json file
The 'image_data_format' parameter in the keras.json file
Which backend Keras should use is defined in the keras.json file, which is located at ~/.keras/keras.json in Linux and Mac OS, and at %USERPROFILE%\.keras\keras.json on Windows.

The default keras.json file (default set to TensorFlow) would look like this,
 {  
   "epsilon": 1e-07,  
   "floatx": "float32",  
   "image_data_format": "channels_last",  
   "backend": "tensorflow"  
 }  
The "backend" parameter should either be "tensorflow", "cntk", or "theano". When switching the backend, make sure to switch the "image_data_format" parameter too. For "tensorflow "or "cntk" backends, it should be “channels_last”. For “theano”, it should be “channels_first”.

Tuesday, August 8, 2017

Using Bottleneck Features for Multi-Class Classification in Keras and TensorFlow

Training an Image Classification model - even with Deep Learning - is not an easy task. In order to get sufficient accuracy, without overfitting requires a lot of training data. If you try to train a deep learning model from scratch, and hope build a classification system with similar level of capability of an ImageNet-level model, then you'll need a dataset of about a million training examples (plus, validation examples also). Needless to say, it's not easy to acquire, or build such a dataset practically.

So, is there any hope for us to build a good image classification system ourselves?

Yes, there is!

Luckily, Deep Learning supports an immensely useful feature called 'Transfer Learning'. Basically, you are able to take a pre-trained deep learning model - which is trained on a large-scale dataset such as ImageNet - and re-purpose it to handle an entirely different problem. The idea is that since the model has already learned certain features from a large dataset, it may be able to use those features as a base to learn the particular classification problem we present it with.

This task is further simplified since popular deep learning models such as VGG16 and their pre-trained ImageNet weights are readily available. The Keras framework even has them built-in in the keras.applications package.

An image classification system built with transfer learning
An image classification system built with transfer learning


The basic technique to get transfer learning working is to get a pre-trained model (with the weights loaded) and remove final fully-connected layers from that model. We then use the remaining portion of the model as a feature extractor for our smaller dataset. These extracted features are called "Bottleneck Features" (i.e. the last activation maps before the fully-connected layers in the original model). We then train a small fully-connected network on those extracted bottleneck features in order to get the classes we need as outputs for our problem.

Wednesday, June 28, 2017

Machine UI : An IDE for Machine Learning, currently in Alpha

Machine UI, or just "Machine" as it's commonly referred, is an IDE for Machine Learning, which is currently in its Alpha stage. It has been designed to work with TensorFlow, and aims at simplifying setting up machine Learning experiments so that you spend more time experimenting, and less time configuring.


The interface of Machine UI
The interface of Machine UI (Note: This is a screenshot from their announcement video)

As per their announcement video, the machine learning experiments are set up visually. The input data, convolutions, and the outputs are placed as nodes on a graph. You can think of it as a more interactive version of the Tensor Board which comes with TensorFlow.

Tuesday, May 9, 2017

image_data_format vs. image_dim_ordering in Keras v2

If you have been using Keras for some time, then you would probably know the image_dim_ordering parameter of Keras. Specially, if you switch between TensorFlow and Theano backends frequently when using Keras.

When I first started using Keras for image classification, most of my experiments failed because I have set the image_dim_ordering incorrectly. Learning from my mistakes, last year I did a post on what image_dim_ordering is and why is it important.

The keras.json file houses the configuration options for Keras
The keras.json file houses the configuration options for Keras


In short, image_dim_ordering instructed Keras to properly rearrange the image data structure when passing to the backend:
Both TensorFlow and Theano expects 4D tensors of image data as input. But, while TensorFlow expects its structure/shape to be (samples, rows, cols, channels), Theano expects it to be (samples, channels, rows, cols). So, setting the image_dim_ordering to 'tf' made Keras use the TensorFlow ordering, while setting it to 'th' made it Theano ordering.

At least, that's how it used to work.

But recently, if you have updated to the latest version of Keras, you might have run into issues with the dimension ordering, even if you're sure that you set the image_dim_ordering correctly.

You may have gotten errors like,
 ValueError: The shape of the input to "Flatten" is not fully defined (got (0, 7,  
  50). Make sure to pass a complete "input_shape" or "batch_input_shape" argument  
  to the first layer in your model.  

It may seem to you that Keras has started to ignore your image_dim_ordering setting.

And you're right.

Thursday, March 9, 2017

Setting up TensorFlow with CUDA on Windows

I did the post about How to setup TensorFlow on Windows about a month back. I only covered setting up the CPU version of TensorFlow there, and promised that I'll do the guide for the GPU version soon. But I haven't had the change to come round to write the guide until now.

I guess better late than never. So, here's how to setup the GPU version of TensorFlow on Windows.

So, what do you need to get TensorFlow working on GPU?
You need to setup the following prerequisites, in that order.
  1. Microsoft Visual Studio 2015 (The free community edition of VS 2015 will work)
  2. CUDA 8.0
  3. cuDNN 5.1 for CUDA 8.0
Start by installing Visual Studio 2015. You can get the free community edition from here. When you install, make sure to select ‘Custom Installation’, and select ‘Visual C++’ in the programming language selection (by default, C++ is not selected). Once installed, check whether you have C++ capability by checking the ‘New Project’ options.

Visual Studio 2015 installed with C++
Visual Studio 2015 installed with C++

Thursday, February 16, 2017

TensorFlow 1.0 Released!

TensorFlow 1.0 has been released!

Just a week back I posted about the announcement of TensorFlow 1.0 and the new features coming. Now, it's finally here. It was released at the first TensorFlow Dev Summit held yesterday (15th Feb) at Mountain View, California.


The official TensorFlow website is now updated for the 1.0 release, which also now has a section for TensorFlow Fold which allows to work with input data of different shapes and sizes, and which has now been made open-source. The installation instructions have also been updated for the latest version, and they have added a guide on how to upgrade your application to TensorFlow 1.0.

The release notes for 1.0 suggests that all the anticipated features are here: The XLA compiler, Debugger, The Java API, Mobile support improvements, and much more. There's also news reports coming in which states that more machine learning models, such as SVMs, will come to TensorFlow. Better support for Keras is also hinted. I'm eagerly waiting for Keras to upgrade to support new features in TensorFlow 1.0.

Related posts:

Related links:

Build Deeper: Deep Learning Beginners' Guide is the ultimate guide for anyone taking their first step into Deep Learning.

Get your copy now!

Friday, February 10, 2017

TensorFlow 1.0 announced with better Mobile support and Java API

TensorFlow, with its r0.12 version gave the (much awaited) compatibility to Windows (check How to setup TensorFlow on Windows), and an Experimental API for the GO language. Now, they have announced the first major version - version r1.0 - of TensorFlow, with several exciting features.


While Python would still be the primary API - and would be the most complete API for TensorFlow - version r1.0 will introduce a new, experimental Java API. While it could be far from complete, and may take a few releases to stabilize, having the Java API may open TensorFlow to new possibilities. Also note that in the current release candidates of r1.0, you will need to compile TensorFlow from source in order to get the Java interface (instructions from TensorFlow GitHub page), and it's still only available for Linux and Mac OS.

Thursday, February 2, 2017

Setting up TensorFlow on Windows

TensorFlow, since it has been released just over an year ago, has gained a huge popularity with its capabilities. But the platforms it worked on was limited - there were only pre-packaged Python packages for Linux initially, while it gained a Mac OS compatible version along the way. Windows users were left wondering when (or if) there will be a Windows version of TensorFlow. There has been few unofficial builds for Windows, and some have attempted to build from source for Windows with various degrees of success. But, there was nothing official.

Now, I'm happy to learn that TensorFlow now officially supports Windows. The release notes suggests that Windows compatibility was present from TensorFlow v0.12.0 RC0, and it has been tested on Windows 7, Windows 10, and Windows Server 2016. It only supports 64-Bit Python 3.5. Both the standard Python distribution (from Python.org) and Anaconda Python are supported.

TensorFlow Logo


This was great news for me, since I was switching between Linux and Windows for my machine learning experiments. So, I gave a try to setup TensorFlow on Windows.

I've been using Anaconda Python for most of my experiments, so I created an Anaconda environment - named 'tensorflow' - with all the basic packages I need.
 conda create --name tensorflow numpy scipy scikit-learn pillow h5py mingw libpython  

Then I activated the environment I just created,
 activate tensorflow  

Now for the big step, installing TensorFlow from pip.

Friday, December 9, 2016

What is a Tensor?

Whether you are new to the field of Deep Learning, or might be coming from bit of a Machine Learning background and just dipping your toes into Deep Learning frameworks, you might be coming across the word "Tensor". Specially, with the increasing popularity of TensorFlow - the 2nd generation Machine Learning library by the Google Brain Team - the concept of the "Tensor" has also gained popularity.

You may be thinking that a Tensor is probably a mysterious and complicated thing, used exclusively in Machine Learning algorithms. But in reality, a Tensor is something quite simple and common.

So, what is a Tensor?

A representation of a Tensor
Image courtesy: Wikipedia - https://commons.wikimedia.org/wiki/File:Components_stress_tensor.svg

A Tensor is simply a data structure. A much more generic representation of any of the data collections we see in programming languages. However, unlike any other data structures, typically the position of a value in a Tensor has a meaning (we'll get to that in a bit).

A Tensor may have zero or more dimensions, which is given as the 'Order' of a Tensor.

A scalar, or a single number is a Tensor of order zero, which can also be thought as an array of dimension zero.

A Tensor of order one would be a Vector, or a one-dimensional array.

A Tensor of order two would be a matrix, or a two-dimensional array.

And so on... Tensors of any order can be made.

As I mentioned above, in typical use cases, the position of a value in a tensor - commonly referred to as the coordinates - has a meaning, and the value would transform according to the rules we define when the coordinates are changed.

For an example, we can define a tensor of order two which contains the pixel values of a grayscale image. In this case, the coordinates of the values of the tensor would represent the X and Y coordinates of the pixels of the image, and the values themselves would be the grayscale values at those coordinates. And, as you expect, if you transform the coordinates, the values would also change.


Related links:
http://www.physlink.com/education/askexperts/ae168.cfm
https://en.wikipedia.org/wiki/Tensor
http://mathworld.wolfram.com/Tensor.html

Build Deeper: Deep Learning Beginners' Guide is the ultimate guide for anyone taking their first step into Deep Learning.

Get your copy now!

Saturday, November 5, 2016

What is the image_dim_ordering parameter in Keras, and why is it important

Update 9/May/2017: With Keras v2, the image_dim_ordering parameter has been renamed to image_data_format. Check my updated post on how to configure it.

If you remember my earlier post about switching Keras between TensorFlow and Theano backends, you would have seen that we switched the image_dim_ordering parameter also when switching the backend. For TensorFlow, image_dim_ordering should be "tf", while for Theano, it should be "th".

The keras.json file contains the Keras configuration options
The keras.json file contains the Keras configuration options


So, what is this parameter, and where does it affect?

It has to do with how each of the backends treat the data dimensions when working with multi-dimensional convolution layers (such as Convolution2D, Convolution3D, UpSampling2D, Copping2D, … and any other 2D or 3D layer). Specifically, it defines where the 'channels' dimension is in the input data.

Tuesday, November 1, 2016

Switching between TensorFlow and Theano on Keras

Keras speeds up the task of building Neural Networks by providing high-level simplified functions to create and manipulate neural models. It itself does not provide the lower level neural and deep learning functions, but it’s rather meant to be run on an engine – which Keras refers to as a “backend” - which would provide such low-level functions.

Currently, Keras supports two such backends – TensorFlow and Theano.

The current version of Keras (v1.1.0 at the time of this writing) uses TensorFlow by default.

Most models written on top of Keras can be switched to a different backend without changes – at least it’s what’s said in the documentation. I’m yet to test this.

Which backend Kesas will use is defined in the Keras config file, which is located in the .keras directory in your home directory:
e.g.: on linux it would be ~/.keras/keras.json and on windows you can get to it on %USERPROFILE%\.keras\keras.json

For the default of using the TensorFlow backend, use the following config,
 {  
   "image_dim_ordering": "tf",  
   "epsilon": 1e-07,  
   "floatx": "float32",  
   "backend": "tensorflow"  
 }  

Notice the "backend" is set to "tensorflow" and "image_dim_ordering" is set to "tf".

To use the Theano backend, use the following,
 {  
   "image_dim_ordering": "th",   
   "epsilon": 1e-07,   
   "floatx": "float32",   
   "backend": "theano"  
 }  

Apart from the obvious "backend": "theano", note that "image_dim_ordering" is set to "th".

See my new post to see what the image_dim_ordering parameter in Keras does, and why is it important to set it properly.

Update: If you use Jupyter notebooks, and need to switch between TensorFlow and Theano backends quite often, fellow blogger desertnaut has a solution to dynamically switch the backend. Check out his solution at: Dynamically switch Keras backend in Jupyter notebooks

Related posts:
What is the image_dim_ordering parameter in Keras, and why is it important

Related links:
https://keras.io/backend/

Build Deeper: Deep Learning Beginners' Guide is the ultimate guide for anyone taking their first step into Deep Learning.

Get your copy now!