Thursday, October 8, 2020

Accuracy of Deep Learning Models Over the Years

Over the years, there were many achievements in deep learning, many of which were directly related to the ImageNet Large Scale Visual Recognition Challenge (ILSVRC, or ImageNet challenge for short). We talked about some of those milestones in deep learning in the past and how their unique innovations have helped shape the deep learning landscape today.

Today let us look at how the accuracy of these significant models has increased over the years.

Deep Learning Models Over the Years
Deep Learning Models Over the Years


When reporting the accuracy of classification models two accuracy measures are typically used: Top-1 Accuracy, and Top-5 Accuracy.
  • Top-1 Accuracy - Where the highest probability/confidence prediction from the model matches the expected class
  • Top-5 Accuracy - Where the expected class is within the top 5 predictions of the model


Based on these two matrices, we can see how the model accuracies on the ImageNet dataset has improved over the years.

Year

Model

Top-1 Accuracy

Top-5 Accuracy

2012

AlexNet

0.633

0.846

2013

ZF Net

0.64

0.853

2014

VGG16

0.713

0.901

2014

VGG19

0.714

0.91

2015

ResNet50

0.749

0.921

2015

ResNet101

0.764

0.928

2015

ResNet152

0.766

0.931

2016

ResNet50V2

0.76

0.93

2016

ResNet101V2

0.772

0.938

2016

InceptionV3

0.779

0.937

2016

ResNet152V2

0.78

0.942

2016

Xception

0.79

0.945

2017

DenseNet121

0.75

0.923

2017

DenseNet169

0.762

0.932

2017

DenseNet201

0.773

0.936

2017

InceptionResNetV2

0.803

0.953

2018

NasNet

0.825

0.96


If we visualize these numbers, we will get a chart such as the following:

Accuracy of Deep Learning Models Over the Years
Accuracy of Deep Learning Models Over the Years



We can see that the models have had a gradual but steady increase in the accuracy - in both Top-5 and Top-1 accuracies - over the years. We can see this more clearly if we graph just the trendlines:

Trendlines of Top-1 and Top-5 Accuracies of Models Over the Years
Trendlines of Top-1 and Top-5 Accuracies of Models Over the Years

Based on these charts, we can make some interesting observations:
  • Within just 6 years - from AlexNet in 2012 to NasNet in 2018 - the top-1 accuracy has increased by around 20% and the top-5 accuracy by around 12%
  • The latest models are performing beyond human level accuracy. (It is measured that the accuracy level of a human in similar classification tasks is around 90-95% - Source:'Deep Residual Learning for Image Recognition')
  • As the accuracy increased, the gap between top-1 accuracy and the top-5 accuracy is also narrowing. This is another indication of the consistent improvements of the models
  • The top-1 accuracies of the latest models are now approaching the top-5 accuracies of the earlier models, showing how much deep learning models have improved in just a few years
There is more analysis we can perform and insights to gain from these models, which I will cover on future articles.

Stats and Referances:










No comments:

Post a Comment