Object Detection : is a computer technology related
to Computer vision and Image Processing that deals with detecting
instances of semantic objects of a certain class (such as humans,
buildings, or cars) in digital images and videos.
Classification : is a process related to
categorization,the process in which ideas and objects are
recognized, differentiated and understood.
Localization : Placing a rectangular box for
classified objects.
Why YOLO?
The R CNN family of techniques used to localize the objects within
the image. The network does not look at the entire image, only at
the parts of the images which have a higher chance of containing an
object.
The yolo framework, on the other hand, deals with object detection
in a different way. It takes the entire image in a single instance
and predicts the bounding box coordinates and class probabilities
for these boxes. The biggest advantage using yolo is its superb
speed – it’s incredibly fast and can process 45 frames per second.
yolo also understands generalized object representation. This is one
of the best algorithms for object detection and has shown a
comparatively similar performance to the R CNN algorithms. So for
better results we are using yolov4 for developing custom object
detection.
How YOLO (You only Look once) Works?
Contents :
- What is YOLO and Why is it Useful?
- How does the YOLO Framework Function?
- How to Encode Bounding Boxes?
- Intersection over Union and Non-Max Suppression
- What is YOLO
Yolo is an algorithm that uses neural networks to provide real-time
object detection. This algorithm is popular because of its speed and
accuracy. It has been used in various applications to detect traffic
signals, people, parking meters, and animals. Yolo model process 45
frames per second then any other detecting models.
How does YOLO work?
After reading an Image YOLO splits the Image into 19 * 19 grids. For
better understanding will take 3 * 3 grid image example
From the above Image 3 * 3 grid each grid will be considered as an
individual image and sent to the Neural Network. If the Images label
founds in the grid will collect the 4 Parameters from the grid
- Height
- Width
- X_Coordinates from the mid Point of the label
- Y_Coor dinates from the mid Point of the label
How YOLO Train Images?
We need to pass the labelled data to the architecture in order to
train it. Suppose we have divided the image into a grid of size 3 *
3 and there are a total of 3 classes which we want the objects to be
classified into. Let’s say the classes are Bus, Car, and Person
respectively. So, for each grid cell, the label y will be an eight
dimensional vector:
Here,
- pc defines whether an object is present in the grid or not (it is the probability)
- Probability range 0 to 1.
- height, Width, x coordinate, y coordinate, specify the bounding box if there is an object
- c1, c2, c3 represent the classes. So, if the object is a car, c2 will be 1 and c1 & c3 will be 0, and so on
Let’s say we select the first grid from the above example:
Since there is no object in this grid, pc will be zero and the y
label for this grid will be:
Means that it doesn’t matter what height, width, x coordinate, y
coordinate, c1, c2, and c3 contain as there is no object in the
grid. Let’s take another grid in which we have a car (c2 = 1):
In the above image, there was a car , So YOLO will take the
mid-point of these objects . The y label for the center left grid
with the car will be:
Since there is an object in this grid, pc will be equal to 1. H , W
, X Coordinate , y Coordinate will be allocated . Since a car is the
second class, c2 = 1 and c1 and c3 = 0. So, for each of the 9 grids,
we will have an eight dimensional output vector. This output will
have a shape of 3 X 3 X 8
How does bounding boxes really work?
If we found a label in one grid then Yolo takes care of finding
their height, width, and both coordinates.
there is a (label) car in one grid
collecting the values from the grid and y will be =
height , width , X and y values are updated according to the label
found inside the grid . This will happen for every grid in an image.
Intersection Over Union:
IOU = AREA OF INTERSECTION / AREA OF UNION
Union over Intersection Measure the overlap between two bounding
boxes During training we calculate the IOU between the predicted
bounding box and the ground truth (the pre labelled bonding box we
aim to match )
If IOU value > 0.5 that predicted bounding is a good boundary or < 0.5 not a good boundary.
Non Max Suppression:
Non max suppression means in an image the grids are 19 * 19 Each
grid will think it is a midpoint. So on every iterations during
training we will get different boundary predicted positions like
So from among these we will select the boundary which has a high Pc
value. High Pc boundaries will be selected and these undergo the
IOU concept with the neighbor boundary in the grid. If The IOU
value is > 0.5 for that particular Pc it is considered as a perfect
boundary.
Yolo V4 on coco dataset:
Yolo V4 model were Trained on coco dataset where there will be 80
different labels. It is very easy to test our own labels using
pretrained weights. For this purpose we used a framework called
darknet.
Ex:
- person
- bicycle
- car
- Motor Bike
- aero plane
- Bus
- train
- truck
- boat
- traffic lights etc..
Finding detections on video :
How to do it for our own data?
For developing our own object detection model we need to select
labels that we want to detect and annotate them in yolo format. For
annotating the images we are using a tool known as labellmg.
labelling own data for object detection
For creating our own object detection we need to collect the data
from external sources. Collected 10 different label data from Kaggle
and some other websites now our main task is to annotate the Images
into Yolo format. For that purpose I am using an tool known as an
Labellmg tool.
In the first step select the directory of the images and apply a
rectangle box for every label in the image.
What happens if we apply a rectangle box?
we will get their x coordinate , y coordinate , height and width .
and wee need to specify class label also for every image . by doing
this process we will get images and their annotated file.
Custom object detection
Multiclass object detection
Developing an custom object detection using 10 Labels
The yolo implementations are amazing tools that can be used to start
detecting common objects in images and videos. However there are
many cases, when the object which we want to detect is not part of
the popular dataset. In such cases we need to create our own
training set and execute our own training. Vehicle and its License
plate detection are one such case. We have used yolov4 to detect the
desired classes.
- car – 0
- truck – 1
- Bus – 2
- Motor Cycle – 3
- Auto – 4
- car LP – 5
- truck LP – 6
- Bus Lp – 7
- Motor Cycle – 8
- Auto LP – 9
Later splitted all the dataset in training and validation set and
stored the path of all the images in file named train.txt and
valid.txt
Configuring Files
Yolov4 needs certain specific files to know how and what to train.
- Obj.data
- Obj.names
- Obj.cfg
obj.data
This basically says that we are training 10 classes, what the train
and validation files are and which file contains the name of the
object we want to detect.During training save the weight in the
backup folder.
- classes = 10
- train = train.txt
- valid = test.txt
- names = obj.names
- backup = backup
obj.cfg
Just copied the yolov4.cfg files and made a few changes in it.
- Set batch=24 to use 24 images for every training step.
- Set subdivisions=8 to subdivide the batch by 8 to speed up the training process.
- Set filters=(classes + 5)*3 , e.g filter=45.
After completing the training part total accuracy score 89%. For
better model performance we need to increase the accuracy score.
Steps used to increase the accuracy
- Data Augmentation
- Increasing the subdivisions for each batch
- Trained the model with 6000 Iterations
- After applying all 3 above steps and once again training has been done. the accuracy has increase by 94%
What is Data Augmentation
Techniques are used to increase the amount of data by adding
slightly modified copies of already existing data or newly created
synthetic data from existing data . So data augmentation involves
creating new and representative data. some images rotated with 40
and some with 180 , 270 etc.
- Rotation_range = 40
- width_shift_range=0.2
- height_shift_range=0.2
- shear_range=0.2
- zoom_range=0.3
Final result on a Video :