Object detection with Pytorch SSD MobileNet re-training Part (4)

colaboratory
2022.04.15

Introduction

Hello, this is swim-lover. Object detection is done with Python and Pytorch. I’ve just started Python, but I’m studying with the concept of “learning while using”.

In Part (2),object detection using SSD was performed.

In Part (3), object detection was performed using YoLov3.

In each case, object detection was performed using the trained data.

This time, we will create training data by re-training and perform object detection.

Reference Web Site

In writing this blog, I refer to the following GitHub.

pytorch-ssd/README.md at master · qfgaohao/pytorch-ssd
MobileNetV1, MobileNetV2, VGG based SSD/SSD-lite implementation in Pytorch 1.0 / Pytorch 0.4. Out-of-box support for retraining on Open Images dataset. ONNX and...
github.com
jetson-inference/docs/pytorch-ssd.md at dev · dusty-nv/jetson-inference
Hello AI World guide to deploying deep-learning inference networks and deep vision primitives with TensorRT and NVIDIA Jetson. - dusty-nv/jetson-inference
github.com

Preparing to download OpenImages

Mount Google Drivre in the Colab environment as before.

from google.colab import drive
drive.mount('/content/drive')

This time, we will build the environment under the machine folder.

cd drive/MyDrive/machine

Clone the ssd sample from github.

!git clone https://github.com/qfgaohao/pytorch-ssd.git
cd pytorch-ssd

Install boto3. It is referenced in the following open_images_downloader.py.

pip install boto3

Check the number of OpenImages images

You can see that there are 601 types of Open Images Classes.

pytorch-ssd/open_images_classes.txt at master · dusty-nv/pytorch-ssd
MobileNetV1, MobileNetV2, VGG based SSD/SSD-lite implementation in PyTorch. Out-of-box support for retraining on Open Images dataset. ONNX and Caffe2 support. E...
github.com

Learn about four classes, “Aircraft”, “Airplane”, “Bird”, and “Helicopter” from 601 types.

By adding option –status-only, you can know the number of data before actually downloading the Image.

!python3 open_images_downloader.py --stats-only --class-names "Aircraft","Airplane","Bird","Helicopter" --data=data/airobject

When I checked the execution results, I found that there were a total of 37584 Images and 77415 Boundingn Box Clounts in 4 classes in advance.

2022-04-13 09:02:36 - Download https://storage.googleapis.com/openimages/2018_04/class-descriptions-boxable.csv.
2022-04-13 09:02:36 - Requested 4 classes, found 4 classes
2022-04-13 09:02:36 - Download https://storage.googleapis.com/openimages/2018_04/train/train-annotations-bbox.csv.
2022-04-13 09:03:02 - Read annotation file data/airobject/train-annotations-bbox.csv
2022-04-13 09:03:23 - Available train images:  31225
2022-04-13 09:03:23 - Available train boxes:   68369

2022-04-13 09:03:23 - Download https://storage.googleapis.com/openimages/2018_04/validation/validation-annotations-bbox.csv.
2022-04-13 09:03:23 - Read annotation file data/airobject/validation-annotations-bbox.csv
2022-04-13 09:03:24 - Available validation images:  1551
2022-04-13 09:03:24 - Available validation boxes:   2238

2022-04-13 09:03:24 - Download https://storage.googleapis.com/openimages/2018_04/test/test-annotations-bbox.csv.
2022-04-13 09:03:26 - Read annotation file data/airobject/test-annotations-bbox.csv
2022-04-13 09:03:28 - Available test images:  4808
2022-04-13 09:03:28 - Available test boxes:   6808

2022-04-13 09:03:28 - Total available images: 37584
2022-04-13 09:03:28 - Total available boxes:  77415


-------------------------------------
 'train' set statistics
-------------------------------------
  Image count:  31225
  Bounding box count:  68369
  Bounding box distribution: 
    Bird:  43869/68369 = 0.64
    Airplane:  19979/68369 = 0.29
    Helicopter:  2799/68369 = 0.04
    Aircraft:  1722/68369 = 0.03
 

-------------------------------------
 'validation' set statistics
-------------------------------------
  Image count:  1551
  Bounding box count:  2238
  Bounding box distribution: 
    Airplane:  1005/2238 = 0.45
    Bird:  917/2238 = 0.41
    Aircraft:  182/2238 = 0.08
    Helicopter:  134/2238 = 0.06
 

-------------------------------------
 'test' set statistics
-------------------------------------
  Image count:  4808
  Bounding box count:  6808
  Bounding box distribution: 
    Airplane:  3213/6808 = 0.47
    Bird:  2636/6808 = 0.39
    Aircraft:  542/6808 = 0.08
    Helicopter:  417/6808 = 0.06
 

-------------------------------------
 Overall statistics
-------------------------------------
  Image count:  37584
  Bounding box count:  77415

Downloading OpenImages with a limited number of images

This time, instead of downloading all 37584 images in 4 classes, I try to limited download to 2000 images in consideration of time and storage size.

!python3 open_images_downloader.py --max-images=2000 --class-names "Aircraft","Airplane","Bird","Helicopter" --data=data/airobject

The image download was completed without any problems.

2022-04-13 09:13:21 - Requested 4 classes, found 4 classes
2022-04-13 09:13:21 - Read annotation file data/airobject/train-annotations-bbox.csv
2022-04-13 09:13:40 - Available train images:  31225
2022-04-13 09:13:40 - Available train boxes:   68369

2022-04-13 09:13:40 - Read annotation file data/airobject/validation-annotations-bbox.csv
2022-04-13 09:13:40 - Available validation images:  1551
2022-04-13 09:13:40 - Available validation boxes:   2238

2022-04-13 09:13:40 - Read annotation file data/airobject/test-annotations-bbox.csv
2022-04-13 09:13:42 - Available test images:  4808
2022-04-13 09:13:42 - Available test boxes:   6808

2022-04-13 09:13:42 - Total available images: 37584
2022-04-13 09:13:42 - Total available boxes:  77415

2022-04-13 09:13:42 - Limiting train dataset to:  1661 images (4199 boxes)
2022-04-13 09:13:42 - Limiting validation dataset to:  82 images (115 boxes)
2022-04-13 09:13:42 - Limiting test dataset to:  255 images (362 boxes)

-------------------------------------
 'train' set statistics
-------------------------------------
  Image count:  1661
  Bounding box count:  4199
  Bounding box distribution: 
    Bird:  2822/4199 = 0.67
    Airplane:  1057/4199 = 0.25
    Aircraft:  176/4199 = 0.04
    Helicopter:  144/4199 = 0.03
 

-------------------------------------
 'validation' set statistics
-------------------------------------
  Image count:  82
  Bounding box count:  115
  Bounding box distribution: 
    Airplane:  58/115 = 0.50
    Bird:  45/115 = 0.39
    Aircraft:  9/115 = 0.08
    Helicopter:  3/115 = 0.03
 

-------------------------------------
 'test' set statistics
-------------------------------------
  Image count:  255
  Bounding box count:  362
  Bounding box distribution: 
    Airplane:  165/362 = 0.46
    Bird:  144/362 = 0.40
    Helicopter:  30/362 = 0.08
    Aircraft:  23/362 = 0.06
 

-------------------------------------
 Overall statistics
-------------------------------------
  Image count:  1998
  Bounding box count:  4676

2022-04-13 09:13:42 - Saving 'train' data to data/airobject/sub-train-annotations-bbox.csv.
2022-04-13 09:13:42 - Saving 'validation' data to data/airobject/sub-validation-annotations-bbox.csv.
2022-04-13 09:13:42 - Saving 'test' data to data/airobject/sub-test-annotations-bbox.csv.
2022-04-13 09:13:42 - Starting to download 1998 images.
2022-04-13 09:13:49 - Downloaded 100 images.
2022-04-13 09:13:54 - Downloaded 200 images.
(Skip)
2022-04-13 09:15:12 - Downloaded 1600 images.
2022-04-13 09:15:18 - Downloaded 1700 images.
2022-04-13 09:15:23 - Downloaded 1800 images.
2022-04-13 09:15:28 - Downloaded 1900 images.
2022-04-13 09:15:34 - Task Done.

Re-training SSD Mobile Net

First, download the trained model.

!wget https://nvidia.box.com/shared/static/djf5w54rjvpqocsiztzaandq1m3avr7c.pth -O models/mobilenet-v1-ssd-mp-0_675.pth

Execute Re-trainig This time, to save time, I set the number of Epoch to 5.

!python3 train_ssd.py --data=data/airobject --model-dir=models/airobject --batch-size=20 --epochs=5

Learning has started.

2022-04-13 09:37:00 - Using CUDA...
2022-04-13 09:37:00 - Namespace(balance_data=False, base_net=None, base_net_lr=0.001, batch_size=20, checkpoint_folder='models/airobject', dataset_type='open_images', datasets=['data/airobject'], debug_steps=10, extra_layers_lr=None, freeze_base_net=False, freeze_net=False, gamma=0.1, lr=0.01, mb2_width_mult=1.0, milestones='80,100', momentum=0.9, net='mb1-ssd', num_epochs=5, num_workers=2, pretrained_ssd='models/mobilenet-v1-ssd-mp-0_675.pth', resume=None, scheduler='cosine', t_max=100, use_cuda=True, validation_epochs=1, weight_decay=0.0005)
2022-04-13 09:37:00 - Prepare training datasets.
2022-04-13 09:37:00 - loading annotations from: data/airobject/sub-train-annotations-bbox.csv
2022-04-13 09:37:00 - annotations loaded from:  data/airobject/sub-train-annotations-bbox.csv
num images:  1661
2022-04-13 09:37:02 - Dataset Summary:Number of Images: 1661
Minimum Number of Images for a Class: -1
Label Distribution:
	Aircraft: 176
	Airplane: 1057
	Bird: 2822
	Helicopter: 144
2022-04-13 09:37:02 - Stored labels into file models/airobject/labels.txt.
2022-04-13 09:37:02 - Train dataset size: 1661
2022-04-13 09:37:02 - Prepare Validation datasets.
2022-04-13 09:37:02 - loading annotations from: data/airobject/sub-test-annotations-bbox.csv
2022-04-13 09:37:02 - annotations loaded from:  data/airobject/sub-test-annotations-bbox.csv
num images:  255
2022-04-13 09:37:02 - Dataset Summary:Number of Images: 255
Minimum Number of Images for a Class: -1
Label Distribution:
	Aircraft: 23
	Airplane: 165
	Bird: 144
	Helicopter: 30
2022-04-13 09:37:02 - Validation dataset size: 255
2022-04-13 09:37:02 - Build network.
2022-04-13 09:37:03 - Init from pretrained ssd models/mobilenet-v1-ssd-mp-0_675.pth
2022-04-13 09:37:03 - Took 0.10 seconds to load the model.
2022-04-13 09:37:13 - Learning rate: 0.01, Base net learning rate: 0.001, Extra Layers learning rate: 0.01.
2022-04-13 09:37:13 - Uses CosineAnnealingLR scheduler.
2022-04-13 09:37:13 - Start training from epoch 0.
/usr/local/lib/python3.7/dist-packages/torch/optim/lr_scheduler.py:134: UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`. In PyTorch 1.1.0 and later, you should call them in the opposite order: `optimizer.step()` before `lr_scheduler.step()`.  Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate
  "https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate", UserWarning)
/usr/local/lib/python3.7/dist-packages/torch/nn/_reduction.py:42: UserWarning: size_average and reduce args will be deprecated, please use reduction='sum' instead.
  warnings.warn(warning.format(ret))
2022-04-13 09:37:32 - Epoch: 0, Step: 10/84, Avg Loss: 10.7709, Avg Regression Loss 3.5393, Avg Classification Loss: 7.2317
2022-04-13 09:37:47 - Epoch: 0, Step: 20/84, Avg Loss: 6.3753, Avg Regression Loss 2.8129, Avg Classification Loss: 3.5624
2022-04-13 09:38:03 - Epoch: 0, Step: 30/84, Avg Loss: 6.0810, Avg Regression Loss 2.8787, Avg Classification Loss: 3.2023
2022-04-13 09:38:17 - Epoch: 0, Step: 40/84, Avg Loss: 4.8146, Avg Regression Loss 1.8806, Avg Classification Loss: 2.9340
2022-04-13 09:38:31 - Epoch: 0, Step: 50/84, Avg Loss: 5.1356, Avg Regression Loss 2.1621, Avg Classification Loss: 2.9735
2022-04-13 09:38:45 - Epoch: 0, Step: 60/84, Avg Loss: 4.5565, Avg Regression Loss 1.8808, Avg Classification Loss: 2.6758
2022-04-13 09:39:00 - Epoch: 0, Step: 70/84, Avg Loss: 4.4294, Avg Regression Loss 1.7599, Avg Classification Loss: 2.6695

(Skip)
2022-04-13 09:47:40 - Epoch: 4, Step: 80/84, Avg Loss: 3.5394, Avg Regression Loss 1.2643, Avg Classification Loss: 2.2750
2022-04-13 09:47:48 - Epoch: 4, Validation Loss: 3.0542, Validation Regression Loss 0.9724, Validation Classification Loss: 2.0818
2022-04-13 09:47:48 - Saved model models/airobject/mb1-ssd-Epoch-4-Loss-3.0541839599609375.pth
2022-04-13 09:47:48 - Task done, exiting program.

The learning results are as follows.

Epoch4 seems to have the least loss.

2022-04-13 09:39:27 - Epoch: 0, Validation Loss: 3.8735, Validation Regression Loss 1.3683, Validation Classification Loss: 2.5053
2022-04-13 09:39:27 - Saved model models/airobject/mb1-ssd-Epoch-0-Loss-3.8735207410959096.pth
2022-04-13 09:41:32 - Epoch: 1, Validation Loss: 3.3729, Validation Regression Loss 1.1192, Validation Classification Loss: 2.2537
2022-04-13 09:41:32 - Saved model models/airobject/mb1-ssd-Epoch-1-Loss-3.3728924898000865.pth
2022-04-13 09:43:38 - Epoch: 2, Validation Loss: 3.3090, Validation Regression Loss 1.0735, Validation Classification Loss: 2.2355
2022-04-13 09:43:38 - Saved model models/airobject/mb1-ssd-Epoch-2-Loss-3.309031156393198.pth
2022-04-13 09:45:44 - Epoch: 3, Validation Loss: 3.1392, Validation Regression Loss 0.9944, Validation Classification Loss: 2.1449
2022-04-13 09:45:44 - Saved model models/airobject/mb1-ssd-Epoch-3-Loss-3.1392355882204495.pth
2022-04-13 09:47:48 - Epoch: 4, Validation Loss: 3.0542, Validation Regression Loss 0.9724, Validation Classification Loss: 2.0818
2022-04-13 09:47:48 - Saved model models/airobject/mb1-ssd-Epoch-4-Loss-3.0541839599609375.pth

Object detection by re-training data

Let’s perform object detection using the retrained data.

!python3 run_ssd_example.py mb1-ssd models/airobject/mb1-ssd-Epoch-4-Loss-3.0541839599609375.pth models/airobject/labels.txt airplane_1.jpg

It recognizes the plane correctly.

Try other images as well.

!python3 run_ssd_example.py mb1-ssd models/airobject/mb1-ssd-Epoch-4-Loss-3.0541839599609375.pth models/airobject/labels.txt airplane_2.jpg

Although it is a spaceship “space shuttle”, it is no wonder that it recognizes an airplane.

Conclusion

This time, I tried to perform object detection by re-learning using Mobile Net, which is an implementation sample of object detection (SSD) with Pytorch. I would like to continue object detection.

コメント

タイトルとURLをコピーしました