Pytorchによる物体検出 SSD MobileNet 再学習 Part(4)

colaboratory
2022.04.14

はじめに

こんにちわ、swim-loverです。PythonとPytorchで物体検出を行っています。 Pythonを始めたばかりですが、「使いながら覚える」をコンセプトに勉強しています。

Part(2)では、SSDを使った物体検出を行いました。

Part(3)では、YoLov3を使った物体検出を行いました。

いずれも学習済みデータを用いて、物体検出を行いました。

今回、再学習によりトレーニングデータを作成し物体検出を行ってみます。

参考WebSite

今回のBlogを作成するにあたり、以下のGitHubを参考にしています。

pytorch-ssd/README.md at master · qfgaohao/pytorch-ssd
MobileNetV1, MobileNetV2, VGG based SSD/SSD-lite implementation in Pytorch 1.0 / Pytorch 0.4. Out-of-box support for retraining on Open Images dataset. ONNX and...
github.com
jetson-inference/docs/pytorch-ssd.md at dev · dusty-nv/jetson-inference
Hello AI World guide to deploying deep-learning inference networks and deep vision primitives with TensorRT and NVIDIA Jetson. - dusty-nv/jetson-inference
github.com

OpenImages ダウンロード準備

Colab環境に前回までと同様にGoogle Drivreをマウントします。

from google.colab import drive
drive.mount('/content/drive')

今回、machine フォルダ以下に環境を構築します。

cd drive/MyDrive/machine

gitbubよりssdのサンプルをCloneします。

!git clone https://github.com/qfgaohao/pytorch-ssd.git
cd pytorch-ssd

boto3 のインストール。次のopen_images_downloader.pyの中で参照されています。

pip install boto3

OpenImages イメージ数の確認

Open Images のClass数は601種類あることが確認できます。

pytorch-ssd/open_images_classes.txt at master · dusty-nv/pytorch-ssd
MobileNetV1, MobileNetV2, VGG based SSD/SSD-lite implementation in PyTorch. Out-of-box support for retraining on Open Images dataset. ONNX and Caffe2 support. E...
github.com

601種類の中から、”Aircraft”,”Airplane”,”Bird”,”Helicopter”の4つのクラスについて扱ってみます。

–status-onlyをつけることで、実際にImageをダウンロードする前にデータ数を知ることができます。

!python3 open_images_downloader.py --stats-only --class-names "Aircraft","Airplane","Bird","Helicopter" --data=data/airobject

実行結果を確認すると、４Classで合計、37584個のImageと 77415個のBoundingn Box Clountがあることが事前にわかりました。

2022-04-13 09:02:36 - Download https://storage.googleapis.com/openimages/2018_04/class-descriptions-boxable.csv.
2022-04-13 09:02:36 - Requested 4 classes, found 4 classes
2022-04-13 09:02:36 - Download https://storage.googleapis.com/openimages/2018_04/train/train-annotations-bbox.csv.
2022-04-13 09:03:02 - Read annotation file data/airobject/train-annotations-bbox.csv
2022-04-13 09:03:23 - Available train images:  31225
2022-04-13 09:03:23 - Available train boxes:   68369

2022-04-13 09:03:23 - Download https://storage.googleapis.com/openimages/2018_04/validation/validation-annotations-bbox.csv.
2022-04-13 09:03:23 - Read annotation file data/airobject/validation-annotations-bbox.csv
2022-04-13 09:03:24 - Available validation images:  1551
2022-04-13 09:03:24 - Available validation boxes:   2238

2022-04-13 09:03:24 - Download https://storage.googleapis.com/openimages/2018_04/test/test-annotations-bbox.csv.
2022-04-13 09:03:26 - Read annotation file data/airobject/test-annotations-bbox.csv
2022-04-13 09:03:28 - Available test images:  4808
2022-04-13 09:03:28 - Available test boxes:   6808

2022-04-13 09:03:28 - Total available images: 37584
2022-04-13 09:03:28 - Total available boxes:  77415


-------------------------------------
 'train' set statistics
-------------------------------------
  Image count:  31225
  Bounding box count:  68369
  Bounding box distribution: 
    Bird:  43869/68369 = 0.64
    Airplane:  19979/68369 = 0.29
    Helicopter:  2799/68369 = 0.04
    Aircraft:  1722/68369 = 0.03
 

-------------------------------------
 'validation' set statistics
-------------------------------------
  Image count:  1551
  Bounding box count:  2238
  Bounding box distribution: 
    Airplane:  1005/2238 = 0.45
    Bird:  917/2238 = 0.41
    Aircraft:  182/2238 = 0.08
    Helicopter:  134/2238 = 0.06
 

-------------------------------------
 'test' set statistics
-------------------------------------
  Image count:  4808
  Bounding box count:  6808
  Bounding box distribution: 
    Airplane:  3213/6808 = 0.47
    Bird:  2636/6808 = 0.39
    Aircraft:  542/6808 = 0.08
    Helicopter:  417/6808 = 0.06
 

-------------------------------------
 Overall statistics
-------------------------------------
  Image count:  37584
  Bounding box count:  77415

OpenImages イメージ数を制限してダウンロード

今回は、４Classで合計、37584個のImageずべてをダウンロードせずに、時間と保存場所のことを考慮して2000個に制限してダウンロードしてみることにしました。

!python3 open_images_downloader.py --max-images=2000 --class-names "Aircraft","Airplane","Bird","Helicopter" --data=data/airobject

問題なくイメージのダウンロードが完了しました。

2022-04-13 09:13:21 - Requested 4 classes, found 4 classes
2022-04-13 09:13:21 - Read annotation file data/airobject/train-annotations-bbox.csv
2022-04-13 09:13:40 - Available train images:  31225
2022-04-13 09:13:40 - Available train boxes:   68369

2022-04-13 09:13:40 - Read annotation file data/airobject/validation-annotations-bbox.csv
2022-04-13 09:13:40 - Available validation images:  1551
2022-04-13 09:13:40 - Available validation boxes:   2238

2022-04-13 09:13:40 - Read annotation file data/airobject/test-annotations-bbox.csv
2022-04-13 09:13:42 - Available test images:  4808
2022-04-13 09:13:42 - Available test boxes:   6808

2022-04-13 09:13:42 - Total available images: 37584
2022-04-13 09:13:42 - Total available boxes:  77415

2022-04-13 09:13:42 - Limiting train dataset to:  1661 images (4199 boxes)
2022-04-13 09:13:42 - Limiting validation dataset to:  82 images (115 boxes)
2022-04-13 09:13:42 - Limiting test dataset to:  255 images (362 boxes)

-------------------------------------
 'train' set statistics
-------------------------------------
  Image count:  1661
  Bounding box count:  4199
  Bounding box distribution: 
    Bird:  2822/4199 = 0.67
    Airplane:  1057/4199 = 0.25
    Aircraft:  176/4199 = 0.04
    Helicopter:  144/4199 = 0.03
 

-------------------------------------
 'validation' set statistics
-------------------------------------
  Image count:  82
  Bounding box count:  115
  Bounding box distribution: 
    Airplane:  58/115 = 0.50
    Bird:  45/115 = 0.39
    Aircraft:  9/115 = 0.08
    Helicopter:  3/115 = 0.03
 

-------------------------------------
 'test' set statistics
-------------------------------------
  Image count:  255
  Bounding box count:  362
  Bounding box distribution: 
    Airplane:  165/362 = 0.46
    Bird:  144/362 = 0.40
    Helicopter:  30/362 = 0.08
    Aircraft:  23/362 = 0.06
 

-------------------------------------
 Overall statistics
-------------------------------------
  Image count:  1998
  Bounding box count:  4676

2022-04-13 09:13:42 - Saving 'train' data to data/airobject/sub-train-annotations-bbox.csv.
2022-04-13 09:13:42 - Saving 'validation' data to data/airobject/sub-validation-annotations-bbox.csv.
2022-04-13 09:13:42 - Saving 'test' data to data/airobject/sub-test-annotations-bbox.csv.
2022-04-13 09:13:42 - Starting to download 1998 images.
2022-04-13 09:13:49 - Downloaded 100 images.
2022-04-13 09:13:54 - Downloaded 200 images.
(Skip)
2022-04-13 09:15:12 - Downloaded 1600 images.
2022-04-13 09:15:18 - Downloaded 1700 images.
2022-04-13 09:15:23 - Downloaded 1800 images.
2022-04-13 09:15:28 - Downloaded 1900 images.
2022-04-13 09:15:34 - Task Done.

再学習 Re-training SSD Mobile Net

まずは、学習済モデルをダウンロードします。

!wget https://nvidia.box.com/shared/static/djf5w54rjvpqocsiztzaandq1m3avr7c.pth -O models/mobilenet-v1-ssd-mp-0_675.pth

再学習を行います。今回は、時間節約のため、Epoch数は、５にしました。

!python3 train_ssd.py --data=data/airobject --model-dir=models/airobject --batch-size=20 --epochs=5

学習が開始されました。

2022-04-13 09:37:00 - Using CUDA...
2022-04-13 09:37:00 - Namespace(balance_data=False, base_net=None, base_net_lr=0.001, batch_size=20, checkpoint_folder='models/airobject', dataset_type='open_images', datasets=['data/airobject'], debug_steps=10, extra_layers_lr=None, freeze_base_net=False, freeze_net=False, gamma=0.1, lr=0.01, mb2_width_mult=1.0, milestones='80,100', momentum=0.9, net='mb1-ssd', num_epochs=5, num_workers=2, pretrained_ssd='models/mobilenet-v1-ssd-mp-0_675.pth', resume=None, scheduler='cosine', t_max=100, use_cuda=True, validation_epochs=1, weight_decay=0.0005)
2022-04-13 09:37:00 - Prepare training datasets.
2022-04-13 09:37:00 - loading annotations from: data/airobject/sub-train-annotations-bbox.csv
2022-04-13 09:37:00 - annotations loaded from:  data/airobject/sub-train-annotations-bbox.csv
num images:  1661
2022-04-13 09:37:02 - Dataset Summary:Number of Images: 1661
Minimum Number of Images for a Class: -1
Label Distribution:
	Aircraft: 176
	Airplane: 1057
	Bird: 2822
	Helicopter: 144
2022-04-13 09:37:02 - Stored labels into file models/airobject/labels.txt.
2022-04-13 09:37:02 - Train dataset size: 1661
2022-04-13 09:37:02 - Prepare Validation datasets.
2022-04-13 09:37:02 - loading annotations from: data/airobject/sub-test-annotations-bbox.csv
2022-04-13 09:37:02 - annotations loaded from:  data/airobject/sub-test-annotations-bbox.csv
num images:  255
2022-04-13 09:37:02 - Dataset Summary:Number of Images: 255
Minimum Number of Images for a Class: -1
Label Distribution:
	Aircraft: 23
	Airplane: 165
	Bird: 144
	Helicopter: 30
2022-04-13 09:37:02 - Validation dataset size: 255
2022-04-13 09:37:02 - Build network.
2022-04-13 09:37:03 - Init from pretrained ssd models/mobilenet-v1-ssd-mp-0_675.pth
2022-04-13 09:37:03 - Took 0.10 seconds to load the model.
2022-04-13 09:37:13 - Learning rate: 0.01, Base net learning rate: 0.001, Extra Layers learning rate: 0.01.
2022-04-13 09:37:13 - Uses CosineAnnealingLR scheduler.
2022-04-13 09:37:13 - Start training from epoch 0.
/usr/local/lib/python3.7/dist-packages/torch/optim/lr_scheduler.py:134: UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`. In PyTorch 1.1.0 and later, you should call them in the opposite order: `optimizer.step()` before `lr_scheduler.step()`.  Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate
  "https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate", UserWarning)
/usr/local/lib/python3.7/dist-packages/torch/nn/_reduction.py:42: UserWarning: size_average and reduce args will be deprecated, please use reduction='sum' instead.
  warnings.warn(warning.format(ret))
2022-04-13 09:37:32 - Epoch: 0, Step: 10/84, Avg Loss: 10.7709, Avg Regression Loss 3.5393, Avg Classification Loss: 7.2317
2022-04-13 09:37:47 - Epoch: 0, Step: 20/84, Avg Loss: 6.3753, Avg Regression Loss 2.8129, Avg Classification Loss: 3.5624
2022-04-13 09:38:03 - Epoch: 0, Step: 30/84, Avg Loss: 6.0810, Avg Regression Loss 2.8787, Avg Classification Loss: 3.2023
2022-04-13 09:38:17 - Epoch: 0, Step: 40/84, Avg Loss: 4.8146, Avg Regression Loss 1.8806, Avg Classification Loss: 2.9340
2022-04-13 09:38:31 - Epoch: 0, Step: 50/84, Avg Loss: 5.1356, Avg Regression Loss 2.1621, Avg Classification Loss: 2.9735
2022-04-13 09:38:45 - Epoch: 0, Step: 60/84, Avg Loss: 4.5565, Avg Regression Loss 1.8808, Avg Classification Loss: 2.6758
2022-04-13 09:39:00 - Epoch: 0, Step: 70/84, Avg Loss: 4.4294, Avg Regression Loss 1.7599, Avg Classification Loss: 2.6695

(Skip)
2022-04-13 09:47:40 - Epoch: 4, Step: 80/84, Avg Loss: 3.5394, Avg Regression Loss 1.2643, Avg Classification Loss: 2.2750
2022-04-13 09:47:48 - Epoch: 4, Validation Loss: 3.0542, Validation Regression Loss 0.9724, Validation Classification Loss: 2.0818
2022-04-13 09:47:48 - Saved model models/airobject/mb1-ssd-Epoch-4-Loss-3.0541839599609375.pth
2022-04-13 09:47:48 - Task done, exiting program.

学習結果は、以下の通りになりました。

Epoch４が最もLossが少ないようです。

2022-04-13 09:39:27 - Epoch: 0, Validation Loss: 3.8735, Validation Regression Loss 1.3683, Validation Classification Loss: 2.5053
2022-04-13 09:39:27 - Saved model models/airobject/mb1-ssd-Epoch-0-Loss-3.8735207410959096.pth
2022-04-13 09:41:32 - Epoch: 1, Validation Loss: 3.3729, Validation Regression Loss 1.1192, Validation Classification Loss: 2.2537
2022-04-13 09:41:32 - Saved model models/airobject/mb1-ssd-Epoch-1-Loss-3.3728924898000865.pth
2022-04-13 09:43:38 - Epoch: 2, Validation Loss: 3.3090, Validation Regression Loss 1.0735, Validation Classification Loss: 2.2355
2022-04-13 09:43:38 - Saved model models/airobject/mb1-ssd-Epoch-2-Loss-3.309031156393198.pth
2022-04-13 09:45:44 - Epoch: 3, Validation Loss: 3.1392, Validation Regression Loss 0.9944, Validation Classification Loss: 2.1449
2022-04-13 09:45:44 - Saved model models/airobject/mb1-ssd-Epoch-3-Loss-3.1392355882204495.pth
2022-04-13 09:47:48 - Epoch: 4, Validation Loss: 3.0542, Validation Regression Loss 0.9724, Validation Classification Loss: 2.0818
2022-04-13 09:47:48 - Saved model models/airobject/mb1-ssd-Epoch-4-Loss-3.0541839599609375.pth

再学習データによる認識処理

再学習したデータを使って、物体検出を実行してみます。

!python3 run_ssd_example.py mb1-ssd models/airobject/mb1-ssd-Epoch-4-Loss-3.0541839599609375.pth models/airobject/labels.txt airplane_1.jpg

正しく飛行機を認識しています。

他の画像も試してみます。

!python3 run_ssd_example.py mb1-ssd models/airobject/mb1-ssd-Epoch-4-Loss-3.0541839599609375.pth models/airobject/labels.txt airplane_2.jpg

宇宙船（スペースシャトル）ですが、さすがにこれは飛行機を認識しても不思議ではありません。

まとめ

今回、Pytorchで物体検出（SSD)の実装サンプルであるMobile Netを使って、再学習を行い、物体認識を行ってみました。次回以降も物体認識をしてみたいと思います。

コメント

タイトルとURLをコピーしました