Using deep lab for semantic segmentation

Time:2020-3-28

introduce

Using deep lab for semantic segmentation

DeepLabGoogle usestensorflowBe based onCNNFour versions of the developed semantic segmentation model have been updated up to now. The latest version isDeepLabv3+In this model, the deep separable convolution is further applied to the pyramid pooling and decoder modules in the pore space, thus forming a faster and more powerful semantic segmentation encoder decoder network. Starting from the official case, this paper introduces how to train and useDeepLabv3+Model.

Using deep lab for semantic segmentation

Model installation

1. Download tensorflow model and rename the extracted file to”models“And move totensorflowFolder.

Using deep lab for semantic segmentation

2. Add environment variable

cd /anaconda3/lib/python3.6/site-packages/tensorflow/models/research
export PYTHONPATH=$PYTHONPATH:`pwd`:`pwd`/slim

3. Test whether the installation is successful

cd /anaconda3/lib/python3.6/site-packages/tensorflow/models/research
python deeplab/model_test.py

The following error occurred:

Using deep lab for semantic segmentation

The solution is as follows:

Using deep lab for semantic segmentation
Using deep lab for semantic segmentation

Preparation data

Download data:

cd /anaconda3/lib/python3.6/site-packages/tensorflow/models/research/deeplab/datasets/
sh download_and_convert_voc2012.sh

Specific steps:

  1. Download and unzipVOC2012data set

Using deep lab for semantic segmentation

  1. Remove colormap

Using deep lab for semantic segmentation

  1. Create in datasetTFRecordfile

Using deep lab for semantic segmentation

Download pre training model

The official provides a variety of models, which can be downloaded and installed according to their actual needs.

Using deep lab for semantic segmentation

Training with Pascal voc2012 dataset

File structure:

Using deep lab for semantic segmentation

tensorflow deeplabFolder:

Using deep lab for semantic segmentation

Training:

cd /anaconda3/lib/python3.6/site-packages/tensorflow/models/research

Fill in the relevant path and perform the training

python deeplab/train.py \
    --logtostderr \
    --training_number_of_steps=30000 \
    --train_split="train" \
    --model_variant="xception_65" \
    --atrous_rates=6 \
    --atrous_rates=12 \
    --atrous_rates=18 \
    --output_stride=16 \
    --decoder_output_stride=4 \
    --train_crop_size=513 \
    --train_crop_size=513 \
    --train_batch_size=1 \
    --dataset="pascal_voc_seg" \
    --tf_initial_checkpoint=${PATH_TO_INITIAL_CHECKPOINT} \
    --train_logdir=${PATH_TO_TRAIN_DIR} \
    --dataset_dir=${PATH_TO_DATASET}

PATH_TO_INITIAL_CHECKPOINT: initial checkpoint file path (migration learning)
PATH_TO_TRAIN_DIR: training model save path
PATH_TO_DATASET: dataset path

Problems to be noted:

  1. When the GPU memory is not enough, use the transfer learning method to train and reduce the learning rate. Set “fine? Tune? Batch? Norm = false“
  2. When “output_strip = 8” and “routes_rates” are set, the value range should be changed from [6,12,18] to [12,24,36].
  3. When you do not want to use decoder structure, you need to comment out “decoder” output “string”.

Verification

cd /anaconda3/lib/python3.6/site-packages/tensorflow/models/research
python deeplab/eval.py \
    --logtostderr \
    --eval_split="val" \
    --model_variant="xception_65" \
    --atrous_rates=6 \
    --atrous_rates=12 \
    --atrous_rates=18 \
    --output_stride=16 \
    --decoder_output_stride=4 \
    --eval_crop_size=513 \
    --eval_crop_size=513 \
    --dataset="pascal_voc_seg" \
    --checkpoint_dir=${PATH_TO_CHECKPOINT} \
    --eval_logdir=${PATH_TO_EVAL_DIR} \
    --dataset_dir=${PATH_TO_DATASET}

PATH_TO_CHECKPOINT: training phase checkpoint file path
PATH_TO_EVAL_DIR: evaluation model save path
PATH_TO_DATASET: dataset path

Visual model output

cd /anaconda3/lib/python3.6/site-packages/tensorflow/models/research
python deeplab/vis.py \
    --logtostderr \
    --vis_split="val" \
    --model_variant="xception_65" \
    --atrous_rates=6 \
    --atrous_rates=12 \
    --atrous_rates=18 \
    --output_stride=16 \
    --decoder_output_stride=4 \
    --vis_crop_size=513 \
    --vis_crop_size=513 \
    --dataset="pascal_voc_seg" \
    --checkpoint_dir=${PATH_TO_CHECKPOINT} \
    --vis_logdir=${PATH_TO_VIS_DIR} \
    --dataset_dir=${PATH_TO_DATASET}

PATH_TO_CHECKPOINT: training phase checkpoint file path
PATH_TO_VIS_DIR: evaluation model save path
PATH_TO_DATASET: dataset path
Problems to be noted:
When you need to save the split results, you need to set”also_save_raw_predictions = True“

Tensorboard

tensorboard --logdir=${PATH_TO_LOG_DIRECTORY}

The results of “train”, “Eval”, “vis” will be displayed at the same time. If you need to display one class separately, you can specify the display category, such as:

tensorboard --logdir train/

aboutvoc2012Such a dataset,TensorboardIt may take several minutes to display the data.

Model export

cd /anaconda3/lib/python3.6/site-packages/tensorflow/models/research
python deeplab/export_model.py \
    --logtostderr \
    --checkpoint_path="${PATH_TO_CHECKPOINT}" \
    --export_path="${PATH_TO_EXPORT_PD}" \
    --model_variant="xception_65" \
    --atrous_rates=6 \
    --atrous_rates=12 \
    --atrous_rates=18 \
    --output_stride=16 \
    --decoder_output_stride=4 \
    --num_classes=21 \
    --crop_size=513 \
    --crop_size=513 \
    --inference_scales=1.0

Path to checkpoint: checkpoint file path
Path to export PD: path to export PD file (suffix. PD)

case

cd /anaconda3/lib/python3.6/site-packages/tensorflow/models/research/deeplab
sh local_test.sh

Google ColabOnline test address, you can select pre training model, test split photos to view model output results.

Using deep lab for semantic segmentation

summary

Compared with the traditional method based on random forest, the semantic segmentation based on deep learning has made great progress. Although there are many ideas in the current method based on deep learning, they are basically based on the evolution of full convolution (FCN). FCN uses convolution instead of full connection layer, so it can accept input of any size and use transposition convolution to restore image size for the following pixel by pixel prediction.
DeepLabIn order to improve the segmentation effect, the model usesatrousConvolution (void convolution),ASPP, full connection condition random field and other technologies.