TF learning CNN_ Separable separable CNN model

Convolutional neural networks

Convolution operation and pooling operation

Deep separable convolution

Data enhancement

Transfer learning

General structure:

• Convolutional neural network

  • (convolution+subsampling)n + fully connected layers,FCm

  • classification

• Full convolution neural network

  • (convolution+subsampling)n + pixelwise classificationK

  • Object segment

Convolution operation:

Solve the problem of too many neural network parameters, for example, the picture size is 10001000, the next layer neuron is 10 ^ 6, and the full link parameter is (10 ^ 3)Too many parameters (10 ^ 6, too many training parameters required) * 10 ^ 3, easy to fit

Features of convolution:

• Local connection

For example, the picture size is 10001000, the neurons in the lower layer are 10 ^ 6, and the local connection range is 1010. The full connection parameter is (10)10)10^6=10^8

• Parameter sharing

For example, the picture size is 10001000, the neurons in the lower layer are 10 ^ 6, and the local connection range is 1010. The full connection parameter is 10 * 10 = 100. PS parameter sharing is used here, which means that the pixel data distribution in the local area is the same. In fact, many local images are the same in the image, so parameter sharing can effectively optimize the data, which may not be as terrible as the optimization of 100 mentioned above

Convolution – calculated per location

Input image:

[

    [1,2,3,4,5],

    [6,7,8,9,10],

    [11,12,13,14,15],

    [16,17,18,19,20],

    [21,22,23,24,25],

]

Convolution kernel:

[

    [1,0,1],

    [0,1,0],

    [1,0,1],

]

Output size = input size convolution kernel size + 1

Output size = 5 – (3-1)

Pooling operation – maximum pooling

Input image:

[

Output:

[

There is also an average pooling:

AVG pool operation

[

    [4,6],

    [14,16],

]

characteristic:

• Try not to overlap or fill in zeros

• There are no parameters for derivation

• Pool layer parameters are not long and pool core size

• It is used to reduce the image size, so as to reduce the amount of calculation

• Dynamic robustness to a certain extent

• Loss of space location

import matplotlib as mpl

import matplotlib.pyplot as plt

import numpy as np

import sklearn

import pandas as pd

import tensorflow as tf

from tensorflow import keras

import os, sys, time 

print(sys.version_info)

for module in tf, mpl, np, pd, sklearn, tf, keras:

    print(module.__name__, module.__version__)

sys.version_info(major=3, minor=6, micro=7, releaselevel='final', serial=0)

tensorflow 2.1.0

matplotlib 3.2.0

numpy 1.18.1

pandas 1.0.1

sklearn 0.22.2.post1

tensorflow 2.1.0

tensorflow_core.python.keras.api._v2.keras 2.2.4-tf

fashion_ mnist = keras. datasets. fashion_ mnist

x_valid.shape, y_valid.shape: (5000, 28, 28) (5000,)

x_train.shape, y_train.shape (55000, 28, 28) (55000,)

x_test.shape, y_test.shape (10000, 28, 28) (10000,)

#Data set sample display

'''

model = keras. models. Sequential()

Model: "sequential"

_________________________________________________________________

Layer (type)                 Output Shape              Param #   

=================================================================

conv2d (Conv2D)              (None, 28, 28, 32)        320       

_________________________________________________________________

separable_conv2d (SeparableC (None, 28, 28, 32)        1344      

_________________________________________________________________

max_pooling2d (MaxPooling2D) (None, 14, 14, 32)        0         

_________________________________________________________________

conv2d_1 (Conv2D)            (None, 14, 14, 64)        18496     

_________________________________________________________________

separable_conv2d_1 (Separabl (None, 14, 14, 64)        4736      

_________________________________________________________________

max_pooling2d_1 (MaxPooling2 (None, 7, 7, 64)          0         

_________________________________________________________________

conv2d_2 (Conv2D)            (None, 7, 7, 128)         73856     

_________________________________________________________________

separable_conv2d_2 (Separabl (None, 7, 7, 128)         17664     

_________________________________________________________________

max_pooling2d_2 (MaxPooling2 (None, 3, 3, 128)         0         

_________________________________________________________________

flatten (Flatten)            (None, 1152)              0         

_________________________________________________________________

dense (Dense)                (None, 128)               147584    

_________________________________________________________________

dense_1 (Dense)              (None, 10)                1290      

=================================================================

Total params: 265,290

Trainable params: 265,290

Non-trainable params: 0

_________________________________________________________________

x_train_scaled.shape[0]

55000

logdir = os. path. join('callbacks')

Using TensorFlow backend.

Train on 55000 samples, validate on 5000 samples

Epoch 1/10

55000/55000 [==============================] - 19s 344us/sample - loss: 2.3026 - accuracy: 0.1008 - val_loss: 2.3027 - val_accuracy: 0.0986

Epoch 2/10

55000/55000 [==============================] - 14s 262us/sample - loss: 2.3026 - accuracy: 0.1007 - val_loss: 2.3027 - val_accuracy: 0.0986

Epoch 3/10

55000/55000 [==============================] - 15s 269us/sample - loss: 2.3025 - accuracy: 0.1024 - val_loss: 2.3024 - val_accuracy: 0.0986

Epoch 4/10

55000/55000 [==============================] - 15s 278us/sample - loss: 2.3023 - accuracy: 0.1045 - val_loss: 2.3022 - val_accuracy: 0.0914

Epoch 5/10

55000/55000 [==============================] - 15s 272us/sample - loss: 2.3016 - accuracy: 0.1108 - val_loss: 2.3004 - val_accuracy: 0.0980

Epoch 6/10

55000/55000 [==============================] - 15s 272us/sample - loss: 2.1338 - accuracy: 0.2406 - val_loss: 1.0289 - val_accuracy: 0.6426

Epoch 7/10

55000/55000 [==============================] - 15s 268us/sample - loss: 0.7877 - accuracy: 0.7085 - val_loss: 0.6717 - val_accuracy: 0.7530

Epoch 8/10

55000/55000 [==============================] - 15s 268us/sample - loss: 0.6077 - accuracy: 0.7766 - val_loss: 0.5327 - val_accuracy: 0.8078

Epoch 9/10

55000/55000 [==============================] - 15s 272us/sample - loss: 0.5246 - accuracy: 0.8053 - val_loss: 0.5014 - val_accuracy: 0.8184

Epoch 10/10

55000/55000 [==============================] - 15s 275us/sample - loss: 0.4697 - accuracy: 0.8273 - val_loss: 0.4354 - val_accuracy: 0.8406

x_train_scaled

array([[[[-0.8105136],

         [-0.8105136],

         [-0.8105136],

         ...,

         [-0.8105136],

         [-0.8105136],

         [-0.8105136]],

        [[-0.8105136],

         [-0.8105136],

         [-0.8105136],

         ...,

         [-0.8105136],

         [-0.8105136],

         [-0.8105136]],

        [[-0.8105136],

         [-0.8105136],

         [-0.8105136],

         ...,

         [-0.8105136],

         [-0.8105136],

         [-0.8105136]],

        ...,

        [[-0.8105136],

         [-0.8105136],

         [-0.8105136],

         ...,

         [-0.8105136],

         [-0.8105136],

         [-0.8105136]],

        [[-0.8105136],

         [-0.8105136],

         [-0.8105136],

         ...,

         [-0.8105136],

         [-0.8105136],

         [-0.8105136]],

        [[-0.8105136],

         [-0.8105136],

         [-0.8105136],

         ...,

         [-0.8105136],

         [-0.8105136],

         [-0.8105136]]],

       [[[-0.8105136],

         [-0.8105136],

         [-0.8105136],

         ...,

         [-0.8105136],

         [-0.8105136],

         [-0.8105136]],

        [[-0.8105136],

         [-0.8105136],

         [-0.8105136],

         ...,

         [-0.8105136],

         [-0.8105136],

         [-0.8105136]],

        [[-0.8105136],

         [-0.8105136],

         [-0.8105136],

         ...,

         [-0.8105136],

         [-0.8105136],

         [-0.8105136]],

        ...,

        [[-0.8105136],

         [-0.8105136],

         [-0.8105136],

         ...,

         [-0.8105136],

         [-0.8105136],

         [-0.8105136]],

        [[-0.8105136],

         [-0.8105136],

         [-0.8105136],

         ...,

         [-0.8105136],

         [-0.8105136],

         [-0.8105136]],

        [[-0.8105136],

         [-0.8105136],

         [-0.8105136],

         ...,

         [-0.8105136],

         [-0.8105136],

         [-0.8105136]]],

       [[[-0.8105136],

         [-0.8105136],

         [-0.8105136],

         ...,

         [-0.8105136],

         [-0.8105136],

         [-0.8105136]],

        [[-0.8105136],

         [-0.8105136],

         [-0.8105136],

         ...,

         [-0.8105136],

         [-0.8105136],

         [-0.8105136]],

        [[-0.8105136],

         [-0.8105136],

         [-0.8105136],

         ...,

         [-0.8105136],

         [-0.8105136],

         [-0.8105136]],

        ...,

        [[-0.8105136],

         [-0.8105136],

         [-0.8105136],

         ...,

         [-0.8105136],

         [-0.8105136],

         [-0.8105136]],

        [[-0.8105136],

         [-0.8105136],

         [-0.8105136],

         ...,

         [-0.8105136],

         [-0.8105136],

         [-0.8105136]],

        [[-0.8105136],

         [-0.8105136],

         [-0.8105136],

         ...,

         [-0.8105136],

         [-0.8105136],

         [-0.8105136]]],

       ...,

       [[[-0.8105136],

         [-0.8105136],

         [-0.8105136],

         ...,

         [-0.8105136],

         [-0.8105136],

         [-0.8105136]],

        [[-0.8105136],

         [-0.8105136],

         [-0.8105136],

         ...,

         [-0.8105136],

         [-0.8105136],

         [-0.8105136]],

        [[-0.8105136],

         [-0.8105136],

         [-0.8105136],

         ...,

         [-0.8105136],

         [-0.8105136],

         [-0.8105136]],

        ...,

        [[-0.8105136],

         [-0.8105136],

         [-0.8105136],

         ...,

         [-0.8105136],

         [-0.8105136],

         [-0.8105136]],

        [[-0.8105136],

         [-0.8105136],

         [-0.8105136],

         ...,

         [-0.8105136],

         [-0.8105136],

         [-0.8105136]],

        [[-0.8105136],

         [-0.8105136],

         [-0.8105136],

         ...,

         [-0.8105136],

         [-0.8105136],

         [-0.8105136]]],

       [[[-0.8105136],

         [-0.8105136],

         [-0.8105136],

         ...,

         [-0.8105136],

         [-0.8105136],

         [-0.8105136]],

        [[-0.8105136],

         [-0.8105136],

         [-0.8105136],

         ...,

         [-0.8105136],

         [-0.8105136],

         [-0.8105136]],

        [[-0.8105136],

         [-0.8105136],

         [-0.8105136],

         ...,

         [-0.8105136],

         [-0.8105136],

         [-0.8105136]],

        ...,

        [[-0.8105136],

         [-0.8105136],

         [-0.8105136],

         ...,

         [-0.8105136],

         [-0.8105136],

         [-0.8105136]],

        [[-0.8105136],

         [-0.8105136],

         [-0.8105136],

         ...,

         [-0.8105136],

         [-0.8105136],

         [-0.8105136]],

        [[-0.8105136],

         [-0.8105136],

         [-0.8105136],

         ...,

         [-0.8105136],

         [-0.8105136],

         [-0.8105136]]],

       [[[-0.8105136],

         [-0.8105136],

         [-0.8105136],

         ...,

         [-0.8105136],

         [-0.8105136],

         [-0.8105136]],

        [[-0.8105136],

         [-0.8105136],

         [-0.8105136],

         ...,

         [-0.8105136],

         [-0.8105136],

         [-0.8105136]],

        [[-0.8105136],

         [-0.8105136],

         [-0.8105136],

         ...,

         [-0.8105136],

         [-0.8105136],

         [-0.8105136]],

        ...,

        [[-0.8105136],

         [-0.8105136],

         [-0.8105136],

         ...,

         [-0.8105136],

         [-0.8105136],

         [-0.8105136]],

        [[-0.8105136],

         [-0.8105136],

         [-0.8105136],

         ...,

         [-0.8105136],

         [-0.8105136],

         [-0.8105136]],

        [[-0.8105136],

         [-0.8105136],

         [-0.8105136],

         ...,

         [-0.8105136],

         [-0.8105136],

         [-0.8105136]]]], dtype=float32)

def plot_learning_curves(history):

    pd.DataFrame(history.history).plot(figsize=(8, 5))

    plt.grid(True)

    plt.gca().set_ylim(0, 3)

    plt.show()

plot_learning_curves(history)

print(y_test.shape)

model.evaluate(x_test_scaled, y_test[:5000])

(10000,)

5000/5000 [==============================] - 1s 119us/sample - loss: 8.8059 - accuracy: 0.1018

[8.805856645965576, 0.1018]

This work adoptsCC agreement, reprint must indicate the author and the link to this article