Little bear paddle exercise book-02 eye disease identification


Little bear paddle exercise book-02 eye disease identification

brief introduction

The development and testing of this project are carried out under Ubuntu 20.04 system.
View the latest code of the project home page:Little bear paddle exercise book
Home page of Baidu PaddlePaddle AI Studio:Little bear paddle exercise book-02 eye disease identification
CUDA reference for Ubuntu system installation:Installation of Ubuntu Baidu PaddlePaddle and CUDA

Document description

file explain Training procedure Test procedure Test program GTK interface Report program Get data to dataset directory Check whether the data in the dataset directory exists
mod/ Alexnet network model
mod/ Analysis of imageCLASS image classification data set
mod/ miscellaneous
mod/ to configure
mod/ Result report
dataset Dataset directory
params Model parameter saving directory
log Visualdl log save directory

data set

The data set comes from the public data set of Baidu PaddlePaddle:Eye disease identification data set ichallenge – finishing version

get data

If it is running on the local computer, after downloading the data, the file will be placed in thedatasetUnder the project directory, run the following script under the project directory.
If running in BaiduAI StudioEnvironment, viewingdataWhether the directory has data, run the following script under the project directory.


Classification label

  • Non eye disease 0
  • Eye disease 1

Check data

After obtaining the data, run the following script in the project directory to check whether the data in the dataset directory exists.


network model

Network model usageAlexnet network modelSource: Baidu PaddlePaddle tutorial and network.
Alexnet network modelreference resources:Baidu PaddlePaddle tutorial

import paddle
import paddle.nn as nn
import paddle.nn.functional as F

#Alexnet network model
class AlexNet(nn.Layer):
    Alexnet network model

    The input image size is 224 x 224
    Pool layer kernel_ Size = 2, the first layer of convolution filling padding = 2
    def __init__(self, num_classes=10, pool_kernel_size=2, conv1_paddling=2, fc1_in_features=9216):
        Alexnet network model

            num_ Classes (int, optional): number of classes. The default is 10
            pool_ kernel_ Size (int, optional): pool layer core size, default 2
            conv1_ Padding (int, optional): the first layer is filled by convolution layer. The default is 2,
                The input image size is 224 x 224 填充 2
            fc1_ in_ Features (int, optional): input the number of features in the first full connection layer. The default is 9216, 
                According to Max_ Pool3 output results, calculated 256 * 6 * 6 = 9216

            Exception: classification quantity num_ Classes must be greater than or equal to 2
        super(AlexNet, self).__init__()
        if num_classes < 2:
            Raise exception ("the number of categories num_classes must be greater than or equal to 2: {}". Format (num_classes))
        self.num_classes = num_classes
        self.pool_kernel_size = pool_kernel_size
        self.fc1_in_features = fc1_in_features
        self.conv1 = nn.Conv2D(
            in_channels=3, out_channels=96, kernel_size=11, stride=4, padding=conv1_paddling)
        self.max_pool1 = nn.MaxPool2D(kernel_size=pool_kernel_size, stride=2)
        self.conv2 = nn.Conv2D(
            in_channels=96, out_channels=256, kernel_size=5, stride=1, padding=2)
        self.max_pool2 = nn.MaxPool2D(kernel_size=pool_kernel_size, stride=2)
        self.conv3 = nn.Conv2D(
            in_channels=256, out_channels=384, kernel_size=3, stride=1, padding=1)
        self.conv4 = nn.Conv2D(
            in_channels=384, out_channels=384, kernel_size=3, stride=1, padding=1)
        self.conv5 = nn.Conv2D(
            in_channels=384, out_channels=256, kernel_size=3, stride=1, padding=1)
        self.max_pool3 = nn.MaxPool2D(kernel_size=pool_kernel_size, stride=2)
        # in_ features 9216 = max_ Pool3 output 256 * 6 * 6
        self.fc1 = nn.Linear(in_features=fc1_in_features, out_features=4096)
        self.drop_ratio1 = 0.5
        self.drop1 = nn.Dropout(self.drop_ratio1)
        self.fc2 = nn.Linear(in_features=4096, out_features=4096)
        self.drop_ratio2 = 0.5
        self.drop2 = nn.Dropout(self.drop_ratio2)
        self.fc3 = nn.Linear(in_features=4096, out_features=num_classes)

    def forward(self, x):
        x = self.conv1(x)
        x = F.relu(x)
        x = self.max_pool1(x)
        x = self.conv2(x)
        x = F.relu(x)
        x = self.max_pool2(x)
        x = self.conv3(x)
        x = F.relu(x)
        x = self.conv4(x)
        x = F.relu(x)
        x = self.conv5(x)
        x = F.relu(x)
        x = self.max_pool3(x)
        #Flatten according to the given start_ Axis and stop_ Axis flattens successive dimensions
        x = paddle.flatten(x, start_axis=1, stop_axis=-1)
        x = self.fc1(x)
        x = F.relu(x)
        #Use dropout to suppress overfitting after full connection
        x = self.drop1(x)
        x = self.fc2(x)
        x = F.relu(x)
        #Use dropout to suppress overfitting after full connection
        x = self.drop2(x)
        x = self.fc3(x)
        return x

Data set analysis

Data set analysis, mainly analysisImage path and label textThen read the image and label according to the image path.

import paddle
import os
import random
import numpy as np
from PIL import Image
import as ppvs

class ImageClass(
    ImageCLASS image classification data set analysis, inheriting padding io. Dataset class

    def __init__(self,
                 dataset_path: str,
                 images_labels_txt_path: str,
        Constructor, defining data sets

            dataset_ Path (STR): data set path
            images_ labels_ txt_ Path (STR): text path of images and labels
            Transform (compose, optional): operation combination for converting data. The default is none
            Shuffle (bool, true): randomly disrupt data. The default is true

        super(ImageClass, self).__init__()
        self.dataset_path = dataset_path
        self.images_labels_txt_path = images_labels_txt_path
        self._ check_ Path (dataset_path, "dataset path error")
        self._ check_ Path (images_labels_txt_path, "wrong text path for images and labels")
        self.transform = transform
        self.image_paths, self.labels = self.parse_dataset(
            dataset_path, images_labels_txt_path, shuffle)

    def __getitem__(self, idx):
        Get individual data and labels

            IDX (any): index

            Image (float32): image
            Label (int): Label
        image_path, label = self.image_paths[idx], self.labels[idx]
        return self.get_item(image_path, label, self.transform)

    def get_item(image_path: str, label: int, transform=None):
        Get individual data and labels

            image_ Path (STR): image path
            Label (int): Label
            Transform (compose, optional): operation combination for converting data. The default is none

            Image (float32): image
            Label (int): Label
        image =
        if transform is not None:
            image = transform(image)
        #Convert image HWC to CHW
        image = np.transpose(image, (2, 0, 1))
        return image.astype("float32"), label

    def __len__(self):
        Data quantity

            int: Data quantity
        return len(self.labels)

    def _check_path(self, path: str, msg: str):
        Check whether the path exists

            Path (STR): Path
            MSG (STR, optional): exception message

            Exception: path error, exception
        if not os.path.exists(path):
            raise Exception("{}: {}".format(msg, path))

    def parse_dataset(dataset_path: str, images_labels_txt_path: str, shuffle: bool):
        Data set analysis

            dataset_ Path (STR): data set path
            images_ labels_ txt_ Path (STR): text path of images and labels

            image_ Paths: image path set
            Labels: classification label set
        lines = []
        image_paths = []
        labels = []
        with open(images_labels_txt_path, "r") as f:
            lines = f.readlines()
        #Scramble random data
        if (shuffle):
        for i in lines:
            data = i.split(" ")
            image_paths.append(os.path.join(dataset_path, data[0]))
        return image_paths, labels

Configuration module

You can view and modifymod/config.pyDocument with detailed description

Start training

functiontrain.pyFile, view the command line parameter plus – H

--Whether CPU uses CPU for calculation. CUDA is used by default
  --Learning rate, default 0.001
  --Epochs training rounds, 2 rounds by default
  --Batch size batch quantity, default 2
  --Default number of threads, num-2
  --No save whether to save the model parameters. It is saved by default. After selection, the model parameters are not saved
  --Load dir reads model parameters and subfolders under params directory. It does not read by default
  --Log whether to output visualdl log. It is not output by default
  --Summary outputs network model information, which is not output by default. After selection, only information is output and training will not be started

test model

functiontest.pyFile, view the command line parameter plus – H

--Whether CPU uses CPU for calculation. CUDA is used by default
  --Batch size batch quantity, default 2
  --Number of num workers threads, default 2
  --Load dir reads model parameters, reads subfolders under params directory, and defaults to best directory

Test model GTK interface

functiontest-gtk.pyFile, this program relies on GTK library and can only run on the local computer.


GTK library installation

python3 -m pip install pygobject

User manual

  • 1. ClickSelect modelButton.
  • 2. In the pop-up file dialog box, select the model. The model is displayed in theparamsOf subdirectories under directorymodel.pdparamsDocuments.
  • 3. ClickRandom testButton, you can see the test image, prediction results and actual results.

View results report

functionreport.pyFile, can displayparamsOf all subdirectories under the directoryreport.json
Add parameter–bestaccording tolossThe smallest model parameters are saved inbestUnder the subdirectory.


report. JSON description

Key name explain
id String ID generated by time
loss Loss value of this training
acc ACC value of this training
epochs Epochs value of this training
batch_size Batch of this training_ Size value
learning_rate Learning in this training_ Rate value

Visualdl visual analysis tool

  • Installation and use instructions refer to:VisualDL
  • Add parameters during training–log
  • If it isAI StudioEnvironmental traininglogDownload the directory, unzip it and put it under the local project directorylogcatalogue
  • Run the following command in the project directory
  • Then, according to the prompted URL, open the browser to access the prompted URL
visualdl --logdir ./log

Recommended Today

Ansible combat MySQL installation

Do the preparatory work first, prepare a client, and then equip a master machine. Check connectivity: [[email protected] ~]# ansible mytest -m ping| SUCCESS => { “ansible_facts”: { “discovered_interpreter_python”: “/usr/bin/python” }, “changed”: false, “ping”: “pong” } [[email protected] ~]# 1. Ansible module search method 1. Ansible’s yum module search yum module online documentation: Example: – […]