The bitter history of the copy siege lion: a tearful “one minute” run through mindspree’s lenet model

Time:2021-3-5

Abstract:A cope siege lion tells you with a personal example: the experience of cope code is good for a while, and bug modification takes half a day.

Preface: This article is the operation practice of r0.7-beta. Why do I often have tears in my eyes? Because I love stepping on the pit deeply. I would like to dedicate this article to my friends who like to step on the pit to commemorate the moment of stepping on the pit.

The bitter history of the copy siege lion: a tearful

Start with a picture, the story is all made up.

Sometimes I often ask myself: as a front-end developer, I don’t have a python foundation. How dare I try to use deep learning framework? Who gives courage, is Liang Jingru? Sometimes I often imply that I “don’t have too many skills to press myself”, and live as free and easy as Ah Q in Zhou Shuren’s works, but the reality is like – I’m a driller, drilling one well after another, because I didn’t insist, I passed by the treasure again and again, and finally came back empty handed; sometimes I often comfort myself: “just do it”, it is so popular, what can I do without it? Isn’t it just another place to move bricks? Set a small goal and run through mindspree’s lenet model first!

install

An excellent open source project of full scene deep learning framework should provide an installation image of docker

  • Ubuntu 18.04.5 LTS
  • Docker version 18.09.6

This installation is CPU version, command:

docker pull mindspore/mindspore-cpu:0.7.0-betadocker run -it mindspore/mindspore-cpu:0.7.0-beta /bin/bash

The bitter history of the copy siege lion: a tearful

One step to the “stomach”, go directly to the home directory, and then copy siege lion will start to perform rough copy*, running through mindspree’s lenet model.

Fork code

Why fork code? Do you expect a copy company with no core technology to develop a deep learning framework mindspree? Code thousands, fork first! No, I’ll fork firstMindsprore official warehouseOne key has a deep learning framework. Of course, after fork, we need to clone the code locally, because my code cloud account is Hu Qi, so the path I want to clone ishttps://gitee.com/hu-qi/minds…

git clone https://gitee.com/hu-qi/mindspore

Because the code cloud is our own. The speed is great. Wait a moment, the delicious food will appear soon.

The bitter history of the copy siege lion: a tearful

Rollover site

ModuleNotFoundError: No module named ‘mindspore.dataset.vision’.

I’m used to tossing around and thinking that I can run it directly train.py It’s still “too young, too simple”. A wave of false demonstrations, followed by the huge pits in the picture:

The bitter history of the copy siege lion: a tearful

Fortunately, predecessors have stepped on the pitRun through Huawei mindspree’s lenet network under Windows system, although it’s windows, it seems to be the same problem. In order to record this time, I decided to use git skills!

Checkout branch

In order to be more impressed, I decided to name the branch 9-12 to commemorate the “9.12 trample pit incident”.

cd /home/mindspore
git checkout -b 9-12

The bitter history of the copy siege lion: a tearful

Then, referring to the previous experience and lessons, we start to modify the local file.

Modification of documents

Two files — lenet are modified this time/ train.py And lenet / SRC/ dataset.py .

  • lenet/src/dataset.py
# ……
#Line 20 new level transformations
import mindspore.dataset.transforms.vision.c_transforms as CV
#Line 22 new level transformations
from mindspore.dataset.transforms.vision import Int
# ……

The bitter history of the copy siege lion: a tearful

  • lenet/train.py
# ……
#Notes line 32 and line 34
# from mindspore.common import set_seed
# set_seed(1)
#Line 55 new settings_ grad=false
net_loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True, reduction="mean", is_grad=False)
# ……

The bitter history of the copy siege lion: a tearful

Execute command confidently in lenet Directory: Python train.py –device_ target=CPU –dataset_ sink_ Mode = false, the result is another error:ValueError: The folder ./Data/train does not exist or permission denied!

The bitter history of the copy siege lion: a tearful

At first, I thought it was a problem of permissions. After some attempts, I found that there was no data directory. Then create a new data directory and subdirectories test and train under lenet.

cd /home/mindspore/model_zoo/official/cv/lenet
mkdir Data
mkdir Data/test && mkdir Data/train

Then, with full confidence, he knocked down the order to carry out the training: Python train.py –device_ target=CPU –dataset_ sink_ mode=False

The bitter history of the copy siege lion: a tearful

The result is another pitUnexpected error. There is no valid data matching the dataset API MnistDataset.Please check file path or dataset API validation first.. What should I do? I realized that the script didn’t automatically download the MNIST dataset for me, and I didn’t understand the code, so I had to download it manually.

Download MNIST dataset

MNIST data set:http://yann.lecun.com/exdb/mn…

Data directory structure:

└─Data
    ├─test
    │      t10k-images.idx3-ubyte
    │      t10k-labels.idx1-ubyte
    │
    └─train
            train-images.idx3-ubyte
            train-labels.idx1-ubyte

Since it’s Linux, let’s start with four wgets!

#Switch to the data directory
cd Data
#Download training pictures
wget  
#Download training tag
wget  
#Download test images
wget  
#Download test tag
wget http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz

The bitter history of the copy siege lion: a tearful

Then there’s another wave of decompression operations. Four gunzips:

gunzip train-images-idx3-ubyte.gz
gunzip train-labels-idx1-ubyte.gz
gunzip t10k-images-idx3-ubyte.gz
gunzip t10k-labels-idx1-ubyte.gz

The bitter history of the copy siege lion: a tearful

Finally, there is another wave of mobile file operations. Here are four Videos:

mv train-images-idx3-ubyte ./train
mv train-labels-idx1-ubyte ./train
mv t10k-images-idx3-ubyte ./test/
mv t10k-labels-idx1-ubyte ./test/

At this moment, I feel that it’s really hard to understand the code. It’s clear that a few lines of code will solve the problem. I need so many sets of “Chek Nao” to get the data set. I will learn from Mingming and strive to get rid of it as soon as possibleA siege lionTo achieve the title ofCode freedom, winGreat achievements of open source, trendTop of the world! Cough, another dish of peanuts, I can still chatter!

Training and verification

The new driver is on the road again. Can I train happily this time?

cd /home/mindspore/model_zoo/official/cv/lenet
python train.py --device_target=CPU --dataset_sink_mode=False

The bitter history of the copy siege lion: a tearful

Then finally saw the dawn of victory, ran up! Run! Run! Seeing the continuous emergence of logs, my eyes filled with tears — “Xiao Hu, what are you doing? You’re running a model at work, don’t you want to do it? Tomorrow, I’ll go to the finance department to collect money… ” Fortunately, brother Wang next door broke through in time — “he’s learning in depth. He won’t cut 1 pixel into 2 pixels in the future.”

Next, we need to verify:

cd /home/mindspore/model_zoo/official/cv/lenet
python eval.py --ckpt_path="ckpt/checkpoint_lenet-10_1875.ckpt" --device_target=CPU

Results of operation:

============== Starting Testing ==============
============== {'Accuracy': 0.9847756410256411} ==============

The bitter history of the copy siege lion: a tearful

I can barely accept it. After all, it’s only 10 epochs.

Upload to remote warehouse

Although we have run through mindspree’s lenet, I still hope to record these pits and finally use git skills:

#Switch to local warehouse directory
cd /home/mindspore
#Set Git
git config --global user.email "[email protected]"
git config --global user.name "hu-qi"
#Check the local branch to make sure my 9-12 is still there
git branch
#See what's changed
git status
#New changes
git add .
#Submit changes to cache repository
git commit -m 'finish LeNet'
#Push branch to remote (log in as prompted)
git push origin 9-12

The bitter history of the copy siege lion: a tearful

A fierce operation like a tiger, a look at the code in place pestle, of course, to switch to9-12This branch is unique.

epilogue

The so-called “one minute” experience took about an hour to step on the pit, and then took about three hours to record. Deficiencies, look forward to your advice!

**[click follow to learn about Huawei’s new cloud technology for the first time ~](https://bbs.huaweicloud.com/b…
)**