MindSpore! I love this just open source deep learning framework!

Time:2021-7-29

[Abstract]This paper mainly adopts two practical application cases: one is MNIST handwritten data recognition based on local jupyter notebook; Second, based on the cifar-10 image classification of Huawei cloud server, this paper introduces the open source framework mindspire.

MindSpore! I love this just open source deep learning framework!

I still remember that at this year’s Huawei Developer Conference HDC 2020, mindspire, a deep learning framework that has always attracted attention, was finally open-source.

I’ve been following mindspire before, and I’m looking forward to it. Mindspire is a unified training and reasoning framework that supports the independence / collaboration of end, edge and cloud. With tensorflow and pytorch   And other popular deep learning frameworks. Mindspire aims to significantly reduce the threshold of AI application development and make artificial intelligence ubiquitous.

MindSpore! I love this just open source deep learning framework!

The biggest feature of mindspire is that the development threshold is greatly reduced and the development efficiency is improved, which can significantly reduce the model development time.

Therefore, the advantages of using mindspire can be summarized as follows:

● simple development experience

● flexible commissioning mode

● give full play to the potential of hardware

● full scenario rapid deployment

Now that it’s open source, get started and try this open source mindspire! In this article, I will introduce the installation and start-up tutorial of mindspire, and run a complete AI training and testing process through a simple image recognition case.

1、 Installation of mindspire

There are many installation methods for the open source framework mindspire, which can be installed on windows, Ubuntu or Huawei ascend 910. For detailed installation methods, see the following link:

https://www.mindspore.cn/install

Here are the two simplest installation methods!

1. Docker installation

Docker installation is the simplest. Please refer to:

https://gitee.com/mindspore/m…

with   0.3.0-alpha   For example:

  • CPU:

    docker pull mindspore/mindspore-cpu:0.3.0-alpha

  • GPU:

    docker pull mindspore/mindspore-gpu:0.3.0-alpha

After installation, you can see the installed image and create your container with the following command:

docker run -it mindspore/mindspore-cpu:0.3.0-alpha /bin/bash

2. Win10+Anaconda+MindSpore

It is also very simple to install using win10 + Anaconda + mindspire. This article will install mindspire in this way.

In the mindspire installation homepage, select the installation related configuration:

  • Version: 0.3.0-alpha
  • Hardware platform: CPU
  • Operating system: windows-64
  • Programming language: Python 3.7.5

First, in win10   Anaconda is an open source Python distribution that contains more than 180 science packages such as CONDA and Python and their dependencies.

Then, create a virtual environment.

1) . open Anaconda prompt terminal in Anaconda component:

MindSpore! I love this just open source deep learning framework!

2) . use the following command to create a virtual environment mindpool (the name can be customized) and enter the virtual environment:

conda create -n mindspore python=3.7.5 
conda activate mindspore

3) Install dependent libraries according tohttps://gitee.com/mindspore/m…The dependent libraries listed are installed using the CONDA command. For example:

conda install numpy

4) . according to the relevant configuration selected previously, on the website:https://www.mindspore.cn/vers…Select the corresponding mindspire version to:

mindspore-0.3.0-cp37-cp37m-win_amd64.whl

You can download the. WHL file locally and install it with PIP (online installation with CONDA command may be slow, so you can choose to download the. WHL file locally and install it with PIP command):

pip install mindspore-0.3.0-cp37-cp37m-win\_amd64.whl

Finally, test whether the installation is successful. Enter the python shell and execute the following commands. If there is no loading error message such as no module named ‘mindspore’, the installation is successful.

MindSpore! I love this just open source deep learning framework!

At this point, the installation is complete!

2、 Implementation based on local jupyter   MNIST handwritten dataset classification

1. Installation   Jupyter   Notebook

First, install Jupiter notebook in the virtual environment mindspot. To do this: open the anaconda component Anaconda navigator.

MindSpore! I love this just open source deep learning framework!

In Anaconda navigator, application on selects the newly created virtual environment mindspot, and click Install under the component Jupiter notebook to install. After installation, see the following figure:

MindSpore! I love this just open source deep learning framework!

Click launch under notebook to open Jupiter notebook.

2. Download dataset

MNIST handwritten data set must be familiar to everyone. It contains numbers 0-9 and consists of 60000 training pictures and 10000 test pictures.

MindSpore! I love this just open source deep learning framework!

MNIST dataset download page:

http://yann.lecun.com/exdb/mn…

Using mindspire, we can directly define a download_ Use the dataset function to automatically download MNIST datasets:

def download\_dataset():
    """Download the dataset from http://yann.lecun.com/exdb/mnist/."""
    print("\*\*\*\*\*\*Downloading the MNIST dataset\*\*\*\*\*\*")
    train\_path = "./MNIST\_Data/train/"
    test\_path = "./MNIST\_Data/test/"
    train\_path\_check = os.path.exists(train\_path)
    test\_path\_check = os.path.exists(test\_path)
    if train\_path\_check == False and test\_path\_check ==False:
        os.makedirs(train\_path)
        os.makedirs(test\_path)
    train\_url = {"http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz", "http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz"}
    test\_url = {"http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz", "http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz"}
    for url in train\_url:
        url\_parse = urlparse(url)
        # split the file name from url
        file\_name = os.path.join(train\_path,url\_parse.path.split('/')\[-1\])
        if not os.path.exists(file\_name.replace('.gz','')):
            file = urllib.request.urlretrieve(url, file\_name)
            unzipfile(file\_name)
            os.remove(file\_name)
    for url in test\_url:
        url\_parse = urlparse(url)
        # split the file name from url
        file\_name = os.path.join(test\_path,url\_parse.path.split('/')\[-1\])
        if not os.path.exists(file\_name.replace('.gz','')):
            file = urllib.request.urlretrieve(url, file\_name)
            unzipfile(file\_name)
            os.remove(file\_name)

This function can automatically download the data set to the local  ./ MNIST_ Data   In the directory, the training sets are placed in the subdirectory  / train   Under, the test set is placed in a subdirectory  / test   Down.

three   Data preprocessing

After the MNIST dataset is ready, the next step is to preprocess the dataset, including adjusting the image size to 32×32 (because we use the lenet-5 network, which will be described later), pixel normalization and batch_ Set the size to 32 (adjustable), and so on.

Mindspire provides mindpore.dataset.mnistdataset to directly define the Minist dataset, which is very convenient. Apply data operations to the dataset using the mindscore.dataset.mnistdataset.map mapping function.

We define create_ Use the dataset() function to create a dataset:

def create\_dataset(data\_path, batch\_size=32, repeat\_size=1,
                   num\_parallel\_workers=1):
    """ create dataset for train or test
    Args:
        data\_path: Data path
        batch\_size: The number of data records in each group
        repeat\_size: The number of replicated data records
        num\_parallel\_workers: The number of parallel workers
    """
    # define dataset
    mnist\_ds = ds.MnistDataset(data\_path)


    # define operation parameters
    resize\_height, resize\_width = 32, 32
    rescale = 1.0 / 255.0
    shift = 0.0
    rescale\_nml = 1 / 0.3081
    shift\_nml = -1 \* 0.1307 / 0.3081


    # define map operations
    resize\_op = CV.Resize((resize\_height, resize\_width), interpolation=Inter.LINEAR)  # Resize images to (32, 32)
    rescale\_nml\_op = CV.Rescale(rescale\_nml, shift\_nml) # normalize images
    rescale\_op = CV.Rescale(rescale, shift) # rescale images
    hwc2chw\_op = CV.HWC2CHW() # change shape from (height, width, channel) to (channel, height, width) to fit network.
    type\_cast\_op = C.TypeCast(mstype.int32) # change data type of label to int32 to fit network


    # apply map operations on images
    mnist\_ds = mnist\_ds.map(input\_columns="label", operations=type\_cast\_op, num\_parallel\_workers=num\_parallel\_workers)
    mnist\_ds = mnist\_ds.map(input\_columns="image", operations=resize\_op, num\_parallel\_workers=num\_parallel\_workers)
    mnist\_ds = mnist\_ds.map(input\_columns="image", operations=rescale\_op, num\_parallel\_workers=num\_parallel\_workers)
    mnist\_ds = mnist\_ds.map(input\_columns="image", operations=rescale\_nml\_op, num\_parallel\_workers=num\_parallel\_workers)
    mnist\_ds = mnist\_ds.map(input\_columns="image", operations=hwc2chw\_op, num\_parallel\_workers=num\_parallel\_workers)


    # apply DatasetOps
    buffer\_size = 10000
    mnist\_ds = mnist\_ds.shuffle(buffer\_size=buffer\_size)  # 10000 as in LeNet train script
    mnist\_ds = mnist\_ds.batch(batch\_size, drop\_remainder=True)
    mnist\_ds = mnist\_ds.repeat(repeat\_size)


    return mnist\_ds

Through the above function, the preprocessing of the MNIST data set just downloaded is completed.

four   Define network

Lenet-5 is a very efficient convolutional neural network for handwritten character recognition. Lenet-5 has 7 layers, excluding input, and each layer contains trainable parameters; Each layer has multiple feature maps, and each feature map extracts an input feature through a convolution filter.

MindSpore! I love this just open source deep learning framework!

1) Model initialization

Initialize the parameters using the mindscore.common.initializer.truncatednormal method, and define conv and FC_ with_ Initialize initializes the volume layer and the full connection layer respectively.

import mindspore.nn as nnfrom mindspore.common.initializer import TruncatedNormal
def conv(in\_channels, out\_channels, kernel\_size, stride=1, padding=0):    """Conv layer weight initial."""    weight = weight\_variable()    return nn.Conv2d(in\_channels, out\_channels,                     kernel\_size=kernel\_size, stride=stride, padding=padding,                     weight\_init=weight, has\_bias=False, pad\_mode="valid")
def fc\_with\_initialize(input\_channels, out\_channels):    """Fc layer weight initial."""    weight = weight\_variable()    bias = weight\_variable()    return nn.Dense(input\_channels, out\_channels, weight, bias)
def weight\_variable():    """Weight initial."""    return TruncatedNormal(0.02)

use   The mindspot.common.initializer.truncatednormal method can easily initialize the network weight coefficient without customizing the initialization function.

2)   Define lenet-5 network

It is also very simple to define the lenet-5 network by mindspire. According to the network structure, you can define the corresponding convolution layer and full connection layer. Initializing functions__ init__ Each layer of neural network is defined, and then the forward construction of neural network is completed by defining construct method.

class LeNet5(nn.Cell):    """Lenet network structure."""    # define the operator required    def \_\_init\_\_(self):        super(LeNet5, self).\_\_init\_\_()        self.conv1 = conv(1, 6, 5)        self.conv2 = conv(6, 16, 5)        self.fc1 = fc\_with\_initialize(16 \* 5 \* 5, 120)        self.fc2 = fc\_with\_initialize(120, 84)        self.fc3 = fc\_with\_initialize(84, 10)        self.relu = nn.ReLU()        self.max\_pool2d = nn.MaxPool2d(kernel\_size=2, stride=2)        self.flatten = nn.Flatten()
    # use the preceding operators to construct networks    def construct(self, x):        x = self.conv1(x)        x = self.relu(x)        x = self.max\_pool2d(x)        x = self.conv2(x)        x = self.relu(x)        x = self.max\_pool2d(x)        x = self.flatten(x)        x = self.fc1(x)        x = self.relu(x)        x = self.fc2(x)        x = self.relu(x)        x = self.fc3(x)        return x

Lenet-5 is a very typical and simple convolutional neural network   The structure of each layer of lenet-5 can be seen in detail in the construct method.

3) Define loss function

The loss functions supported by mindspire include softmaxcrossentropywithlogits, l1loss, mselos, etc. The softmaxcrossentropwithlogits cross entropy loss function is used here.

from mindspore.nn.loss import SoftmaxCrossEntropyWithLogits
# define the loss functionnet\_loss = SoftmaxCrossEntropyWithLogits(is\_grad=False, sparse=True, reduction='mean')

4) Define network gradient descent algorithm

Mindspire supports gradient descent algorithms such as Adam, Adam weightdecay, momentum, etc. The popular momentum algorithm is used here. The learning rate is set to 0.01 and the momentum parameter is set to 0.9.

# learning rate setting
lr = 0.01
momentum = 0.9
# define the optimizer
net\_opt = nn.Momentum(network.trainable\_params(), lr, momentum)

5. Training network

1) Model saving

The mindspot.train.callback.modelcheckpoint method can save the network model and parameters.

config\_ck = CheckpointConfig(save\_checkpoint\_steps=1875, keep\_checkpoint\_max=10)
# save the network model and parameters for subsequence fine-tuning
ckpoint\_cb = ModelCheckpoint(prefix="checkpoint\_lenet", config=config\_ck)

2) Training network

The training network is conducted using the model.train method. Here put epoch_ When the size is set to 1, the data set is trained in one iteration. The change of loss value will be printed during training.

from mindspore.nn.metrics import Accuracyfrom mindspore.train.callback import LossMonitorfrom mindspore.train import Model
def train\_net(args, model, epoch\_size, mnist\_path, repeat\_size, ckpoint\_cb, sink\_mode):    """define the training method"""    print("============== Starting Training ==============")    #load training dataset    ds\_train = create\_dataset(os.path.join(mnist\_path, "train"), 32, repeat\_size)    model.train(epoch\_size, ds\_train, callbacks=\[ckpoint\_cb, LossMonitor()\], dataset\_sink\_mode=sink\_mode)

epoch\_size = 1
mnist\_path = "./MNIST\_Data
# group layers into an object with training and evaluation features
model = Model(network, net\_loss, net\_opt, metrics={"Accuracy": Accuracy()})
train\_net(args, model, epoch\_size, mnist\_path, repeat\_size, ckpoint\_cb)

Where MNIST_ Path is the path to the MNIST dataset.

3) Hardware information

In the main function, don’t forget to configure the hardware information that mindspire runs. Because we are in the CPU environment, so ‘– device_ Target ‘is set to “CPU”.

parser = argparse.ArgumentParser(description='MindSpore LeNet Example')
parser.add\_argument('--device\_target', type=str, default="CPU", choices=\['Ascend', 'GPU', 'CPU'\],
                        help='device where the code will be implemented (default: CPU)')
args = parser.parse\_args(args=\[\])
context.set\_context(mode=context.GRAPH\_MODE, device\_target=args.device\_target)

‘– device’ here_ Target ‘defaults to “CPU”. You can also select “ascend” or “GPU” according to the hardware. The graph pattern “context. Graph” is used_ MODE”。

4) Model training

Execute the program and the model training begins. The loss value will be printed during the training:

...
epoch: 1 step: 262, loss is 1.9212162
epoch: 1 step: 263, loss is 1.8498616
epoch: 1 step: 264, loss is 1.7990671
epoch: 1 step: 265, loss is 1.9492403
epoch: 1 step: 266, loss is 2.0305142
epoch: 1 step: 267, loss is 2.0657792
epoch: 1 step: 268, loss is 1.9582214
epoch: 1 step: 269, loss is 0.9459006
epoch: 1 step: 270, loss is 0.8167224
epoch: 1 step: 271, loss is 0.7432692
...

It can be seen that the loss will gradually decrease and the accuracy will gradually improve. The final loss is 0.067.

After the training, the saved model file is obtained:

checkpoint_lenet-1_1875.ckpt

6. Model test

After getting the model file, use model. Eval ()   The interface reads in the test data set and runs the test data set through the model. Define test function test_ net():

def test\_net(args, network, model, mnist\_path):
    """Define the evaluation method."""
    print("============== Starting Testing ==============")
    # load the saved model for evaluation
    param\_dict = load\_checkpoint("checkpoint\_lenet-1\_1875.ckpt")
    # load parameter to the network
    load\_param\_into\_net(network, param\_dict)
    # load testing dataset
    ds\_eval = create\_dataset(os.path.join(mnist\_path, "test"))
    acc = model.eval(ds\_eval, dataset\_sink\_mode=False)
    print("============== Accuracy:{} ==============".format(acc))

Run test network:

test\_net(args, network, model, mnist\_path)

\============== Starting Testing ==============
============== Accuracy:{'Accuracy': 0.9663461538461539} ==============

Finally, we can see that the accuracy of the newly trained lenet-5 network model on the test set is 96.63%, and the effect is very good.

So far, we have completed training the lenet-5 model using the mindspire framework. MNIST handwritten dataset classification based on local jupyter is realized. In general, mindspire provides many modular methods for model building and training, which is very convenient for us to quickly build a neural network model. You can try to build your own neural network according to your actual needs.

Complete code of this section:

https://gitee.com/mindspore/d…_code/lenet.py

3、 Using mindspire on ECS

In addition to using the mindspire framework locally, we can also use mindspire on Huawei ECs. Another advantage of using mindspire on Huawei cloud is that we can apply to use the shengteng AI processor resource pool as hardware.

Modelarts is a one-stop AI development platform for developers provided by Huawei cloud, and mindspire is integrated. Next, we will use resnet-50 network to identify cifar-10 pictures under modelarts.

1. Prepare modelarts

1) Enter the Huawei cloud official website and register your account.

Specific operation:

https://support.huaweicloud.c…_08_0001.html

2)   Obtain the access key and complete the modelarts configuration.

Specific operation:

https://support.huaweicloud.c…_08_0002.html

3)   Create OBS bucket

Specific operation:

https://support.huaweicloud.c…_08_0003.html

2. Apply for server upgrade AI processor resources

In order to use Huawei cloud shengteng AI processor on modelarts, we need to apply for experience qualification. The application method is also very simple. We can apply on the following website:

https://console.huaweicloud.c…

MindSpore! I love this just open source deep learning framework!

You can fill in the application carefully. Generally, it will be approved within two working days.

three   Data preparation

1)   Download cifar-10 dataset

CIFAR-10   There are 60000 color images in the data set. These images are 32 * 32 and are divided into 10 categories, with 6000 images in each category.

MindSpore! I love this just open source deep learning framework!

Cifar-10 dataset download address:

http://www.cs.toronto.edu/~kr…

Note download   CIFAR-10   binary version   edition.

2)   Create your own OBS bucket (for example: Mine MS dataset)

Modelarts uses object storage service (OBS) for data storage. Therefore, before starting the training task, it is necessary to upload the data to obs.

First, log in to the OBS management console:

https://storage.huaweicloud.c…

Create OBS bucket   Mine MS dataset (the name can be modified, similar below).

Then, in the newly created OBS bucket, create a folder for storing data: click the bucket to be operated in the bucket list, click object in the left navigation bar, and create a new folder   mine-cifar-10。

Finally, upload the downloaded cifar-10 data set to the data directory according to the following directory structure   mine-cifar-10   Medium:

└ - object store / mine MS dataset / mine-cifar-10
    ├─train
    │      data\_batch\_1.bin
    │      data\_batch\_2.bin
    │      data\_batch\_3.bin
    │      data\_batch\_4.bin
    │      data\_batch\_5.bin
    │
    └─eval
        test\_batch.bin

4. Procedure preparation

Create a new OBS bucket (e.g. mine-resnet50-train) and create a code directory (e.g. mine-resnet50) in the bucket_ cifar10_ train)。 Create in this bucket at the same time   output   Catalog and   log   Directory, used to store models and logs.

The web address will be:

https://gitee.com/mindspore/d…_code/sample_for_cloud/

Two. Py files in   dataset.py   and   resnet50_ train.py   Download and upload to the code directory   mine-resnet50_ cifar10_ train   Yes.

Code directory   mine-resnet50_ cifar10_ train   The structure is as follows:

└ - object store / mine resnet50 train
    ├─mine-resnet50\_cifar10\_train
    │      dataset.py
    │      resnet50\_train.py
    │
    ├─output
    └─log

five   Create training task

After you have prepared the data and executed the script, you can create training tasks on the ECS.

1)   Enter modelarts console

Open Huawei cloud modelarts Homepage

https://www.huaweicloud.com/p…

Click “enter console”.

2) Create training assignments using mindspire as a common framework

Select “Training Management > training assignments” in the left navigation bar to enter the “training assignments” list by default.

In the training assignment list, click “create” in the upper left corner to enter the “create training assignment” page.

On the create training assignment page, the training assignment name can be customized, such as   mine-resnet50-trainjob。 Fill in relevant parameters of training operation. The specific configuration parameters are as follows:

MindSpore! I love this just open source deep learning framework!

It is worth noting that the algorithm source is often selected by the framework   Ascend powered engine, because the hardware we use is Huawei cloud shengteng AI processor. Mindspire version selection   MindSpore-0.1-python3.7-aarch64   Just.

After configuration, click Next – > submit – > return to the training job list. You can see that the training job mine-resnet50-trainjob is running:

MindSpore! I love this just open source deep learning framework!

The whole operation process is about   8 and a half minutes, it shows that the operation is successful, indicating that the model training test is completed.

MindSpore! I love this just open source deep learning framework!

Finally, click the training operation mine-resnet50-trainjob. In the log, you can see that the accuracy of the model on the test set is 92.49%, indicating that the model has a good effect.

MindSpore! I love this just open source deep learning framework!

We can also download log files from OBS and view them.

The above is a simple tutorial on using mindspire on the cloud.

4、 Summary:

This paper mainly introduces the open source framework mindspire through two practical application cases. One is MNIST handwritten data recognition based on local jupyter notebook; The second is cifar-10 image classification based on Huawei ECs. Both cases revolve around and use mindspire.

From my personal experience, mindspire is very easy to use, and the function encapsulation is relatively simple and convenient to use. Through the hands-on tutorial, you can do it yourself and feel the charm of the open source framework mindspire.

You can also use mindspire to build a neural network model and solve practical problems according to your specific application scenarios and practical cases. Whether it is computer vision or natural language processing, I believe mindspire can bring you a smooth experience.

reference material:

https://www.mindspore.cn/

https://www.mindspore.cn/tuto…

https://support.huaweicloud.c…

Click focus to learn about Huawei cloud’s new technologies for the first time~