Three most popular methods of copying files in Go

Time:2019-4-10

This article will show you how to use GO programming language. Although there are more than three methods of copying files in Go, this article will introduce three most common methods: these three methods have their own advantages and disadvantages, we only need to choose the most appropriate application, without blindly pursuing performance.
Use io. Copy () to call functions from GO libraries; read input files at once and write them to another file; and use buffers to copy files in small chunks.

Method 1: Using io. Copy ()

The first version of the utility will use the functions of the io. Copy () standard GO library. The logic of the utility can be found in the implementation. The functions of copy () are as follows:

In addition to testing the existence of the file to be copied (os. Stat (src)), and is a regular file (sourceFileStat. Mode (). IsRegular ()) so that you can open it to read, all the work is declared by io. Copy (destination, source). This, this, that, the io. Copy () function returns the number of bytes duplicated and the first error message that occurred during duplication. In Go, if there is no error message, the value of the error variable will be nil.

You can learn more about io. Copy () on the IO encapsulated file page.

Executing cp1.go generates the next output:


$ run cp1
Please provide two command line arguments
$ run cp1 fileCPtxt tmpfileCPCOPY
Copied bytes
$ diff fileCPtxt tmpfileCPCOPY

It’s not always a bad thing that this technology is as simple as possible, but it doesn’t give developers flexibility. However, sometimes developers need or want to decide how to read files.

Method 2: Use ioutil. WriteFile () and ioutil. ReadFile ()

The second way to copy files is to use ioutil. ReadFile () and ioutil. WriteFile () functions. The first function reads the contents of the entire file into the byte slice, and the second function writes the contents of the byte slice into the file.
The logic of the utility can be found in the following GO code:

In addition to these two if blocks, which are part of the way the program works, you can see that the functions of the program are stated in ioutil. ReadFile () and ioutil. WriteFile ().

Executing cp2.go generates the next output:


$ run cp2
Please provide two command line arguments
$ run cp2 fileCPtxt tmpcopyFileCP
$ diff fileCPtxt tmpcopyFileCP

Note that although this technique will copy a file, it may not be effective when you want to copy a large file, because ioutil. ReadFile () will also be huge.

Method 3: Use os. Read () and os. Write ()

The third way to copy files in GO is to use the utilities that cp3.go will develop in this section. It takes three parameters: the file name of the input file, the file name of the output file and the size of the buffer.

The most important part of cp3.go resides in the following for loop, which can be used in copy ():

This technique uses os. Read () to read a small portion of the input file into a file called buf and os. Write () to write the contents of the buffer to the file. When reading errors or reaching the end of the file, the copy process stops (io.EOF).

Executing cp3.go generates the next output:


$ run cp3
usage cp3 source destination BUFFERSIZE
$ run cp3 fileCPtxt tmpbuf10 
Copying fileCPtxt to tmpbuf10
$ run cp3 fileCPtxt tmpbuf20 
Copying fileCPtxt to tmpbuf20

As you can see, the size of the buffer greatly affects cp3.go.

Do some benchmarks

The last part of this article will try to compare these three programs and cp3. go. For different buffer sizes, use the time (1) command line utility.

The following output shows cp1.go, cp2.go, and cp3.go when copying 500 MB files:

The output results show that the performance of these three utilities is very similar, which means that the functions of the standard GO library are very smart and optimized.

Now let’s test how buffer size affects cp3. go. If the size of the buffer is 10, 20, and 1,000 bytes to copy 500 MB files on a fairly fast computer, the output generated shows that the larger the buffer, the more or less the cp3.go utility is expected. In addition, copying large files with a buffer size of less than 20 bytes is a very slow process and should be avoided.

summary

Above is the whole content of this article. I hope that the content of this article has a certain reference learning value for everyone’s study or work. Thank you for your support for developpaer. If you want to know more about it, please check the links below.