Potential risks of defer close() in golang

Time:2021-12-22

As a gopher, it’s easy to form a programming convention: whenever an implementation is implementedio.CloserObject of interfacexWhen you get the object and check for errors, it is used immediatelydefer x.Close()To ensure that when the function returnsxObject is closed. Here are two examples of idiomatic writing.

  • HTTP request
resp, err := http.Get("https://golang.google.cn/")
if err != nil {
    return err
}
defer resp.Body.Close()
// The following code: handle resp
f, err := os.Open("/home/golangshare/gopher.txt")
if err != nil {
    return err
}
defer f.Close()
// The following code: handle f

Existing problems

In fact, there are potential problems with this writing.defer x.Close()Its return value is ignored, but in executionx.Close()We cannot guarantee thatxIt must close normally. What if it returns an error? This way of writing makes it possible for the program to make errors that are very difficult to check.

So,Close()What error does the method return? In POSIX Operating Systems, such as Linux or maxos, close the fileClose()The function finally calls the system methodclose(), we canman closeManual, viewingclose()What errors might be returned

ERRORS
     The close() system call will fail if:

     [EBADF]            fildes is not a valid, active file descriptor.

     [EINTR]            Its execution was interrupted by a signal.

     [EIO]              A previously-uncommitted write(2) encountered an
                        input/output error.

errorEBADFIndicates an invalid file descriptor FD, which is independent of the situation in this article;EINTRRefers to UNIX signal interruption; So what are the possible errors in this articleEIO

EIOWhat is your mistakeUncommitted read, what’s wrong?

Potential risks of defer close() in golang

EIOAn error is a of a filewrite()Called before the read of is committedclose()method.

The figure above is a classic computer memory hierarchy. In this hierarchy, from top to bottom, the access speed of devices becomes slower and slower, and the capacity becomes larger and larger. The main idea of memory hierarchy is that the upper layer memory is used as the cache of the lower layer memory.

The CPU accesses registers very fast. In contrast, accessing ram is very slow. Accessing disk or network means wasting time. If eachwrite()If all calls submit data to disk synchronously, the overall performance of the system will be extremely reduced, and our computer will not work like this. When we callwrite()When, the data is not immediately written to the target carrier. Each carrier of the computer memory is caching the data. At the right time, brush the data to the next carrier, which turns the synchronous, slow and blocked synchronization of the write call into a fast and asynchronous process.

So,EIOMistakes are indeed mistakes we need to guard against. This means that if we try to save data to disk, indefer x.Close()When executing, the operating system has not brushed the data to the disk. At this time, we should get the error prompt(As long as the data has not been saved, the data will not be persistent successfully, and it may be lost. For example, in case of power failure, this part of the data will disappear forever, and we will not know it)。 However, according to the conventional writing above, what our program gets isnilWrong.

Solution

We discuss several feasible transformation schemes according to the situation of closing documents

  • The first option is not to use defer
func solution01() error {
    f, err := os.Create("/home/golangshare/gopher.txt")
    if err != nil {
        return err
    }

    if _, err = io.WriteString(f, "hello gopher"); err != nil {
        f.Close()
        return err
    }

    return f.Close()
}

This way of writing requires us toio.WriteStringExplicitly called when execution failsf.Close()Close. However, this scheme requires closing statements to be added to every errorf.Close(), if yesfThere are many write cases, which is prone to the risk of missing and closing files.

  • The second option is to handle this by naming the return value err and closure
func solution02() (err error) {
    f, err := os.Create("/home/golangshare/gopher.txt")
    if err != nil {
        return
    }

    defer func() {
        closeErr := f.Close()
        if err == nil {
            err = closeErr
        }
    }()

    _, err = io.WriteString(f, "hello gopher")
    return
}

This solution solves the risk of forgetting to close files in solution 1. If there are moreif err !=nilThis mode can effectively reduce the number of lines of code.

  • The third option is to display a call to f. close() before the last return statement of the function
func solution03() error {
    f, err := os.Create("/home/golangshare/gopher.txt")
    if err != nil {
        return err
    }
    defer f.Close()

    if _, err := io.WriteString(f, "hello gopher"); err != nil {
        return err
    }

    if err := f.Close(); err != nil {
        return err
    }
    return nil
}

This solution can be used inio.WriteStringWhen an error occurs, due todefer f.Close()The existence of can be obtainedcloseCall. Can also be inio.WriteStringWhen no error occurs, but the cache is not flushed to disk, geterr := f.Close()And becausedefer f.Close()It doesn’t return an error, so I don’t worry about twiceClose()The call overwrites the error.

  • The last option is to execute f.sync() when the function returns
func solution04() error {
    f, err := os.Create("/home/golangshare/gopher.txt")
    if err != nil {
        return err
    }
    defer f.Close()

    if _, err = io.WriteString(f, "hello world"); err != nil {
        return err
    }

    return f.Sync()
}

Due to callclose()This is the last chance to get the error returned by the operating system, but when we close the file, the cache will not necessarily be flushed to the disk. Then we can callf.Sync()(it calls system functions internally)fsync)Force the kernel to persist the cache to disk.

// Sync commits the current contents of the file to stable storage.
// Typically, this means flushing the file system's in-memory copy
// of recently written data to disk.
func (f *File) Sync() error {
    if err := f.checkValid("sync"); err != nil {
        return err
    }
    if e := f.pfd.Fsync(); e != nil {
        return f.wrapErr("sync", e)
    }
    return nil
}

becausefsyncThis pattern can be well avoidedcloseAppearingEIO。 Predictably, due to the mandatory disk brushing, although this scheme can well ensure the data security, it will greatly reduce the implementation efficiency.