Microsoft’s favorite rust language, is its security really reliable

Time:2021-2-22

Abstract:In recent years, rust language has gained a lot of attention with its rapid growth. Its feature is to ensure high security at the same time, to achieve the same performance as C / C + +. After rust is used in many projects, how about its actual security performance?

In recent years, rust language has gained a lot of attention with its rapid growth. Its feature is to ensure high security at the same time, get the same performance as C / C + +, so that the field of system programming has a rare new choice full of hope. After rust is used in many projects, how about its actual security performance? In June this year, five scholars from three universities published a research result at the ACM SIGPLAN International Conference (pldi’20), which conducted a comprehensive investigation on the security defects of open source projects using rust language in recent years.

This study investigated five software systems developed in rust language, five widely used rust libraries, and two vulnerability databases. A total of 850 unsafe code usage, 70 memory security defects and 100 thread security defects were investigated.

Microsoft's favorite rust language, is its security really reliable

In the survey, the researchers not only looked at the defects reported in all vulnerability databases and software public reports, but also looked at the submission records in all open source software code repositories. Through manual analysis, they define the types of bugs to be fixed and classify them into the corresponding memory safety / thread safety issues. All the investigated questions are put into the open git warehouse:https://github.com/system-pclub/rust-study

Analysis of memory security

The study investigated 70 memory security issues. For each problem, researchers carefully analyzed the root cause and effect of the problem. The root cause of the problem is defined by the patch code submitted when modifying the problem, that is, where the coding error occurs; the effect of the problem refers to the location of the observable error caused by the code running, such as the location of the code where the buffer overflow occurs. Because there is a transmission process from root cause to effect, the two are sometimes far apart. According to the different code regions where the root cause and effect are located, the researchers divide the errors into four categories: safe > safe, safe > unsafe, unsafe > safe, unsafe > unsafe. For example, if a coding error occurs in the safe code, but the effect is reflected in the unsafe code, it is classified as safe > unsafe.

On the other hand, according to the traditional classification of memory problems, problems can be divided into two categories: wrong access and lifetime violation, which can be further subdivided into buffer overflow, null pointer dereferencing and reading uninitialized memory Memory, invalid free, use after free, double free and so on. According to the two dimensions, the statistical data of the questions are as follows:

Microsoft's favorite rust language, is its security really reliable

It can be seen from the statistical results that there is only one memory security problem that does not involve unsafe code at all. Further investigation found that this problem occurred in the early version of rust v0.3, and later stable versions of the compiler have been able to intercept this problem. Therefore, it can be said that:The safe code mechanism of rust language can effectively avoid memory security problems. All memory security problems found in stable versions are related to unsafe code.

However, this doesn’t mean that we can find the problem effectively just by checking all the unsafe code segments. Because sometimes the root cause of the problem will appear in the safe code, but the effect will appear in the unsafe code. An example is given in the paper: (hi3ms has no rust code editing function, so it can only make do with other languages.)

CSS code

    pub fn sign(data: Option<&[u8]>) {
        let p = match data {
            Some(data) => BioSlice::new(data).as_ptr(),
            None => ptr::null_mut(),
        };
        unsafe {
            let cms = cvt_p(CMS_sign(p));
        }
    }

In this code, P is the raw pointer type. In the safe code, when data contains a value (some branch), the branch attempts to create a bioslice object and assign the object pointer to P. However, according to rust’s life cycle rule, the newly created bioslice object is released at the end of the match expression, and P is passed to CMS_ The sign function is a wild pointer.There is no problem with the unsafe code segment in this example. If you only look at the unsafe code, you can’t find the error after releasing.The revised code for this problem is as follows:

CSS code

    pub fn sign(data: Option<&[u8]>) {
        let bio = match data {
            Some(data) => Some(BioSlice::new(data)),
            None => None,
        };
        let p = bio.map_or(ptr::null_mut(),|p| p.as_ptr());
        unsafe {
            let cms = cvt_p(CMS_sign(p));
        }
    }

The modified code correctly extends the life cycle of bio. All the changes only occur in the safe code segment, not the unsafe code.

Since all problems involve unsafe code, can we avoid the problem by eliminating unsafe code? The researchers further investigated all the bug modification strategies and found that most of the modifications involved unsafe code, but only a few of them completely removed unsafe code. This shows that unsafe code can not be completely avoided.

What is the value of unsafe? Why can’t it be completely removed? According to the survey of 600 unsafe uses, 42% of them are for reusing existing code (such as converting rust code from existing C code, or calling C library functions), 22% are for improving performance, and the remaining 14% are for realizing functions and bypassing various verifications of rust compiler.

Further studies have shown that,Use the unsafe method to access the offset memory (such as slice:: get)_ Compared with the subscript method of safe, the speed of unsafe can be 4-5 times faster.This is due to the run-time verification of buffer overrun brought by rust, so in some performance critical areas, the role of unsafe is indispensable.

It should be noted that unsafe code segments do not necessarily contain unsafe operations. Researchers found that there are five unsafe codes, even if the unsafe tag is removed, there will be no compilation errors – that is, it can be regarded as safe code from the compiler perspective. It is marked as unsafe code to prompt users with key calling contracts, which cannot be checked by the compiler. A typical example is string:: from in the rust standard library_ utf8_ Unchecked() function, which has no unsafe operation inside, but is marked as unsafe. The reason is that this function constructs a string object directly from a piece of memory provided by the user, but it does not check whether the content is a legal UTF-8 encoding. Rust requires that all string objects must be legal UTF-8 encoding strings. In other words, string:: from_ utf8_ The unsafe tag of unchecked() function is only used to pass logical calling contract, which has no direct relationship with memory security. However, if the contract is violated, it may lead to memory security problems in other places (possibly safe code). This kind of unsafe label cannot be removed.

Microsoft's favorite rust language, is its security really reliable

Even so, if possible, eliminating unsafe code fragments is an effective way to improve security. The researchers investigated 130 modification records that removed unsafe, and found that 43 of them completely changed unsafe code segments into safe code through code refactoring, while the remaining 87 ensured security by encapsulating unsafe code into safe interface.

Analysis of thread safety

This study investigated 100 thread safety issues. Problems can be divided into two types: blocking problems (causing deadlock) and non blocking problems (causing data competition). There are 59 blocking problems, 55 of which are related to synchronization primitives (mutex and condvar)

Microsoft's favorite rust language, is its security really reliable

Although rust claims to be able to program without fear of concurrency, it also provides well-designed synchronization primitives to avoid concurrency problems. However, just using safe code may lead to deadlock caused by repeated locking, and worse still, it is not easy to use safe code,Some problems, even caused by the unique design of rust, will not appear in other languages.An example is given in the paper

CSS code

    fn do_request() {
        //client: Arc<RwLock<Inner>>
        match connect(client.read().unwrap().m) {
            Ok(_) => {
                let mut inner = client.write().unwrap();
                inner.m = mbrs;
            }
            Err(_) => {}
        };
    }

In this code, the client variable is protected by a rwlock. Rwlock’s methods read() and write() will automatically lock variables and return the lockresult object, which will be automatically unlocked at the end of the lockresult object’s life cycle.

Obviously, the author of the code thinks that client.read The temporary lockresult object returned by () is released and unlocked before the match branch inside the match, so it can be used again in the match branch client.write () lock it. However, the life cycle rules of rust language make it difficult to understand client.read The actual life cycle of the object returned by () is extended to the end of the match statement, so the actual result of this code is to try to obtain the write () lock when the read () lock has not been released, resulting in a deadlock.

This kind of temporary object life cycle rule is a very obscure rule in rust language. For a detailed explanation, please refer to this article.

According to the correct usage of life cycle, the code was later modified as follows:

CSS code

    fn do_request() {
        //client: Arc<RwLock<Inner>>
        let result = connect(client.read().unwrap().m);
        match result {
            Ok(_) => {
                let mut inner = client.write().unwrap();
                inner.m = mbrs;
            }
            Err(_) => {}
        };
    }

After modification, client.read The temporary object returned by () is released immediately after the end of the line statement and will not be locked to the inside of the match statement.

For 41 non blocking problems, 38 of them are caused by improper protection of shared resources. According to different protection methods for shared resources and whether the code is safe, these problems are further classified as follows:

Microsoft's favorite rust language, is its security really reliable

Of the 38 problems, 23 occurred in unsafe code and 15 occurred in safe code. Although rust sets strict data borrowing and access rules, because concurrent programming depends on the logic and semantics of the program, even safe code can not completely avoid the problem of data competition. An example is given in the paper

CSS code

    impl Engine for AuthorityRound {
        fn generate_seal(&self) -> Seal {
            if self.proposed.load() { return Seal::None; }
            self.proposed.store(true);
            return Seal::Regular(...);
        }
    }

In this code, the proposed member of the authorityround structure is a boolean type atomic variable. Load() will read the value of the variable, and store() will set the value of the variable. Obviously, this code wants to return seal:: regular (…) only once during concurrent operations, and then seal:: none. However, the operation of atomic variables is not handled correctly. If two threads execute the if statement and read the false result at the same time, this method may return seal:: regular (…) to both threads.

The modified code for this problem is as follows. Compare is used here_ and_ Swap () method ensures that the reading and writing of atomic variables are completed together in a non preemptive atomic operation.

CSS code

    impl Engine for AuthorityRound {
        fn generate_seal(&self) -> Seal {
            if !self.proposed.compare_and_swap(false, true) {
                return Seal::Regular(...);
            }
            return Seal::None;
        }
    }

This data contention problem does not involve any unsafe code, and all operations are completed in safe code.This also shows that even if the strict concurrency checking rules are set in rust language, programmers still need to manually ensure the correctness of concurrent access in coding

Suggestions for rust defect checking tool

Obviously, from the previous investigation, the check of rust compiler itself is not enough to avoid all problems, and even some obscure life cycle may trigger new problems. The researchers suggest adding the following checking tools to rust language:
1. Improve ide. When the programmer selects a variable, the scope of its life cycle will be displayed automatically, especially for the life cycle of the object returned by the lock() method. This can effectively solve the coding problem caused by improper understanding of life cycle.
2. Static check of memory security. Researchers have implemented a static scanning tool to check the memory security problems after release. After scanning the rust project involved in the research, the tool found four new memory security problems that had not been found before. It shows that this static checking tool is necessary.
3. Static check for repeated locking. Researchers have implemented a static scanning tool to detect the problem of repeated locking by analyzing whether the variables returned by the lock () method are locked again in their life cycle. After scanning the rust project involved in the research, the tool found six new deadlock problems that had not been found before.

The paper also puts forward some suggestions on the application of dynamic detection and fuzzy testing.

conclusion

1. The safe code of rust language is very effective for checking space and time memory security problems. All memory security problems in stable versions are related to unsafe code.

2. Although memory security problems are related to unsafe code, a large number of problems are also related to safe code. Some of the problems are even caused by coding errors in safe code, rather than unsafe code.

3. Thread safety problems, whether blocking or non blocking, can occur in safe code, even if the code completely conforms to the rules of rust language.

4. A large number of problems are caused by the coder’s incorrect understanding of the life cycle rules of rust language.

5. It is necessary to establish a new defect detection tool for typical problems in rust language.

Click follow to learn about Huawei’s new cloud technology for the first time~

Recommended Today

Redis design and implementation 4: Dictionary Dict

In redis, the dictionary is the infrastructure. Redis database data, expiration time and hash type all take the dictionary as the underlying structure. Structure of dictionary Hashtable The implementation code of hash table is as follows:dict.h/dictht The dictionary of redis is implemented in the form of hash table. typedef struct dictht { //Hash table array, […]