Programming is a craft, and the experience of nginx community is shared

Time:2021-7-27

preface

About me: NJS core developer, unit contributor, nginx expert
Programming is a craft. It is a series. I share my thinking on programming, especially what I learned in nginx community.

Programming is the creation of logic and strives for clarity

This is my understanding of programming. Code reflects the essence of business, which can also be said to be an abstraction. Since it is logic, clarity is the most important.
Take Linus in Ted as an example: delete the elements in the list.
Bad logic: traverse the list and find the element entry. If the entry has no pre node prev, point the header to the next of the entry; If so, point the next of prev to the next of entry.
Programming is a craft, and the experience of nginx community is shared
Good logic: traverse the list, find the reference of the element entry, and point the reference to the next of the entry.
Programming is a craft, and the experience of nginx community is shared
Although both can achieve the goal, it is obvious that the latter is clearer and the code is more concise. Unclear logic often has more special processing, which is reflected in more if branches in the code. If you want to exercise better logic, try optimizing the if branch in the code.
A few years ago, when the author of nginx was still in the NJS project, he had more contact with him than now. NJS is a pure C JS engine designed and implemented by him. Its code has nothing to do with nginx. Both software have the implementation of red black tree. The version of NJS is the best red black tree implementation I’ve ever seen. I also mentioned a small improvement to him. I personally like the data structure of red black tree. The logic of NJS is clearer, if you look at its implementation. The advantage of this is that its API is easier to use. Compare the use of:

ngx_rbtree_init(&cache->sh->rbtree, &cache->sh->sentinel,
 ngx_http_file_cache_rbtree_insert_value);
ngx_rbtree_insert(&cache->sh->rbtree, &fcn->node);
njs_rbtree_init(&tree, rbtree_unit_test_comparison);
njs_rbtree_insert(&tree, &items[i].node);

The only difference is rbtree_ In init, the NJS version is simpler, and the nginx version requires a sentinel sub node called sentinel. The implementation logic of NJS is clearer and the special processing is optimized.
In short, the code should be logically clear. It can also be understood as a further abstraction.

Criteria for good code

@Drxp mentioned a method that the ratio of increasing the number of rows to deleting the number of rows is 4:1. This is very intuitive and effective, especially for stable projects. For new projects, this ratio can be increased to 3:1, because the design of new projects is often not considered enough and more refactoring actions are required. Take the current project NJS as an example.

xeioex#1xeioex
eight hundred and five   commits    Ignoring the data, a large number of files are renamed and reconstructed

igorsysoev#2igorsysoev
313 commits  58,714 ++  15,181 --

lexborisov#3lexborisov
124 commits  29,307 ++  11,163 --

hongzhidao#4hongzhidao
108 commits  12,864 ++  10,028 --

@Igor contributed the basic library of the whole software and the early JS syntax. The number of lines of code is about 4:1.
@Xeioex is the absolute core and realizes a large number of JS functions.
@Experts in lexborisov language analysis have made many improvements.
@Hongzhidao I was the third one to participate in, realizing some JS functions and reconstructing some basic libraries.
It can be seen that @ lexborisov and I have done more refactoring, and the increase and deletion are very high.
This method is one I am deeply impressed with, especially pointed out for reference only.

Refactoring is the most practical habit to improve ability

The following is an example of Igor’s reconfiguration on nginx. It’s like cleaning the room to make the room cleaner.
Programming is a craft, and the experience of nginx community is shared

Refactoring is a skill that makes the code more extensible through internal optimization without changing the original behavior.
Be sure to remember that you can’t change your original behavior. The purpose of refactoring is to make the code more extensible, which is to make it easier to add new functions. Refactoring can do many things. There are several common things from small to large:

  • Minor: correct the wrong words; Rename variables, functions, files, etc. to make the code more readable.
  • Logic Optimization: for example, Linus’s previous example makes the logic clearer.
  • Design improvement: for example, the implementation of red black tree from nginx to NJS.

There will be an in-depth study of refactoring, especially in my community practice.

Test cases, indispensable

Only test cases for code are discussed here. The following is the test case I wrote to implement the Lua version of radius tree.

function test_get()
 local tree = radix_tree.new();
 tree:insert('foo', 1);
 tree:insert('bar', 2);
 tree:insert('zoo', 3);
 
 assert(tree:get('bar') == 2, 'get bar');
 assert(tree:get('noop') == nil, 'get non');
end

NJS is better than nginx in that basic libraries such as red and black trees have corresponding test cases. I think this is very important and many projects have not done well. Once there are problems in these implementations, it will be a large-scale avalanche. Of course, many will write their own test cases in private, but they are not open source.
The refactoring mentioned above is impossible without test cases. Any changed code may have problems. How to ensure that there is no problem? Running test cases is the best way. In the NJS project, I and the other two often do some refactoring when fixing bugs and adding functions. Sometimes, if a bug is found on the same day, it is urgently repaired on the same day and then released. We dare to do such a thing because we have a large number of test cases. Test cases will also have a special space to continue to be introduced in depth.

Naming is a great knowledge

There are two difficulties in programming, naming and caching. Naming this seemingly simple thing is not easy to do well. This topic is placed in another article in this series.

Design, real kung fu

Good design can reduce the implementation cost of code. The most complex part of nginx code should be buf. For example, if the response is a streaming content, you want to filter it. That’s what the SSI module does.
C is different from other languages, almost all dealing with memory. Nginx has a memory pool, but it can only be allocated and released uniformly. You need to save the contents of the above examples. They are stored in places like buf in nginx, and the buf itself needs to occupy memory. Therefore, in order to reuse buf, nginx has made a very complex implementation. In contrast, NJS implements a dynamically allocated memory pool. Since memory can be dynamically allocated and recycled, there is no need for buf reuse. I estimate that if nginx has such an implementation, the code can be simplified by at least 30%. As I said before, it is a pity that nginx cannot introduce such improvements because of a large number of third-party modules. I refer to NJS many good designs in my nginx Lua module.
Therefore, good design can not only save a lot of implementation cost, but also make the software more robust and scalable. This often depends on the comprehensive ability of the implementer itself, the so-called talent.

How to really improve programming ability

I think professional programmers are reflected in two aspects: overall design ability and detail processing ability. When I first submitted the code to the community, more than 60% of it was changed and adjusted, and the overall design and details were not handled well. Now more than 90% will be directly incorporated without any change. So I think practice is the best way. It’s better to have an expert review for you. He will review your design and improve your details.
Practice is the best way, and then you have to find good code or good projects. Realize it yourself, then compare it, or find an expert to help you see it. Good code or projects do not need more, one is enough. Find what you are interested in, or use what I recommend.
Find the basic library to practice, which is the most simple and effective. Now many companies have to do questions first in the interview, and many people brush questions on leetcode. In fact, the essence is to assess your code ability.
For example, the following is my implementation of JS_ Functions required when importing syntax such as “.. /.. /. / AA / BB / cc /… / d.js”.

void
njs_file_dirname(const njs_str_t *path, njs_str_t *name)
{
 const u_char  *p, *end;
 if (path->length == 0) {
 goto current_dir;
 }
 p = path->start + path->length - 1;
 /* Stripping basename. */
 while (p >= path->start && *p != '/') { p--; }
 end = p + 1;
 if (end == path->start) {
 goto current_dir;
 }
 /* Stripping trailing slashes. */
 while (p >= path->start && *p == '/') { p--; }
 p++;
 if (p == path->start) {
 p = end;
 }
 name->start = path->start;
 name->length = p - path->start;
 return;
current_dir:
 *name = njs_str_value(".");
}

This example is very good. It involves naming, annotation granularity and API design. This is very suitable for practicing. The other is to study and write projects by yourself. Recommend Utopia, a 1000 line API gateway. Open source soon. I will always share good code for training, and welcome to the official account.

Programming is a craft, a series of shared programming

This article is the beginning. More programming skills such as naming, refactoring and testing are in their respective chapters.

[nginx-lua-module] https://github.com/hongzhidao…
[utopia] https://github.com/hongzhidao…(open source soon)
Programming is a craft, and the experience of nginx community is shared
More quality articles, official account

Recommended Today

Implementation example of go operation etcd

etcdIt is an open-source, distributed key value pair data storage system, which provides shared configuration, service registration and discovery. This paper mainly introduces the installation and use of etcd. Etcdetcd introduction etcdIt is an open source and highly available distributed key value storage system developed with go language, which can be used to configure sharing […]