During the three years after graduation, how can I improve my single server architecture?

Time:2019-12-2

It’s almost 3 years since graduation. I don’t know how to play so many technologies suddenly. Net has become a PHP full stack engineer. Although it’s not in-depth, general software development can still cope. Sometimes when I have an interview in Guangzhou or in Shenzhen, I will be more resistant to written examination, because I haven’t read the terms they said. I always go directly to GitHub to review and refer to it. If I don’t understand it, I will go to Google.

As for how I started to build the architecture, I had to start from graduation.

When I graduated, I went to an advertising company to receive orders from pig Bajie, that is, an outsourcing company, for two months because of my incomplete confidence and experience. I feel that this is not the development direction I want, and then I left without saying goodbye, because I am young and don’t understand. Then he went to a company selling tea. At first, he blew the sharing to the sky. Then he visited the “micro three cloud” enterprise and bought a set of source code. Then teach me to maintain, I just graduated from the people, business do not know how to deal with, you ask me to maintain a listed company’s source code? , you’re teasing me, and then I don’t feel comfortable a month later, I quit. Then I went to a traditional financial industry without any developers. It’s more interesting here. What I learned in university can be played by myself, but there is a problem in traditional enterprises. They are eager for success. Two months later, they found that they have no income, and then they disbanded.

Then came the turning point of my life. It was also the mentor I met in my career. There were two people, one was my boss, the other was my manager. Before meeting the manager again, I worked with my boss in another enterprise, and then the company moved to a very large store. In that store, the company can be said to be the only technology company. At that time, the store manager came to visit. Later, we knew that the manager was born in Baidu. After the boss and technical director of the conflict, but also hit the company. Then it didn’t happen. The company was facing dissolution again. Then my boss and I joined our manager and wanted to make a “big business” with him. In this way, three of us have formed a team.

Because the boss is getting older and has a family, so many times, I have to take on more things. The front end is me, PHP, and the server building is me, hahaha. At that time, when dealing with business and sensitive business, my boss would not deal with it for me, so PHP was at the crud level. My boss gave me the job of selecting a technology line, so I played all the frameworks you mentioned, no matter angular.js, angular, react, Vue, Omi, ThinkPHP, laravel, yii2, mysql, node.js, egg.js,
Express. JS… And so on. This is what junior architects should do. Finally, according to the team’s situation, I chose laravel + (Vue or react) as my own technology stack, back-end or layui in the middle specification. I don’t need to ask why or what a jQuery can’t do? If there are two! Because the back end doesn’t know any front-end framework, but they will definitely jQuery.

You can browse my short book, and you will find that in 2017, the most popular blog I wrote was server building. Yes, I started to build the server architecture in one year. And then there’s a gogs

You can also run PHP environment at this stage. At the same time, you can start to analyze the requirements given by the manager and sort out the business with your boss. Then, the system scheme is very simple at the beginning. In addition to dump, there is no redis cache mechanism. The cache is handled by the local file mechanism.
In the first edition: the server environment is like this

All the code runs on one host. Because the business was not big at that time. I didn’t choose CentOS as the server system, but Ubuntu. Because of the support of Ubuntu’s software library, I think the cloud computing from other places will play a very important role.

Later, we need to use the cache mechanism, so our environment architecture also needs to be updated and supported. There will be various wonderful problems on windows. For example, the installation of PHP extension is the headache of windows. Now, windows 10 has a solution, but at that time we did not have such a condition. PHP image processing needs to be expanded, and everything needs to be expanded. At that time, I began to propose using virtual machine as our development environment, using VM virtual machine, using VM virtual sharing file mechanism, plus the mapping of hosts to virtual machine. After this treatment, our enterprise’s PHP environment guarantees one to one.

At this time, the development environment becomes

With my familiarity with Linux, and the gradual deepening of environment construction. 16 years can be said to be the year when Vue broke out. I have contacted the framework of Vue in 14 years. However, people at that time yearned for anguar.js. I know that Chinese people’s urination, Vue is a trend sooner or later. In 16 years, Vue broke out, and the ecology is perfect. At the same time, the emergence of small programs is understood. It is found that small programs are a bit like Vue, and there are similar Vue frameworks behind them. That’s when I realized that before and after The importance of separation. This is the time to build a real entry-level architecture.

At this time, it’s a kind of standard software development processing method. Later, MySQL database needs to be separated from reading and writing. How do ordinary people deal with it? It is also true to deal with the configuration of read-write separation, in the program to make some judgments!!! It means that when you write, you can read the database, and when you write the database, you can’t do that either. But later, your system will be very painful to decouple. Maybe you forget to write a nickname for that place sometimes. I chose to use middleware link pool to manage my database read-write separation.

At this time, I have started to know swoole. I accidentally found smproxy, a database link pool based on swoole, in open source China. Therefore, the following situations began to appear.

With the subsequent traffic, MySQL can’t keep up with the speed of reading, because you know the running speed of laravel. I won’t say much about it. At that time, I knew that laravel + swoole can improve the performance, but I had to go to the server to restart the service every time. At that time, swoole was still in version 1.9, and there were many pits, so in order to be conservative, it was in mysq L and API layer have done load balancing.
You see the following.

Why do I need to use keepalived to listen for MySQL written? If the MySQL write is suspended, once it is detected by keepalived, a read MySQL will be written to the database on the left. Make sure your system can continue.

Here, even if your computer is 16g running such a virtual machine, it will inevitably have some cards. Although docker has been used before, it was not familiar with Linux at that time. When setting up the environment, it would encounter various problems, and the dockerfile could not be written out. Until April of this year, I began to realize that I need docker, and the docker of 19 years has been very perfect, dock With the appearance of Er compose, you can simulate one click deployment of the environment. I first went to GitHub to find several docker-compose versions to see and absorb their essence, and now deploy their own docker-composer project environment.

The UI layer has always been maintained by me. I chose react as my technology stack, for nothing else, because it supports ts, writes code and is comfortable, and has the support of antd. But our UI layer is divided into Applet / express.js (PC rendering layer SEO) / mobile response. It’s good to say that the applet and react are static resources, so express is a bit troublesome, and the service needs to be restarted, so PM2 access is needed at this time. At the same time, the improvement of swoole enters into swoole4.

Because of the high performance of swoole, the previous load balancing can be left.

As the UI layer and swoole update the code every iteration, they need to do a tedious operation, package and compile.
So that’s when Jenkins came in.

Jenkins is simply an automated deployment tool. Now with docker, it’s easy to install..

Here, almost all software development can cope. Of course, this is just environment building. There are some business carding architectures that are not described in detail. If you can build the above server environment, then your project will not be possible, just stay in the simple code writing, because now is the era of data-driven industry, we need to analyze the user behavior!!!

Often our market says the most is PV UV, if it is you, what would you do?

If you haven’t contacted the code farm of nignx log, you will generally give two schemes:
First, use Webmaster Tools
Second, using Ajax + to build database model. Then use program + MySQL for statistical analysis.

The second is that in the advanced version, redis + scheduled tasks are often used to write to the database. If not, do not cluster your writes to the database? Wait for ha ha.

If you have ever contacted with nginx log, many PHP or node developers will be fooled in the first time when PHP faces this massive data. Yes, if you hear the massive data, your first reaction is hadopp. At this time, we need to jump out of the circle where we live for a long time (PHP + node). Why choose Hadoop instead of the second way What about it? Because Hadoop provides a lot of algorithms for dealing with massive data, and python + Hadoop data analysis was used when writing crawlers before. Do you still worry that you can’t analyze things?

So we get the following log structure

Wait for some user behavior of nginx.

With the above log analysis, we will find that sometimes there are peers or some people who scan our servers maliciously. I should need API gateway at this time.

My choice is Kong. In fact, you can use nginx, but you also know PHP / node birth, where to understand the underlying things… Unless you can go Lang.

Kong is managed through the admin API, not in the configuration like nginx. You don’t need to be stupid with management tools? Besides, Kong supports Lua plug-in mechanism, oauth2.0, black and white list, ACL, JWT, SSL, limited rate limiting and manageable rest API interaction.

So it’s changed

Later, your data volume will become larger and larger. At this time, you are still using MySQL full-text search index, which is a very unrealistic building.

At this time you have to think about using search engines. What foreigners use more isElastic SearchOf course, laravel is rightElastic SearchSupport. Unfortunately, this thing consumes too much resources. In the end, we used xunsearch. Although sometimes the search of this thing is not accurate, at least it can be used…. it is not easy to configure it without elastic search.

Here, search engines are used for search products. You need to synchronize the relationship between xunsearch and MySQL with queues or scheduled tasks.

In September 2019, this is an awkward time. Due to the decline of the real economy, the system is too stable. Big boss doesn’t want to develop the system. The technology department is facing disbandment, after my boss and manager confessed. Embark on the journey of recruitment again.

I don’t know if I can be qualified for the post of architect, because after reading the description, almost all of them have bachelor’s degree. I think I can do the front / PHP / full stack jobs.

I began to vote, but how to say. I’ve also brought people with me. Although I’m not very skilled in technology, I also have business processing ability. You can’t squeeze my salary because of my education and youth.

At the end of September, I decided to step out of Zhongshan. I went to two enterprises in Guangzhou for an interview. I feel that I can’t do what I used to do. If I want to be a returned yard farmer, I will have fewer and fewer things to play.

Finally, I don’t know why. I met an invitation from a start-up company.
So I went back to my previous work

Now the boss is quite successful in setting up the demand analysis business carding architecture. He can make the platform to the peak of 300wpv.

I can see that PV can still have 10V every day. However, there are many bugs in the previous system, and the system has not been developed by using the design pattern. It’s all heap code. I feel that I can’t take this hole when the technical director leaves.

Apply for refactoring the code, and sort out some unreasonable businesses again
Then the architecture changes to the following state

Starting from scratch again, poor programmer silly dream silly, now the problem is that when docker is deployed in different machines, it is difficult to start port mapping, so we need to learn k8s in the future, and MySQL used to use ORM more, but now in order to improve this convenient knowledge, occasionally write SQL API layer to reduce the number of queries.

Now I use Xiaomi based soar web for SQL.