Five questions, three strategies, hand in hand teach you to customize app performance monitoring scheme

Time:2021-4-29

Introduction:To sum up, we can count the number of errors, error rate, number of affected users, proportion of affected users and other indicators of different types of errors in a certain time range. In the detailed classification of indicators, we can also define monitoring with different dimensions, such as version number.

Author: Youmeng + u-apm team

Why? Why should we do application performance monitoring?

First of all, we need to know what application performance monitoring means? And purpose:

Monitoring is a complete set of “monitoring + alarm” system. For app developers like us, application performance monitoring is the first hurdle to measure the app. If the quality of the application is not good, it will bring the most direct experience harm to users. After the app goes online, developers can’t get users’ use and experience in 7 * 24 real time, so they need a set of high-quality monitoring tools.

So, what indicators do we need to monitor?

There are many differences between Android and IOS in terms of client monitoring indicators. For example, Android needs JAVA, native, anr errors and so on, while IOS needs Objective-C, swift, C + + layer errors and so on.

In defining error indicators, the most basic is the number of different types of errors. If you consider the comparison between the number of errors and the overall application usage, you can consider using the ratio method. For example, you can define the error rate:

Five questions, three strategies, hand in hand teach you to customize app performance monitoring scheme

To sum up, we can count the number of errors, error rate, number of affected users, proportion of affected users and other indicators of different types of errors in a certain time range. In the detailed classification of indicators, we can also define monitoring with different dimensions, such as version number.

If we want to pay attention to the number of errors and the number of users affected by errors, we can calculate the number of users affected by errors according to the number of users.

How to define independent users? We can consider using the device ID to identify, such as IMEI, IDFA, Android ID, etc. if the information is difficult to obtain, we can also use the business user ID, such as login account, member name, etc. In addition, using the device identification definition id provided by the third-party SDK is also a good choice. After using this kind of ID, you can get the number of users affected by the error.

If we know the number of users affected by the error, but we are unable to determine the proportion of its impact area, we can look at the following indicator:

Five questions, three strategies, hand in hand teach you to customize app performance monitoring scheme

To sum up, we can count the number of errors, error rate, number of affected users, proportion of affected users and other indicators of different types of errors in a certain time range. In the detailed classification of indicators, we can also define monitoring with different dimensions, such as version number.

How? How to make your alarm plan flexibly?

Let’s ask you to do a quiz to judge your monitoring alarm type (a total of 5 questions, only 1.5 minutes)

The rules are as follows: 5 points for option a, 10 points for option B, 15 points for option C, and 20 points for option D

Q1: what stage is your product in?

A: It has been online and is in a relatively stable state, so the demand for monitoring alarm is low

B: In the development stage, we need to catch some errors in the test, and the demand for monitoring alarm is general

C: It has just been online, and it is relatively stable on the whole, with high demand for monitoring alarm

D: Just online, the effect is unknown, it needs 7 * 24-hour real-time attention, and the demand for monitoring alarm is very high

Q2: what is your position in your company / department?

A: Leaders, focus on the quality of application

B: Operation and maintenance personnel, the online problem supervisor responsible for monitoring the overall application performance

C: The tester is responsible for the quality control before the release of the application

D: Android / IOS client developers

Q3: how many people in your team are concerned about application performance quality and involved in it?

A: 1. The bare commander works by himself

B: 2-5 people, small development team

C: 6-25 people, cooperate with each other to optimize the application quality

D: 25 +, super large development team, not modestly speaking, is the industry leader

Q4: which application performance monitoring indicators do you pay attention to

A: The most basic error number is OK

B: Considering the range of users affected by the client, we need to monitor the number and proportion of users affected on the basis of the above

C: On the basis of the number of errors mentioned above and the impact on users, we should also consider the distribution of each version

D: Combined alarm rules need to be formulated: for example, when the number of errors is more than 100 and the error rate is more than 1%, or the number of users affected is 1% more than one day ago, the alarm will be triggered, and the version distribution should also be considered

Q5: do you have any requirements for fine setting of alarm notification mode?

A: No request, as long as you can receive it

B: There are some demands on time. I don’t want to be disturbed in the middle of the night

C: There are some requirements on the channel, such as email or specific office chat software

D: There are requirements for time and access channel

What? So how to set the alarm plan?

To sum up the above scores, please determine your total score (5 for option a, 10 for option B, 15 for option C, and 20 for option d) to see which monitoring alarm demand level your app is in: (data in which range? Which level is the monitoring alarm

Hot blood bronze (25-50 points): you are the primary stage user of monitoring alarm. You don’t need to check the occurrence status of various errors very carefully in your daily work. Maybe it’s because your application is still in the initial stage, or you’re in a high position. You don’t need to repair the alarm information yourself, just need to monitor the whole system. Please see scenario 1 below

Heroic gold (50-75 points): you are an intermediate stage user of monitoring alarm. You or your team already have the awareness of monitoring alarm, and will pay attention to the real-time application quality in daily work. You can set alarms with some fine rules. Please go to scheme 2

King of glory (75-100 points): you are a high-energy player in the field of monitoring and warning. With a little guidance, you can become a “super ace” in the field of monitoring and warning

According to the score of the above test, you can judge the difficulty of the alarm setting you need. It can be divided into the following schemes as a whole, from easy to difficult. If you want to learn the most comprehensive alarm setting function, please go to scheme 3

Scheme 1: simple – overall application quality control

As the primary alarm setting, you only need to consider two issues:

a. When should I receive the alarm?

b. How can I receive application alert messages?

To solve the first problem, you can consider the simplest state. As long as there is an error, I will receive an early warning. Then you can solve it by setting the condition that the number of errors > 0. If you feel that you are disturbed so much, you can set the alarm rule of error number > XX according to your own application

Five questions, three strategies, hand in hand teach you to customize app performance monitoring scheme

To solve the second problem, you need to have a medium that can receive messages. The simplest one is mailbox:

Five questions, three strategies, hand in hand teach you to customize app performance monitoring scheme

A simple monitoring alarm plan is set up like this

Scheme 2: advanced application quality monitoring

You can set different alarm messages for a single application, which can be distinguished according to the monitoring indicator type or version. For example, our requirement for the new version is that if the number of users affected is more than 10, an alarm will be triggered. The requirement for the old version is that the overall error rate should not increase by more than 5% compared with last week. Then we can set it as follows:

a. New version of alarm rules:

Five questions, three strategies, hand in hand teach you to customize app performance monitoring scheme

b. Old alarm rules:

Five questions, three strategies, hand in hand teach you to customize app performance monitoring scheme

In this scheme, we apply threshold type and contrast type alarm trigger conditions respectively. The definitions of these two rules are as follows;

Threshold rule

You can select an indicator (number of errors, error rate, number of users affected, percentage of users affected), and select “greater than” a certain value or a certain percentage

Contrast rule

You can select an indicator (number of errors, error rate, number of affected users, proportion of affected users), and select the time period of “compared” history to increase the proportion. The calculation method is: (past 1 hour value – Historical 1 hour value) / historical 1 hour value. If it is greater than or equal to the selected value, an alarm will be sent

Scheme 3: King type – combined index monitoring

You can skillfully set up the monitoring alarm, so through the following hints, I believe you can flexibly make your alarm plan according to your daily work needs

a. Flexible setting of alarm effective time:

Five questions, three strategies, hand in hand teach you to customize app performance monitoring scheme

You can add alarm effective time period, such as every Monday to Friday from 9:00 to 19:00, weekend from 12:00 to 20:00, flexibly set your working hours, not disturbed by invalid information

b. Key error type / single error alarm

You can choose the type of error that you need to focus on
Five questions, three strategies, hand in hand teach you to customize app performance monitoring scheme

Or directly for a fixed error continuous attention alarm

Five questions, three strategies, hand in hand teach you to customize app performance monitoring scheme

c. Combined alarm trigger conditions

Five questions, three strategies, hand in hand teach you to customize app performance monitoring scheme

You can flexibly set the alarm trigger conditions you want through a variety of indicators, threshold type or contrast type rules, in the combination of intersection / Union

d. Multiple alarm access channels

Five questions, three strategies, hand in hand teach you to customize app performance monitoring scheme

If you also have requirements for the touch channel of monitoring alarm, you can consider using the company’s office software for group touch, and work with other colleagues in your group to pay attention to and repair the application problems.

All the monitoring alarm setting functions mentioned in this scheme can be experienced by u-apm and alarm plan can be made in 2 minutes.
Original link
This article is the original content of Alibaba cloud and cannot be reproduced without permission.