Purpose of burial point
For this modification of the data statistical analysis program buried point, just for the statistical data in some difficult to find errors, all let the program take the initiative to run out. However, as long as the statistics are actively thrown out, they are all buried points.
Data burying point is the preparation work of data analysis, only burying point can obtain data for analysis. Only after doing enough data embedding points can we get the desired target data
- Behavior data: time, place, person, interaction, interaction content;
- Quality data: Browser loading, error exception, etc;
- Environment data: Browser related metadata, geography, operators, etc;
- Operation data: PV, UV, conversion rate, retention rate (very intuitive data)
After adjusting the specification of Python project as a whole, adding the error data buried point belongs to the quality data part.
Buried point scheme
Burying points are divided into front burying point and back burying point, and the common burying point schemes are as follows
- Front end code burying point: directly embed code in front end to collect target data
- Front end no buried point: common front end buried point technology. Collect all, select on demand
- Back end code burying point: directly embed code in the back end to collect target data
This time, we mainly focus on the back-end code burying of projects that do not use the third-party data analysis platform (applicable to the scenario of data statistical analysis monitoring program)
Rear burying pointThe following points must be taken into consideration in the design of the project
- High flexibility: it can be buried by hand anytime and anywhere
- Small impact: less intrusion into source code
- Updated: when the buried point content changes, you only need to modify the buried point interface, but not the code
In some public libraries or encapsulated custom frameworks, there is no burying point. All burying points are in the business layer. For example, when the number of connections overflows in the database connection pool, the error needs to be handled, and then the relevant information is thrown to the business layer according to the user-defined unified style, so that the business layer can judge whether there are errors in the execution of the public library and the user-defined framework.
Specific implementation scheme of back end buried point
Directly write the buried point code at the designated location of the back end to collect data. In each method, there are some differences in the area of sharing common information.
- Each file that needs to send a message will import a configuration information, and then manually bury the point according to the configuration information where necessary.
- Encapsulate the message sending into a mixin class, and mix it into each class with inheritance. Each time you need to establish a connection, send a message to disconnect.
- Encapsulate the buried point information into a module, provide global initialization function and buried point message generation function, call the initialization function of initialization module at the entrance of the program to initialize, and then generate message function anywhere in the program. This method is similar to using the third-party data analysis platform to provide the SDK.
- Because the project is written in Python, the buried point data is sent to Kafka, and the encapsulated third-party Python module can collect all the information it needs. Therefore, using the last method, written in the form of pluggable third-party SDK, can directly encapsulate some new methods, including some usable information, such as automatically collecting the number of rows of error information.
- For non Python projects, provide Python scripts for other language programs to call. For example, in the Erlang push round service, invoking Python script and throwing wrong information and other data.
- Shence data: bury deep, do not drop “pit”
- The next evolution of data analysis products: Based on buried point without buried point
- Practice of front end traceless burying point of meituan review
Remember to praise me!
We have carefully arranged video courses and e-books from entry-level, advanced and practical in all directions of computer. We can always find the learning materials you need according to the reasonable classification of the catalogue. What are you waiting for? Pay attention to download it!!!
If you don’t forget, there will be an echo. Please give me a compliment. Thank you very much.
I am a bright brother in the workplace, YY Senior Software Engineer, with four years of working experience. I refuse to be a leading slash programmer.
Listen to me, more progress, a shuttle of procedural life
If you are lucky enough to help you, please give me a “like” to pay attention to it. If you can comment on it and give me encouragement, I will be very grateful.
List of articles of workplace bright brother:More articles
All my articles and answers have cooperation with the copyright protection platform. The copyright belongs to brother Liang in the workplace. Without authorization, reprint must be investigated!