Use Baidu brain and human body analysis technology to help you supervise and standardize fitness

Time:2020-2-24

[use strategy] [recognition of key points of human body]

1. Scheme introduction
Baidu launched human key point recognition, detected the human body in the image and returned to the human body rectangular frame position, accurately positioned 21 core key points, including the main joints of the head, facial features, neck and limbs, supporting complex scenes such as multi-person detection and large movements. It supports multi body detection, detects all human bodies in the image, and marks the coordinate position of each human body; it does not limit the number of human bodies, and adapts to light occlusion and truncation of human body.

According to the information of the key points of the human body, the function characteristics of the recognition of the key points of the human body can be applied to the physical fitness. According to the information of the key points of the human body, the body posture, the movement track, the movement angle, etc. can be analyzed to assist the athletes in physical training, analyze the effect of the fitness exercise, and improve the teaching efficiency. In these applications, the determination of human body posture is a key. Through the determination of posture, we can judge whether the movements of athletes or sports lovers meet the requirements. For gymnastics, yoga and other sports activities with high requirements for posture, even the sense of body games are very helpful.

2. Scheme idea and Implementation
Through the key points of human body, this strategy calculates 8 included angles of left and right arm, left and right shoulder, left and right leg and left and right hip, and then compares with the required standard included angle and allowable error range. So as to realize the judgment and scoring of human posture. The function I developed provides not only attitude scoring, but also rendering and movement analysis based on the original image, which is convenient for athletes to analyze. The main contents include the following aspects:

2.1 platform access
The specific access method is relatively simple. You can refer to another post of mine, which will not be repeated here:
http://ai.baidu.com/forum/topic/show/943327

2.2 Baidu human key function call:
Please refer to another article of mine: the fourth issue [Baidu brain new experience] human body key point recognition
https://ai.baidu.com/forum/topic/show/954123

2.3 other aspects:
Included angle calculation: calculated by cosine.
Key point filtering: key points with different confidence are displayed in different colors on the rendering and analysis charts, with green for those with more than 0.9 points, yellow for those with 0.5-0.9 points, and red for those with less than 0.5 points. Convenient for user identification.
Rendering: rendering on the original image, showing the key points and main lines, including the body, limbs, head and face, adding a central axis from the top of the head to the nose to the neck.
Analysis chart: show the key points and main lines on the new picture, including the body, limbs, head and face, add a central axis from the top of the head to the nose to the neck, so as to facilitate the user to view the direction of the face. At the same time, display the included angle of each position and whether it meets the requirements. If it meets the requirements, display green, if it does not meet the requirements, display red. It is convenient for athletes to analyze.

Judgment score: 100 * number of angles passed / total number of angles recognized

3 implementation effect and test:

Case 1: a classic gymnastics saddle horse movement:

The four limbs should be straight, the right leg should be straight with the body, and the left leg should be about 75 degrees. The following are the specific angle requirements:

required_pose1={
‘shoulder_left_angle’:[15,10],
‘shoulder_right_angle’:[60,10],
‘hip_left_angle’:[180,10],
‘hip_right_angle’:[75,10],

‘arm_left_angle’:[180,10],
‘arm_right_angle’:[180,10],
‘leg_left_angle’:[180,10],
‘leg_right_angle’:[180,10]
}

The first value of each group above is the required angle, and the second is the allowable error space.

The following is the rendering, analysis and report:

 

Processing time: 0.71 seconds
person_num: 1
Left shoulder: 13.0, in accordance with
Right shoulder: 57.5, in accordance with
Left hip: 175.6, in accordance with
Right hip: 71.6, in accordance with
Left arm: 171.2, in accordance with
Right arm: 173.1, in accordance with
Left leg: 171.9, in accordance with
Right leg: 178.8, in accordance with
Action score: 100.0

 

Case 2: multiple hands opening action:

Action template, hands open:

required_pose2={
‘shoulder_left_angle’:[90,10],
‘shoulder_right_angle’:[90,10],
‘hip_left_angle’:[170,10],
‘hip_right_angle’:[170,10],

‘arm_left_angle’:[180,10],
‘arm_right_angle’:[180,10],
‘leg_left_angle’:[180,10],
‘leg_right_angle’:[180,10]
}

Rendering:

Analysis chart:

Presentation:

Processing time: 4.00 seconds
person_num: 2
Left shoulder: 93.6, in accordance with
Right shoulder: 99.5, in accordance with
Left hip: 165.8, in accordance with
Right hip: 171.7, in accordance with
Left arm: 171.9, in accordance with
Right arm: 180.0, in accordance with
Left leg: 174.5, in accordance with
Right leg: 175.2, in accordance with
Action score: 100.0

Left shoulder: 116.6, inconsistent
Right shoulder: 108.4, inconsistent
Left hip: 156.8, inconsistent
Right hip: 164.7, in accordance with
Left arm: 175.2, in accordance with
Right arm: 180.0, in accordance with
Left leg: Nan, discrepancy
Right leg: 175.2, in accordance with
Action score: 50.0

Case 3, Yoga:

Action template:

required_pose3={
‘shoulder_left_angle’:[180,10],
‘shoulder_right_angle’:[80,10],
‘hip_left_angle’:[110,10],
‘hip_right_angle’:[80,10],

‘arm_left_angle’:[45,10],
‘arm_right_angle’:[45,10],
‘leg_left_angle’:[30,10],
‘leg_right_angle’:[55,10]
}

Rendering:

Analysis chart:

Presentation:

Processing time: 0.82 seconds
person_num: 1
Left shoulder: 172.1, in accordance with
Right shoulder: 79.0, in accordance with
Left hip: 108.1, in accordance with
Right hip: 81.9, in accordance with
Left arm: 37.4, in accordance with
Right arm: 36.9, in accordance with
Left leg: 9.0, inconsistent
Right leg: 48.2, in accordance with
Action score: 87.5

4 key code

import numpy as np
Seeking angle
def GetCrossAngle(l1p1,l1p2,l2p1,l2p2):
    arr_0 = np.array([(l1p2['x'] - l1p1['x']), (l1p2['y'] - l1p1['y'])])
    arr_1 = np.array([(l2p2['x'] - l2p1['x']), (l2p2['y'] - l2p1['y'])])
    cos_value = (float(arr_0.dot(arr_1)) / (np.sqrt(arr_0.dot(arr_0)) * np.sqrt(arr_1.dot(arr_1))))   
    result = 180-(np.arccos(cos_value) * (180/np.pi))
    result = round(result,1)
    return result

#Judge whether the angle meets the requirements
def angle_if(d,beginkey,middlekey,endkey,draw,color,setFont,text='',required=None):
    total=0
    score=0
    if ((beginkey in d)&(endkey in d)&(middlekey in d)):
        total=1
        angle=GetCrossAngle(d[beginkey],d[middlekey],d[middlekey],d[endkey])
        result=text+str(angle)
        if required:
            #print(str(required[0]))
            if abs(angle-required[0])

5. Test results and comparison with competitive products

The overall test found that Baidu human body key point recognition has a strong recognition ability for human body key points. On this basis, the human body posture judgment scoring function constructed has a good effect and a fast speed. It can be widely used in physical fitness and entertainment interaction.

At present, in addition to Baidu, which makes the key products of human body, there are also commercial soup and face + +, and a simple comparison is also made here.

FACE++

The function name corresponding to face + + is: human bone key point detection API. The effect is to detect the human body and the key points of the skeleton in the incoming picture, locate and return the 14 key points (head, neck, shoulder, elbow, hand, hip, knee, foot) of the human body. Support skeleton detection of all human bodies in the picture

The specific points are as follows:

The analysis effect is shown in the figure:

Shangtang Technology:

It supports 14 key points positioning of human body, and provides the positions of human head, shoulder, waist and leg in super real time through RGB image input. The analysis effect is as shown in the figure:

Through comparison, we can find that, in terms of function, face + + and Shangtang technology are detecting 14 points, and Baidu is 21 points. Baidu’s many points are mainly on the face, which is very helpful to determine the face orientation and facial expression (you can compare the rendering map, Baidu can render the face orientation, the other two can’t). This is very useful for some application scenarios. In terms of cost performance, baidu now has 50000 times / day of free volume adjustment, which is enough for testing and small applications. The price of commercial application is also very low, so Baidu has a great advantage in cost performance.

 

6. Optimization suggestions

Just as the hand has key point recognition and gesture recognition, it is recommended that Baidu directly establish a new function of human body gesture recognition, directly returning the results of human body gesture recognition. More content can be added on the basis of included angle. For users with this need, it is more convenient to use.

Author: I waste 99