Use opencv and tensorflow to track the trajectory of volleyball


By Constantin toporov
Compile Flin
Source: towardsdatascience


After the experience of applying artificial intelligence to sports, I was inspired to move on. Family practice is not enough to be my goal. My goal is team competition.

Artificial intelligence in sports is a very new thing. Here are some interesting works:

I’m a volleyball fan. Let’s talk about the last website. This is the website of an Austrian Institute. He analyzes the local amateur league matches.

There are some documents to read, and more importantly, open the video dataset.

Volleyball is a complex sport with many different aspects. So I started with a small but important part – the ball.

Tracking the ball is a very famous task. Google provides many links, but many of them are just a simple demonstration. Obviously, the recognition and tracking of a large colored ball in front of the camera can not be compared with the real game ball detection, because the ball in the real world is very small, moving fast, and integrated into the background.

Finally, we want to get the following results:

Before we start, let’s pay attention to some details of the video dataset:

  • The camera is stationary and located behind the court

  • The technical level of players is not so high. We can see the ball freely (professionals hit the ball too hard, and it is almost impossible to see the ball without TV playback)

  • The color of the ball: blue and yellow, with little contrast with the floor. This makes all color based methods meaningless


By far the most obvious method – using color – doesn’t work. I use a moving ball.

Let’s find a moving object and recognize it as a ball. It sounds simple.

Opencv contains tools for detecting moving objects with background removal:

mask = backSub.apply(frame)     
mask = cv.dilate(mask, None)     
mask = cv.GaussianBlur(mask, (15, 15),0)     
ret,mask = cv.threshold(mask,0,255,cv.THRESH_BINARY | cv.THRESH_OTSU)

Put such a picture

Convert to:

In this example, the ball is on top and can be easily detected by the human brain and eyes. How did we decide? Some laws can be inferred from the figure:

  • The ball is ball shaped

  • This is the highest spot in the picture

The second rule doesn’t work. For example, in this photo, the highest spot is the referee’s shoulder.

However, the highest blob method provides further initial data.

We can collect these spots and train the classifier to distinguish the ball.

This dataset is as follows:

In artificial intelligence – this is a binary color image, very similar to the cats vs dogs challenge.

There are many implementation methods, but the most popular method is to use VGg neural network.

One problem – the picture of the ball is very small, so it is not suitable to use multiple convolution layers. So I have to simplify VGg into a very simple architecture:

model = Sequential([
Convolution2D(32,(3,3), activation='relu',       input_shape=input_shape),         MaxPooling2D(),            Convolution2D(64,(3,3), activation='relu'),         
Dense(64, activation='relu'),         
Dense(2, activation='softmax')       

model.compile(loss="categorical_crossentropy",   optimizer=SGD(lr=0.01), metrics=["accuracy"])

The model is simple and the results are general: about 20% false positive and 30% false negative.

It’s better than nothing. Of course, it’s not enough.

Models applied to games produce many “fake balls”:

There are actually two kinds of fake balls:

  • They appear at random locations in random time

  • This model always makes mistakes and regards other things as a ball


Next, there is an idea that the ball will not move randomly, but follow a parabolic or linear trajectory.

Verifying the motion of water droplets on this geometry will cut off random and consistent errors.

There is an example of recording the trajectory of a play:

The directed path is blue, the static path is green, and the random path is gray.

Only blue tracks are useful. They consist of at least three points and have one direction. Direction is very important because with direction, the next point can be predicted in case it is lost in the actual flow and no new path is detected.

This logic is applied to a real trace:


GitHub repository

Original link:

Welcome to panchuang AI blog:

Official Chinese document of sklearn machine learning:

Welcome to panchuang blog resources summary station: