Best practice of building real-time game data analysis based on Flink + Clickhouse

Time:2021-9-18

Introduction:This practice introduces how to quickly collect massive user behavior data, realize real-time user behavior analysis of second response, deeply mine and analyze through real-time stream computing, cloud database Clickhouse and other technologies, obtain user characteristics and portraits, and realize personalized system recommendation services.

Direct best practices:【Best practice of building real-time game data analysis based on Flink + Clickhouse
Best practices channel:【Best practices channel
There are rich enterprise cloud best practices. Starting from typical scenarios, we provide a series of project practice schemes to reduce the threshold of Enterprise Cloud and meet your needs at the same time!

Scene description

In the Internet and game industries, it is often necessary to analyze user behavior logs to better support business operations through data mining, such as user trajectory, heat map, login behavior analysis, real-time business screen, etc. When the volume of business data reaches 100 billion, it often leads to non real-time analysis, with an average response time of 10 minutes, affecting the normal operation and development of the business.

Program advantages

  • The original Presto data warehouse is replaced by the cloud database Clickhouse, which improves the performance of Presto by 20 times compared with the open source presto.
  • Using the extreme analysis performance of the cloud database Clickhouse, 100 billion level data analysis is shortened from 10 minutes to 30 seconds.
  • The cloud database Clickhouse has high batch writing efficiency and supports the writing of 23 billion user data per hour at business peak.
  • The cloud database Clickhouse is out of the box, free of operation and maintenance, deployed in multiple regions around the world, and quickly supports the opening of new games.

Product list

  • VPC
  • EIP of EIP
  • ECS
  • Message queuing Kafka version
  • Cloud database Clickhouse
  • Real time computing Flink Edition
  • Quick BI data visualization analysis platform

Business architecture

Best practice of building real-time game data analysis based on Flink + Clickhouse

Best practices

Copyright notice:The content of this article is spontaneously contributed by Alibaba cloud real name registered users, and the copyright belongs to the original author. Alibaba cloud developer community does not own its copyright or bear corresponding legal liabilities. Please refer to Alibaba cloud developer community user service agreement and Alibaba cloud developer community intellectual property protection guidelines for specific rules. If you find any content suspected of plagiarism in the community, fill in the infringement complaint form to report. Once verified, the community will immediately delete the content suspected of infringement.