• Stepping.js ——Two steps to complete the front and back end separation architecture design


    One week ago, I participated in a workshop on “modeling and design” of the company. In a sense, this is a workshop on architecture design and software modeling. And then there was an inspiration Stepping.js 。 When we end the event storm, we need to take pictures or other means to record the relevant content. […]

  • Enhanced Excel to JSON tool


    Let excel support complex JSON format (array, object, object array, string, number, bool, etc.), and convert excel file to JSON.The original intention of writing tools was that when we were playing games, game planning used Excel to write forms. We converted the forms into JSON for use in games, but it was not convenient for […]

  • What did Flink do to conquer hungry engineers?


    Platform status The following is the current architecture of the platform: Data from multiple data sources are written to Kafka. The computing engines are storm, spark and Flink. The result data from the computing engine is then landed on various storage. At present, there are more than 100 storm tasks and about 50 spark tasks. […]

  • EMQ edge messaging server management suite EMQ x storm release


    Following the release of EMQ’s first edge computing product, EMQ x edge (EMQ x edge messaging server), on January 25, 2019, EMQ released a new product, EMQ x storm, which works together with EMQ x edge, on April 19. The product aims to build a convenient, efficient and stable centralized cloud management suite. This is […]

  • 58 group’s secret of smooth migration of large storm tasks to Flink


    Flink storm is a beta tool officially provided by Flink for Flink compatible storm program, and the relevant code is removed after release 1.8. This paper focuses on how to optimize the Flink storm of 58 real-time computing platform and realize the smooth migration of large-scale storm tasks in real scenes based on Flink storm. […]

  • The Secret of Smooth Migration of 58 Group Storm Tasks to Flink


    Flink-Storm is Flink’s official beta tool for Flink-compatible Storm programs, and the relevant code is removed after Release 1.8. This paper mainly describes how to optimize Flink-Storm for 58 real-time computing platform and how to smoothly migrate Flink for large-scale Storm tasks in real scenes based on Flink-Storm. background 58 Real-time computing platform is designed […]

  • Big Data Learning Route


    I. Large Data Processing Flow The figure above is a simplified flow chart of large data processing. The main flow chart of large data processing includes data collection, data storage, data processing and data application. Next, we will explain the technology stack for each link one by one. 1.1 Data collection The first step in […]