Tag:Checkpoint

  • How to deploy pytorch lightning model to production

    Time:2020-11-20

    A complete guide to the large scale service pytorch lightning model. Throughout the field of machine learning, one of the main trends is the proliferation of projects focusing on the application of software engineering principles to machine learning. For example,CortexIt reproduces the experience of deploying server free but reasoning pipeline. Similarly, DVC implements modern version […]

  • spark:checkpoint

    Time:2020-11-12

    Previously, I recorded the checkpoint mechanism of Flink in the process of learning Flink. Today, I will mainly record the checkpoint mechanism of spark and analyze the overall process with the source code. checkpoint As a fault-tolerant mechanism, checkpoint will be used in many scenarios. For example, in the previously recorded Flink, the checkpoint mechanism […]

  • Error 6003 when calling HMS SDK interface

    Time:2020-10-15

    For games or common applications integrated with Huawei HMS SDK, the calling interface (such as the getcurrentplayer login interface) always reports an error of 6003. The error codeOfficial documentsThe signature certificate fingerprint is mainly used to verify the authenticity of the application and ensure that the application comes from a trusted source. In view of […]

  • [pbft] practical Byzantine fault tolerance

    Time:2020-10-5

    1、 The necessity of view replacement   view replacement is a method to ensure the availability of the system due to the primary failure. Availability means that the operation can be completed within the effective time.checkpoint, stable checkpoint the states produced by the excution of these requests : checkpointa checkpoint with a proof : stable […]

  • Flink weekly weekly update – March 25, 2020

    Time:2020-10-4

    The content covers the answers to user’s questions in the mailing list, the progress of community development and proposals, community news and other activities, blog articles, etc., and we welcome your continuous attention Hello everyone, this is the tenth issue of Flink weekly, which is organized by ACE. The main contents include: recent community development […]

  • Chapter 5 spark streaming Programming Guide (2)

    Time:2020-10-3

    Chapter 4 spark streaming Programming Guide (1)The implementation mechanism of spark streaming, transformations and output operations, spark streaming data sources and spark streaming sinks are discussed. This paper will continue the previous part, mainly including the following contents: Stateful computation Window operation based on time Persistence Checkpoint Using dataframes & SQL to process stream data […]

  • Flink state best practices

    Time:2020-8-31

    This article mainly shares and exchanges some experience and experience in the process of using Flink status. Of course, the title is named “best practice”. I hope that the content of the article can bring some dry goods to the readers. The content of this paper is to review the related concepts of state, and […]

  • Optimization of MySQL 8.0 redo log system

    Time:2020-8-21

    background Now the mainstream database system fault recovery logic is based on the classic Aries protocol, that is, based on undo log + redo log. The redo log is a physical log, which generally adopts the wal (write ahead logging) mechanism, so it is also known as the wal log. The redo log records all […]

  • python GIL

    Time:2020-8-13

    Python does not support true multithreading. Python provides multithreaded packages, but if you want to speed up your code through multithreading, it’s not a good idea to use multithreaded packages. Python has a thing called global interpreter lock (GIL), which ensures that only one of your multiple threads is executed at any time. The execution […]

  • [20200401] confusion of optimization 5.txt

    Time:2020-7-15

    [20200401] confusion of optimization 5.txt –//Before the Spring Festival, optimize a production system and use ash after completion_ wait_ chains.sql After checking, it is found that there are a lot of control file parallel writes.–//At that time, I didn’t care. I always felt that there was something wrong with the disk IO of this system. […]

  • Learn about Flink state backends

    Time:2020-7-9

    When we use Flink for stream computing, we usually produce various forms of intermediate results, which we call state. If there is state generation, state storage must be involved. What forms of state storage are defined in Flink? Let’s introduce them one by one. State Backends MemoryStateBackend FsStateBackend RocksDBStateBackend MemoryStateBackend As the name implies, the […]

  • How to install Debian on mobile hard disk

    Time:2020-6-29

    Installing Debian’s grinding pump Microsoft instigating VMware strike In the past two days, I just want to make trouble. I dismantle my dear Ubuntu and install Debian But because of joiningMicrosoft’s insider version I was embarrassed to find that when I opened my VMware Workstation Pro 15 Look at the faceGreen screenTAT,Green screenYou know, orz […]