Tag:The Conduit

  • Go uses the blocking mechanism of Chan to realize the start, blocking and return of the coprocessor


    1、 Usage scenarios The background is to read oplog from Kafka for incremental processing, but when I want to send a command to block the incremental process, and then start a full synchronization, I will continue the incremental processing. Therefore, it is necessary to control multiple coroutines. 2、 Using knowledge 1. Reading from an uninitialized […]

  • Channel: Notes on go expert programming


    Channel concept The way of communication between CO programs provided by go in language layer initialization Var Chan int // pipe declaration //The value is nil Ch1: = make (Chan int) // unbuffered pipe CH2: = make (Chan int, 2) // pipeline with buffer operation Operator The operator “< -” represents the data flow direction. […]

  • Shell: pipeline and redirection


    YesshellAnyone who knows something about it knows that pipeline and redirection are very useful IPC mechanisms in Linux. stayShellWe usually use the symbol “|” to represent the pipeline, and the symbols “>” and “<” to indicate redirection. So what are the real meanings (definitions) of pipes and redirections? Definition of pipeline A pipeline is a […]

  • Swote source code analysis — pipe pipe of basic module


    preface Pipes are interprocess communicationIPCThere are two types of pipes: named pipes and anonymous pipes. Anonymous pipes are specially used for processes with blood relationship to complete data transfer. Named pipes can be used between any two processes.swooleThe pipes in are anonymous. stayswooleIn, there are three different types of pipes, among whichswPipeBaseIt’s the most basic […]

  • Combing beanstalkd


    Beanstalkd A high performance, lightweight distributed memory queue characteristic Priority, delay, persistence, reservation, task retransmission Maintenance class Stats: current statusListtubes: current pipelineStatstube: View pipeline detailsUsetube: Specifies the pipe to useStatsjob: task detailsPeek: get the task by ID Production Putintube: encapsulatedPut: no encapsulationUsetube: Specifies the pipe to use Consumption Watch: set the monitoring pipelineIgnore: remove the […]

  • Building text classification model with spark NLP


    AuthorCompile | VKSource | analytics vidhya summary Spark NLP of John Snow laboratory is established on AWS electronic medical record, and the library is used for simple text classification of BBC articles. introduce Natural language processing is one of the important processes of global data science team. With the continuous growth of data, most organizations […]

  • Spark NLP text classification based on Bert and general sentence coding


    By veysel kocamanCompile | VKSource: toward Data Science Natural language processing (NLP) is a key component of many data science systems that must understand or reason text. Common use cases include text classification, question answering, interpretation or summary, sentiment analysis, natural language Bi, language modeling and disambiguation. NLP is becoming more and more important in […]

  • linux_ Introduction to xargs_ And is different from the for loop command


    Please listen to the question. There is a file with 20 URLs. Please write a command to request these URLs respectively. The contents of the file are as follows: file name( urls.txt )。 https://www.baidu.comhttps://www.google.comhttps://www.alibaba.comhttps://www.qq.comhttps://www.mi.com…. We just learned that every programmer should know something about curl, so the first solution we think of is to loop […]

  • How to use watch command with pipeline on Linux


    Command line Basics – Watch There are several log files in Linux system. Paying close attention to these log files may be one of the important tasks for Linux system administrators. You canusetailcommandEasily view the end of the log file. However, if you want to monitor the file all day, enter it every few minutestailCommand […]

  • Capacity understanding and example explanation of coroutine channel in droole


    First go to the official websitehttps://wiki.swoole.com/#/coroutine/channel。 About channel: channel, used for communication between coroutines, supporting multi producer and multi consumer collaborations. The lower layer automatically realizes the switch and scheduling of coprocess. Its construction method: swote / coroutine / Channel – >__ Construct (int $capacity = 1) has a capacity parameter. At first, I didn’t […]

  • How to use grep to find the string from the pipeline |


    cause Today, we have a problem and need to view the business log. The scenario is like this: each request will generate multiple lines of business log. The first line of log records the request parameter uid, and each line of business log will have a traceid. First, we need to find the business log […]

  • Neural network training of Leela zero with pytorch


    By Peter YuCompile | FlinSource: towards science Recently, I’ve been looking for ways to speed up my research and manage my experiments, especially around writing training pipelines and managing experiment profiles, and I’ve found these two new projects are pytorch lightning and hydra. Pytorch lightning can help you write training pipelines quickly, while hydra can […]