• Translation of mit6.824 distributed system course & learning notes (1) Introduction & MapReduce


    explain This series of articles is the translation supplement and learning summary of the famous mit6.824 distributed system course, which is recorded while learning. If there is any omission, please correct:) Keep updating… Translation & Supplement content This course mainly involves application infrastructure, including: storage signal communication calculation Ultimate goal Hide the abstraction of distributed […]

  • Build a high availability architecture for micro services


    With the rapid development of microservice and cloud computing in recent years, the machine has gradually changed from physical machine to virtual machine, and the application service has gradually changed from a huge single application to an application cluster composed of several microservices. The speed of update iteration has doubled. The traditional deployment mode can […]

  • Virtual machine CentOS cluster construction


    When we build a redis cluster, a zookeeper cluster, a Hadoop cluster, or a kafaka cluster, we need to use multiple machines, so building a cluster server is a basic literacy of a back-end programmer. The goal of this paper is to prepare to build five Linux virtual machines 1. Software preparation CentOS-7-x86_64-Minimal-1611.iso、 2. Installation […]

  • 7 open source data science projects


    Author Pranav darCompile VKSource: analytics vidhya summary Open source data science project will add a lot of value to your resume and help you stand out in the interview Here are seven open source data science projects introduce I have a suggestion for you. I hope someone gave me this advice when I started my […]

  • Docker machine


    A series of articles: Installing docker in CentOS Docker machine How to create an image Dockerfile syntax Using docker to deploy a WordPress Publishing of docker image brief introduction Docker machine is a tool that allows you to install docker on a virtual host and use the docker machine command to manage the host. Docker […]

  • Top 5 programming languages in 2021


    introduce By 2020, there are about 700 programming languages available [1]. Some of them tend to apply only to specific areas, while others are widely appreciated for their ability to work in a variety of applications. In the past decade, the application of software has grown almost steadily, and new languages have been developed to […]

  • Spark standalone mode deployment case


    precondition: Environmental Science rhel 7.2 jdk-8u102-linux-x64 spark-2.0.2-bin-hadoop2.7 Scala 2.11, note: version 2.11. X is not compatible, see the official website:http://spark.apache.org/docs/latest/。 Prepare the master and worker extensions Server1 machine:, master Server2 machine:, worker Server3 machine:, worker Modify host [[email protected] ~]# vim /etc/hosts localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 […]

  • Distributed session consistency solution


    Consistency of distributed session In the early days, due to the small scale of users, many websites adopted the stand-alone deployment mode, using only one server to carry the user’s requests. At this time, the session exists on the same server, so it is easy to track and maintain the session. However, with the expansion […]

  • Discussion on distributed unique ID generation scheme from mongoid generation


    background Mongodb must have been used by everyone. After the data is loaded into the disk, when you query the data, you will find that it will automatically generate a_ “Id”, such as: db.test.insert({“name”:”tom”}) Query results: { “_id” : ObjectId(“5fd049327fbb28868f4660a5”), “name” : “tom” } Mongoid is used as the primary key index. Even in the […]

  • Installing NVM in Linux CentOS


    1. Install version management tool Git yum install git View git version git –version 1. Download NVM through Git git clone git://github.com/creationix/nvm.git ~/nvm 2. Join the system environment after downloading source ~/.bashrc 3. View NVM version list nvm list-romote 4. Install the required node version nvm install v11.0.0 5. View the installed version number of […]

  • Great! The AWS award helps China and the world to promote the scientific research of machine learning together!


    In recent years, the rapid development of artificial intelligence and big data is inseparable from the machine learning technology. However, it is still a long process to accelerate the development of machine learning technology. In order to accelerate the development of machine learning technology,In October 2017, Amazon Web services (AWS) set up the AWS Machine […]

  • Hadoop up and down datanode


    The HDFS file system in Hadoop can dynamically add and delete nodes, which is easier to add. After configuring various XML files of Hadoop on the new node, you can start it directly. To balance the balance of each datanode node in the cluster, you can execute the balancer command in HDFS. The usage is […]