• Week 26, 2018 – anatomy MapReduce job


    Hadoop architecture Preview Apache HadoopIs an open source software framework for large-scale storage and computation of datasets on cheap hardware. Here are five modules that make up Hadoop. clusterIs a collection of hosts (callednodes) Nodes can be subdivided intoracks。 These are the hardware parts of the infrastructure.YARN Infrastructure (Yet Another Resource Negotiator)Is a framework that […]

  • Four stages of MapReduce


    1. Split phase: In this stage, each input file is input to map in pieces. If a file has 200m, it will be divided into two pieces by default, because the default maximum value of each piece is the same as 128M of each piece. If the input is a large number of small files, […]

  • Learning records of mongodb master class (day 18)


    Eighteenth days The chapter to be learned today is “21 transaction development: read operation transaction II”. Continue with yesterday’s topic. Yesterday, we talked about where to read the readpreference, and today, we talked about what kind of data readconcert to read. What is readconcert? After the specified node is selected by readpreference, readconcert determines which […]

  • Learning records of mongodb master class (day 19)


    Nineteenth days The chapter to be learned today is 22 transaction development: multi document transaction, which mainly deals with multi document transaction management. Explain Mongodb started to fully support multi document transactions in 4.2 The principle of using transactions should be: don’t use them as much as possible. By properly designing the document model, we […]

  • Optimization Practice of tens of times performance improvement of oppo million level high concurrency mongodb cluster (2)


    This article is from oppo Internet technology team. Please note the author for reprint. At the same time, welcome to pay attention to the public number of OPPO Internet technology team: OPPO_tech, share with you OPPO frontier Internet technology and activities. Previously: not long ago, oppo Internet technology was launchedOptimization Practice of tens of times […]

  • Data inconsistency caused by data migration to mongodb and Solutions


    Story background Present situation of enterprises At the beginning of 2019, I received a mysterious phone call, which actually accurately said my nickname: Shanghai xiaopang. I don’t think it’s easy. I’m xiaopang. How do you do? “I just added your wechat XXX” Oh… He just reported my wechat nickname With in-depth communication, it is understood […]

  • With all due respect, mongodb may only be 30%


    Mongodb’s debut Self introduction. Mongodb is a database based on distributed file storage. Written in C + +. It aims to provide scalable high-performance data storage solutions for web applications. Mongodb is a product between relational database and non relational database. It has the most abundant functions and is the most like relational database. The […]

  • What are thread scheduler and time slicing?


    Thread scheduler is an operating system service that allocates CPU time to threads in runnable state.Once we create a thread and start it, its execution depends on the implementation of the thread scheduler. In the same problem, thread scheduling is not controlled by Java virtual machine, so it is better to control it by application […]

  • Keping, a quark chain: how to create the ultimate compatibility public chain


    Good afternoon, everyone. Today, “linkmaker talk” invited “Keping”, the R & D director of “quark chain” in Beijing, to the scene of linkmaker. Welcome to Mr. Ke. Before the activity, I’d like to introduce today’s activity flow: The first part is the question and answer session of host and guest; The second part is free […]

  • The second part of sub database and sub table


    The second part of sub database and sub table 2. Quick start to sharding JDBC 2.1 requirement description 2.2. Environmental construction 2.2.1 environmental description 2.2.2 create database 2.2.3 dating Maven dependency 2.3 preparation procedure 2.3.1 fragment rule configuration 2.3.2 data operation 2.3.3 test 2.4. Process analysis 2.5 other integration methods   2. Quick start to […]

  • Database middleware dble learning (I) basic introduction and fast building


    Introduction to dble basic architecture Dble isShanghai aikesheng Information Technology Co., LtdBe based onmysqlHigh availability extensible distributed middleware. Jianghu people give nicknamesMyCat Plus。 Open source address Let’s first look at the architecture diagram. External applications connect through NiO / AIO. First of all, we need to introduce the concept of NiO / AIO. Bio, the […]

  • Section 3 of sub database and sub table


    Section 3 of sub database and sub table 3. Sharding JDBC execution principle 3.1 basic concepts 3.2. SQL parsing 3.3. SQL routing 3.4. SQL rewrite 3.6. Merging of results 3.7 summary   3. Sharding JDBC execution principle 3.1 basic concepts Before you know how sharding JDBC works, you need to understand the following concepts:Logic tableThe […]