Tag：Structured
-
Time：2021-9-16
1、 Introduction to spark SQL Spark SQL is a module used by spark to process structured data. It provides a programming abstraction called dataframe and acts as a distributed SQL query engine. Why learn spark SQL? We have learned about hive, which converts hive SQL into MapReduce and then submits it to the cluster for […]
-
Time：2021-9-14
Jsonconsole in asp.net core 5.0 Intro New log in asp.net core 5.0JsonConsole, or output log toConsole, but it will be appliedJsonOne of the formatsFormatterFormat the log into JSON and output it to the console Sample Let’s take a look at an example. Take a small project of mine as an example to demonstrate that the […]
-
Time：2021-8-17
Zhengcai cloud front end tabloid No. 87 For more past tabloids, please visit:https://weekly.zoo.team Understand what VLAN, layer 3 switch, gateway, DNS, subnet mask and MAC address are The most understandable guide to VLAN, layer 3 switch, gateway and other knowledge in history. Understand typescript generics and Applications I think typescript generics are a little difficult. […]
-
Time：2021-8-11
follow“Full stack of Java backend Technology”** Reply to “interview” to obtain a full set of interview materials What is big data? The most famous answer is “5V characteristics”: 1) Volume: large amount of data, including collection, storage and calculation. The starting measurement unit of big data is at least p (1000 t), e (1 million […]
-
Time：2021-7-29
Structured log, as the name suggests, is no longer a free-form log, but follows a certain structure: each line of log is a JSON structure. The benefits are obvious: simplify log parsing and make the subsequent processing, analysis or query of logs convenient and efficient. In contrast to the structured log, the traditional log is […]
-
Time：2021-7-15
Introduction:Data is the core asset driving business innovation. Different types of data, such as unstructured data (videos, pictures, etc.) and structured data (orders, tracks), need to choose the appropriate storage engine for different business requirements, which can really play the value of data. For massive structured / semi-structured data of massive non strong transactions, table […]
-
Time：2021-7-2
Translation link:https://www.infoworld.com/art… Logging is a basic function of an application. Logging is especially useful for finding, checking and debugging bugs,SerilogIs a third-party open source class library, it can be structured log output to the console, files and a number of other categories of data storage. This article mainly discusses how to write structured logs to […]
-
Time：2021-6-23
Introduction:Best practices, taking DLA as an example. DLA is committed to helping customers build a low-cost, easy-to-use and flexible data platform, saving at least 50% of the cost compared with traditional Hadoop. Among them, DLA meta supports a unified view of 15 + data sources (OSS, HDFS, DB, DW) on the cloud, introduces multi tenant, […]
-
Time：2021-6-2
One of the functions of logging is to facilitate the tracking and debugging of applications, which is introduced in practical applicationsLog frameworkBut if yourlog fileIf it contains unstructured data, it will be a nightmare to query, so we need to use structured method when recording logs. Structured logs can be more easily queried and analyzed […]
-
Time：2021-5-10
Author Bai Xueyu This paper is shared by Bai Xueyu, the research and Development Engineer of big data platform of Zhongyuan bank. It mainly introduces the application of real-time financial data Lake in Zhongyuan bank. The main contents include: 1. Background 2. Real time financial data Lake architecture 3. Scene practice 1、 Background First of […]
-
Time：2021-3-16
In most cases, we can complete the database calculation with SQL (stored procedure), but if we encounter some complex operations that SQL is not good at, we can only read the data out of the database with other programming languages, and then complete the calculation outside the database. Such programming languages often appear in the […]
-
Time：2021-3-12
summary If you’ve ever used Uber, you’ll notice how easy it is to operate. You call a car with one click, then the car comes to you, and finally the payment is automatically completed, and the whole process is flowing. However, behind this simple process, it is actually supported by a complex basic big data […]
