Tag：data processing
-
Time：2021-12-2
Big data contains great value and has attracted great attention from all walks of life. There are many sources of big data. The data collected from the real world are generally incomplete and inconsistent dirty data, which cannot be directly mined and analyzed, or the results of analysis and mining are not satisfactory. In order […]
-
Time：2021-11-28
preface Previously, I saw such a question as “is it necessary to separate the service layer in the yii2 framework?”. I also got the answer from others’ answers, but I think it still needs a living corn to have more clarity and strong persuasion. If you are interested in my actual combat experience, continue to […]
-
Time：2021-9-25
1. Preface Hello, I’m Ango! Python data processing. Up to now, I have written a total of 6 articles. Interested partners can learn about it! The most complete summary | talk about Python data processing (MySQL) The most complete summary | talk about Python data processing (SQLite) The most complete summary | talk about Python […]
-
Time：2021-9-19
Today is the first day to record my pit experience. It has been Java back-end development before, but after real work, it began to contact the big data processing aspect of Flink. There have been many small pits developed by Flink before, but they have not been recorded, so now I am ready to fully […]
-
Time：2021-9-9
Series articles： ThinkPHP 5 model usage history – inherit first and then extend ThinkPHP 5 model usage history – replication ThinkPHP 5 model usage history – global definition ThinkPHP 5 model usage history – ide friendly ThinkPHP 5 model usage history – Single Responsibility ThinkPHP 5 model usage history – sorting code ThinkPHP 5 model […]
-
Time：2021-8-16
With the continuous integration of big data and AI services, there are more and more business scenarios for big data processing of unstructured data (such as pictures, audio and text) through in-depth learning technology in the process of big data analysis and processing. This paper will introduce how spark works with the deep learning framework […]
-
Time：2021-8-16
Pytorch adopts the data saving method of dataset in data and needs to inherit the data.dataset class. If you need to process data yourself, you need to implement two basic methods. :. Getitem: returns a piece of data or a sample, obj [index] = obj. Getitem (index).:. Len: returns the number of samples. len(obj) = […]
-
Time：2021-8-14
Author: Dongge take offFirst appeared in the official account.Python Data Science aboutPandasThe methods to improve the running speed have been introduced many times before, which are often mentionedDask, many friends haven’t touched it and may not understand it. Today I recommend this artifact. 1. What is dask? PandasandNumpyEveryone is familiar with it. After the code […]
-
Time：2021-6-24
Development Notes: Statistics of today’s, 7 days’ and 30 days’ data by time period 1. Front end Vue usevue-highcharts <highcharts :options=”options”></highcharts> data() { return { options: { title: { text: ” }, xAxis: { categories: [] }, yAxis: { title: { text: ” }, plotLines: [{ value: 0, width: 1, color: ‘#808080’ }] }, legend: […]
-
Time：2021-5-14
problem In the process of dataframe data processing, sometimes the elements of a column are all lists or tensors, such as: import pandas as pd import numpy as np data_test = { “a”:[[1,2],[3,4],[5,6]], “b”:[4,5,6], “c”:[7,8,9] } df_test = pd.DataFrame(data_test) At this time, if the data is saved in the CSV format df_ test.to_ CSV (“. […]
-
Time：2021-4-29
Hadoop and spark are the mainstream big data frameworks at present, but with the advantages of spark in speed and ease of use, some domestic and foreign experts gradually praise spark technology, and think that spark is the future of big data. This paper will analyze the development process of Hadoop ecology and the technical […]
-
Time：2021-4-23
Hadoop and spark are the mainstream big data frameworks at present, but with the advantages of spark in speed and ease of use, some domestic and foreign experts gradually praise spark technology, and think that spark is the future of big data. This paper will analyze the development process of Hadoop ecology and the technical […]
