Tag：Mantissa

Time：20201019
Read some IEEE 754 implementation of floatingpoint arithmetic related articles IEEE 754 (IEEE 7542019) Floatingpoint arithmetic Significand JavaScript floating point number trap and solution Basic field: on floating point numbers In depth analysis of floating point numbers What is the difference between quiet NaN and signaling NaN? Why is the most secure integer in JavaScript […]

Time：2020106
Since the release of Java, the basic data types have been part of the Java languagebyte, short, int, long, char, float, double, boolean. Among them: Integer: byte, short, int, long Character type: char Float, double Boolean: Boolean Before we introduce it in detail, let’s talk about it first. The smallest calculation unit in Java isbyte, […]

Time：2020828
1: The size of several basic data types in Java keyword type Bits (8 bits, 1 byte) Value range (indicated range) byte integer 8 2^7 ~ 2^71 short integer 16 2^15 ~ 2^151 int integer 32 2^31 ~ 2^311 long integer 64 2^63 ~ 2^631 float Floating point number 32 3.402823e+38 ~ 1.401298e45 double Floating […]

Time：2020521
Come on, sit down and talk Previously, in the previous article “the boss said: who wants to use double to define the amount of goods, just pack up the things and go”, he had a heartbreaking chat: Dealing with things likeOrder transaction、Currency calculation, andCommodity amountBe careful with floating point numbers（double/float）To define variables, otherwise you may […]

Time：2020425
First look at the output of the following calculation: 0.1 + 0.2 Obviously 0.3. But in JavaScript, what is the result? 0.30000000000000004 This is an accuracy problem that is easy to appear in the numerical calculation of the program language, as shown in the figure below, the amount on the bill page is displayed. Causes of problems […]

Time：2020414
[20191013] Oracle number type storage conversion script.txt –//Test to see if you can use the output of BC obase = 100 to solve the problem. In addition, before the script forgot to consider the rounding of mantissa.–//Maybe programming is like this. There are always some details that are not taken into account–//The code is num2raw_5.sh […]