Jpg learning notes 2 (with complete code)

Time:2021-3-6

We have got the data that need to compress RGB from BMP. We need to transform the original data from RGB domain to YCbCr domain, and then down sample the YCbCr data. For students who don’t need to read the article, the source code is given here directly.https://github.com/Cheemion/JPEG_COMPRESS

Image reference “compressed image file formats, JPEG, PNG, GIF, XBM, BMP – John Miano” [1]

1. RGB domain and YCbCr domain

RGB stands for red, green and blue. We can get the color we see through the superposition of three colors. 0 – to 255 represent colors from light to dark.

Y   =  0.299   * red + 0.587  *  green + 0.114  *  blue;
Cb = -0.1687 * red – 0.3313 * green + 0.5    *    blue + 128;
Cr  =  0.5       * red – 0.4187 * green –  0.0813 * blue + 128;

Y is the weighted average of RGB, which is called luminance

CB is the difference between B component and brightness, which is called chromaticity (CB)

Cr is the difference between R component and brightness, which is called chrominance (CR)

The following code converts RGB to YCbCr. Why convert RGB to YCbCr? Because the human eye is more sensitive to the change of brightness (y), I can downsampling Cr and CB (compression, for example, one byte represents one pixel data, and one byte represents four pixels data after compression), so as to retain the complete Y component as much as possible. In this way, we can further compress the data.

void JPG::convertToYCbCr() {
    for(uint i = 0; i < height; i++) {
        for(uint j = 0; j < width; j++) {
            YCbCr temp = BMPData[i * width + j];
            BMPData[i * width + j].Y  =  0.299  * temp.red + 0.587 * temp.green  + 0.114  * temp.blue;
            BMPData[i * width + j].Cb = -0.1687 * temp.red - 0.3313 * temp.green + 0.5    * temp.blue + 128;
            BMPData[i * width + j].Cr =  0.5    * temp.red - 0.4187 * temp.green - 0.0813 * temp.blue + 128;
        }
    }
}

2. Sampling

Sampling is usually to sample the continuous signal. For example, in the figure below, blue is the continuous signal x (T), red is the signal obtained after sampling, x [n] = x (t * n), t is the sampling interval, and 1 / T is the sampling frequency.

In JPEG, we sample the discrete data, and the sampling value in JPEG is the relative sampling value. The sampling value relative to the highest sampling frequency.

As shown in the figure on the left

The horizontal sampling frequency (H) and vertical sampling frequency (V) of Y component are both 4, which is the highest sampling frequency. The highest sampling frequency is equivalent to retaining the Y component of the original image without down sampling.

The horizontal and vertical sampling frequency of CB component is 2, which is equal to half of the highest sampling frequency. So the horizontal sampling is once every two points, and the vertical sampling is once every two points.

The horizontal and vertical sampling frequency of Cr component is 1, which is equal to 1 / 4 of the highest sampling frequency. So one point is sampled every four points horizontally and vertically.

The sum of the three components yields the value of our pixel.

Image reference “compressed image file formats, JPEG, PNG, GIF, XBM, BMP – John Miano” [1]

2. The storage of YCbCr data in JPEG

JPEG stipulates that all data are changed and stored in the form of a 8 * 8 block (data unit). The 8 * 8 block can be regarded as the minimum storage unit.

MCU is the smallest unit composed of complete blocks of Y, CB and Cr, which can completely restore a range of colors. What do you mean by that?

Suppose our picture is 10 * 10 in size.

If the horizontal and vertical sampling frequencies of Y, CB and Cr are all 1, then the original image is composed of four MCU (each MCU contains a Y block, a CB block and a CR block, and the size of each MCU is 8 * 8). The edge blank can be replaced by 0, and the edge value can also be repeated.

The values of the 4 * 4 small block in the upper left corner are different

pixel[0,0] = y[0,0] + cb[0,0] + cr[0,0]

pixel[0,1] = y[0,1] + cb[0,1] + cr[0,1]

pixel[1,0] = y[1,0] + cb[1,0] + cr[1,0]

pixel[1,1] = y[1,1] + cb[1,1] + cr[1,1]

If the horizontal and vertical sampling frequency of Y is 2, and the sampling frequency of CB and Cr is 1, the original image is composed of one MCU (size is 16 * 16). MCU contains four y blocks (2 * 2), one CB and one CR. There are 6 blocks in total, and the size is only half of the original block.

The values of the 4 * 4 small block in the upper left corner are different

pixel[0,0] = y[0,0] + cb[0,0] + cr[0,0]

pixel[0,1] = y[0,1] + cb[0,0] + cr[0,0]

pixel[1,0] = y[1,0] + cb[0,0] + cr[0,0]

pixel[1,1] = y[1,1] + cb[0,0] + cr[0,0]

 

Conclusion: MCU size = vertical maximum sampling value * horizontal maximum sampling value, a MCU contains y blocks of Y horizontal sampling value * y vertical sampling value (if y horizontal sampling value is 2 and Y vertical sampling value is 2, then a MUC has 4 yblocks). The other components are the same

1.3 define JPG class code

//Define blockusing block = int [64]; // define YCbCr, and this structure is used to display RGB data
struct YCbCr {
    union
    {
        double Y;
        double red;
    };
    union 
    {
        double Cb;
        double green;
    };
    union {
        double Cr;
        double blue;
    };
};

 

struct MCU {
    Block* y;
    Block* cb;
    Block* cr;
};
//Define JPG class to compress pictures
class JPG
{
Public: // RGB go to YCbCr
    Void converttoycbcr(); // downsampling
    Void subsampling(); // change
    Void discretecosinetransform(); // quantization
    Void quantification(); // Huffman
    Void huffmancoding(); // output
    void output(std::string path);
Public: MCU * data; block * blocks; // bmpdata stores RGB data of BMP image
    YCbCr* BMPData;
    uint blockNum;

    //Pixels of the original image
    uint width;
    uint height;

    //How many is the length of MCU
    uint mcuWidth;
    uint mcuHeight;

    //The number of horizontal and vertical pixels of a complete muc
    uint mcuVerticalPixelNum;
    uint mcuHorizontalPixelNum;

    //For subsampling
    // only support 1 or 2
    byte YVerticalSamplingFrequency;
    byte YHorizontalSamplingFrequency;
    byte CbVerticalSamplingFrequency;
    byte CbHorizontalSamplingFrequency;
    byte CrVerticalSamplingFrequency;
    byte CrHorizontalSamplingFrequency;
    byte maxVerticalSamplingFrequency;
    byte maxHorizontalSamplingFrequency;
public:
    JPG(uint width, uint height,const RGB* const rgbs,
        byte YVerticalSamplingFrequency, byte YHorizontalSamplingFrequency, 
        byte CbVerticalSamplingFrequency, byte CbHorizontalSamplingFrequency,
        byte CrVerticalSamplingFrequency, byte CrHorizontalSamplingFrequency
       )
        :width(width), height(height),
         YVerticalSamplingFrequency(YVerticalSamplingFrequency), YHorizontalSamplingFrequency(YHorizontalSamplingFrequency),
         CbVerticalSamplingFrequency(CbVerticalSamplingFrequency), CbHorizontalSamplingFrequency(CbHorizontalSamplingFrequency),
         CrVerticalSamplingFrequency(CrVerticalSamplingFrequency), CrHorizontalSamplingFrequency(CrHorizontalSamplingFrequency)
        {   
            maxHorizontalSamplingFrequency = std::max({YHorizontalSamplingFrequency, CbHorizontalSamplingFrequency, CrHorizontalSamplingFrequency});
            maxVerticalSamplingFrequency = std::max({YVerticalSamplingFrequency, CbVerticalSamplingFrequency, CrVerticalSamplingFrequency});
            //The number of MCU
            mcuWidth = (width + (maxHorizontalSamplingFrequency * 8 - 1)) / (maxHorizontalSamplingFrequency * 8);       
            mcuHeight = (height + (maxVerticalSamplingFrequency * 8 - 1)) / (maxVerticalSamplingFrequency * 8);   
            
            mcuVerticalPixelNum = maxVerticalSamplingFrequency * 8;
            mcuHorizontalPixelNum = maxHorizontalSamplingFrequency * 8;
            //How many MCU in total    
            data = new MCU[mcuWidth * mcuHeight];
            //How many blocks does a MCU have
            blockNum = (YVerticalSamplingFrequency * YHorizontalSamplingFrequency + CbVerticalSamplingFrequency * CbHorizontalSamplingFrequency + CrHorizontalSamplingFrequency * CrVerticalSamplingFrequency);
           
            //Allocate block memory space
            blocks = new Block[mcuHeight * mcuHeight * blockNum];

            //Map memory to the structure for
            for (uint i = 0; i < mcuHeight; i++) {
                for (uint j = 0; j < mcuWidth; j++) {
                    data[i * mcuWidth + j].y = &blocks[(i * mcuWidth + j) * blockNum]; 
                    data[i * mcuWidth + j].cb = data[i * mcuWidth + j].y + YVerticalSamplingFrequency * YHorizontalSamplingFrequency;
                    data[i * mcuWidth + j].cr = data[i * mcuWidth + j].cb + CbVerticalSamplingFrequency * CbHorizontalSamplingFrequency;
                }
            }
            //BMP data is used to store the original image data of BMP
            Bmpdata = new YCbCr [width * height]; // temporarily store BMP data in bmpdata
            for(uint i = 0; i < height; i++) {
                for(uint j = 0; j < width; j++) {
                    BMPData[i * width + j].red = static_cast(rgbs[i * width + j].red);
                    BMPData[i * width + j].blue = static_cast(rgbs[i * width + j].blue);
                    BMPData[i * width + j].green = static_cast(rgbs[i * width + j].green);
                }
            }  
        }
    ~JPG() {
        delete[] data;
        delete[] blocks;
        delete[] BMPData;
    }

};

 

 

1.6 down sampling code

//Here, the point on the top left is taken as the subsampling point
//The average value can also be taken
void JPG::subsampling() {
    //Traversing MCU
    for (uint i = 0; i < mcuHeight; i++) {
        For (uint J = 0; J < mcuwidth; j + +) {// MCU
            MCU & currentmcu = data [I * mcuwidth + J]; // the starting coordinate point of each MCU
            uint heightOffset = i * maxVerticalSamplingFrequency * 8;
            uint widthOffset = j * maxHorizontalSamplingFrequency * 8;
            //Iterate over every component Y, CB, Cr
            for (uint componentID = 1; componentID <= 3; componentID++) {
                //Traverse the block and take the block from the muc
                for(uint ii = 0, yOffSet = heightOffset; ii < getVerticalSamplingFrequency(componentID); ii++, yOffSet = yOffSet + 8) {
                    For (uint JJ = 0, xoffset = widthoffset; JJ < gethorizontalsamplingfrequency (componentid); JJ + +, xoffset = xoffset + 8) {// get the specific block object
                        Block& currentBlock = currentMCU[componentID][ii * getHorizontalSamplingFrequency(componentID) + jj];
                        //Traverse the block every pixels, and sample and assign values
                        for(uint y = 0; y < 8; y++) {
                            For (uint x = 0; x < 8; X + +) {// get the coordinates of the sampled point
                                uint sampledY = yOffSet + y *  maxVerticalSamplingFrequency / getVerticalSamplingFrequency(componentID);
                                uint sampledX = xOffset + x * maxHorizontalSamplingFrequency / getHorizontalSamplingFrequency(componentID);
                                //cannot find in original pictures;
                                if(sampledX >= width || sampledY >= height) {
                                    currentBlock[y * 8 + x] = 0;
                                } else {
                                    currentBlock[y * 8 + x] = BMPData[sampledY * width + sampledX][componentID];
                                }
                            }
                        }
                    }
                }             
            }
        }
    }  
}

Complete codehttps://github.com/Cheemion/JPEG_COMPRESS/tree/main/Day2

end

  Thanks for reading.

  wish you have a good day.

>>>>Jpg learning notes 3 (with complete code)

reference material

[1]https://github.com/Cheemion/JPEG_COMPRESS/blob/main/resource/Compressed%20Image%20File%20Formats%20JPEG%2C%20PNG%2C%20GIF%2C%20XBM%2C%20BMP%20-%20John%20Miano.pdf