How to solve the problem that when uiimage loads a large image, the memory is too large, resulting in cash?

Time:2022-1-11

In this article, we share with you the problems we face, what is the root cause of high memory utilization, and how we can use simple fixes recommended by Apple engineers to reduce the memory footprint of applications.

problem

As mentioned earlier, when we start loading HD images on the screen, the memory usage of our application will surge. To illustrate the problems we face, I created a sample application that loads high-definition images into the image view when a button is clicked.

Initial memory png

    @IBAction func loadImage(_ sender: Any) {
        image.image = UIImage(named: "image.JPG")
    }

Screenshot: 11.55.17 am, June 30, 2021 png

Memory after loading pictures png

Please note that I use size 6240   × 4160px image with a file size of 9mb.

As you can see, memory usage soared from 9mb to 155Mb after loading images.

This makes no sense! We are loading a 9mb image file. Why is the memory consumption so high? Where does all this memory use come from?

Memory usage ≠ file size

One method we use to solve this problem is to reduce the image and draw it on the screen according to the frame size of the image view. Unfortunately, this does not seem to solve our problem.

Without any clue about how to solve the problem, I turned to the Internet. After Google searched several times, I got an amazing WWDC video, which introduced a very important concept we don’t know:

Memory usage is related to the size of the image, not the file size.

In order to display the image on the screen, IOS first needs to decode and decompress the image. Usually, one pixel of the decoded image will occupy 4 bytes of memory

  • 1 byte for red,
  • 1 byte for green,
  • 1 byte for blue,
  • 1 byte for alpha component.

Our size is 6240   ×   example picture of 4160px:

(6240  ×   4160) * 4 bytes = 103833600 bytes  
Almost 103.8336 MB

This is really close to what we want to see in the memory report (there will be a deviation due to the format problem of the picture scene)

Downsampling to save

We have figured out the source of memory usage, but how can we reduce memory usage to an acceptable level?

As I mentioned earlier, we tried to shrink and redraw the image, but it didn’t seem to help. that is becauseUIImageResizing and resizing are expensive. During resizing, IOS still decodes and decompresses the original image, resulting in unnecessary memory peaks.

Down sampling using imageio

Fortunately, wwdc18 provides a good solution to the problems we face. We can use thisImageIOThe frame adjusts its size before the image is displayed on the screen. suchImageIOThe great thing about this method is that we can now resize the image by paying the cost of resizing the image. We call this method “down sampling”.

The following code snippet demonstrates how to perform down sampling on an image:

func downsample(imageAt imageURL: URL,
                to pointSize: CGSize,
                scale: CGFloat = UIScreen.main.scale) -> UIImage? {

    // Create an CGImageSource that represent an image
    let imageSourceOptions = [kCGImageSourceShouldCache: false] as CFDictionary
    guard let imageSource = CGImageSourceCreateWithURL(imageURL as CFURL, imageSourceOptions) else {
        return nil
    }
    
    // Calculate the desired dimension
    let maxDimensionInPixels = max(pointSize.width, pointSize.height) * scale
    
    // Perform downsampling
    let downsampleOptions = [
        kCGImageSourceCreateThumbnailFromImageAlways: true,
        kCGImageSourceShouldCacheImmediately: true,
        kCGImageSourceCreateThumbnailWithTransform: true,
        kCGImageSourceThumbnailMaxPixelSize: maxDimensionInPixels
    ] as CFDictionary
    guard let downsampledImage = CGImageSourceCreateThumbnailAtIndex(imageSource, 0, downsampleOptions) else {
        return nil
    }
    
    // Return the downsampled image as UIImage
    return UIImage(cgImage: downsampledImage)
}

Let’s first look at the parameters of the function:

  • Imageurl: image URL. It can be a web URL or a local image path.
  • Pointsize: the required size of the down sampled image. Typically, this will be the frame size of uiimageview.
  • Scale: down sampling scale factor. Typically, this will be a screen related scale factor (we usually call it @ 2x or @ 3x). This is why you can see that its default value is set to uiscreen main. scale.

Option flag

Next, I would like to draw your attention to the use of the function of option Flag 3-kCGImageSourceShouldCachekCGImageSourceShouldCacheImmediatelyandkCGImageSourceCreateThumbnailWithTransform

First, let’s talkkCGImageSourceShouldCache。 When this flag is set tofalseWe let the core graphics framework know that we only need to create a reference to the image source and don’t want toCGImageSourceDecodes the image immediately when the object is created.

Tip: if you cannot access the image source path, you canCGImageSourceuseCGImageSourceCreateWithData()The initializer creates an object.

Next in the list iskCGImageSourceShouldCacheImmediately。 This flag indicates that the core graphics framework should decode the image at the moment we start the down sampling process.

Therefore, by using bothkCGImageSourceShouldCacheandkCGImageSourceShouldCacheImmediatelyOption flag, we can completely control when to occupy CPU for image decoding.

FinallykCGImageSourceCreateThumbnailWithTransformOption flag. Set this flag totrueThis is important because it lets the core graphics framework know that you want the down sampled image to have the same orientation as the original image.

Let’s see the results

Now let’s try to load our sample image on the screen again, but this time down sampling is enabled.

let filePath = Bundle.main.url(forResource: "image", withExtension: "JPG")!
let downsampledLadyImage = downsample(imageAt: filePath, to: imageView.bounds.size)
imageView.image = downsampledLadyImage

Final memory png

You’ve seen the result of down sampling. It’s amazing. But that doesn’t mean you should start bundling HD images into your application and down sampling them when needed.

Remember that down sampling is a CPU intensive process. Therefore, it is better to use an appropriately compressed image source rather than down sampling the HD image.
In other words, you should consider using down sampling only if you need to display an image that is much larger than the required size on the screen.

This article’s demo GitHub address:https://github.com/stevendinggang/UIImageMemory

Reference:https://developer.apple.com/wwdc18/219