How to optimize today’s headline IOS installation package

Time:2021-5-10

preface

Today’s headlines: IOS has focused on the problem of installation package size since 2016, and started the optimization of package size. In 2017, we published our experience at that time as a technical article “dry goods – today’s headlines IOS installation package size optimization – ideas and practice” [1].

Now three years have passed. Today’s headlines have practiced more ideas when they continue to explore the optimization of package size, including building configuration, image compression__ Text segment migration, binary segment compression, etc. These optimization items bring significant package size benefits to today’s headlines under the premise of less business intrusion. At the same time, the whole industry has also produced more solutions in packet size optimization. Therefore, we update the article and look forward to sharing with you about packet size optimization.

Table: list of optimization items and revenue of today’s headlines

1、 Composition of installation package

When we get an IPA file after app slicing by building, we can unzip it by using the zip decompression method, and after entering the. App file, we can see the contents of the installation package intuitively.

An installation package often contains resources and the executable file mach-o file on IOS. Resources can be divided into assets.car file and other resources. Among them, assets.car and mach-o are the parts that we put more efforts into optimization.

1.1, assets.car file

Assets.car file is the product of asset catalog construction in engineering. Actool in Xcode tool chain is responsible for building assets. Car. In the process of building assets. Car, actool will select the encoding algorithm according to a certain strategy and re encode the PNG image.

Figure: asset catalog

1.2. Mach-o file

Mach-o file is an executable file on IOS, which is obtained by compiling and static linking the code source file. Mach-o files after app slicing often contain only a single schema. Using tools such as machoview, we can intuitively understand the contents contained in mach-o.

At the same time, link map file can further help us analyze the composition of mach-o file.

Open in build settingsLD_GENERATE_MAP_FILESwitch. In the process of building the app, a TXT file called link map will be generated, which can show the distribution and size of each segment, section and function in mach-o. This information is often used in packet size optimization.

2、 Resource size optimization

“Compressed resources” is often the most easily associated package size optimization scheme, but in practice, it also contains skills. Today’s headlines have made many attempts to optimize resources.

2.1 use appropriate resource compression configuration

The minimum supported IOS system version for today’s headlines is IOS 9. However, most of the pod libraries are specified in the podspec filedeployment_target(minimum supported version) is still IOS 8 because it has not been modified in time, which leads to the failure of theresource_bundlesWhen building assets.car, IOS 8 is the lowest supported version.

We found that: 1

1. Upgrade the minimum supported version of pod library and main project from IOS 8.0 to IOS 9.0

2. Open pod library and Xcode build settingsASSETCATALOG_COMPILER_OPTIMIZATION spaceoption

These two settings can change the encoding and compression algorithm selected by actor when building assets. Car, and reduce the packet size. We can use itxcrun assetutil --info Assets.carCommand to check the encoding and compression algorithm used for each image in assets. Car. In today’s headlines, the results are as follows:

As the encoding and compression algorithm of PNG images in assets.car has been changed, these two configurations have achieved a packet size gain of 2.31mb when they hit the headlines today.

2.2 use RGB with palette to compress pictures

In the early days of today’s headline investment package size optimization, we tried to do lossless compression for PNG images in asset catalog, but after practice, we found that although the image size in asset catalog has been significantly reduced, the size of the constructed product has almost no change.

After exploration, we found that in Xcode, the tool for building asset catalog, actool, will first decode the PNG image in asset catalog to get the bitmap data, and then use the coding compression algorithm of actool to carry out coding compression processing. Lossless compression reduces the size of the image by transforming the encoding compression algorithm of the image, but does not change the bitmap data. For actool, the input it receives does not change, so lossless compression cannot optimize the size of assets. Car.

Is there any other compression method that can optimize the size of assets. Car? We guess that it’s a good idea to do lossy compression for images.

So we tried the RGB with palette encoding method [2]. The byte stream encoded by RGB with palette first maintains a color array. Each member of the color array maintains a color with four RGBA components. Each pixel in the image stores the subscript of the color array to represent the color of the point. The type and number of colors maintained by the color array are determined by the image. At the same time, the upper limit of the type of colors maintained by the color array can be artificially limited. The default value is 256. This encoding is just like its name: palette.

Although most of the pictures in the app use many kinds of colors, most of these colors are very close, which is difficult to distinguish visually, such as a large number of flat style icons. This type of image is very suitable for lossy compression by using palette encoding and reducing the size of color array. It can not only reduce the number of colors to achieve lossy compression, but also ensure that the retained colors are close to the original image, making the lossy compressed image look lossless. We landed in today’s headlines and got 3.15mb packet size revenue.

In the specific implementation, we use imageoptim tool to change the image coding mode to RGB with palette:


imageoptim -Q --no-imageoptim --imagealpha --number-of-colors 16 --quality 40-80 ./1.png

among--number-of-colors  Control color array to maintain the number of colors;--quality  Control the quality of the image to the original percentage. Our experience shows that when--number-of-colors  Starting from 16, adjust upward,--qualityMaintain 40-80, can significantly reduce the size of the package while maintaining the quality changes invisible to the naked eye. After the pixel eye examination of UI students, it is confirmed that the pictures before and after optimization look the same.

2.3. Assets.car merger

Today’s headline uses cocoapods for component integration, and the asset catalog file carried by each component is in the form of resource in podspec_ The introduction of bundles will eventually be reflected in the installation package in the form of assets. Car file under bundles.

Taking version 7.9.4 as an example, there are 106 bundles in the installation package, including the assets.car file

The assets.car file is essentially a BOM file. At the same time, Xcode will also bring some optimization operations when building assets.car files with actor, such as automatically merging several small pictures into a packed image. Therefore, merging several assets. Cars can reduce the number of duplicate BOM blocks and maximize the optimization effect of the tool itself.

In the process of building, today’s headline adds scripts to build phases, merges the images in asset catalog from multiple libraries into one asset catalog, and then builds assets.car product through actool. This optimization results in a packet size gain of 2.1MB. At the same time, theoretically, this optimization can also reduce the parsing operation of assets. Car at run time, and has a positive benefit to the response time of image reading.

2.4. Text file compression

In addition to the largest proportion of picture resources, there are many text file resources in today’s Toutiao installation package, such as JSON files, HTML files, etc. The compression of these text files can also optimize the packet size.

Today’s headline landing text file compression scheme consists of three parts:

1. Compression stage: add scripts in build phase, and zip the text files in the whitelist during construction;

2. Decompression stage: in the app startup stage, the decompression operation is performed in the asynchronous thread, and the decompression products are stored in the sandbox;

3. Reading stage: when the app is running, hook reads these files by changing the reading path from bundle to the corresponding path in the sandbox;

This scheme can complete the compression optimization under the premise of less business intrusion. First, we apply this scheme to the JSON file of Lottie animation, and the package size is 400KB. This scheme can be further extended to more types of files.

3、 Mach-o file optimization

At the same time of resource optimization, we are also concerned that mach-o files always occupy about 80% of the volume of today’s headlines installation package. The optimization of mach-o file is essential. Next, we will introduce our mach-o file optimization items in chronological order.

3.1. Compile with – OZ

Oz is a new compiler optimization option for Xcode 11. Oz has been introduced in WWDC 2019 what’s new in clang and llvm [3]. The core principle of Oz is to reuse the repeated continuous machine instructions into function, which is opposite to the principle of “inline function”. Therefore, when Oz is turned on, the size of binary can be reduced, but at the same time, it will bring extra consumption of execution efficiency in theory. Performance (CPU) – sensitive code usage needs to be evaluated.

Apple’s reference data is 4.5% of the package volume revenue.

After evaluating the execution efficiency, stack parsing, stability and compilation speed, we turn on oz compilation for most of the source code and reduce the package size by more than 4MB.

3.2 optimize LTO when using link

Link time optimization is a compile / link parameter of Xcode. According to WWDC 2016 “what’s new in llvm” [4], LTO has a positive impact on packet size and operation efficiency. With incremental LTO enabled in the compilation and linking of today’s headlines, the package size is reduced by 6.5mb.

3.3. Modify exported symbols configuration

In Xcode build settingsEXPORTED_SYMBOLS_FILEConfiguration, controlling mach-o__LINKEDITSection. When dynamic linker dyld is doing symbol binding, it will read the export info information of the bound dynamic library or executable file to get the actual calling address corresponding to a symbol. If the symbol being bound is missing in the export info of the target dynamic library, dyld will throw an exception, showing that the app crashes.

In principle, the information in export info is indispensable. However, for a mach-o file, not all symbols need to be exposed to other dynamic libraries or executable files. Ideally, private symbols should be encoded with__attribute__((visibility(hidden)))modification. However, when it is difficult to add modifiers one by one to the history code, the exported symbols configuration gives the project an opportunity to maintain the white list of public symbols. If a validEXPORTED_SYMBOLS_FILEConfiguration, dynamic library or executable file will remove the symbols outside the white list during static link, which can reduce the package size and increase the reverse difficulty.

Today’s Toutiao has reduced the package size by 2.1MB after using the exported symbols configuration.

3.4. Dynamic attributes

Attribute is one of the most common concepts in OC. However, an attribute is not as small as we think. By analyzing the mach-o file, we find that an attribute can be divided into three parts

(1) Member variable part: the essence of a member variable is a structure with a size of 32B. The contents pointed to by the three pointers (offset, name and type) in the structure are 8b, 10B and 10B respectively. The size of the contents pointed to by the name and type pointers is related to the type and name length of the member variable. The total size is about 60b.


@interface presentViewController ()
@property (nonatomic,strong) UIImageView *imageView;
@property (nonatomic,strong) UIButton *button;
@property (nonatomic,strong) NSString *name;
@end

(2) Auto generated set / get method part: the essence of set / get method is a 24B structure, which contains three pointers: name, type and implementation, and the content size is about 10B, 10B and 20b. The size of a method is about 64b, while the size of set and get methods is 128B.

(3) Property part: the essence of property is still a structure, with the size of 16b. The size of the two pointers in the structure pointing to the content is about 10B and 10B respectively, which is related to the name and type of the property. The total size is about 36B.

That is, the packet size occupied by an attribute is about 224b.

If we use@dynamicModify an attribute without generating member variables and get / set methods, then an attribute can be reduced from 224b to 36B, that is, it only contains the size of the property part.

At the same time, there are a lot of scripts generated automatically in the codeJSONModelSubclasses, which often have a large number of properties. There is space for packet size optimization.

So we generate it by modifying itJSONModelThe script of subclass implements the following functions:

1. All properties are used@dynamicModify the basic variable to generate Ivar

2. AllJSONModelThe subclass of inherits from the new parent class, and the new parent class implementsresolveInstanceMethodIn this method, theclass_addMethodGet / set methods are added to properties. The attributes of the object type are accessed in the way of associated objects, and the attributes of the base type are accessed in the way of extra generated Ivar.

The packet size gain of 800KB is achieved by this optimization, and the performance impact on read and write is acceptable.

3.5、__ Text segment migration

If the download size of the compressed installation package exceeds 200 MB, the downloading of apps in the cellular network will be restricted, which will have a great impact on the new addition. In the second half of 2020, we will explore and practice__ Text segment migration technology: used in link phase-rename_pOptions will__TEXT,__textMove to__BD_TEXT,__textTo reduce the scope of Apple’s encryption of executable files, improve the compression efficiency of executable files, so as to reduce the download size.

Using this scheme, we finally reduce the download size of 60 MB and the install size of 2 Mb. Detailed principles can be referred to: “today’s headline Optimization Practice: IOS package size binary optimization, one line of code to reduce the download size of 60 MB” [5].

3.6 binary segment compression

Mach-o file accounts for a large proportion of install size, but not every segment / section in the file is used at the first time of program startup. This part of the mach-o file can be compressed in the construction process, and then decompressed into memory before these segments are used, which can achieve the effect of reducing the package size and ensuring the normal operation of the program. Due to some limitations of apple, we only compress the__TEXT,__gcc_except_tabAnd__TEXT,__objc_methtypeTwo sections, then in the_dyld_register_func_for_add_imageIt is decompressed in the callback of. This scheme optimizes 3.5 MB install size.

4、 Summary

While the above optimization items are implemented, we also cooperate with the business to further optimize the size of the installation package by mining useless code and resources. So that today’s headlines in high-speed business iterations, packet size can still remain stable.

The above is how to optimize the details of today’s headlines IOS installation package. For more information about today’s headlines IOS installation package size optimization, please pay attention to other related articles of developer!

Recommended Today

Looking for frustration 1.0

I believe you have a basic understanding of trust in yesterday’s article. Today we will give a complete introduction to trust. Why choose rust It’s a language that gives everyone the ability to build reliable and efficient software. You can’t write unsafe code here (unsafe block is not in the scope of discussion). Most of […]