Android custom plugin replaces repetitive labor

Time:2021-12-4

In the previous article, we said that we should do a practical battle to customize the gradle plug-in. This article records the practice in two scenarios. The practice content belongs to the entry-level and is relatively simple. First, find the same activity name in multiple modules; Second: find out the duplicate PNG pictures in the picture resources.

Scenario 1: find the same activity

Because of multi module development, it is inevitable that there will be such a scene. Different people are responsible for different modules, and it is inevitable that the same names will appear, which also has an impact on the statistical buried points. Our purpose is to find out these same names. The ASM bytecode manipulation framework is applied here, which directly operates bytecode and then changes Java classes. The following is the concept of ASM

ASM is a Java bytecode manipulation framework. It can be used to dynamically generate classes or enhance the functionality of existing classes. ASM can directly generate binary class files, or dynamically change the class behavior before the class is loaded into the Java virtual machine.

        The underlying principle is that. Java files are compiled into. Class files through javac. Although the contents of. Class files are different, they all have the same format. ASM scans the contents of. Class files from beginning to end according to the unique format of. Class files by using visitor mode, You can do some operations on the. Class file.

However, on Android, the. Class file also needs to be converted into a DEX file. Here, a schematic diagram of the packaging process of APK is attached

Android custom plugin replaces repetitive labor

Apk packaging process

        Therefore, we need to scan the class and modify it before converting it into a DEX file. Gradle provides the transform API here to make it easier for us to complete this operation (in this scenario, we only do scanning and no operation)

1. Inherit transform and implement the following methods
class ScanDuplicateTransform(var mProjectDir:File) : Transform() {

    /**
     *Set the task name corresponding to our custom transform. Gradle will display this name on the console when compiling
     * @return String
     */
    override fun getName(): String = "ScanDuplicateTransform"

    /**
     *There will be files in various formats in the project. This method can set the file type received by transform
     *Specific value range
     * CONTENT_ Class. Class file
     * CONTENT_ Jars jar package
     * CONTENT_ The resources resource contains java files
     * CONTENT_NATIVE_LIBS native lib
     * CONTENT_ DEX file
     * CONTENT_ DEX_ WITH_ Resources DEX file
     * @return
     */
    override fun getInputTypes(): MutableSet<QualifiedContent.ContentType> = TransformManager.CONTENT_CLASS


    /**
     *Define the scope of transform retrieval
     *Project retrieves only the project content
     * SUB_ Projects only check the contents of subprojects
     * EXTERNAL_ Libraries has only external libraries
     * TESTED_ Code the code tested by the current variable, including dependencies
     * PROVIDED_ Only provides only local or remote dependencies for
     * @return
     */
    //Retrieve item content only
    override fun getScopes(): MutableSet<in QualifiedContent.Scope> = TransformManager.PROJECT_ONLY

    /**
     *Indicates whether the current transform supports incremental compilation and returns true. It is not required to support the current test plug-in
     * @return Boolean
     */
    override fun isIncremental(): Boolean = false
    //Retrieve the item class
    override fun transform(transformInvocation: TransformInvocation) {
        Println ("transform method call")

        //Get all input file collections
        val transformInputs = transformInvocation.inputs
        val transformOutputProvider = transformInvocation.outputProvider

        transformOutputProvider?.deleteAll()

        transformInputs.forEach { transformInput ->
            //Caused by: java.lang.classnotfoundexception: didn't find class "Android. Appcompat. R $drawable" on path
            //R classes above gradle 3.6.0 will not be converted into. Class files, but will be converted into jars. Therefore, a separate copy is required in the transformation implementation, transforminvocation.inputs.jarinputs
            //Jar file processing
            transformInput.jarInputs.forEach { jarInput ->
                val file = jarInput.file

                val dest = transformOutputProvider.getContentLocation(jarInput.name, jarInput.contentTypes, jarInput.scopes, Format.JAR)
                println("find jar input:$dest")
                FileUtils.copyFile(file, dest)
            }
            //Source file processing
            //Directoryinputs represents all directory structures involved in project compilation in the form of source code and the source files in their directories
            transformInput.directoryInputs.forEach { directoryInput ->
                //Traverse all files and folders to find the end of class
                directoryInput.file.walkTopDown()
                        .filter { it.isFile }
                        .filter { it.extension == "class" }
                        .forEach { file ->
//                            println("find class file:${file.name}")
                            val classReader = ClassReader(file.readBytes())
                            val classWriter = ClassWriter(classReader, ClassWriter.COMPUTE_MAXS)
                            //Bytecode stake processing
                            //2. Class reads the incoming ASM visitor
                            val scanDuplicateClassVisitor = ScanDuplicateClassVisitor(mProjectDir,classWriter)
                            //3. Processing through classvisitor API
                            classReader.accept(scanDuplicateClassVisitor,ClassReader.EXPAND_FRAMES)
                            //4. Process the bytecode successfully modified
                            val bytes = classWriter.toByteArray()
                            //Write back to file
                            val fos =  FileOutputStream(file.path)
                            fos.write(bytes)
                            fos.close()
                        }
                //Copy to corresponding directory
                val dest = transformOutputProvider.getContentLocation(directoryInput.name,directoryInput.contentTypes,directoryInput.scopes, Format.DIRECTORY)
                FileUtils.copyDirectory(directoryInput.file,dest)
            }
        }
    }
}

        The bytecode operation process is written in the code. In fact, our scene only needs to be scanned and will not move to the class file. Here, we just borrow it and return it immediately, mainly throughScanDuplicateClassVisitorThis class to scan

val scanDuplicateClassVisitor=ScanDuplicateClassVisitor(mProjectDir,classWriter)             
classReader.accept(scanDuplicateClassVisitor,ClassReader.EXPAND_FRAMES)
2. Scan class files through classvisitor in ASM framework
class ScanDuplicateClassVisitor( file: File,classVisitor: ClassVisitor?) : ClassVisitor(Opcodes.ASM5, classVisitor) {

    private var className:String? = null
    private var superName:String? = null

    private var mFile:File? =null
    init {
        mFile = File(file,"activity_name_c.txt")
    }

    override fun visit(version: Int, access: Int, name: String?, signature: String?, superName: String?, interfaces: Array<out String>?) {
        super.visit(version, access, name, signature, superName, interfaces)
        this.className = name
        this.superName = superName
        if (superName == "xxx"){
            getLastName(name)
        }
        if (superName == "yyy"){
            getLastName(name)
        }

    }


    override fun visitModule(name: String?, access: Int, version: String?): ModuleVisitor {

        println("------visitModule------ "+name)
        return super.visitModule(name, access, version)
    }

    private fun getLastName(name: String?){
        val pos = name?.lastIndexOf("/")!!+1
        if (name.isEmpty().not()){
            val lastName = name.substring(pos,name.length)
            println(lastName)
            writeFileName(lastName)
        }
    }


    private fun writeFileName(name: String?){


        val bytes: ByteArray = name!!.toByteArray()
        val fos = FileOutputStream(mFile,true)
        fos.apply {
            write(bytes)
            flush()
            close()
        }
    }

    override fun visitMethod(access: Int, name: String, descriptor: String?, signature: String?, exceptions: Array<out String>?): MethodVisitor {
        val methodVisitor = cv.visitMethod(access,name,descriptor,signature,exceptions)
        //Find the activity class under the Android x package

        return methodVisitor
    }

    override fun visitEnd() {
        super.visitEnd()

    }
}

Using the classvisit of ASM framework, you can scan all class file names and even method names, and then do whatever you want. It’s very intersting. First, I print out the page names of all inherited activities and save them in the activity_ name_ C file, here I just learned some Python fur before, looking for the same name. I use the following

import shutil

def openFile():
   f = open("E:\\MyTestSpace\\kplugin\\app\\build\\activity_name_c.txt","r")
   new_file = open("E:\\MyTestSpace\\kplugin\\app\\build\\duplicate_activity.txt","a")
   new_file.write("")
   files = f.readlines()
   words_dic = {}
   for line in files:
       if line in words_dic:
          words_dic[line]+=1
       else:
          words_ DIC [line] = 1# the first occurrence of the word, we assign its value to 1

   for (key,value) in words_dic.items():
       if(value > 1):
          print(key)
          
   print(words_dic)
   f.close()

openFile()

It’s a little troublesome. It’s mainly to practice by yourself. It’s not written in real combat. It’s wild. In the future, we will have the opportunity to go deep into this area and improve it

3. Customize the plugin and register scanduplicatetransform
import com.android.build.gradle.AppExtension
import com.android.build.gradle.LibraryExtension
import com.android.build.gradle.internal.api.BaseVariantImpl
import com.kunsan.plugin.utils.Md5Util
import com.kunsan.plugin.utils.PathUtils
import com.sun.imageio.plugins.common.ImageUtil
import org.gradle.api.Plugin
import org.gradle.api.Project
import java.io.File
import java.util.*
import java.util.stream.Collectors
import kotlin.collections.ArrayList

class MyPlugin : Plugin<Project> {

    override fun apply(target: Project) {
////////////////////////////////Scene 1///////////////////////////////////////////////
        val asmTransform = target.extensions.getByType(LibraryExtension::class.java)

        val transform = ScanDuplicateTransform(target.buildDir)
        asmTransform.registerTransform(transform)

    }
}
Scenario 2: find duplicate PNG resources

        The same purpose of this is to develop by multiple people and avoid importing the same PNG image. Of course, this involves the company’s development specifications. This should not happen normally. In fact, Scene 1 is more similar to scene 2. They are both scanning, but the application tools are different. We directly use the methods provided by gradleallRawAndroidResources.filesYou can get all the pictures. First find the pictures of the same size, and then calculate their MD5 value, you can know which pictures are the same, and then delete them manually (it’s safer)

import com.android.build.gradle.AppExtension
import com.android.build.gradle.LibraryExtension
import com.android.build.gradle.internal.api.BaseVariantImpl
import com.kunsan.plugin.utils.Md5Util
import com.kunsan.plugin.utils.PathUtils
import com.sun.imageio.plugins.common.ImageUtil
import org.gradle.api.Plugin
import org.gradle.api.Project
import java.io.File
import java.util.*
import java.util.stream.Collectors
import kotlin.collections.ArrayList

class MyPlugin : Plugin<Project> {

    override fun apply(target: Project) {
////////////////////////////////Scene 2///////////////////////////////////////////////
        //check is library or application
        val hasAppPlugin = target.plugins.hasPlugin("com.android.application")
        val variants = if (hasAppPlugin) {
            (target.property("android") as AppExtension).applicationVariants
        } else {
            (target.property("android") as LibraryExtension).libraryVariants
        }

        //Get resources
        target.afterEvaluate {
            variants.all{  variant ->
                val mergeResourceTask = variant.mergeResourcesProvider.get()
                val mcPicTask = target.task("KsImage${variant.name.capitalize()}")
                mcPicTask.doLast{
                    val dir = variant.allRawAndroidResources.files
                    for (channelDir: File in dir) {
                        traverseResDir(channelDir)
                    }
                    sameLengthFileMap.forEach {
                        it.value.forEach{ name ->
                            if (sameMd5List.contains(Md5Util.getMD5Str(File(name)))){
                                Println ("==================================== + pathutils.getlastname (name))
                            }
                            sameMd5List.add(Md5Util.getMD5Str(File(name)))
                        }
                    }
                }
            }
        }

    }

    /**
     *Key - > picture size
     *Value - > collection of pictures with the same size
     */
    var sameLengthFileMap = hashMapOf<Long,ArrayList<String>>()

    /**
     *Used to determine whether there are pictures with the same MD5 value
     */
    var sameMd5List = arrayListOf<String?>()


    /**
     *Recursive res folder
     */
    private fun traverseResDir(file: File){

        if (file.isDirectory){
            file.listFiles().forEach {  it ->
                if (it.isDirectory){
                    if (it.absolutePath.contains(".gradle\\caches")){
                        [email protected]
                    }else{
                        traverseResDir(it)
                    }
                }else{
                    if (ImageUtils.isImage(it) && !it.absolutePath.contains("ic_launcher")){
                        filterImage(it)
                    }
                }
            }

        }
    }

    /**
     *Put pictures of the same size into the map
     *Do the first filter
     */
    private fun filterImage(file: File){

        var mList = sameLengthFileMap[file.length()]
        if (mList == null){
            mList =  ArrayList()
            sameLengthFileMap[file.length()] = mList
        }
        val imageName = file.absolutePath
        if (!mList.contains(imageName)){
                mList.add(imageName)
        }

    }
}

summary

The above is that the flag has finally been completed. It’s a good introduction. As a basis for further opportunities in the future, make a note for yourself, otherwise it’s easy to forget. At the same time, I hope my little partner can give me some advice.