内容简介:年前,微信开源了整体代码结构比较清晰,主要包括三部分:目的:解压Apk,解析Class混淆规则、Res混淆规则,并输出apk中每个entry原始大小、zip包中压缩后的大小。主要存储了一些原始数据,为后续的Task做准备。
年前,微信开源了 Matrix 项目,提供了Android、ios的APM实现方案。对于Android端实现,主要包括 APK Checker
、 Resource Canary
、 Trace Canary
、 SQLite Lint
、 IO Canary
五部分。本文主要介绍 APK Checker
的源码实现,其他部分的源码分析将在后续推出。
代码框架分析
整体代码结构比较清晰,主要包括三部分: ApkJob
、 Task
、 Result
。 ApkJob
是表示整体这个apk检测任务, Task
表示每一步细分的检测任务、 Result
表示检测任务的结果。总体流程如下: ApkJob
读取配置信息,实例化相关的 Task
任务;相关 Task
任务执行之后输出 Result
到文件(默认为 MMTaskJsonResult
)。
Task任务实现分析
UnZipTask
目的:解压Apk,解析Class混淆规则、Res混淆规则,并输出apk中每个entry原始大小、zip包中压缩后的大小。主要存储了一些原始数据,为后续的Task做准备。
@Override public TaskResult call() throws TaskExecuteException { try { //apk文件 ZipFile zipFile = new ZipFile(inputFile); ... //Result输出对象 TaskResult taskResult = TaskResultFactory.factory(getType(), TASK_RESULT_TYPE_JSON, config); ... //apk总大小 ((TaskJsonResult) taskResult).add("total-size", inputFile.length()); //读取Class的mapping规则,并存储到config对象中 readMappingTxtFile(); config.setProguardClassMap(proguardClassMap); //读取Res的mapping规则,并存储到config对象中 readResMappingTxtFile(); config.setResguardMap(resguardMap); Enumeration entries = zipFile.entries(); JsonArray jsonArray = new JsonArray(); String outEntryName = ""; while (entries.hasMoreElements()) { ZipEntry entry = (ZipEntry) entries.nextElement(); outEntryName = writeEntry(zipFile, entry); if (!Util.isNullOrNil(outEntryName)) { JsonObject fileItem = new JsonObject(); //输出Apk中每个item的名字、压缩后的大小 fileItem.addProperty("entry-name", outEntryName); fileItem.addProperty("entry-size", entry.getCompressedSize()); jsonArray.add(fileItem); //Map:解压后文件(相对路径)-> (未压缩Size,压缩后Size) entrySizeMap.put(outEntryName, Pair.of(entry.getSize(), entry.getCompressedSize())); //Map:Apk中文件名 -> :解压后文件(相对路径) entryNameMap.put(entry.getName(), outEntryName); } } //存储到config对象 config.setEntrySizeMap(entrySizeMap); config.setEntryNameMap(entryNameMap); //输出到Result ((TaskJsonResult) taskResult).add("entries", jsonArray); taskResult.setStartTime(startTime); taskResult.setEndTime(System.currentTimeMillis()); return taskResult; } catch (Exception e) { throw new TaskExecuteException(e.getMessage(), e); } } 复制代码
重点讲解一下Task任务中mapping文件的解析规则:
- Class Mapping
class mapping文件截取片段:
... android.arch.core.executor.ArchTaskExecutor$1 -> android.arch.a.a.a$1: 42:42:void <init>() -> <init> 45:46:void execute(java.lang.Runnable) -> execute android.arch.core.executor.ArchTaskExecutor$2 -> android.arch.a.a.a$2: 50:50:void <init>() -> <init> 53:54:void execute(java.lang.Runnable) -> execute android.arch.core.executor.DefaultTaskExecutor -> android.arch.a.a.b: java.lang.Object mLock -> a java.util.concurrent.ExecutorService mDiskIO -> b android.os.Handler mMainHandler -> c 31:33:void <init>() -> <init> 40:41:void executeOnDiskIO(java.lang.Runnable) -> a 45:54:void postToMainThread(java.lang.Runnable) -> b 58:58:boolean isMainThread() -> b ... 复制代码
* 原始类名 -> 混淆后类名 (顶格) * 原始字段名 -> 混淆后字段名 (行首预留一个Tab) * 原始函数名 -> 混淆后函数名 (行首预留一个Tab) 复制代码
- Res Mapping
res mapping文件截取片段:
res path mapping: res/layout-v22 -> r/a res/drawable -> r/b res/color-night-v8 -> r/c res/xml -> r/d res/layout -> r/e ... res id mapping: com.example.app.R.attr.avatar_border_color -> com.example.app.R.attr.a com.example.app.R.attr.actualImageScaleType -> com.example.app.R.attr.b com.example.app.R.attr.backgroundImage -> com.example.app.R.attr.c com.example.app.R.attr.fadeDuration -> com.example.app.R.attr.d com.example.app.R.attr.failureImage -> com.example.app.R.attr.e 复制代码
* 原始资源目录 -> 混淆后资源目录 * 原始资源名 -> 混淆后资源名 复制代码
ManifestAnalyzeTask
目的:解析Manifest文件、arsc文件
public TaskResult call() throws TaskExecuteException { try { ManifestParser manifestParser = null; //创建Manifest解析对象 if (!FileUtil.isLegalFile(arscFile)) { manifestParser = new ManifestParser(inputFile); } else { manifestParser = new ManifestParser(inputFile, arscFile); } TaskResult taskResult = TaskResultFactory.factory(getType(), TASK_RESULT_TYPE_JSON, config); if (taskResult == null) { return null; } long startTime = System.currentTimeMillis(); JsonObject jsonObject = manifestParser.parse(); //输出Manifest解析结果 ((TaskJsonResult) taskResult).add("manifest", jsonObject); taskResult.setStartTime(startTime); taskResult.setEndTime(System.currentTimeMillis()); return taskResult; } catch (Exception e) { throw new TaskExecuteException(e.getMessage(), e); } } 复制代码
此处讲解下arsc文件。arsc文件以二进制形式存在,存储了资源的索引信息,基本文件格式如下图(图片来源网络):
用二进制 工具 查看arsc文件的内容:
arsc的详细文件格式暂时不展开,可参考文章,此处仅简单分析一下二进制工具中可视化展示的一些信息。
- ResTable_header部分
- ResStringPool部分
- ResTablePackage
关于arsc文件解析的相关内容,详见文章
ShowFileSizeTask
目的:统计超过阈值的文件。
public TaskResult call() throws TaskExecuteException { ... long startTime = System.currentTimeMillis(); //获取UnZipTask中记录的 文件名->(文件压缩后大小,文件压缩前大小) map Map<String, Pair<Long, Long>> entrySizeMap = config.getEntrySizeMap(); if (!entrySizeMap.isEmpty()) { for (Map.Entry<String, Pair<Long, Long>> entry : entrySizeMap.entrySet()) { final String suffix = getSuffix(entry.getKey()); Pair<Long, Long> size = entry.getValue(); // 记录超出阈值的文件 if (size.getFirst() >= downLimit * ApkConstants.K1024) { if (filterSuffix.isEmpty() || filterSuffix.contains(suffix)) { entryList.add(Pair.of(entry.getKey(), size.getFirst())); } } } } ... //排序 JsonArray jsonArray = new JsonArray(); for (Pair<String, Long> sortFile : entryList) { JsonObject fileItem = new JsonObject(); fileItem.addProperty("entry-name", sortFile.getFirst()); fileItem.addProperty("entry-size", sortFile.getSecond()); jsonArray.add(fileItem); } //输出到结果 ((TaskJsonResult) taskResult).add("files", jsonArray); taskResult.setStartTime(startTime); taskResult.setEndTime(System.currentTimeMillis()); return taskResult; } catch (Exception e) { throw new TaskExecuteException(e.getMessage(), e); } } 复制代码
MethodCountTask
目的:统计在本dex文件内定义的方法数、未在本dex文件内定义的方法数。
public TaskResult call() throws TaskExecuteException { try { ... long startTime = System.currentTimeMillis(); JsonArray jsonArray = new JsonArray(); for (int i = 0; i < dexFileList.size(); i++) { RandomAccessFile dexFile = dexFileList.get(i); //计算dex中的方法信息 countDex(dexFile); //dex内能找到定义的方法 int totalInternalMethods = sumOfValue(classInternalMethod); //跨dex的方法 int totalExternalMethods = sumOfValue(classExternalMethod); JsonObject jsonObject = new JsonObject(); jsonObject.addProperty("dex-file", dexFileNameList.get(i)); //按Class维度聚合 if (JobConstants.GROUP_CLASS.equals(group)) { List<String> sortList = sortKeyByValue(classInternalMethod); JsonArray classes = new JsonArray(); for (String className : sortList) { JsonObject classObj = new JsonObject(); classObj.addProperty("name", className); classObj.addProperty("methods", classInternalMethod.get(className)); classes.add(classObj); } jsonObject.add("internal-classes", classes); //按package维度聚合 } else if (JobConstants.GROUP_PACKAGE.equals(group)) { String packageName; for (Map.Entry<String, Integer> entry : classInternalMethod.entrySet()) { packageName = ApkUtil.getPackageName(entry.getKey()); if (!Util.isNullOrNil(packageName)) { if (!pkgInternalRefMethod.containsKey(packageName)) { pkgInternalRefMethod.put(packageName, entry.getValue()); } else { pkgInternalRefMethod.put(packageName, pkgInternalRefMethod.get(packageName) + entry.getValue()); } } } List<String> sortList = sortKeyByValue(pkgInternalRefMethod); JsonArray packages = new JsonArray(); for (String pkgName : sortList) { JsonObject pkgObj = new JsonObject(); pkgObj.addProperty("name", pkgName); pkgObj.addProperty("methods", pkgInternalRefMethod.get(pkgName)); packages.add(pkgObj); } jsonObject.add("internal-packages", packages); } jsonObject.addProperty("total-internal-classes", classInternalMethod.size()); jsonObject.addProperty("total-internal-methods", totalInternalMethods); if (JobConstants.GROUP_CLASS.equals(group)) { List<String> sortList = sortKeyByValue(classExternalMethod); JsonArray classes = new JsonArray(); for (String className : sortList) { JsonObject classObj = new JsonObject(); classObj.addProperty("name", className); classObj.addProperty("methods", classExternalMethod.get(className)); classes.add(classObj); } jsonObject.add("external-classes", classes); } else if (JobConstants.GROUP_PACKAGE.equals(group)) { String packageName = ""; for (Map.Entry<String, Integer> entry : classExternalMethod.entrySet()) { packageName = ApkUtil.getPackageName(entry.getKey()); if (!Util.isNullOrNil(packageName)) { if (!pkgExternalMethod.containsKey(packageName)) { pkgExternalMethod.put(packageName, entry.getValue()); } else { pkgExternalMethod.put(packageName, pkgExternalMethod.get(packageName) + entry.getValue()); } } } List<String> sortList = sortKeyByValue(pkgExternalMethod); JsonArray packages = new JsonArray(); for (String pkgName : sortList) { JsonObject pkgObj = new JsonObject(); pkgObj.addProperty("name", pkgName); pkgObj.addProperty("methods", pkgExternalMethod.get(pkgName)); packages.add(pkgObj); } jsonObject.add("external-packages", packages); } jsonObject.addProperty("total-external-classes", classExternalMethod.size()); jsonObject.addProperty("total-external-methods", totalExternalMethods); jsonArray.add(jsonObject); } ((TaskJsonResult) taskResult).add("dex-files", jsonArray); taskResult.setStartTime(startTime); taskResult.setEndTime(System.currentTimeMillis()); return taskResult; } catch (Exception e) { throw new TaskExecuteException(e.getMessage(), e); } } 复制代码
这段代码的重点是如何对dex文件进行静态分析的
private void countDex(RandomAccessFile dexFile) throws IOException { classInternalMethod.clear(); classExternalMethod.clear(); pkgInternalRefMethod.clear(); pkgExternalMethod.clear(); DexData dexData = new DexData(dexFile); //加载dex数据 dexData.load(); MethodRef[] methodRefs = dexData.getMethodRefs(); ClassRef[] externalClassRefs = dexData.getExternalReferences(); //获取混淆的Class maping规则 Map<String, String> proguardClassMap = config.getProguardClassMap(); String className = null; for (ClassRef classRef : externalClassRefs) { className = ApkUtil.getNormalClassName(classRef.getName()); if (proguardClassMap.containsKey(className)) { //混淆前的原始className className = proguardClassMap.get(className); } if (className.indexOf('.') == -1) { continue; } classExternalMethod.put(className, 0); } for (MethodRef methodRef : methodRefs) { className = ApkUtil.getNormalClassName(methodRef.getDeclClassName()); if (proguardClassMap.containsKey(className)) { className = proguardClassMap.get(className); } if (!Util.isNullOrNil(className)) { if (className.indexOf('.') == -1) { continue; } if (classExternalMethod.containsKey(className)) { classExternalMethod.put(className, classExternalMethod.get(className) + 1); } else if (classInternalMethod.containsKey(className)) { classInternalMethod.put(className, classInternalMethod.get(className) + 1); } else { classInternalMethod.put(className, 1); } } } //remove 0-method referenced class Iterator<String> iterator = classExternalMethod.keySet().iterator(); while (iterator.hasNext()) { if (classExternalMethod.get(iterator.next()) == 0) { iterator.remove(); } } } 复制代码
理解上述代码之前,先介绍下dex文件格式。 dex文件可分为Header部分、String索引表、类型索引表、方法原型索引表、字段索引表、方法索引表、类定义、Data数据区。
- Header部分
- String索引表
- 类型索引表
- 方法原型索引表
- 字段索引表
- 方法索引表
- 类定义
通过二进制工具,大概讲解了dex的文件格式。再回过头看代码,代码中有一个 classInternalMethod
和 classExternalMethod
的区别;首先在解析TypeId的时候会有一个 internal
字段表示这个类型是否定义在这个dex文件内;
/** * Holds the contents of a type_id_item. * * This is chiefly a list of indices into the string table. We need * some additional bits of data, such as whether or not the type ID * represents a class defined in this DEX, so we use an object for * each instead of a simple integer. (Could use a parallel array, but * since this is a desktop app it's not essential.) */ static class TypeIdItem { public int descriptorIdx; // index into string_ids public boolean internal; // defined within this DEX file? } 复制代码
internal
字段的赋值操作如下:
/** * Sets the "internal" flag on type IDs which are defined in the * DEX file or within the VM (e.g. primitive classes and arrays). */ void markInternalClasses() { for (int i = mClassDefs.length - 1; i >= 0; i--) { mTypeIds[mClassDefs[i].classIdx].internal = true; } for (int i = 0; i < mTypeIds.length; i++) { String className = mStrings[mTypeIds[i].descriptorIdx]; if (className.length() == 1) { // primitive class mTypeIds[i].internal = true; } else if (className.charAt(0) == '[') { mTypeIds[i].internal = true; } //System.out.println(i + " " + // (mTypeIds[i].internal ? "INTERNAL" : "external") + " - " + // mStrings[mTypeIds[i].descriptorIdx]); } } 复制代码
在ClassDef中定义的类型都属于internal,同时转换后的className长度为1的类型(基础数据类型)也属于interal,最后数组类型的也属于internal。
classInternalMethod
和 classExternalMethod
的具体划分规则如下:
private void countDex(RandomAccessFile dexFile) throws IOException { ... ... for (ClassRef classRef : externalClassRefs) { className = ApkUtil.getNormalClassName(classRef.getName()); if (proguardClassMap.containsKey(className)) { className = proguardClassMap.get(className); } if (className.indexOf('.') == -1) { continue; } //将类定义不在本dex文件中的类名加入map classExternalMethod.put(className, 0); } for (MethodRef methodRef : methodRefs) { className = ApkUtil.getNormalClassName(methodRef.getDeclClassName()); if (proguardClassMap.containsKey(className)) { className = proguardClassMap.get(className); } if (!Util.isNullOrNil(className)) { if (className.indexOf('.') == -1) { continue; } //根据类名加入不同的分类 if (classExternalMethod.containsKey(className)) { classExternalMethod.put(className, classExternalMethod.get(className) + 1); } else if (classInternalMethod.containsKey(className)) { classInternalMethod.put(className, classInternalMethod.get(className) + 1); } else { classInternalMethod.put(className, 1); } } } //remove 0-method referenced class Iterator<String> iterator = classExternalMethod.keySet().iterator(); while (iterator.hasNext()) { if (classExternalMethod.get(iterator.next()) == 0) { iterator.remove(); } } } 复制代码
ResProguardCheckTask
目的:判断apk是否执行了资源混淆。
@Override public TaskResult call() throws TaskExecuteException { File resDir = new File(inputFile, ApkConstants.RESOURCE_DIR_PROGUARD_NAME); ... if (resDir.exists() && resDir.isDirectory()) { Log.d(TAG, "find resource directory " + resDir.getAbsolutePath()); //有名为r的文件夹,执行了支援混淆 ((TaskJsonResult) taskResult).add("hasResProguard", true); } else { resDir = new File(inputFile, ApkConstants.RESOURCE_DIR_NAME); if (resDir.exists() && resDir.isDirectory()) { File[] dirs = resDir.listFiles(); boolean hasProguard = true; for (File dir : dirs) { //任意文件夹不符合资源混淆的命名规则,则未执行资源混淆 if (dir.isDirectory() && !fileNamePattern.matcher(dir.getName()).matches()) { hasProguard = false; Log.i(TAG, "directory " + dir.getName() + " has a non-proguard name!"); break; } } ((TaskJsonResult) taskResult).add("hasResProguard", hasProguard); ... } 复制代码
FindNonAlphaPngTask
目的:检测出没有透明度的png文件(应该使用jpg替换,占用空间会更小)
private void findNonAlphaPng(File file) throws IOException { if (file != null) { if (file.isDirectory()) { File[] files = file.listFiles(); for (File tempFile : files) { findNonAlphaPng(tempFile); } } else if (file.isFile() && file.getName().endsWith(ApkConstants.PNG_FILE_SUFFIX) && !file.getName().endsWith(ApkConstants.NINE_PNG)) { BufferedImage bufferedImage = ImageIO.read(file); //没有alpha信息 if (!bufferedImage.getColorModel().hasAlpha()) { String filename = file.getAbsolutePath().substring(inputFile.getAbsolutePath().length() + 1); if (entryNameMap.containsKey(filename)) { filename = entryNameMap.get(filename); } long size = file.length(); if (entrySizeMap.containsKey(filename)) { size = entrySizeMap.get(filename).getFirst(); } if (size >= downLimitSize * ApkConstants.K1024) { nonAlphaPngList.add(Pair.of(filename, file.length())); } } } } } 复制代码
MultiLibCheckTask
目的:检测lib文件夹中是否有多文件夹存在。
@Override public TaskResult call() throws TaskExecuteException { try { TaskResult taskResult = TaskResultFactory.factory(getType(), TASK_RESULT_TYPE_JSON, config); if (taskResult == null) { return null; } long startTime = System.currentTimeMillis(); JsonArray jsonArray = new JsonArray(); if (libDir.exists() && libDir.isDirectory()) { File[] dirs = libDir.listFiles(); for (File dir : dirs) { if (dir.isDirectory()) { jsonArray.add(dir.getName()); } } } ((TaskJsonResult) taskResult).add("lib-dirs", jsonArray); if (jsonArray.size() > 1) { ((TaskJsonResult) taskResult).add("multi-lib", true); } else { ((TaskJsonResult) taskResult).add("multi-lib", false); } taskResult.setStartTime(startTime); taskResult.setEndTime(System.currentTimeMillis()); return taskResult; } catch (Exception e) { throw new TaskExecuteException(e.getMessage(), e); } } 复制代码
UncompressedFileTask
目的:比对apk压缩包里每一个entry的压缩后大小、压缩前大小;若大小一样,则表示文件未压缩。
@Override public TaskResult call() throws TaskExecuteException { try { ... if (!entrySizeMap.isEmpty()) { //take advantage of the result of UnzipTask. for (Map.Entry<String, Pair<Long, Long>> entry : entrySizeMap.entrySet()) { final String suffix = getSuffix(entry.getKey()); Pair<Long, Long> size = entry.getValue(); if (filterSuffix.isEmpty() || filterSuffix.contains(suffix)) { if (!uncompressSizeMap.containsKey(suffix)) { uncompressSizeMap.put(suffix, size.getFirst()); } else { uncompressSizeMap.put(suffix, uncompressSizeMap.get(suffix) + size.getFirst()); } if (!compressSizeMap.containsKey(suffix)) { compressSizeMap.put(suffix, size.getSecond()); } else { compressSizeMap.put(suffix, compressSizeMap.get(suffix) + size.getSecond()); } } else { // Log.d(TAG, "file: %s, filter by suffix.", entry.getKey()); } } } for (String suffix : uncompressSizeMap.keySet()) { //大小比对 if (uncompressSizeMap.get(suffix).equals(compressSizeMap.get(suffix))) { JsonObject fileItem = new JsonObject(); fileItem.addProperty("suffix", suffix); fileItem.addProperty("total-size", uncompressSizeMap.get(suffix)); jsonArray.add(fileItem); } } ((TaskJsonResult) taskResult).add("files", jsonArray); taskResult.setStartTime(startTime); taskResult.setEndTime(System.currentTimeMillis()); return taskResult; } catch (Exception e) { throw new TaskExecuteException(e.getMessage(), e); } } 复制代码
CountRTask
目的:统计R文件数量。
@Override public TaskResult call() throws TaskExecuteException { try { TaskResult taskResult = TaskResultFactory.factory(type, TaskResultFactory.TASK_RESULT_TYPE_JSON, config); long startTime = System.currentTimeMillis(); Map<String, String> classProguardMap = config.getProguardClassMap(); for (RandomAccessFile dexFile : dexFileList) { DexData dexData = new DexData(dexFile); dexData.load(); ClassRef[] defClassRefs = dexData.getInternalReferences(); for (ClassRef classRef : defClassRefs) { String className = ApkUtil.getNormalClassName(classRef.getName()); if (classProguardMap.containsKey(className)) { className = classProguardMap.get(className); } //去掉内部类 String pureClassName = getOuterClassName(className); //识别R文件 if (pureClassName.endsWith(".R") || "R".equals(pureClassName)) { if (!classesMap.containsKey(pureClassName)) { classesMap.put(pureClassName, classRef.getFieldArray().length); } else { classesMap.put(pureClassName, classesMap.get(pureClassName) + classRef.getFieldArray().length); } } } } JsonArray jsonArray = new JsonArray(); long totalSize = 0; Map<String, String> proguardClassMap = config.getProguardClassMap(); for (Map.Entry<String, Integer> entry : classesMap.entrySet()) { JsonObject jsonObject = new JsonObject(); if (proguardClassMap.containsKey(entry.getKey())) { jsonObject.addProperty("name", proguardClassMap.get(entry.getKey())); } else { jsonObject.addProperty("name", entry.getKey()); } jsonObject.addProperty("field-count", entry.getValue()); totalSize += entry.getValue(); jsonArray.add(jsonObject); } ((TaskJsonResult) taskResult).add("R-count", jsonArray.size()); ((TaskJsonResult) taskResult).add("Field-counts", totalSize); ((TaskJsonResult) taskResult).add("R-classes", jsonArray); taskResult.setStartTime(startTime); taskResult.setEndTime(System.currentTimeMillis()); return taskResult; } catch (Exception e) { throw new TaskExecuteException(e.getMessage(), e); } } 复制代码
DuplicateFileTask
目的:通过计算md5,判断apk中是否存在完全一样的文件。
private void computeMD5(File file) throws NoSuchAlgorithmException, IOException { if (file != null) { if (file.isDirectory()) { File[] files = file.listFiles(); for (File resFile : files) { computeMD5(resFile); } } else { MessageDigest msgDigest = MessageDigest.getInstance("MD5"); BufferedInputStream inputStream = new BufferedInputStream(new FileInputStream(file)); byte[] buffer = new byte[512]; int readSize = 0; long totalRead = 0; while ((readSize = inputStream.read(buffer)) > 0) { msgDigest.update(buffer, 0, readSize); totalRead += readSize; } inputStream.close(); if (totalRead > 0) { final String md5 = Util.byteArrayToHex(msgDigest.digest()); String filename = file.getAbsolutePath().substring(inputFile.getAbsolutePath().length() + 1); if (entryNameMap.containsKey(filename)) { filename = entryNameMap.get(filename); } if (!md5Map.containsKey(md5)) { md5Map.put(md5, new ArrayList<String>()); if (entrySizeMap.containsKey(filename)) { fileSizeList.add(Pair.of(md5, entrySizeMap.get(filename).getFirst())); } else { fileSizeList.add(Pair.of(md5, totalRead)); } } //md5相同的文件列表 md5Map.get(md5).add(filename); } } } } 复制代码
@Override public TaskResult call() throws TaskExecuteException { ... ... for (Pair<String, Long> entry : fileSizeList) { //md5相同的文件 if (md5Map.get(entry.getFirst()).size() > 1) { JsonObject jsonObject = new JsonObject(); jsonObject.addProperty("md5", entry.getFirst()); jsonObject.addProperty("size", entry.getSecond()); JsonArray jsonFiles = new JsonArray(); for (String filename : md5Map.get(entry.getFirst())) { jsonFiles.add(filename); } jsonObject.add("files", jsonFiles); jsonArray.add(jsonObject); } } ((TaskJsonResult) taskResult).add("files", jsonArray); taskResult.setStartTime(startTime); taskResult.setEndTime(System.currentTimeMillis()); } catch (Exception e) { throw new TaskExecuteException(e.getMessage(), e); } return taskResult; } 复制代码
MultiSTLCheckTask
目的:判断so是否带有多份stl标准库。
@Override public TaskResult call() throws TaskExecuteException { try { ... for (File libFile : libFiles) { if (isStlLinked(libFile)) { Log.d(TAG, "lib: %s has stl link", libFile.getName()); jsonArray.add(libFile.getName()); } } ((TaskJsonResult) taskResult).add("stl-lib", jsonArray); if (jsonArray.size() > 1) { ((TaskJsonResult) taskResult).add("multi-stl", true); } else { ((TaskJsonResult) taskResult).add("multi-stl", false); } taskResult.setStartTime(startTime); taskResult.setEndTime(System.currentTimeMillis()); return taskResult; } catch (Exception e) { throw new TaskExecuteException(e.getMessage(), e); } } 复制代码
private boolean isStlLinked(File libFile) throws IOException, InterruptedException { ProcessBuilder processBuilder = new ProcessBuilder(toolnmPath, "-D", "-C", libFile.getAbsolutePath()); Process process = processBuilder.start(); BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream())); String line = reader.readLine(); while (line != null) { String[] columns = line.split(" "); // Log.d(TAG, "%s", line); if (columns.length >= 3 && columns[1].equals("T") && columns[2].startsWith("std::")) { return true; } line = reader.readLine(); } reader.close(); process.waitFor(); return false; } 复制代码
UnusedResourcesTask
目的:检测出在代码、资源文件中未被引用的资源。
@Override public TaskResult call() throws TaskExecuteException { try { TaskResult taskResult = TaskResultFactory.factory(type, TaskResultFactory.TASK_RESULT_TYPE_JSON, config); long startTime = System.currentTimeMillis(); readMappingTxtFile(); readResourceTxtFile(); //添加所有声明的资源 unusedResSet.addAll(resourceDefMap.values()); Log.d(TAG, "find resource declarations %d items.", unusedResSet.size()); //找到所有代码中使用的资源 decodeCode(); Log.d(TAG, "find resource references in classes: %d items.", resourceRefSet.size()); //找到所有资源中引用的资源 decodeResources(); Log.d(TAG, "find resource references %d items.", resourceRefSet.size()); //去掉被引用的资源 unusedResSet.removeAll(resourceRefSet); Log.d(TAG, "find unused references %d items", unusedResSet.size()); JsonArray jsonArray = new JsonArray(); for (String name : unusedResSet) { jsonArray.add(name); } ((TaskJsonResult) taskResult).add("unused-resources", jsonArray); taskResult.setStartTime(startTime); taskResult.setEndTime(System.currentTimeMillis()); return taskResult; } catch (Exception e) { throw new TaskExecuteException(e.getMessage(), e); } } 复制代码
private void readMappingTxtFile() throws IOException { // com.tencent.mm.R$string -> com.tencent.mm.R$l: // int fade_in_property_anim -> aRW if (mappingTxt != null) { BufferedReader bufferedReader = new BufferedReader(new FileReader(mappingTxt)); String line = bufferedReader.readLine(); boolean readRField = false; String beforeClass = "", afterClass = ""; try { while (line != null) { if (!line.startsWith(" ")) { String[] pair = line.split("->"); if (pair.length == 2) { beforeClass = pair[0].trim(); afterClass = pair[1].trim(); afterClass = afterClass.substring(0, afterClass.length() - 1); if (!Util.isNullOrNil(beforeClass) && !Util.isNullOrNil(afterClass) && ApkUtil.isRClassName(ApkUtil.getPureClassName(beforeClass))) { // Log.d(TAG, "before:%s,after:%s", beforeClass, afterClass); readRField = true; } else { readRField = false; } } else { readRField = false; } } else { if (readRField) { String[] entry = line.split("->"); if (entry.length == 2) { String key = entry[0].trim(); String value = entry[1].trim(); if (!Util.isNullOrNil(key) && !Util.isNullOrNil(value)) { String[] field = key.split(" "); if (field.length == 2) { // Log.d(TAG, "%s -> %s", afterClass.replace('$', '.') + "." + value, getPureClassName(beforeClass).replace('$', '.') + "." + field[1]); //添加 R.java中混淆后的全路径field -> R.java混淆前的全路径field rclassProguardMap.put(afterClass.replace('$', '.') + "." + value, ApkUtil.getPureClassName(beforeClass).replace('$', '.') + "." + field[1]); } } } } } line = bufferedReader.readLine(); } } finally { bufferedReader.close(); } } } 复制代码
private void readResourceTxtFile() throws IOException { //读取R.txt BufferedReader bufferedReader = new BufferedReader(new FileReader(resourceTxt)); String line = bufferedReader.readLine(); try { while (line != null) { String[] columns = line.split(" "); if (columns.length >= 4) { final String resourceName = "R." + columns[1] + "." + columns[2]; if (!columns[0].endsWith("[]") && columns[3].startsWith("0x")) { //int styleable ActionBar_title 27 if (columns[3].startsWith("0x01")) { Log.d(TAG, "ignore system resource %s", resourceName); } else { final String resId = parseResourceId(columns[3]); if (!Util.isNullOrNil(resId)) { //资源id 资源名称 映射 resourceDefMap.put(resId, resourceName); } } } else { //int[] styleable ActionMode { 0x7f030034, 0x7f030036, 0x7f030056, 0x7f0300ad, 0x7f030168, 0x7f03019e } Log.d(TAG, "ignore resource %s", resourceName); if (columns[0].endsWith("[]") && columns.length > 5) { Set<String> attrReferences = new HashSet<String>(); for (int i = 4; i < columns.length; i++) { if (columns[i].endsWith(",")) { attrReferences.add(columns[i].substring(0, columns[i].length() - 1)); } else { attrReferences.add(columns[i]); } } //style映射 styleableMap.put(resourceName, attrReferences); } } } line = bufferedReader.readLine(); } } finally { bufferedReader.close(); } } 复制代码
解析dex文件中的smali代码:
private void decodeCode() throws IOException { for (String dexFileName : dexFileNameList) { DexBackedDexFile dexFile = DexFileFactory.loadDexFile(new File(inputFile, dexFileName), Opcodes.forApi(15)); BaksmaliOptions options = new BaksmaliOptions(); List<? extends ClassDef> classDefs = Ordering.natural().sortedCopy(dexFile.getClasses()); for (ClassDef classDef : classDefs) { String[] lines = ApkUtil.disassembleClass(classDef, options); if (lines != null) { readSmaliLines(lines); } } } } 复制代码
private void readSmaliLines(String[] lines) { if (lines == null) { return; } for (String line : lines) { line = line.trim(); if (!Util.isNullOrNil(line)) { if (line.startsWith("const")) { String[] columns = line.split(","); if (columns.length == 2) { final String resId = parseResourceId(columns[1].trim()); //从id获取资源名 if (!Util.isNullOrNil(resId) && resourceDefMap.containsKey(resId)) { resourceRefSet.add(resourceDefMap.get(resId)); } } } else if (line.startsWith("sget")) { String[] columns = line.split(" "); if (columns.length == 3) { //获取资源名称 final String resourceRef = parseResourceNameFromProguard(columns[2]); if (!Util.isNullOrNil(resourceRef)) { //Log.d(TAG, "find resource reference %s", resourceRef); if (styleableMap.containsKey(resourceRef)) { //reference of R.styleable.XXX for (String attr : styleableMap.get(resourceRef)) { resourceRefSet.add(resourceDefMap.get(attr)); } } else { resourceRefSet.add(resourceRef); } } } } } } } 复制代码
private String parseResourceNameFromProguard(String entry) { if (!Util.isNullOrNil(entry)) { // sget v6, Lcom/tencent/mm/R$string;->chatting_long_click_menu_revoke_msg:I // sget v1, Lcom/tencent/mm/libmmui/R$id;->property_anim:I // sput-object v0, Lcom/tencent/mm/plugin_welab_api/R$styleable;->ActionBar:[I // const v6, 0x7f0c0061 String[] columns = entry.split("->"); if (columns.length == 2) { int index = columns[1].indexOf(':'); if (index >= 0) { final String className = ApkUtil.getNormalClassName(columns[0]); final String fieldName = columns[1].substring(0, index); if (!rclassProguardMap.isEmpty()) { String resource = className.replace('$', '.') + "." + fieldName; if (rclassProguardMap.containsKey(resource)) { return rclassProguardMap.get(resource); } else { return ""; } } else { if (ApkUtil.isRClassName(ApkUtil.getPureClassName(className))) { return (ApkUtil.getPureClassName(className) + "." + fieldName).replace('$', '.'); } } } } } return ""; } 复制代码
UnusedAssetsTask
目的:检测出apk中未被使用的asset资源(代码实现仅覆盖了字符串常量的情况,会有遗留)。
@Override public TaskResult call() throws TaskExecuteException { try { TaskResult taskResult = TaskResultFactory.factory(type, TaskResultFactory.TASK_RESULT_TYPE_JSON, config); long startTime = System.currentTimeMillis(); File assetDir = new File(inputFile, ApkConstants.ASSETS_DIR_NAME); //找到所有asset文件 findAssetsFile(assetDir); generateAssetsSet(assetDir.getAbsolutePath()); Log.d(TAG, "find all assets count: %d", assetsPathSet.size()); //解析代码中的asset引用 decodeCode(); Log.d(TAG, "find reference assets count: %d", assetRefSet.size()); //移除被引用的资源 assetsPathSet.removeAll(assetRefSet); JsonArray jsonArray = new JsonArray(); for (String name : assetsPathSet) { jsonArray.add(name); } ((TaskJsonResult) taskResult).add("unused-assets", jsonArray); taskResult.setStartTime(startTime); taskResult.setEndTime(System.currentTimeMillis()); return taskResult; } catch (Exception e) { throw new TaskExecuteException(e.getMessage(), e); } } 复制代码
private void generateAssetsSet(String rootPath) { HashSet<String> relativeAssetsSet = new HashSet<String>(); for (String path : assetsPathSet) { int index = path.indexOf(rootPath); if (index >= 0) { String relativePath = path.substring(index + rootPath.length() + 1); //Log.d(TAG, "assets %s", relativePath); relativeAssetsSet.add(relativePath); if (ignoreAsset(relativePath)) { Log.d(TAG, "ignore assets %s", relativePath); //获取asset使用时的相对路径 assetRefSet.add(relativePath); } } } assetsPathSet.clear(); assetsPathSet.addAll(relativeAssetsSet); } 复制代码
private void readSmaliLines(String[] lines) { if (lines == null) { return; } for (String line : lines) { line = line.trim(); // invoke-virtual {p0}, Lcom/ss/android/alog/App;->getAssets()Landroid/content/res/AssetManager; //move-result-object v1 //const-string v2, "video" //invoke-virtual {v1, v2}, Landroid/content/res/AssetManager;->open(Ljava/lang/String;)Ljava/io/InputStream; //:try_end_13 //.catch Ljava/io/IOException; {:try_start_a .. :try_end_13} :catch_1a //这个const-string判断不是很完善,只能判断写死的值 if (!Util.isNullOrNil(line) && line.startsWith("const-string")) { String[] columns = line.split(","); if (columns.length == 2) { String assetFileName = columns[1].trim(); assetFileName = assetFileName.substring(1, assetFileName.length() - 1); if (!Util.isNullOrNil(assetFileName)) { //再判断这个常量是否在asset文件名集合中 for (String path : assetsPathSet) { if (path.endsWith(assetFileName)) { assetRefSet.add(path); } } } } } } } 复制代码
UnStrippedSoCheckTask
目的:检测出apk中未裁剪的so。
@Override public TaskResult call() throws TaskExecuteException { try { ... if (libDir.exists() && libDir.isDirectory()) { File[] dirs = libDir.listFiles(); for (File dir : dirs) { if (dir.isDirectory()) { File[] libs = dir.listFiles(); for (File libFile : libs) { if (libFile.isFile() && libFile.getName().endsWith(ApkConstants.DYNAMIC_LIB_FILE_SUFFIX)) { libFiles.add(libFile); } } } } } for (File libFile : libFiles) { //判断是否裁剪 if (!isSoStripped(libFile)) { Log.d(TAG, "lib: %s is not stripped", libFile.getName()); jsonArray.add(libFile.getName()); } } ((TaskJsonResult) taskResult).add("unstripped-lib", jsonArray); taskResult.setStartTime(startTime); taskResult.setEndTime(System.currentTimeMillis()); return taskResult; } catch (Exception e) { throw new TaskExecuteException(e.getMessage(), e); } } 复制代码
通过命令行判断so是否被裁剪
private boolean isSoStripped(File libFile) throws IOException, InterruptedException { ProcessBuilder processBuilder = new ProcessBuilder(toolnmPath, libFile.getAbsolutePath()); Process process = processBuilder.start(); BufferedReader reader = new BufferedReader(new InputStreamReader(process.getErrorStream())); String line = reader.readLine(); if (!Util.isNullOrNil(line)) { //Log.d(TAG, "%s", line); String[] columns = line.split(":"); if (columns.length == 3 && columns[2].trim().equalsIgnoreCase("no symbols")) { return true; } } reader.close(); process.waitFor(); return false; } 复制代码
CountClassTask
目的:统计类的数量。
@Override public TaskResult call() throws TaskExecuteException { try { TaskResult taskResult = TaskResultFactory.factory(type, TaskResultFactory.TASK_RESULT_TYPE_JSON, config); long startTime = System.currentTimeMillis(); Map<String, String> classProguardMap = config.getProguardClassMap(); JsonArray dexFiles = new JsonArray(); for (int i = 0; i < dexFileList.size(); i++) { RandomAccessFile dexFile = dexFileList.get(i); DexData dexData = new DexData(dexFile); dexData.load(); ClassRef[] defClassRefs = dexData.getInternalReferences(); Set<String> classNameSet = new HashSet<>(); for (ClassRef classRef : defClassRefs) { String className = ApkUtil.getNormalClassName(classRef.getName()); if (classProguardMap.containsKey(className)) { className = classProguardMap.get(className); } if (className.indexOf('.') == -1) { continue; } classNameSet.add(className); } JsonObject jsonObject = new JsonObject(); jsonObject.addProperty("dex-file", dexFileNameList.get(i)); //Log.d(TAG, "dex %s, classes %s", dexFileNameList.get(i), classNameSet.toString()); Map<String, Set<String>> packageClass = new HashMap<>(); if (JobConstants.GROUP_PACKAGE.equals(group)) { String packageName = ""; for (String clazzName : classNameSet) { packageName = ApkUtil.getPackageName(clazzName); if (!Util.isNullOrNil(packageName)) { if (!packageClass.containsKey(packageName)) { packageClass.put(packageName, new HashSet<String>()); } //按package聚合 packageClass.get(packageName).add(clazzName); } } JsonArray packages = new JsonArray(); for (Map.Entry<String, Set<String>> pkg : packageClass.entrySet()) { JsonObject pkgObj = new JsonObject(); pkgObj.addProperty("package", pkg.getKey()); JsonArray classArray = new JsonArray(); for (String clazz : pkg.getValue()) { classArray.add(clazz); } //单个package下的所有class pkgObj.add("classes", classArray); packages.add(pkgObj); } jsonObject.add("packages", packages); } dexFiles.add(jsonObject); } ((TaskJsonResult) taskResult).add("dex-files", dexFiles); taskResult.setStartTime(startTime); taskResult.setEndTime(System.currentTimeMillis()); return taskResult; } catch (Exception e) { throw new TaskExecuteException(e.getMessage(), e); } } 复制代码
总结
Matrix静态apk扫描部分的代码逻辑比较简单;初步理解dex文件格式、arsc文件格式之后,代码理解上就不会有太大的问题了。
以上就是本文的全部内容,希望对大家的学习有所帮助,也希望大家多多支持 码农网
猜你喜欢:- 以太坊源码分析(36)ethdb源码分析
- [源码分析] kubelet源码分析(一)之 NewKubeletCommand
- libmodbus源码分析(3)从机(服务端)功能源码分析
- [源码分析] nfs-client-provisioner源码分析
- [源码分析] kubelet源码分析(三)之 Pod的创建
- Spring事务源码分析专题(一)JdbcTemplate使用及源码分析
本站部分资源来源于网络,本站转载出于传递更多信息之目的,版权归原作者或者来源机构所有,如转载稿涉及版权问题,请联系我们。