Swift bottom layer exploration (III): pointer

Time:2021-12-8

IOS memory partition

Swift bottom layer exploration (III): pointer

image.png

Stack

The stack area is to store the current local variables and the context during the operation of the function.

func test() {
    var age: Int = 10
    print(age)
}

test()
(lldb) po withUnsafePointer(to: &age){print($0)}
0x00007ffeefbff3d8
0 elements

(lldb) cat address 0x00007ffeefbff3d8
&0x00007ffeefbff3d8, stack address (SP: 0x7ffeefbff3b0 FP: 0x7ffeefbff3e0) SwiftPointer.test() -> ()
(lldb)

Directly define a local variable, and thenlldbTake a look and you can see that it is aStack address

Heap

For heap space, throughnew & mallocKeyword to apply for memory space, discontinuous, similar to linked list data structure.

class HotpotCat {
    var age: Int = 18
}

func test() {
    var hotpot = HotpotCat()
    print(hotpot)
}

test()
(lldb) po hotpot
<HotpotCat: 0x101157190>

(lldb) cat address 0x101157190
&0x101157190, (char *) $4 = 0x00007ffeefbff250 "0x101157190 heap pointer, (0x20 bytes), zone: 0x7fff88aa8000"
  • The hotpot variable is stored in the stack area
  • The address stored in the hotpot variable is located in the heap

Global area (static area)

//Global initialized variables
int age = 10;
//Global uninitialized variable
int age1;

//Global static variable
static int age2 = 30;

int main(int argc, const char * argv[]) {
    // insert code here...
    char *p = "Hotpot";
    printf("%d",age);
    printf("%d",age1);
    printf("%d",age2);
    return 0;
}
(lldb) po &age
0x0000000100008010

(lldb) cat address 0x0000000100008010
&0x0000000100008010,  age <+0> CTest.__DATA.__data
(lldb) 

ageLocated in executable__DATA.__datain

Swift bottom layer exploration (III): pointer

image.png
(lldb) po &age1
0x0000000100008018

(lldb) cat address 0x0000000100008018
&0x0000000100008018,  age1 <+0> CTest.__DATA.__common

age1be located__DATA.__commonYes. Here, uninitialized and initialized variables are stored separately (more finely divided) to facilitate symbol positioning.

//If age2 is not called, Po & will not find the address.
(lldb) po &age2
0x0000000100008014

(lldb) cat address 0x0000000100008014

age2be located__DATA.__dataYes.
Look closely at the addresses of the three variables and you can seeageandage2Close to the forehead. This verifies that initialized and uninitialized ratio variables are stored globally.age1The address is relatively large, so
In the global area, initialized variables are located below uninitialized variables (by address size)

&age
0x0000000100008010
 &age1
0x0000000100008018
age2
0x0000000100008014

Constant area

//Global static constant
static const int age3 = 30;

int main(int argc, const char * argv[]) {
    int age4 = age3;
    return 0;
}
(lldb) po &age3
0x00000001004f8440

(lldb) cat address 0x00000001004f8440
Couldn't find address, reverting to "image lookup -a 0x00000001004f8440"
(lldb) 

Following the example above, define astatic constEmbellishedage3, it is found that the address cannot be found, and the breakpoint debugging finds that the direct assignment is30。 That is, Noage3This symbol directly stores30This value.

Swift bottom layer exploration (III): pointer

image.png

Then remove itstatic

(lldb) po &age3
0x0000000100003fa0

(lldb) cat address 0x0000000100003fa0
&0x0000000100003fa0,  age3 <+0> CTest.__TEXT.__const

findage3stay__TEXT.__const

Swift bottom layer exploration (III): pointer

image.png

Verify again*p

Swift bottom layer exploration (III): pointer

image.png

__text

Is the instruction we are currently executing.

stayswiftDuring debugging, the global variables are initialized before__commonSegment, after initialization or in__commonParagraph, it should be in__dataParagraph.

Summary: zoning (artificial and general division) andmachoA file is not a thingsegmentofsectionIn, the symbol division is finer). It can be simply understood asWindowsandMacOS

Method scheduling

Structure static call (direct call). After the compilation is completed, the address of the method is determined. In the process of executing the code, it directly jumps to the address execution method. For methods in a class, stored inV-TableDuring the execution, goV-tableFind it in.

V-table (function table)

V-tableIs an array structureExtension is called directlystaySILIt is expressed in this way:

decl ::= sil-vtable
sil-vtable ::= 'sil_vtable' identifier '{' sil-vtable-entry* '}'
sil-vtable-entry ::= sil-decl-ref ':' sil-linkage? sil-function-name

identifierAn identifier is a class,sil-decl-refStatement,sil-function-nameMethod name.
Look directly at oneSILfile

class HotpotCat {
  func test()
  func test1()
  func test2()
  func test3()
  init()
  @objc deinit
}

sil_vtable HotpotCat {
  #HotpotCat.test: (HotpotCat) -> () -> () : @main.HotpotCat.test() -> ()   // HotpotCat.test()
  #HotpotCat.test1: (HotpotCat) -> () -> () : @main.HotpotCat.test1() -> () // HotpotCat.test1()
  #HotpotCat.test2: (HotpotCat) -> () -> () : @main.HotpotCat.test2() -> () // HotpotCat.test2()
  #HotpotCat.test3: (HotpotCat) -> () -> () : @main.HotpotCat.test3() -> () // HotpotCat.test3()
  #HotpotCat.init!allocator: (HotpotCat.Type) -> () -> HotpotCat : @main.HotpotCat.__allocating_init() -> main.HotpotCat    // HotpotCat.__allocating_init()
  #HotpotCat.deinit!deallocator: @main.HotpotCat.__deallocating_deinit  // HotpotCat.__deallocating_deinit
}

sil_vtableIs a keyword,HotpotCatRepresentative isHotpotCat classFunction table for. This table is essentially an array, which is declared in theclassInternal methods are continuously stored in our current address space without any keyword modification.

Breakpoint observation

Visually observe the breakpoint. First, get familiar with the assembly instructions:

Arm64 assembly instruction

  • BLR: jump instruction with return to the address saved in the following register after the instruction
  • Mov: copy the value of one register to another register (only forRegister and register/Registers and constantsPass value between, cannot be used for memory address)
    For example:mov x1,x0Registerx0The value of is copied to the registerx1Yes.
  • LDR: read the value in memory into the register
    For example:ldr x0, [x1, x2]Registerx1And registerx2Add as the address, take the value of the memory address and put it intox0Yes.
  • STR: writes the value in the register to memory
    For example:str x0, [x0, x8]Registerx0Save values to memory[x0 + x8]Office.
  • BL: jump to an address

Direct breakpoint debugging

Swift bottom layer exploration (III): pointer

image.png

The x9 address is obtained from the X8 offset 0x60. Read the X8 content, and X8 is hotpot

(lldb) register read x8
      x8 = 0x00000002837ad610
(lldb) x/8g 0x00000002837ad610
0x2837ad610: 0x0000000104b4dca8 0x0000000200000003
0x2837ad620: 0x000098b3fd5dd620 0x00000002097b0164
0x2837ad630: 0x000098b3fd5dd630 0x0000000209770165
0x2837ad640: 0x000098b3fd5dd640 0x0000000209770167
(lldb) 
Swift bottom layer exploration (III): pointer

image.png
  • Hotpot address offset X60 and jump. X60 is test2

    Swift bottom layer exploration (III): pointer

    image.png

    You can see that test, test1, test2 and test3 are consecutive addresses, which verifies that the methods in class are stored continuously.

Source code observation

searchinitClassVTableAnd make a breakpoint

Swift bottom layer exploration (III): pointer

image.png
  if (description->hasVTable()) {
    auto *vtable = description->getVTableDescriptor();
    auto vtableOffset = vtable->getVTableOffset(description);
    auto descriptors = description->getMethodDescriptors();
    for (unsigned i = 0, e = vtable->VTableSize; i < e; ++i) {
      auto &methodDescription = descriptors[i];
      swift_ptrauth_init(&classWords[vtableOffset + i],
                         methodDescription.Impl.get(),
                         methodDescription.Flags.getExtraDiscriminator());
    }
  }

As you can see from the source code, it is a for loop that writes the v-table to memory. Sizevtable->VTableSize, offset isvtableOffset。 fromdescription->getMethodDescriptors()Get putclassWordsYes.

Method call in extension

extension HotpotCat {
    func test4() {
        print("test4")
    }
}

Swift bottom layer exploration (III): pointer

image.png

As you can see, it becomes a direct call.
why?

class HotpotCat {
    func test() {
        print("test")
    }
    func test1() {
        print("test1")
    }
    func test2() {
        print("test2")
    }
    func test3() {
        print("test3")
    }
}

extension HotpotCat {
    func test4() {
        print("test4")
    }
}

class HotpotCatChild: HotpotCat {
    func test5() {
        print("test5")
    }
}

Hotpotcatchild inherits from hotpotcat. Let’s take a look at the SIL file directly

Swift bottom layer exploration (III): pointer

image.png

You can see that hotpotcatchild inherits all hotpotcat methods and inserts its own method after the parent methodtest5

If the extension is also placed in the v-table, the extension may be placed in any file. When compiling, the extension is only known when it is compiled. At this time, the method in the extension can be inserted into the parent class, but cannot be inserted into the child class v-table. Because there is no pointer to record which block is the parent method and which block is the child method, and there may be many inheritance levels, it is unrealistic to add pointer records. (OC is synthesized by moving memory addresses).

OC method list synthesis

This is why the OC category method “overrides” the method with the same name.

Swift bottom layer exploration (III): pointer

image.png
  • V-table is an array structure
  • The method in extension becomes a static call

Difference between static call and v-table

  • Static call direct call address.
  • V-table needs to find the base address first, and then find the function address according to the offset.
    Call speed:Static call > v-table > dynamic call

SILuseclass_method, super_method, objc_method, andobjc_super_methodOperation to implement the dynamic dispatch of class methods

There are also overridetable, PWT (protocol witness table) and vwt (value witness table). Vwt and PWT manage the memory management (initialization, copy, destruction) and method calls of protocol type instances through division of labor.

final

The final modification method becomes a direct call.

  • Final modifies a class (can only modify a class), and this class cannot be inherited;
  • Final modifies the method, which cannot be overridden;
  • Static and final cannot be used together. Static modifies final

@objc

Marked for OC call, and the scheduling method is not modified.
Because swift is a static language, OC is a dynamic language. In the hybrid project, in order to ensure security, the methods in swift that may be called by OC are marked with @ objc.

  • @Objc tag
  • Inherited from nsobject. (in swift3, the compiler will add @ objc to all methods inherited from nsobject by default. After swift4, it will only add @ objc to functions that implement OC interfaces and override OC methods)

In this way, you can optimize and delete unused swift functions and reduce the package size.
https://www.hangge.com/blog/cache/detail_1839.html

The compiler generates the conversion file itself

Swift bottom layer exploration (III): pointer

image.png
SWIFT_CLASS("_TtC7SwiftOC9HotpotCat")
@interface HotpotCat : NSObject
- (void)test1;
- (nonnull instancetype)init OBJC_DESIGNATED_INITIALIZER;
@end
class HotpotCat {
    @objc func test(){
        print("test")
    }
}

var hotpot = HotpotCat()
hotpot.test()

SIL file to see what is done in this process

Swift bottom layer exploration (III): pointer

image.png

@The objc marked method generates two methods, one is the original method of swift, and the other is to generate the method for OC to call. The method called by OC calls the method of swift internally
More details:https://blog.csdn.net/weixin_33816300/article/details/91370145

dynamic

class HotpotCat {
    dynamic func test(){
        print("test")
    }
}

var hotpot = HotpotCat()
hotpot.test()
Swift bottom layer exploration (III): pointer

image.png

You can see that it is still function table scheduling.

  • Dynamic does not change the scheduling mode
  • The dynamic attribute enables the dynamic forwarding function of objc;
  • Dynamic is only used for classes, not for structs and enumerations, because they have no inheritance mechanism, and objc’s dynamic forwarding implements forwarding according to the inheritance relationship.
  • KVO can also be used in swift, but only in the subclasses of nsobject. This is understandable because KVO is implemented based on KVC (key value coding) and dynamic distribution technology, which are the concepts of Objective-C runtime. In addition, because swift disables dynamic dispatch by default for efficiency, we need to do additional work to implement KVO with swift, that is, mark the objects we want to observe as dynamic.
  • The @ dynamic keyword in OC tells the compiler not to synthesize getter and setter methods for attributes.
  • The dynamic keyword in swift can be used to modify variables or functions. Its meaning is completely different from OC. It tells the compiler to use dynamic distribution instead of static distribution.
  • Variables / functions marked as dynamic will implicitly add the @ objc keyword, which will use OC’s runtime mechanism.
    https://www.cnblogs.com/feng9exe/p/9084788.html
    https://www.jianshu.com/p/49b8e6f6a51d

Swift replacement

class HotpotCat {
    dynamic func test(){
        print("test")
    }
}

extension HotpotCat {
    @_dynamicReplacement(for: test)
    func test1(){
        print("test1")
    }
}

class ViewController: UIViewController {

    override func viewDidLoad() {
        super.viewDidLoad()
        var hotpot = HotpotCat()
        hotpot.test()
    }
}

lldb
test1

Calling test here actually calls test1.

Since methods can be replaced, can attributes be replaced?
First of all, be clearextensionStorage properties cannot be added to. Only calculated properties can be added. To change the behavior of a property is to change itsetandgetJust do it. So this adds a storage property to the extension and binds it toageUpper:

class HotpotCat {
    dynamic var age: Int = 18
    dynamic func test() {
        print("test")
    }
}

extension HotpotCat {
    @_dynamicReplacement(for: age)
    var age2: Int {
        set {
            age = newValue
        }
        get {
            10
        }
    }
    @_dynamicReplacement(for: test)
    func test1() {
        print("test1")
    }
}
var hotpot = HotpotCat()

hotpot.test()

print(hotpot.age)

output

test1
10
(lldb) 

@objc+dynamic

class HotpotCat {
    @objc  dynamic func test(){
        print("test")
    }
}
Swift bottom layer exploration (III): pointer

image.png

You can see that the scheduling mode has changed to dynamic forwarding.

Pointer

SwiftThe middle pointer is divided intoraw pointerData type and pointer are not specifiedtyped pointerSpecifies a pointer to the data type.

  • raw pointerexpressUnsafeRawPointer
  • typed pointerexpressUnsafePointer<T>

Correspondence between Swift pointer and OC:

Swift Objective-c explain
unsafeRawPointer const void * The pointer and the memory content pointed to (unknown) are immutable
unsafeMutableRawPointer void * Pointer to unknown content
unsafePointer<T> const T * The pointer and what it refers to are immutable
unsafeMutablePointer<T> T * The pointer and the memory content pointed to can be changed

Raw pointer

If we want to store four consecutive shaped data in memory, useraw pointerTo handle

//Manually manage the memory. Allocate should match deallocate
let p = UnsafeMutableRawPointer.allocate(byteCount: 32, alignment: 8)

for i in 0 ..< 4 {
    //Advanced step size, equivalent to p forward I * 8, storing I + 1
    p.advanced(by: i * 8).storeBytes(of: i + 1, as: Int.self)
}

for i in 0 ..< 4 {
    let value = p.load(fromByteOffset: i * 8, as: Int.self)
    print("index:\(i),value:\(value)")
}

p.deallocate()

UnsafeMutableRawPointerSource code

Swift bottom layer exploration (III): pointer

image.png

BuiltinSwift standard module, matchingLLVMThe types and methods of reduce the burden of swift runtime.

Typed pointer (specify data type pointer)

typed pointerSimple usage
Mode 1

var age = 18

//The value of P is determined by the return value of the closure expression
let p = withUnsafePointer(to: &age) {$0}
//Pointee is equivalent to * P = age
print(p.pointee)

//Pointee cannot be modified in unsafepointer
age = withUnsafePointer(to: &age) { p in
    p.pointee + 10
}
print(age)
//Pointee can be modified in unsafemutablepointer
withUnsafeMutablePointer(to: &age) { p in
    p.pointee += 10
}

print(age)

Mode II

//A capacity of 1 means 8 bytes
let ptr = UnsafeMutablePointer<Int>.allocate(capacity: 1)
//Initialize corresponds to deinitialize and allocate corresponds to deallocate
ptr.initialize(to: age)

ptr.deinitialize(count: 1)

ptr.pointee += 10

print(ptr.pointee)

ptr.deallocate()

You can use either value or create, except the first methodageThe value of is changed in the second wayageThe value has not been changed.

Pointer to create structure

struct HPTeacher {
    var age = 10
    var height = 1.85
}

//Create hpteacher with pointer
let ptr = UnsafeMutablePointer<HPTeacher>.allocate(capacity: 2)
ptr.initialize(to: HPTeacher())
//Here advanced is 1 because PTR here is type pointer. Know the type, we don't need to pass the size. Just pass the steps.
ptr.advanced(by: 1).initialize(to: HPTeacher(age: 18, height: 1.80))

print(ptr[0])
print(ptr[1])

print(ptr.pointee)
print((ptr + 1).pointee)
//The essential purpose of success is to move forward equivalent (PTR + 1). Pointee
print(ptr.successor().pointee)

ptr.deinitialize(count: 2)
ptr.deallocate()
  • The memory management of pointers requires manual management,allocateAnddeallocateMatch,initializeAnddeinitializeMatch.

application

Heapobject structure

struct HeapObject {
    var kind: UnsafeRawPointer
    var strongRef: UInt32
    var unownedRef: uint32
}

class HPTeacher {
    var age = 18
}

//Instance variable memory address
var t = HPTeacher()

//let ptr1 = withUnsafePointer(to: &t){$0}
//print(t)//SwiftPointer.HPteacher
//print(ptr1)//0x00000001000081e0
//print(ptr.pointee)//SwiftPointer.HPteacher

//What you get here is the value of the instance object. Both raw pointer and type pointer need to manage the memory by ourselves. Unmanaged can host pointers.
//Passunretained here is somewhat similar to the interaction between OC and C__ Conversion of bridge ownership.
//Toopaque returns an opaque pointer unsafemutablerawpointer
let ptr = Unmanaged.passUnretained(t as AnyObject).toOpaque()
//Rebind raw pointer to unsafemutablepointer of heapobject
//Bindmemory changes the pointer type of PTR and binds it to the specific memory pointer of heapobject. If PTR is not bound, it will be bound to heapobject for the first time. If it is bound, it will be re bound.
let heapObject = ptr.bindMemory(to: HeapObject.self, capacity: 1)

print(heapObject.pointee.kind)
print(heapObject.pointee.strongRef)
print(heapObject.pointee.unownedRef)

heretandptrDifferences between:

(lldb) po withUnsafePointer(to: &t){print($0)}
0x00000001000081e0
0 elements

(lldb) po ptr
▿ 0x0000000101817860
  - pointerValue : 4320229472

(lldb) x/8g 0x00000001000081e0
0x1000081e0: 0x0000000101817860 0x0000000101817860
0x1000081f0: 0x0000000101817860 0x0000000000000000
0x100008200: 0x0000000000000000 0x0000000000000000
0x100008210: 0x0000000000000000 0x0000000000000000
(lldb) 

amount toptrpointtMemory address ofmetadataAddress.

Class structure

struct HeapObject {
    var kind: UnsafeRawPointer
    var strongRef: UInt32
    var unownedRef: uint32
}

struct hp_swift_class {
    var kind: UnsafeRawPointer
    var superClass:UnsafeRawPointer
    var cacheData1:UnsafeRawPointer
    var cacheData2:UnsafeRawPointer
    var data:UnsafeRawPointer
    var flags:UInt32
    var instanceAddressOffset:UInt32
    var instanceSize:UInt32
    var instanceAlignMask:UInt16
    var reserved:UInt16
    var classSize:UInt32
    var classAddressOffset:UInt32
    var description:UnsafeRawPointer
}

class HPTeacher {
    var age = 18
}

var t = HPTeacher()
//Get pointer to hpteacher
let ptr = Unmanaged.passUnretained(t as AnyObject).toOpaque()
//Bind PTR to heapobject (instance object)
let heapObject = ptr.bindMemory(to: HeapObject.self, capacity: 1)
//Bind kind (metadata) of heapobject to HP_ swift_ Class. The memory structure here is essentially the same, so you can get the data correctly.
let metaPtr = heapObject.pointee.kind.bindMemory(to: hp_swift_class.self, capacity: 1)

print(metaPtr.pointee)

You can see the printing information as follows:

hp_swift_class(kind: 0x0000000100008140, superClass: 0x00007fff88a6f6f8, cacheData1: 0x00007fff201d3af0, cacheData2: 0x0000802000000000, data: 0x000000010054b672, flags: 2, instanceAddressOffset: 0, instanceSize: 24, instanceAlignMask: 7, reserved: 0, classSize: 136, classAddressOffset: 16, description: 0x0000000100003c3c)

Pointer conversion

Tuple pointer conversion

var tul = (10,20)

func testPointer(_ p : UnsafePointer<Int>) {
    print(p.pointee)
    print(p[1])
}


withUnsafePointer(to: &tul) { (tulPtr : UnsafePointer<(Int,Int)>) in
    //Assumingmemorybound assumes memory binding and tells the compiler that tulptr has bound int.self type, so there is no need to compile and check.
    testPointer(UnsafeRawPointer(tulPtr).assumingMemoryBound(to: Int.self))
}
10
20
(lldb) 

The reason for successful binding here is that tuples also store data continuously in memory. The < int, int > tuples here are actually stored continuously for 8 bytes in memory. The application scenario is that some interfaces need to be converted when they are incompatible.

Structure pointer conversion

The principle is to pass the address of the first int attribute.

struct HeapObject {
    var strongRef: Int
    var unownedRef: Int
}

func testPointer(_ p : UnsafePointer<Int>) {
    print(p.pointee)
}

var hp = HeapObject(strongRef: 10, unownedRef: 20)

withUnsafeMutablePointer(to: &hp) { (ptr : UnsafeMutablePointer<HeapObject>) in
    //1. By taking the address of strongref, PTR and strongrefptr are the same. It should be noted that the outer layer should be changed to withunsafemutablepointer, otherwise ptr.pointee.strongref cannot be modified&
//    let strongRefPtr = withUnsafePointer(to: &ptr.pointee.strongRef){$0}
//    testPointer(strongRefPtr)
    //2. Pass the advance step of advance 0. Since the PTR step of 1 skips HP, it needs to be converted to unsaferawpointer and then advanced
//    let strongRefPtr = UnsafeRawPointer(ptr).advanced(by: 0)
//    testPointer(strongRefPtr.assumingMemoryBound(to: Int.self))
    
    //3. PTR address + offset, where PTR can be directly transmitted. Because the first attribute of the heapobject structure address is strongref
    //The parameter of offset is keypath. If the path is changed to unownedref, the value of unownedref will be output.
//    let strongRefPtr = UnsafeRawPointer(ptr) + MemoryLayout<HeapObject>.offset(of: \HeapObject.strongRef)!
//Testpointer (strongrefptr. Assumpingmemorybound (to: int.self)) // equivalent to 4 here
   //4. Direct PTR address
    testPointer(UnsafeRawPointer(ptr).assumingMemoryBound(to: Int.self))
}

withMemoryRebound

withMemoryReboundIt is used to temporarily change the pointer type. It is used when we do not want to modify the pointer type.

var age = 10
func testPointer(_ value: UnsafePointer<UInt64>){
    print(value.pointee)
}

let ptr = withUnsafePointer(to: &age){$0}
//Temporarily modify the PTR type, only when the closure expression is unsafepointer < Uint64 >, and the scope is still unsafepointer < int >.
ptr.withMemoryRebound(to: UInt64.self, capacity: 1) { (ptr: UnsafePointer<UInt64>) in
    testPointer(ptr)
}

Temporarily modify the PTR type, only when the closure expression is unsafepointer < Uint64 >, and the scope is still unsafepointer < int >.

  • withMemoryRebound: temporarily changes the memory binding type.
  • bindMemory<T>(to type: T.Type, capacity count: Int): change the type of memory binding. If there is no binding before, it is the first binding; If it is bound, it will be re bound to this type.
  • assumingMemoryBound: assuming memory binding, this is to tell the compiler that the type has been bound, and there is no need to compile and check.

Memory operations are unsafe and need to be managed by ourselves.

reference resources:
https://www.jianshu.com/p/358f0cd7d823

Recommended Today

Simple use of Xray scanner

Introduction to Xray   Xray It is a powerful security assessment tool (official website address:https://xray.cool/), Xray community is a free white hat tool platform launched by Changting technology. At present, Xray vulnerability scanner and radius crawler tool are available in the community. In August 2020, the rad browser crawler was released, and the advanced version of Xray […]