Exploration of IOS a/b test scheme

Time:2022-6-9

Forward article:Exploration of IOS a/b test scheme

Introducer

At the end of 2016 and the beginning of 2017, in an Internet company that makes travel products, the product manager frantically raised the demand for a/btest, so that the company’s program apes turned pale when talking about ab. the evil product manager made the program apes terrified and miserable Cough, cough, pull away.

Recently, the team has made a lot of AB test business requirements. When such requirements are increasingly common, we have to improve our code organization to adapt to or better maintain our code on such requirements. Therefore, with this article, this article mainly expounds some ideas and ideas of the business team in doing AB test. They are lack of talent and learning, and are not able to teach.

A/B Test

What is a/b test?

Since the product manager is crazy about the output of a/b test, we need to figure out what is a/btest? Why are product managers so obsessed with a/b test?

A/b test is forSame goalformulateTwo options(for example, the pages of two websites and apps)Some usersUsing scheme a,Other usersUsing scheme B,recordThe user’s usage,seeWhich scheme is closer to the desired result of the test, and we are sure that the conclusion is credible when it is extended to all traffic.

Please pay attention to the bold words in the above paragraph, which will be the core value of AB test.

In fact, a/b test is a control test that we often do in chemical experiment class in middle school. This control test is moved to the Internet. By changing the experimental group with a single variable and the original control group, we can compare the data indicators to see which scheme can improve the user experience (conversion rate);

What are the advantages of AB test (for the product)?

Advantage 1 Grayscale Publishing

Grayscale publishing refers to a publishing method that can smoothly transition between black and white. A/b test is a grayscale publishing method, which allows some users to continue using a and some users to start using B. if users have no objection to B, gradually expand the scope and migrate all users to B. Gray level release can ensure the stability of the overall system. Problems can be found and adjusted at the initial gray level to ensure its impact.

Advantage 2 Reversible scheme

The reversible scheme is somewhat similar to the previous gray-scale release, except that the control power of non gray-scale is stronger. When we find that the scheme of the experimental group has a serious fault after release, or there is a great difference in the amount of comparison data, we can fully switch back to the original control group, ensuring the stability of the online environment and not affecting the normal use of users.

This is a possibility of trial and error for products. Consider that in the previous era of lack of APP dynamics, the release of app is the water thrown by the married daughter. Once gone, it is impossible for users of released products to go back to the previous version after updating. From this point on, product managers love a/b test!

Advantage 3 Data driven

Data driven, which I think is very important. In the current big data era in which user data is the business soil, a product is data-driven, which will be able to more forcefully support the full release of the product, and is also an important trump card for product managers to promote new solutions. Before the launch of a new product, either call it a reference competitor (no objection to plagiarism, plagiarism is the fastest means to catch up with competitors, but it is not a means to surpass), or open your mind and think that a new scheme or interactive experience can bring more conversion rates. This method is not explained by data. Only through the post evaluation after the project is launched can we determine whether the goal has been really achieved as the product manager wishes.

Through a/b test, it can determine which scheme is superior in a short time by comparing the illumination with the data of the test group under the condition that the normal operation of the line is not fully affected, so as to improve the credibility of the product conversion rate in a short time. This is also the essence of the product manager to convince the boss and demonstrate his ability value! So, great love!

Things that development engineers need to pay attention to

Vi. ask the product manager

Before AB test, there are several questions to ask the product manager:

  1. What is the goal?
  2. What is the AB version?
  3. How large is the sample size?
  4. How do users split?
  5. How long is the test?
  6. How to measure the effect?

This is actually the emphasis of the bold text in the above paragraph. Of course, some problems need to be concerned by the server, such as problems 3 and 4.

So what are the concerns of client development?

What is the goal?

First question, what is the goal? What is the purpose? This is what we need to ask. For the client, a/b test requires the client to maintain two sets of code for the same business. This workload is simply understood as the previous double. Since it will double the workload, we need to ask clearly what is the purpose of doing a/b test this time? Assess if it’s really worth it? Although sometimes the arm can’t wring the thigh, you may not need to do a/b test for some needs under your analysis. For example, the competitive products have made a scheme for a long time (don’t tell me that you are not confident in copying), or it is obvious that the UI changes are better than the previous scheme, and so on.

What is the a/b test version? How long is the test?

Second, what is the a/b test version? How long is the test? In fact, these two questions are to confirm when the a/b test scheme will go online and when it will go offline. We need to be clear about the time when we go online and offline, because during this period, we all need to maintain two sets of code. Moreover, in the context of such a tight app size and everyone’s slimming down, your installation package is too large or needs is the reason why users do not choose your product from the beginning! For a/b test scheme, the code is deleted as soon as it is written. When to delete the code depends on when the a/b test scheme is offline and how long it takes for QA test engineers to test after deleting the code. These are all to be arranged.

How to measure the effect?

For some development, you have to say five times or more every day: “this (demand) is to calculate the (R & D) cost.” In this way, the R & D cost is deducted, the demand with low value and low income is cut down as much as possible, and the demand with unclear income is ranked behind, which is equivalent to saving 2-3 development engineers on the basis of almost unchanged output. This is also the trick to maintain the team for a long time, to simplify from the source, rather than demanding superhuman programmers.

How to measure the effect is to judge whether this demand is a project with low value, low return or unclear. We all want to do something valuable, not a function that is ready to be cut off at any time. I hope the product manager will dare to think about it and think about it!

Exploration of IOS a/b test scheme

Well, after the product section, let’s get to the point.
Since the original set of code has two logics or two UI styles, it needs to be removed from the original logic. The inevitable result is that there is an additional if judgment statement. If there are more judgments, let’s do the same if, if, if, if, I It’s too out of standard. As the saying goes: writing business code and being good at it are the basic requirements of programmers. Next, let’s talk about Xiaosheng’s a/b test program exploration.

Program exploration process

Let’s briefly introduce the business background of this exploration:

A/b test scheme background introduction

  • Scheme a online scheme, full quantity;
  • Scheme B is a subset of scheme a, which is applicable to one case in a;
  • The non-standard a/b test is only a transition, because scheme a is a full-scale scheme and cannot be dropped. Scheme B is part of scheme a;

Let’s take the protocol functions of delegate and datasource in the typical uitabelview in IOS as the a/b scheme;

Basic functions a/b

Exploration of IOS a/b test scheme
Exploration of IOS a/b test scheme

As I just said, scheme a is a total scheme, so the switch here will have a default scheme. However, this way of writing is too low. Each calling function judges a/b once, which will affect the efficiency for the time being. Maintenance is also a pit. Seeing the function list page in the second figure, I think the header is large, and it also leads to the controller being too large. If there is another C scheme, it will explode? So this scheme is not desirable.

Method selector + dictionary, cached a/b

Exploration of IOS a/b test scheme
Exploration of IOS a/b test scheme

Due to the runtime dynamic characteristics of Objective-C, we can cache the method selection sub in a dictionary, judge once at the place where the a/b scheme needs to be determined, and get the method cache Dictionary of the corresponding scheme. When calling, we only need to call in the corresponding cache dictionary. Of course, we need to extend the- (id)performSelector:(SEL)aSelector withObject:(id)object;Make it support the passing of multiple parameters.

- (id)fperformSelector:(SEL)selector withObjects:(NSArray *)objects
{
    NSMethodSignature *methodSignature = [[self class] instanceMethodSignatureForSelector:selector];

    if(methodSignature == nil)
    {
        @Throw [nsexception exceptionwithname:@ "throw exception error" reason:@ "there is no such method, or the method name is wrong" userinfo:nil];
        return nil;
    }
    else
    {
        NSInvocation *invocation = [NSInvocation invocationWithMethodSignature:methodSignature];
        [invocation setTarget:self];
        [invocation setSelector:selector];
        //The number of method parameters in the signature, including self and_ CMD, so the parameter starts from the third
        NSInteger  signatureParamCount = methodSignature.numberOfArguments - 2;
        NSInteger requireParamCount = objects.count;
        NSInteger resultParamCount = MIN(signatureParamCount, requireParamCount);
        for (NSInteger i = 0; i < resultParamCount; i++) {
            id  obj = objects[i];
            [invocation setArgument:&obj atIndex:i+2];
        }
        [invocation invoke];
        //Return value processing
        id callBackObject = nil;
        if(methodSignature.methodReturnLength)
        {
            [invocation getReturnValue:&callBackObject];
        }
        return callBackObject;
    }
}

This scheme is only a little better than the previous scheme, that is, we do not judge a/b in each function, but only judge it once. However, the controller is still too large to be extended gracefully. It also introduces a new problem, that is, the additional overhead when forwarding runtime messages, andperformSelectorThe return value needs to be transferred to the awkward type.

Strategy mode of design mode

Exploration of IOS a/b test scheme
Exploration of IOS a/b test scheme
Exploration of IOS a/b test scheme

As shown in the figure, through the policy mode, the method that needs to be divided into a/b is abstracted into a protocol, and then a policy parent class is abstracted to follow the protocol, and its two a/b subclasses also follow the protocol. In this way, the controller only needs to initialize the corresponding policy class at the place where the a/b policy is judged to be called, and call the protocol method of the subclass through the parent class pointer to achieve the execution of the a/b function. In this way, the object-oriented inheritance and polymorphic mechanism are adopted to complete a perfect a/b function execution. The AB strategy can be switched freely, avoiding the use of multiple condition judgments. At the same time, the open and close principle is met. It is open to extensions (adding new policy classes) and closed to modifications.

Protocol protocol distributor, applied to a/b test scheme

Protocol distribution can be simply understood as handing over the protocol proxy to multiple object implementations, similar to multicast delegation.

Protocol proxy is frequently used in development. Developers often encounter a problem – continuous transmission of events. For example, in order to isolate the encapsulation, developers may often extract the delegate or datesource of tableview from independent objects. When other objects (such as VC) need to obtain some delegate events, they can only pass them through the second pass of events. Is there a simpler way? The protocol dispenser can come in handy.

Since the multicast delegated message distribution can be realized, isn’t the message of a/b test classified as a/b distribution by the specified distribution recipient during message distribution?

First of all, I’d like to present the dry goods to you,LJFABTestProtocolDispatcherIs a protocol distributor, which can easily distribute protocol events to multiple implementers and specify which implementers to call. For example, the most common uitableviewdelegate and uitableviewdatasource protocols areLJFABTestProtocolDispatcherIt can be easily distributed to multiple objects, and can specify a/b scheme implementation. For details, please refer toDemo

Principle analysis

The principle is not complicated. The protocol dispatcher does not implement the protocol protocol. It only needs to distribute the corresponding protocol events to different implementers. How to implement distribution?

Nsobject objects respond to unimplemented selector function calls mainly through the following functions

  • Scheme 1: dynamic analysis

    + (BOOL)resolveInstanceMethod:(SEL)sel;
    + (BOOL)resolveClassMethod:(SEL)sel;
    
    
  • Scheme 2: fast forwarding

    //Return the message forwarding object that implements the method
    - (id)forwardingTargetForSelector:(SEL)aSelector OBJC_AVAILABLE(10.5, 2.0,9.0, 1.0);
  • Scheme 3: slow forwarding

    //Function signature
    - (NSMethodSignature *)methodSignatureForSelector:(SEL)aSelector
    //Function call
    - (void)forwardInvocation:(NSInvocation *)anInvocation     OBJC_SWIFT_UNAVAILABLE("");

Therefore, the protocol dispatcher can pass the call of selector in protocol to implementer implemertor in this function, and implementer implemertor can implement the specific selector function, while the a/b call specified in reality needs to pass in the subscript of all implementer organizations to specify the call

/**
 The protocol dispatcher can pass the call of selector in protocol to implementer implemertor in this function, and implementer implemertor can implement the specific selector function
 */
- (void)forwardInvocation:(NSInvocation *)anInvocation
{
    SEL aSelector = anInvocation.selector;
    if (!ProtocolContainSel(self.prococol, aSelector))
    {
        [super forwardInvocation:anInvocation];
        return;
    }

    if (self.indexImplemertor)
    {
        for (NSInteger i = 0; i < [self.implemertors count]; i++)
        {
            ImplemertorContext *implemertorContext = [self.implemertors objectAtIndex:i];
            if (i == self.indexImplemertor.integerValue && [implemertorContext.implemertor respondsToSelector:aSelector])
            {
                [anInvocation invokeWithTarget:implemertorContext.implemertor];
            }
        }
    }
    else
    {
        for (ImplemertorContext *implemertorContext in self.implemertors)
        {
            if ([implemertorContext.implemertor respondsToSelector:aSelector])
            {
                [anInvocation invokeWithTarget:implemertorContext.implemertor];
            }
        }
    }
}

Design key

How to distribute only the selector function calls in the protocol is the key to the design. The system provides functions

objc_method_description protocol_getMethodDescription(Protocol *p, SEL aSel, BOOL isRequiredMethod, BOOL isInstanceMethod)

The following methods can be used to determine whether the selector belongs to a protocol

struct objc_method_description MethodDescriptionForSELInProtocol(Protocol *protocol, SEL sel) {
    struct objc_method_description description = protocol_getMethodDescription(protocol, sel, YES, YES);
    if (description.types) {
        return description;
    }
    description = protocol_getMethodDescription(protocol, sel, NO, YES);
    if (description.types) {
        return description;
    }
    return (struct objc_method_description){NULL, NULL};
}

BOOL ProtocolContainSel(Protocol *protocol, SEL sel) {
    return MethodDescriptionForSELInProtocol(protocol, sel).types ? YES: NO;
}

Also, the protocol dispenser is not a singleton, but a local variable. How can we prevent the delayed release of a local variable? The idea of “self release” is used here. Look at the source code:

- (instancetype)initWithProtocol:(Protocol *)protocol
            withIndexImplemertor:(NSNumber *)indexImplemertor
                  toImplemertors:(NSArray *)implemertors
{
    if (self = [super init])
    {
        self.prococol = protocol;
        self.indexImplemertor = indexImplemertor;
        NSMutableArray *implemertorContexts = [NSMutableArray arrayWithCapacity:implemertors.count];
        [implemertors enumerateObjectsUsingBlock:^(id implemertor, NSUInteger idx, BOOL * _Nonnull stop){
            ImplemertorContext *implemertorContext = [ImplemertorContext new];
            implemertorContext.implemertor = implemertor;
            [implemertorContexts addObject:implemertorContext];
            //Why is a protocoldispatcher attribute associated?
            //"Self release". The protocoldispatcher is not a singleton, but a local variable. When the implementor is released, it will trigger the release of the protocoldispatcher.
            //The key must be random. Otherwise, when there are two distributors, the key will be overwritten and the first distributor will be released. So key =_ CMD does not work.
            void *key = (__bridge void *)([NSString stringWithFormat:@"%p",self]);
            objc_setAssociatedObject(implemertor, key, self, OBJC_ASSOCIATION_RETAIN_NONATOMIC);
        }];
        self.implemertors = implemertorContexts;
    }
    return self;
}

matters needing attention

The protocol distributor needs to know how to handle functions with return values, such as

- (NSInteger)tableView:(UITableView *)tableView numberOfRowsInSection:(NSInteger)section

We know that in IOS, the result returned by function execution exists in register R0, and the later execution will overwrite the first execution result. Therefore, when a function with a return result is encountered, the return result of the function executed after the return result is the final value.

thank

The protocol protocol dispenser was not invented by me. I also read this articleProtocol distributorGet the inspiration to use in a/b test. Thank the author and the open source community here.

Exploration of a/b test components in business modules

With more and more a/b test codes, the Componentization of a/b test in the business module is nothing more than to facilitate the up and down business a/b test codes, improve work efficiency, and make writing and deleting codes a happy thing.

There are also many articles about IOS componentization on the Internet. There is no need to cook cold rice here. You can search for some definitions and experiences about componentization.

Now that the entire client has been componentized, can the business programmers who are not in the architecture group try to solve the Componentization of a/b test in the business module? Most of IOS componentization is carried out around cocoapods. Therefore, under the framework of highly componentized IOS based on cocoapods, let’s ask some technical questions first.

Can different static libraries of the same architecture be merged?

This problem is mainly based on the current entire client architecture. Each business line provides its own static library to the shell project,
Most of the time (during packaging) we will merge the same static libraries of different architectures. Can we merge different static libraries of the same architecture?

The answer is yes.

When merging the same static libraries of different architectures, use the following commands:

  • View the CPU architectures supported by the static library

    lipo -info libname. A (or libname.framework/libname)
  • Merge static libraries

    Lipo -create static inventory drop path 1 static inventory drop path 2- Output storage path after integration
  • Static library split

    Lipo static library source file path -thin CPU architecture name -output file storage path after splitting

So how do you merge different static libraries of the same architecture?

Static library files, also known as “document files”, are some O collection of files. Use the tool “ar” to maintain and manage it in Linux (Unix). It contains several members O documentation. Except O file, and a special member whose name is__.SYMDEF。 It contains valid symbols (function name, variable name) defined by all members in the static library. Therefore, when you add a member to the library, you need to update the member accordingly__.SYMDEFOtherwise, all symbols defined in the added member will not be located by the connector. The command to complete the update is:

ranlib libname.a

for instance:
We have two static librarieslibFlight.aandlibHotel.a, merge into onelibFlight_Hotel.a

  • Take out lib a。
    First view the static libraryFlight.aArchitecture of:

    lipo -info Flight.a
    
    

    You can see:

    Input file /users/f.li/desktop/ consolidation of different static libraries of the same architecture /libflight a is not a fat file
    Non fat file: /users/f.li/desktop/ consolidation of different static libraries of the same architecture /libflight a is architecture: x86_ sixty-four

    libFlight. A is not a fat file and libflight a is architecture: x86_ sixty-four

    Fat file means that the package supports multiple platforms. Not a fat file does not support multiple platforms. The architecture is x86_ 64。

    Of course, if it is a fat file, we need to take out the library of the same platform architecture.

    lipo libFlight.a -thin x86_64 -output libFlight.a
    
    

    This will remove x86_ 64 architecturelibFlight.a

  • View the list of files contained in the library.

    AR -t /users/f.li/desktop/ consolidation of different static libraries of the same architecture /libflight a
    __.SYMDEF SORTED
    Flight.o

    noticelibFlight.aThere are two files,__.SYMDEF SORTEDandFlight.o

  • Extract the object file (i.e. the.O suffix file).

    ~libFlight_o ar xv /Users/f.li/Desktop/libFlight.a
    x - __.SYMDEF SORTED
    x - Flight.o
    
    

    So, inlibFlight_oIn the folder__.SYMDEF SORTEDandFlight.oThese two files.
    Again, inlibHotel_oGet in folder__.SYMDEF SORTEDandHotel.o

  • Merge and repack.
    hold__.SYMDEF SORTEDandFlight.o, andHotel.oMove tolibFlight_Hotel_oFolder. Repackage the object file;

    ar rcs libFlight_Hotel.a /Users/f.li/Desktop/libFlight_Hotel_o/*o
    
    

    So you getlibFlight_Hotel.a

  • to update__.SYMDEFDocuments.
    Actually, we putHotel.oJoined libflight A. finally, it needs to be updated__.SYMDEFDocuments.

    ranlib libFlight_Hotel.a
    
    

    If the header file is included, put the header file into a file and use libflight_ Hotel. A project.

But this is obviously too much trouble.

Xcode subproject?

Xcode subproject is actually to help us cooperate with git submodule to carry out modular development in a project.
Organize your thoughts.

  • Create an Xcode project with the target (flight\u hotel\u project) as the application as the parent project. And gitization.
  • Create an Xcode project with a tag (flight\u subproject) of static library as a subproject and gitize it.
  • Add git submodule for the parent project. Refer to git for details.
  • Drag the child project folder into the parent project.
  • Add flight in the link binary with library of the parent project_ SubProject. a
  • Add the header file search path in the header search paths of the parent project$(SRCROOT)/Flight_SubProject/Flight_SubProjectWhere $(srcroot) macro represents your project file directory.
  • Compile and run.

In this way, it actually returns to the previous state of the architecture. It is unable to call decoupling and has serious interdependence.

What is the concept of subspecs in cocoapods? Does subspec have its own independent git repository? Is it a child pod that can be understood as pod?

The answer is that subspec is not an independent code base, but is compiled separately, and finally forms a product with pod.

Why ask cocoapods subspecs? Because after componentization based on cocoapods architecture, the business provides a static library type pod to the outside.

The source code type is subspec. When importing pod, you can choose to import the subspec directory, or set the default subsepc of podspec. There can also be dependencies between subspecs.

What is the final solution?

The internal split of the business line can be made into multiple pods. Finally, a pod is provided that depends on all the internal components of the business. This does not affect the packaging of the external architecture, and the business line can be flexibly modified.

Finally, the pod, which depends on all internal components of the business, provides a static library externally. The internal component pod of the business does not need to provide a static library, but it also has an independent GIT.

Of course, this internal a/b test componentization scheme is currently in the exploration stage, because the code volume of our a/b test has not reached the point where we need to split it. All this stage is still in the stage of technology development and research.

Summary

There are so many explorations about IOS a/b test in Xiaosheng at present. A/b test is indeed a good solution for products, especially reversible and data-driven. Of course, Xiaosheng views a/b test from the perspective of development. Since it is a favorable solution for the product, our code should be the trend of the times. After all, technology serves the business.

While watching sunny live broadcast some time ago, I talked about the advanced speed of IOS development

Pure daily development < pure reading, blog < self experiment, demo < blogging < systematic sharing and discussion < provide a complete open source solution

In the past, my advanced speed only reached the stage of blogging. In the past six months, I have launched a wave of technology sharing and team blogging in the team. I hope I can score against the two high stages of systematic sharing, discussion and complete open source solutions. This time, I have completed a scoring attempt in combination with my recent business and some of my own ideas and practices. I hope I can be more courageous in scoring!