Sonic cloud real machine directly obtains poco control actual combat

Time:2022-5-11

preface

Sonic official website
Community user communication

Effect display

Sonic cloud real machine directly obtains poco control actual combat
Sonic cloud real machine directly obtains poco control actual combat
Sonic cloud real machine directly obtains poco control actual combat
Sonic cloud real machine directly obtains poco control actual combat

At present, all game engines (unity3d, cocos2dx series, egret and UE4) of poco-sdk have been supported. Before using, the game needs to be connected to the SDK. v1. 4.0-beta can be used after updating ~ (it is expected to be updated after May Day)

What is poco

Poco is a cross engine automated testing framework based on UI control search. Support mainstream game engines: cocos2d-x, unity3d, Android native applications
https://poco-chinese.readthed…

Implementation process

In the past, poco controls were mostly obtained by using the python client or airtest ide of airtest framework, but sonic is the main development language of Java as the back-end. It is wasteful to only obtain controls and introduce Python environment, so try to find the answer from the SDK level. SDK github
Take unity3d as an example:
PocoManager.cs
We can see from this file that the tcpserver exposed by the SDK is an RPC Protocol. The exposed methods are:

rpc = new RPCParser();
        rpc.addRpcMethod("isVRSupported", vr_support.isVRSupported);
        rpc.addRpcMethod("hasMovementFinished", vr_support.IsQueueEmpty);
        rpc.addRpcMethod("RotateObject", vr_support.RotateObject);
        rpc.addRpcMethod("ObjectLookAt", vr_support.ObjectLookAt);
        rpc.addRpcMethod("Screenshot", Screenshot);
        rpc.addRpcMethod("GetScreenSize", GetScreenSize);
        rpc.addRpcMethod("Dump", Dump);
        rpc.addRpcMethod("GetDebugProfilingData", GetDebugProfilingData);
        rpc.addRpcMethod("SetText", SetText);
        rpc.addRpcMethod("GetSDKVersion", GetSDKVersion);

Obviously, the dump method is what we need.
If you don’t say much, the server will open up.

// Call a method in the server
    public string formatRequest(string method, object idAction, List<object> param = null)
    {
        Dictionary<string, object> data = new Dictionary<string, object>();
        data["jsonrpc"] = "2.0";
        data["method"] = method;
        if (param != null)
        {
            data["params"] = JsonConvert.SerializeObject(param, settings);
        }
        // if idAction is null, it is a notification
        if (idAction != null)
        {
            data["id"] = idAction;
        }
        return JsonConvert.SerializeObject(data, settings);
    }

It can be seen from here that tcpserver obtains the request body and maps it to different methods according to the method field.

// Send a response from a request the server made to this client
    public string formatResponse(object idAction, object result)
    {
        Dictionary<string, object> rpc = new Dictionary<string, object>();
        rpc["jsonrpc"] = "2.0";
        rpc["id"] = idAction;
        rpc["result"] = result;
        return JsonConvert.SerializeObject(rpc, settings);
    }

The returned ID is the one-to-one correspondence of the requested ID, and then the result is the return content of the corresponding method. The information we need is extracted here.

Specific communication details

We know from here that tcpserver cs

public class SimpleProtocolFilter : ProtoFilter
    {
        /*Simple protocol filter
        The protocol is packaged and unpacked according to the format of [effective data bytes] [effective data]
        [number of valid data bytes] length header_ Size byte
        [valid data] length valid data byte digital section
        In this way, this class takes out data from the data stream in sequence for splicing. Once a complete protocol packet is received, it will return the protocol packet
        After receiving the [valid data] field, it will be decoded according to UTF-8, because it is encoded with UTF-8 during transmission
        All encoding and decoding operations are completed in this class
        */

        private byte[] buf = new byte[0];
        private int HEADER_SIZE = 4;
        private List<string> msgs = new List<string>();

        public void input(byte[] data)
        {
            buf = Combine(buf, data);

            while (buf.Length > HEADER_SIZE)
            {
                int data_size = BitConverter.ToInt32(buf, 0);
                if (buf.Length >= data_size + HEADER_SIZE)
                {
                    byte[] data_body = Slice(buf, HEADER_SIZE, data_size + HEADER_SIZE);
                    string content = System.Text.Encoding.Default.GetString(data_body);
                    msgs.Add(content);
                    buf = Slice(buf, data_size + HEADER_SIZE, buf.Length);
                }
                else
                {
                    break;
                }
            }
        }

      public byte[] pack(String content)
        {
            int len = content.Length;
            byte[] size = BitConverter.GetBytes(len);
            if (!BitConverter.IsLittleEndian)
            {
                //reverse it so we get little endian.
                Array.Reverse(size);
            }
            byte[] body = System.Text.Encoding.Default.GetBytes(content);
            byte[] ret = Combine(size, body);
            return ret;
        }

Whether sending or receiving a message, the length of the message body is sent once as the head, and then the request body is sent. The same is true when receiving messages.

Sonic communication is as follows:

                poco = new Socket("localhost", port);
                inputStream = poco.getInputStream();
                outputStream = poco.getOutputStream();
                int len = jsonObject.toJSONString().length();
                ByteBuffer header = ByteBuffer.allocate(4);
                header.put(BytesTool.intToByteArray(len), 0, 4);
                header.flip();
                ByteBuffer body = ByteBuffer.allocate(len);
                body.put(jsonObject.toJSONString().getBytes(StandardCharsets.UTF_8), 0, len);
                body.flip();
                ByteBuffer total = ByteBuffer.allocate(len + 4);
                total.put(header.array());
                total.put(body.array());
                total.flip();
                outputStream.write(total.array());

Process summary

To sum up, the communication process is not as complicated as expected.

  1. The first step is to splice the request body we need.

    JSONObject jsonObject = new JSONObject();
         jsonObject.put("jsonrpc", "2.0");
         jsonObject.put("params", Arrays.asList(true));
         jsonObject.put("id", UUID.randomUUID().toString());
         jsonObject.put("method", "Dump");
    //Some engines are lower case dump
  2. Send to socket according to protocol rules
  3. Receive information

               while (poco.isConnected() && !Thread.interrupted()) {
                     byte[] buffer = new byte[1024];
                     int realLen;
                     realLen = inputStream.read(buffer);
                     if (buffer.length != realLen && realLen >= 0) {
                         buffer = subByteArray(buffer, 0, realLen);
                     }
                     if (realLen >= 0) {
                         s.append(new String(buffer));
                         if (s.toString().getBytes(StandardCharsets.UTF_8).length == headLen) {
                             result.set(s.toString());
                             break;
                         }
                     }
                 }
  4. Send the result to the front end for parsing
  5. Compatible with different engine protocols. Some engines are specially compatible with websocket.

epilogue

Poco is a high demand of users. At present, the function is only to obtain game controls. Later, poco will be used to do more work on game automation. Thank you for paying attention to sonic.