Source code analysis of spool — shared memory of memory module

preface

We know, becausePHPThere is no multithreading model, soswooleMore use of multi process model, so the code is relatively more concise, reducing the blocking and synchronization of various thread locks, but it also brings a new problem: data synchronization. Compared with multithreading, process memory can be shared directly, and data synchronization between processes depends on shared memory. This article will explainswooleShared memory source code.

Pre knowledge:

• mmapFunction usage:APUE learning notes – Advanced IO
• Shared memory:APUE learning notes — interprocess communication

Shared memory data structure

typedef struct _swShareMemory_mmap
{
    size_t size;
    char mapfile[SW_SHM_MMAP_FILE_LEN];
    int tmpfd;
    int key;
    int shmid;
    void *mem;
} swShareMemory;

• be carefulmemIt’s avoidPointer of type, used to store the first address of shared memory. This member variable is equivalent to thethisPointer, through which you can accessswShareMemoryEach member of.
• sizeRepresents the size of shared memory (excludingswShareMemoryStructure size),mapfile[]The name of the memory mapping file that represents the shared memory usage,tmpfdDescriptor for the memory mapped file.keyRepresentative useSystem VOfshmShared memory created by a series of functionskeyValue,shmidbyshmShared memory created by a series of functionsid(similar to FD), these two are notPOSIXStandard definedapiIt has limited use.

Application and creation of shared memory

swooleThe function that is often called when applying for shared memory issw_shm_mallocThis function can anonymously request a large block of continuous shared memory for a process

void* sw_shm_malloc(size_t size)
{
    swShareMemory object;
    void *mem;
    size += sizeof(swShareMemory);
    mem = swShareMemory_mmap_create(&object, size, NULL);
    if (mem == NULL)
    {
        return NULL;
    }
    else
    {
        memcpy(mem, &object, sizeof(swShareMemory));
        return mem + sizeof(swShareMemory);
    }
}

• fromsw_shm_mallocFunction can be seen, although we are applying forsizeHowever, the actual application memory is slightly larger, because it needs to be addedswShareMemoryThis structure. When the function returns, it will not directly return the first memory address of the application, but copy itobjectAfter the value of each member variable, addswShareMemoryThe size of the.

void *swShareMemory_mmap_create(swShareMemory *object, size_t size, char *mapfile)
{
    void *mem;
    int tmpfd = -1;
    int flag = MAP_SHARED;
    bzero(object, sizeof(swShareMemory));

#ifdef MAP_ANONYMOUS
    flag |= MAP_ANONYMOUS;
#else
    if (mapfile == NULL)
    {
        mapfile = "/dev/zero";
    }
    if ((tmpfd = open(mapfile, O_RDWR)) < 0)
    {
        return NULL;
    }
    strncpy(object->mapfile, mapfile, SW_SHM_MMAP_FILE_LEN);
    object->tmpfd = tmpfd;
#endif

#if defined(SW_USE_HUGEPAGE) && defined(MAP_HUGETLB)
    if (size > 2 * 1024 * 1024)
    {
        flag |= MAP_HUGETLB;
    }
#endif

    mem = mmap(NULL, size, PROT_READ | PROT_WRITE, flag, tmpfd, 0);
#ifdef MAP_FAILED
    if (mem == MAP_FAILED)
#else
    if (!mem)
#endif
    {
        swWarn("mmap(%ld) failed. Error: %s[%d]", size, strerror(errno), errno);
        return NULL;
    }
    else
    {
        object->size = size;
        object->mem = mem;
        return mem;
    }
}

• becauseswooleAll the processes in the process are controlled bymasterThe process is established, that is, there is a kinship between the processes, soswShareMemory_mmap_createFunction directly withanonymous mapping 、（/dev/zeroHow to use (equipment)mmapThere’s no way to build shared memoryopenSpecific shared memory file, or callshm_openopenPOSIX IPCname.
• It is worth noting thatMAP_HUGETLBThis islinuxkernel2.6.32An introductionflagsTo allocate shared memory using large pages. Big pages are relatively traditional4KFor small pages, generally speaking, common architectures provide two kinds of large page sizes, such as common2MBig page and1GBig page. Using a large page can reduce the number of page table entries, thus reducing the number of pagesTLB MissIn order to improve the memory access performance of the system. Of course, there are advantages and disadvantages. Using large pages reduces the granularity and flexibility of memory management. If the program does not use a large amount of memory, using large pages may also cause a waste of memory.

Shared memorycalloc

callocAndmallocIt’s the same, but one morenumparameter

void* sw_shm_calloc(size_t num, size_t _size)
{
    swShareMemory object;
    void *mem;
    void *ret_mem;
    int size = sizeof(swShareMemory) + (num * _size);
    mem = swShareMemory_mmap_create(&object, size, NULL);
    if (mem == NULL)
    {
        return NULL;
    }
    else
    {
        memcpy(mem, &object, sizeof(swShareMemory));
        ret_mem = mem + sizeof(swShareMemory);
        bzero(ret_mem, size - sizeof(swShareMemory));
        return ret_mem;
    }
}

Shared memoryrealloc

reallocFunction is used to modify the size of the applied memory. The logic is very simple. First apply for new memory, copy it, and then release the old memory

void* sw_shm_realloc(void *ptr, size_t new_size)
{
    swShareMemory *object = ptr - sizeof(swShareMemory);
    void *new_ptr;
    new_ptr = sw_shm_malloc(new_size);
    if (new_ptr == NULL)
    {
        return NULL;
    }
    else
    {
        memcpy(new_ptr, ptr, object->size);
        sw_shm_free(ptr);
        return new_ptr;
    }
}

Permission to modify shared memory

After the memory mapping is completed, read, write and execute permissions are markedPROT_READ、PROT_WRITEandPROT_EXECAnd other permissions can still be usedmprotectSystem call.

int sw_shm_protect(void *addr, int flags)
{
    swShareMemory *object = (swShareMemory *) (addr - sizeof(swShareMemory));
    return mprotect(object, object->size, flags);
}

Release of shared memory

void sw_shm_free(void *ptr)
{
    swShareMemory *object = ptr - sizeof(swShareMemory);
    swShareMemory_mmap_free(object);
}

int swShareMemory_mmap_free(swShareMemory *object)
{
    return munmap(object->mem, object->size);
}