Swoole source code analysis — shared memory table swoole of memory module_ table

preface

swoole_tableA super high performance, concurrent data structure based on shared memory and lock. It is used to solve the problem of multi process / multi thread data sharing and synchronous locking.

`swoole_table`Data structure of

swoole_tableIn fact, it is a hash table implemented by the open chain method,memoryIt is an array composed of hash keys and specific data. If the hash conflicts (different key values correspond to the same hash), it will be deleted from thepoolTo assign an element as the end of the linked list of array elements
sizeIs the maximum number of rows set when creating a shared memory table;conflict_proportionIs the maximum proportion of hash conflicts, beyond which the shared memory table is not allowed to add new row elements;iteratorIt is the iterator of memory table, which can be used to browse the data of memory table;columnsIs the memory table column element collection, because the memory table column element is also a collectionkey-valueFormat, so it is also a hash tableswHashMapType;maskStore is (maximum number of rows – 1), dedicated to hash and arrayindexThe transformation of the traditional culture;item_sizeIs the total memory size of all column elements;

typedef struct
{
    uint32_t absolute_index;
    uint32_t collision_index;
    swTableRow *row;
} swTable_iterator;

typedef struct
{
    swHashMap *columns;
    uint16_t column_num;
    swLock lock;
    size_t size;
    size_t mask;
    size_t item_size;
    size_t memory_size;
    float conflict_proportion;

    /**
     * total rows that in active state(shm)
     */
    sw_atomic_t row_num;

    swTableRow **rows;
    swMemoryPool *pool;

    swTable_iterator *iterator;

    void *memory;
} swTable;

swTableRowIs the row element of a memory table, wherelockIt is row lock;activeRepresents whether the line is enabled or not;nextIs a hash conflict linked list;keyIs the key value of the line, that is, the original key value before the hash;dataIs the real row data, which will load the value of each column element

typedef struct _swTableRow
{
#if SW_TABLE_USE_SPINLOCK
    sw_atomic_t lock;
#else
    pthread_mutex_t lock;
#endif
    /**
     * 1:used, 0:empty
     */
    uint8_t active;
    /**
     * next slot
     */
    struct _swTableRow *next;
    /**
     * Hash Key
     */
    char key[SW_TABLE_KEY_SIZE];
    char data[0];
} swTableRow;

swTableColumnIs a single column element of a memory table,nameIs the name of the column;typeIs the data type of the column. The optional parameters areswoole_table_type；indexDescribes the position of the current column element in the table column;sizeIs the amount of memory occupied by the data type of the column

enum swoole_table_type
{
    SW_TABLE_INT = 1,
    SW_TABLE_INT8,
    SW_TABLE_INT16,
    SW_TABLE_INT32,
#ifdef __x86_64__
    SW_TABLE_INT64,
#endif
    SW_TABLE_FLOAT,
    SW_TABLE_STRING,
};

typedef struct
{
   uint8_t type;
   uint32_t size;
   swString* name;
   uint16_t index;
} swTableColumn;

`swoole_table`The structure of

swoole_table->__construct(int $size, float $conflict_proportion = 0.2)The creation of the shared memory table object corresponds to the following function
The hash conflict percentage is set to a minimum of 0.2 and a maximum of 1
The number of rows is not strictly based on user-defined data, ifsizeIt’s not for twoNPower, such as1024、8192,65536And so on, the bottom layer will automatically adjust to a number close to, if less than1024Then default to1024, i.e1024It’s the minimum
The meaning of each member variable in the creation process can be seen in the previous section

swTable* swTable_new(uint32_t rows_size, float conflict_proportion)
{
    if (rows_size >= 0x80000000)
    {
        rows_size = 0x80000000;
    }
    else
    {
        uint32_t i = 10;
        while ((1U << i) < rows_size)
        {
            i++;
        }
        rows_size = 1 << i;
    }

    if (conflict_proportion > 1.0)
    {
        conflict_proportion = 1.0;
    }
    else if (conflict_proportion < SW_TABLE_CONFLICT_PROPORTION)
    {
        conflict_proportion = SW_TABLE_CONFLICT_PROPORTION;
    }

    swTable *table = SwooleG.memory_pool->alloc(SwooleG.memory_pool, sizeof(swTable));
    if (table == NULL)
    {
        return NULL;
    }
    if (swMutex_create(&table->lock, 1) < 0)
    {
        swWarn("mutex create failed.");
        return NULL;
    }
    table->iterator = sw_malloc(sizeof(swTable_iterator));
    if (!table->iterator)
    {
        swWarn("malloc failed.");
        return NULL;
    }
    table->columns = swHashMap_new(SW_HASHMAP_INIT_BUCKET_N, (swHashMap_dtor)swTableColumn_free);
    if (!table->columns)
    {
        return NULL;
    }

    table->size = rows_size;
    table->mask = rows_size - 1;
    table->conflict_proportion = conflict_proportion;

    bzero(table->iterator, sizeof(swTable_iterator));
    table->memory = NULL;
    return table;
}

`swoole_table`Add a new column

swoole_table->column(string $name, int $type, int $size = 0)Corresponding to the following function
After the column element is created and initialized successfully, theswHashMap_addFunction to add column elements to thetable->columnsin

int swTableColumn_add(swTable *table, char *name, int len, int type, int size)
{
    swTableColumn *col = sw_malloc(sizeof(swTableColumn));
    if (!col)
    {
        return SW_ERR;
    }
    col->name = swString_dup(name, len);
    if (!col->name)
    {
        sw_free(col);
        return SW_ERR;
    }
    switch(type)
    {
    case SW_TABLE_INT:
        switch(size)
        {
        case 1:
            col->size = 1;
            col->type = SW_TABLE_INT8;
            break;
        case 2:
            col->size = 2;
            col->type = SW_TABLE_INT16;
            break;
#ifdef __x86_64__
        case 8:
            col->size = 8;
            col->type = SW_TABLE_INT64;
            break;
#endif
        default:
            col->size = 4;
            col->type = SW_TABLE_INT32;
            break;
        }
        break;
    case SW_TABLE_FLOAT:
        col->size = sizeof(double);
        col->type = SW_TABLE_FLOAT;
        break;
    case SW_TABLE_STRING:
        col->size = size + sizeof(swTable_string_length_t);
        col->type = SW_TABLE_STRING;
        break;
    default:
        swWarn("unkown column type.");
        swTableColumn_free(col);
        return SW_ERR;
    }
    col->index = table->item_size;
    table->item_size += col->size;
    table->column_num ++;
    return swHashMap_add(table->columns, name, len, col);
}

`swoole_table`Creation of

adoptswTable_get_memory_sizeFunction to calculate the total amount of memory needed by the whole shared memory table, which contains the extra memory required by hash conflict.
Appliedmemory_sizeThe first address is assigned to thetable->rowsIt is worth noting thattable->rowsyesswTableRow **Type, and then assign the initial address to each line element through a loop
In order to reduce the time consumption of row lock, set row lock toPTHREAD_PRIO_INHERITTo increase the priority of row lock (if the thread with higher prioritythrd1Is blocked by one or more mutexes owned by thePTHREAD_PRIO_INHERITInitialized, thenthrd1The running priority of is prioritypri1And prioritypri2The one with the highest priority in the list, If there is no priority inheritance, the thread with the bottom priority may not be scheduled for a long time, which will lead to the high priority thread waiting for the lock held by the low priority thread waiting for a long time (because the low priority thread cannot run, so it cannot release the lock, so the high priority thread can only continue to block on the lock). Using priority inheritance can improve the priority of low priority thread in a short time, so that it can be scheduled as soon as possible, and then release the lock. The low priority thread will resume its priority after releasing the lock. )
PTHREAD_MUTEX_ROBUST_NP: if the holder of a mutex “dies”, or if the process holding such a mutexunmapThe shared memory where the mutex is located or the process holding the mutex is executedexecCall, the mutex is unlocked. The next holder of the mutex gets the mutex and returns an errorEOWNWERDEAD。
table->rowsAfter creating successfully, it is necessary to allocate address space to the row element with hash conflict. As you can see, the first address of the hash conflict row element ismemory += row_memory_size * table->sizeAnd use the existing memory to buildFixedPoolRandom memory pool,row_memory_sizeSize as an internal element of the memory pool

int swTable_create(swTable *table)
{
    size_t memory_size = swTable_get_memory_size(table);
    size_t row_memory_size = sizeof(swTableRow) + table->item_size;

    void *memory = sw_shm_malloc(memory_size);
    if (memory == NULL)
    {
        return SW_ERR;
    }

    table->memory_size = memory_size;
    table->memory = memory;

    table->rows = memory;
    memory += table->size * sizeof(swTableRow *);
    memory_size -= table->size * sizeof(swTableRow *);

#if SW_TABLE_USE_SPINLOCK == 0
    pthread_mutexattr_t attr;
    pthread_mutexattr_init(&attr);
    pthread_mutexattr_setpshared(&attr, PTHREAD_PROCESS_SHARED);
    pthread_mutexattr_setprotocol(&attr, PTHREAD_PRIO_INHERIT);
    pthread_mutexattr_setrobust_np(&attr, PTHREAD_MUTEX_ROBUST_NP);
#endif

    int i;
    for (i = 0; i < table->size; i++)
    {
        table->rows[i] = memory + (row_memory_size * i);
        memset(table->rows[i], 0, sizeof(swTableRow));
#if SW_TABLE_USE_SPINLOCK == 0
        pthread_mutex_init(&table->rows[i]->lock, &attr);
#endif
    }

    memory += row_memory_size * table->size;
    memory_size -= row_memory_size * table->size;
    table->pool = swFixedPool_new2(row_memory_size, memory, memory_size);

    return SW_OK;
}

Calculate the memory size of the entire shared memory table:

(number of memory table rows + number of hash conflict rows) * (size of row elements + sum of sizes of all column elements) + size of hash conflict memory pool head + size of row element pointer * number of memory table rows

It’s hard to understand the last oneRow element size * number of memory table rowsThis is actually creatingtable->rows[table->size]This array of pointers, as we said beforetable->rowsIs a two-dimensional array, this array is stored in multipleswTableRow*Type of data, such astable->rows[0]And so on,table->rows[0]If you don’t have this pointer array, you need to calculate the first address of each row element every time you go to get the row element. The efficiency is not so fast.

size_t swTable_get_memory_size(swTable *table)
{
    /**
     * table size + conflict size
     */
    size_t row_num = table->size * (1 + table->conflict_proportion);

    /*
     * header + data
     */
    size_t row_memory_size = sizeof(swTableRow) + table->item_size;

    /**
     * row data & header
     */
    size_t memory_size = row_num * row_memory_size;

    /**
     * memory pool for conflict rows
     */
    memory_size += sizeof(swMemoryPool) + sizeof(swFixedPool) + ((row_num - table->size) * sizeof(swFixedPool_slice));

    /**
     * for iterator, Iterate through all the elements
     */
    memory_size += table->size * sizeof(swTableRow *);

    return memory_size;
}

`swoole_table`Add new data

To add a new element to a shared memory table, you need to call three functions, which areswTableRow_setSet the number of rowskeyValueswTableColumn_getGets the column element,swTableRow_set_valueFunction based on the data type of the columnrow->dataThe assignment process is as follows:

static PHP_METHOD(swoole_table, set)
{
    zval *array;
    char *key;
    zend_size_t keylen;

    if (zend_parse_parameters(ZEND_NUM_ARGS() TSRMLS_CC, "sa", &key, &keylen, &array) == FAILURE)
    {
        RETURN_FALSE;
    }

    swTable *table = swoole_get_object(getThis());

    swTableRow *_rowlock = NULL;
    swTableRow *row = swTableRow_set(table, key, keylen, &_rowlock);

    swTableColumn *col;
    zval *v;
    char *k;
    uint32_t klen;
    int ktype;
    HashTable *_ht = Z_ARRVAL_P(array);

    SW_HASHTABLE_FOREACH_START2(_ht, k, klen, ktype, v)
    {
        col = swTableColumn_get(table, k, klen);

        else if (col->type == SW_TABLE_STRING)
        {
            convert_to_string(v);
            swTableRow_set_value(row, col, Z_STRVAL_P(v), Z_STRLEN_P(v));
        }
        else if (col->type == SW_TABLE_FLOAT)
        {
            convert_to_double(v);
            swTableRow_set_value(row, col, &Z_DVAL_P(v), 0);
        }
        else
        {
            convert_to_long(v);
            swTableRow_set_value(row, col, &Z_LVAL_P(v), 0);
        }
    }
    swTableRow_unlock(_rowlock);
}

`swTableRow_set`function

Let’s look at it firstswTableRow_setFunction, from the following code point of view, the main role of this function is to determine the newly addedkeyWhether the hash conflict is caused, if there is no conflict（row->active=0）So directtable->row_numAuto increment, settingrow->keyThat’s it.
If there is a hash conflict, loop the linked list of the current row elements until (1) finds the same hashkeyValue, indicating that there is not a hash conflict, but that if the user wants to modify the existing row data, he will jump out of the function and change itrow->data(2) the same value was not foundkeyValue, indicating that there is a hash conflict, differentkeyValue corresponds to the same hash value. At this time, the loop has reached the end of the linked list, and a hash value needs to be constructed from the memory poolswTableRowLine elements, put at the end of the list

swTableRow* swTableRow_set(swTable *table, char *key, int keylen, swTableRow **rowlock)
{
    if (keylen > SW_TABLE_KEY_SIZE)
    {
        keylen = SW_TABLE_KEY_SIZE;
    }

    swTableRow *row = swTable_hash(table, key, keylen);
    *rowlock = row;
    swTableRow_lock(row);

#ifdef SW_TABLE_DEBUG
    int _conflict_level = 0;
#endif

    if (row->active)
    {
        for (;;)
        {
            if (strncmp(row->key, key, keylen) == 0)
            {
                break;
            }
            else if (row->next == NULL)
            {
                table->lock.lock(&table->lock);
                swTableRow *new_row = table->pool->alloc(table->pool, 0);

#ifdef SW_TABLE_DEBUG
                conflict_count ++;
                if (_conflict_level > conflict_max_level)
                {
                    conflict_max_level = _conflict_level;
                }

#endif
                table->lock.unlock(&table->lock);

                if (!new_row)
                {
                    return NULL;
                }
                //add row_num
                bzero(new_row, sizeof(swTableRow));
                sw_atomic_fetch_add(&(table->row_num), 1);
                row->next = new_row;
                row = new_row;
                break;
            }
            else
            {
                row = row->next;
#ifdef SW_TABLE_DEBUG
                _conflict_level++;
#endif
            }
        }
    }
    else
    {
#ifdef SW_TABLE_DEBUG
        insert_count ++;
#endif
        sw_atomic_fetch_add(&(table->row_num), 1);
    }

    memcpy(row->key, key, keylen);
    row->active = 1;
    return row;
}

Now let’s look at the codeswTable_hashHow can this function calculate the hash value? We find that there are two kinds of hash functions:
- swoole_hash_phpyesphpThe classic hash function oftime33/DJBalgorithm
- swoole_hash_austinyesMurmurHashHash algorithm is widely used in many fieldsredis、MemcachedAnd so on
After the hash calculation, we find that the hash value andtable->maskThe purpose of logic and calculation is to get a value less than or equal totable->mask(rows_size - 1）As the number of line elementsindex

static sw_inline swTableRow* swTable_hash(swTable *table, char *key, int keylen)
{
#ifdef SW_TABLE_USE_PHP_HASH
    uint64_t hashv = swoole_hash_php(key, keylen);
#else
    uint64_t hashv = swoole_hash_austin(key, keylen);
#endif
    uint64_t index = hashv & table->mask;
    assert(index < table->size);
    return table->rows[index];
}

Next, let’s look at the locking function of row lock
- If it’s a normal mutex, use it directlypthread_mutex_lockThat is, if it is not a mutex, the program implements a spin lock
- If it’s a spin lock, callswooleCustom spin lock lock

static sw_inline void swTableRow_lock(swTableRow *row)
{
#if SW_TABLE_USE_SPINLOCK
    sw_spinlock(&row->lock);
#else
    pthread_mutex_lock(&row->lock);
#endif
}

`swTableColumn_get`function

From multiple column elementshashMapMedium basiscolumn_keyQuickly find the corresponding column elements

static sw_inline swTableColumn* swTableColumn_get(swTable *table, char *column_key, int keylen)
{
    return swHashMap_find(table->columns, column_key, keylen);
}

`swTableRow_set_value`function

According to the type of column element data, therow->dataIt is worth noting thatdefaultActually, it meansSW_TABLE_STRINGIn this case, the string length will be stored first, and then the string value will be stored:

static sw_inline void swTableRow_set_value(swTableRow *row, swTableColumn * col, void *value, int vlen)
{
    int8_t _i8;
    int16_t _i16;
    int32_t _i32;
#ifdef __x86_64__
    int64_t _i64;
#endif
    switch(col->type)
    {
    case SW_TABLE_INT8:
        _i8 = *(int8_t *) value;
        memcpy(row->data + col->index, &_i8, 1);
        break;
    case SW_TABLE_INT16:
        _i16 =  *(int16_t *) value;
        memcpy(row->data + col->index, &_i16, 2);
        break;
    case SW_TABLE_INT32:
        _i32 =  *(int32_t *) value;
        memcpy(row->data + col->index, &_i32, 4);
        break;
#ifdef __x86_64__
    case SW_TABLE_INT64:
        _i64 =  *(int64_t *) value;
        memcpy(row->data + col->index, &_i64, 8);
        break;
#endif
    case SW_TABLE_FLOAT:
        memcpy(row->data + col->index, value, sizeof(double));
        break;
    default:
        if (vlen > (col->size - sizeof(swTable_string_length_t)))
        {
            swWarn("[key=%s,field=%s]string value is too long.", row->key, col->name->str);
            vlen = col->size - sizeof(swTable_string_length_t);
        }
        memcpy(row->data + col->index, &vlen, sizeof(swTable_string_length_t));
        memcpy(row->data + col->index + sizeof(swTable_string_length_t), value, vlen);
        break;
    }
}

`swoole_table`get data

Three functions need to be called to get line elements according to key values:swTableRow_getGets the row object element. If only a specific field is selected, thephp_swoole_table_get_field_valueIf all fields need to be deleted, thephp_swoole_table_row2array：

static PHP_METHOD(swoole_table, get)
{
    char *key;
    zend_size_t keylen;

    char *field = NULL;
    zend_size_t field_len = 0;

    if (zend_parse_parameters(ZEND_NUM_ARGS() TSRMLS_CC, "s|s", &key, &keylen, &field, &field_len) == FAILURE)
    {
        RETURN_FALSE;
    }

    swTableRow *_rowlock = NULL;
    swTable *table = swoole_get_object(getThis());

    swTableRow *row = swTableRow_get(table, key, keylen, &_rowlock);
    if (field && field_len > 0)
    {
        php_swoole_table_get_field_value(table, row, return_value, field, (uint16_t) field_len);
    }
    else
    {
        php_swoole_table_row2array(table, row, return_value);
    }
    swTableRow_unlock(_rowlock);
}

swTableRow_getfunction

utilizekeyCalculation of travel elementsindexValue, when there is a Hash list, we should constantly comparekeyUntil an exactly equal key value is found

swTableRow* swTableRow_get(swTable *table, char *key, int keylen, swTableRow** rowlock)
{
    if (keylen > SW_TABLE_KEY_SIZE)
    {
        keylen = SW_TABLE_KEY_SIZE;
    }

    swTableRow *row = swTable_hash(table, key, keylen);
    *rowlock = row;
    swTableRow_lock(row);

    for (;;)
    {
        if (strncmp(row->key, key, keylen) == 0)
        {
            if (!row->active)
            {
                row = NULL;
            }
            break;
        }
        else if (row->next == NULL)
        {
            row = NULL;
            break;
        }
        else
        {
            row = row->next;
        }
    }

    return row;
}

php_swoole_table_get_field_valuefunction

First of allswHashMap_findFunction based onfieldDetermine the field type. If it is a string, you need to obtain the length of the string first

static inline void php_swoole_table_get_field_value(swTable *table, swTableRow *row, zval *return_value, char *field, uint16_t field_len)
{
    swTable_string_length_t vlen = 0;
    double dval = 0;
    int64_t lval = 0;

    swTableColumn *col = swHashMap_find(table->columns, field, field_len);
    
    if (col->type == SW_TABLE_STRING)
    {
        memcpy(&vlen, row->data + col->index, sizeof(swTable_string_length_t));
        SW_ZVAL_STRINGL(return_value, row->data + col->index + sizeof(swTable_string_length_t), vlen, 1);
    }
    else if (col->type == SW_TABLE_FLOAT)
    {
        memcpy(&dval, row->data + col->index, sizeof(dval));
        ZVAL_DOUBLE(return_value, dval);
    }
    else
    {
        switch (col->type)
        {
        case SW_TABLE_INT8:
            memcpy(&lval, row->data + col->index, 1);
            ZVAL_LONG(return_value, (int8_t) lval);
            break;
        case SW_TABLE_INT16:
            memcpy(&lval, row->data + col->index, 2);
            ZVAL_LONG(return_value, (int16_t) lval);
            break;
        case SW_TABLE_INT32:
            memcpy(&lval, row->data + col->index, 4);
            ZVAL_LONG(return_value, (int32_t) lval);
            break;
        default:
            memcpy(&lval, row->data + col->index, 8);
            ZVAL_LONG(return_value, lval);
            break;
        }
    }
}

`php_swoole_table_row2array`function

Compared with the previous function, this function only uses theswHashMap_eachTraversal column elements, and then use the column element value process, after the value, there are also useadd_assoc_stringl_exetc.zendOfAPI, continuously converting values toPHPArray:

#define sw_add_assoc_string                   add_assoc_string
#define sw_add_assoc_stringl_ex               add_assoc_stringl_ex
#define sw_add_assoc_stringl                  add_assoc_stringl
#define sw_add_assoc_double_ex                add_assoc_double_ex
#define sw_add_assoc_long_ex                  add_assoc_long_ex
#define sw_add_next_index_stringl             add_next_index_stringl

static inline void php_swoole_table_row2array(swTable *table, swTableRow *row, zval *return_value)
{
    array_init(return_value);

    swTableColumn *col = NULL;
    swTable_string_length_t vlen = 0;
    double dval = 0;
    int64_t lval = 0;
    char *k;

    while(1)
    {
        col = swHashMap_each(table->columns, &k);
        if (col == NULL)
        {
            break;
        }
        if (col->type == SW_TABLE_STRING)
        {
            memcpy(&vlen, row->data + col->index, sizeof(swTable_string_length_t));
            sw_add_assoc_stringl_ex(return_value, col->name->str, col->name->length + 1, row->data + col->index + sizeof(swTable_string_length_t), vlen, 1);
        }
        else if (col->type == SW_TABLE_FLOAT)
        {
            memcpy(&dval, row->data + col->index, sizeof(dval));
            sw_add_assoc_double_ex(return_value, col->name->str, col->name->length + 1, dval);
        }
        else
        {
            switch (col->type)
            {
            case SW_TABLE_INT8:
                memcpy(&lval, row->data + col->index, 1);
                sw_add_assoc_long_ex(return_value, col->name->str, col->name->length + 1, (int8_t) lval);
                break;
            case SW_TABLE_INT16:
                memcpy(&lval, row->data + col->index, 2);
                sw_add_assoc_long_ex(return_value, col->name->str, col->name->length + 1, (int16_t) lval);
                break;
            case SW_TABLE_INT32:
                memcpy(&lval, row->data + col->index, 4);
                sw_add_assoc_long_ex(return_value, col->name->str, col->name->length + 1, (int32_t) lval);
                break;
            default:
                memcpy(&lval, row->data + col->index, 8);
                sw_add_assoc_long_ex(return_value, col->name->str, col->name->length + 1, lval);
                break;
            }
        }
    }
}

`swoole_table->incr`Field value auto increment

static PHP_METHOD(swoole_table, incr)
{
    char *key;
    zend_size_t key_len;
    char *col;
    zend_size_t col_len;
    zval* incrby = NULL;

    if (zend_parse_parameters(ZEND_NUM_ARGS() TSRMLS_CC, "ss|z", &key, &key_len, &col, &col_len, &incrby) == FAILURE)
    {
        RETURN_FALSE;
    }

    swTableRow *_rowlock = NULL;
    swTable *table = swoole_get_object(getThis());

    swTableRow *row = swTableRow_set(table, key, key_len, &_rowlock);

    swTableColumn *column;
    column = swTableColumn_get(table, col, col_len);
    if (column->type == SW_TABLE_STRING)
    {
        swTableRow_unlock(_rowlock);
        swoole_php_fatal_error(E_WARNING, "can't execute 'incr' on a string type column.");
        RETURN_FALSE;
    }
    else if (column->type == SW_TABLE_FLOAT)
    {
        double set_value = 0;
        memcpy(&set_value, row->data + column->index, sizeof(set_value));
        if (incrby)
        {
            convert_to_double(incrby);
            set_value += Z_DVAL_P(incrby);
        }
        else
        {
            set_value += 1;
        }
        swTableRow_set_value(row, column, &set_value, 0);
        RETVAL_DOUBLE(set_value);
    }
    else
    {
        int64_t set_value = 0;
        memcpy(&set_value, row->data + column->index, column->size);
        if (incrby)
        {
            convert_to_long(incrby);
            set_value += Z_LVAL_P(incrby);
        }
        else
        {
            set_value += 1;
        }
        swTableRow_set_value(row, column, &set_value, 0);
        RETVAL_LONG(set_value);
    }
    swTableRow_unlock(_rowlock);
}

`swoole_table->incr`Field value self subtraction

static PHP_METHOD(swoole_table, decr)
{
    char *key;
    zend_size_t key_len;
    char *col;
    zend_size_t col_len;
    zval *decrby = NULL;

    if (zend_parse_parameters(ZEND_NUM_ARGS() TSRMLS_CC, "ss|z", &key, &key_len, &col, &col_len, &decrby) == FAILURE)
    {
        RETURN_FALSE;
    }

    swTableRow *_rowlock = NULL;
    swTable *table = swoole_get_object(getThis());

    swTableRow *row = swTableRow_set(table, key, key_len, &_rowlock);

    swTableColumn *column;
    column = swTableColumn_get(table, col, col_len);
    if (column->type == SW_TABLE_STRING)
    {
        swTableRow_unlock(_rowlock);
        swoole_php_fatal_error(E_WARNING, "can't execute 'decr' on a string type column.");
        RETURN_FALSE;
    }
    else if (column->type == SW_TABLE_FLOAT)
    {
        double set_value = 0;
        memcpy(&set_value, row->data + column->index, sizeof(set_value));
        if (decrby)
        {
            convert_to_double(decrby);
            set_value -= Z_DVAL_P(decrby);
        }
        else
        {
            set_value -= 1;
        }
        swTableRow_set_value(row, column, &set_value, 0);
        RETVAL_DOUBLE(set_value);
    }
    else
    {
        int64_t set_value = 0;
        memcpy(&set_value, row->data + column->index, column->size);
        if (decrby)
        {
            convert_to_long(decrby);
            set_value -= Z_LVAL_P(decrby);
        }
        else
        {
            set_value -= 1;
        }
        swTableRow_set_value(row, column, &set_value, 0);
        RETVAL_LONG(set_value);
    }
    swTableRow_unlock(_rowlock);
}

`swoole_table->del`Deletion of list data

The data deletion of shared memory table is slightly complicated, which can be divided into the following situations:

The row element to be deleted has no linked list with hash conflict
- If the key values are the same, use thebzeroInitialize the row element to reduce the number of shared table rows
- If the key value is inconsistent, it means that there is no such line of data, then directly unlock and return
If there is a hash conflict linked list in the row elements to be deleted, we need to loop the linked list to find out the row elements with consistent key values
- If the traversal list is not found, then directly unlock return
- If it is found to be the head element of the linked list, the data of the second element of the linked list is assigned to the head element, and then the second element of the linked list is released from the memory pool to reduce the number of shared table rows
- If it is the middle element of the linked list, it is consistent with the common method of deleting the linked list node, reducing the number of rows in the shared list

int swTableRow_del(swTable *table, char *key, int keylen)
{
    if (keylen > SW_TABLE_KEY_SIZE)
    {
        keylen = SW_TABLE_KEY_SIZE;
    }

    swTableRow *row = swTable_hash(table, key, keylen);
    //no exists
    if (!row->active)
    {
        return SW_ERR;
    }

    swTableRow_lock(row);
    if (row->next == NULL)
    {
        if (strncmp(row->key, key, keylen) == 0)
        {
            bzero(row, sizeof(swTableRow) + table->item_size);
            goto delete_element;
        }
        else
        {
            goto not_exists;
        }
    }
    else
    {
        swTableRow *tmp = row;
        swTableRow *prev = NULL;

        while (tmp)
        {
            if ((strncmp(tmp->key, key, keylen) == 0))
            {
                break;
            }
            prev = tmp;
            tmp = tmp->next;
        }

        if (tmp == NULL)
        {
            not_exists:
            swTableRow_unlock(row);
            return SW_ERR;
        }

        //when the deleting element is root, we should move the first element's data to root,
        //and remove the element from the collision list.
        if (tmp == row)
        {
            tmp = tmp->next;
            row->next = tmp->next;
            memcpy(row->key, tmp->key, strlen(tmp->key));
            memcpy(row->data, tmp->data, table->item_size);
        }
        if (prev)
        {
            prev->next = tmp->next;
        }
        table->lock.lock(&table->lock);
        bzero(tmp, sizeof(swTableRow) + table->item_size);
        table->pool->free(table->pool, tmp);
        table->lock.unlock(&table->lock);
    }

    delete_element:
    sw_atomic_fetch_sub(&(table->row_num), 1);
    swTableRow_unlock(row);

    return SW_OK;
}

`swoole_table->del`Traversal of list data

swoole_tableClass to implement an iterator, you can use theforeachTo traverse.

void swoole_table_init(int module_number TSRMLS_DC)
{
    #ifdef HAVE_PCRE
    zend_class_implements(swoole_table_class_entry_ptr TSRMLS_CC, 2, spl_ce_Iterator, spl_ce_Countable);
#endif
}

As you can see,swooleYesswoole_tableWhen initializing, it is inherited for this classspl_iteratorThis interface, as we know, controls the classes that inherit this interfaceforeachInstead of triggering the traversal of the original object member variable, thespl_iteratorOfrewind、nextAnd so on

#ifdef HAVE_PCRE
static PHP_METHOD(swoole_table, rewind);
static PHP_METHOD(swoole_table, next);
static PHP_METHOD(swoole_table, current);
static PHP_METHOD(swoole_table, key);
static PHP_METHOD(swoole_table, valid);
#endif

About whyPCREThe dependence of this regular expression library, I am very confused, hope someone can solve the problem.

`rewind`

static PHP_METHOD(swoole_table, rewind)
{
    swTable *table = swoole_get_object(getThis());
    if (!table->memory)
    {
        swoole_php_fatal_error(E_ERROR, "the swoole table does not exist.");
        RETURN_FALSE;
    }
    swTable_iterator_rewind(table);
    swTable_iterator_forward(table);
}

void swTable_iterator_rewind(swTable *table)
{
    bzero(table->iterator, sizeof(swTable_iterator));
}

rewindFunction is to return the data iterator to the starting positionswTableIn other words, it means that theabsolute_index、collision_index、rowWait until reset to 0.
swTable_iterator_forwardIs to take the iterator one step forward, whereabsolute_indexA row index similar to a shared table,collision_indexSimilar to the column index of a shared table. The difference is that for the row without hash conflict, the column index has only one 0. For the row with hash conflict, the column index is the linked list index of the open chain method

static sw_inline swTableRow* swTable_iterator_get(swTable *table, uint32_t index)
{
    swTableRow *row = table->rows[index];
    return row->active ? row : NULL;
}

void swTable_iterator_forward(swTable *table)
{
    for (; table->iterator->absolute_index < table->size; table->iterator->absolute_index++)
    {
        swTableRow *row = swTable_iterator_get(table, table->iterator->absolute_index);
        if (row == NULL)
        {
            continue;
        }
        else if (row->next == NULL)
        {
            table->iterator->absolute_index++;
            table->iterator->row = row;
            return;
        }
        else
        {
            int i = 0;
            for (;; i++)
            {
                if (row == NULL)
                {
                    table->iterator->collision_index = 0;
                    break;
                }
                if (i == table->iterator->collision_index)
                {
                    table->iterator->collision_index++;
                    table->iterator->row = row;
                    return;
                }
                row = row->next;
            }
        }
    }
    table->iterator->row = NULL;
}

`current`

currentThe method is very simple. Take out the line element of the current iterator, and then convert it tophpArray

static PHP_METHOD(swoole_table, current)
{
    swTable *table = swoole_get_object(getThis());
    if (!table->memory)
    {
        swoole_php_fatal_error(E_ERROR, "the swoole table does not exist.");
        RETURN_FALSE;
    }
    swTableRow *row = swTable_iterator_current(table);
    swTableRow_lock(row);
    php_swoole_table_row2array(table, row, return_value);
    swTableRow_unlock(row);
}

swTableRow* swTable_iterator_current(swTable *table)
{
    return table->iterator->row;
}

`key`

Get the key value of the current iterator:

static PHP_METHOD(swoole_table, key)
{
    swTable *table = swoole_get_object(getThis());
    if (!table->memory)
    {
        swoole_php_fatal_error(E_ERROR, "the swoole table does not exist.");
        RETURN_FALSE;
    }
    swTableRow *row = swTable_iterator_current(table);
    swTableRow_lock(row);
    SW_RETVAL_STRING(row->key, 1);
    swTableRow_unlock(row);
}

`next`

nextThe iterator goes further

static PHP_METHOD(swoole_table, next)
{
    swTable *table = swoole_get_object(getThis());
    if (!table->memory)
    {
        swoole_php_fatal_error(E_ERROR, "the swoole table does not exist.");
        RETURN_FALSE;
    }
    swTable_iterator_forward(table);
}

`valid`

Verify that the current row element is empty:

static PHP_METHOD(swoole_table, valid)
{
    swTable *table = swoole_get_object(getThis());
    if (!table->memory)
    {
        swoole_php_fatal_error(E_ERROR, "the swoole table does not exist.");
        RETURN_FALSE;
    }
    swTableRow *row = swTable_iterator_current(table);
    RETURN_BOOL(row != NULL);
}

Swoole source code analysis — shared memory table swoole of memory module_ table

preface

`swoole_table`Data structure of

`swoole_table`The structure of

`swoole_table`Add a new column

`swoole_table`Creation of

`swoole_table`Add new data

`swTableRow_set`function

`swTableColumn_get`function

`swTableRow_set_value`function

`swoole_table`get data

`php_swoole_table_row2array`function

`swoole_table->incr`Field value auto increment

`swoole_table->incr`Field value self subtraction

`swoole_table->del`Deletion of list data

`swoole_table->del`Traversal of list data

`rewind`

`current`

`key`

`next`

`valid`

Recommended Today

How to Build a Cybersecurity Career

Swoole source code analysis — shared memory table swoole of memory module_ table

preface

swoole_tableData structure of

swoole_tableThe structure of

swoole_tableAdd a new column

swoole_tableCreation of

swoole_tableAdd new data

swTableRow_setfunction

swTableColumn_getfunction

swTableRow_set_valuefunction

swoole_tableget data

php_swoole_table_row2arrayfunction

swoole_table->incrField value auto increment

swoole_table->incrField value self subtraction

swoole_table->delDeletion of list data

swoole_table->delTraversal of list data

rewind

current

key

next

valid

Recommended Today

How to Build a Cybersecurity Career

`swoole_table`Data structure of

`swoole_table`The structure of

`swoole_table`Add a new column

`swoole_table`Creation of

`swoole_table`Add new data

`swTableRow_set`function

`swTableColumn_get`function

`swTableRow_set_value`function

`swoole_table`get data

`php_swoole_table_row2array`function

`swoole_table->incr`Field value auto increment

`swoole_table->incr`Field value self subtraction

`swoole_table->del`Deletion of list data

`swoole_table->del`Traversal of list data

`rewind`

`current`

`key`

`next`

`valid`