On the meaning of Python descriptors

Time:2021-3-9

You may often hear the concept of “descriptor“, but since most programmers rarely use it, you may not understand its principle very well.

But if you want to take your career to a higher level and become more proficient in using python, I think you should be more proficient in itdescriptor This concept has a clear understanding, which is of great help to your future development, but also conducive to your deeper understanding of Python design in the future.

Although in the process of development, we have not used descriptors directly, but its use in the underlying is very frequent. For example:

  • functionbound methodunbound method
  • It’s a devicepropertystaticmethodclassmethod

    Are these familiar?

    In fact, these are closely related to descriptors. Let’s discuss the working principle behind descriptors through the following articles.

What is a descriptor?

Before we know what a descriptor is, let’s look for an example

class A:
    x = 10

print(A.x) # 10

This example is very simple. Let’s start with the classADefine aClass propertiesxAnd get its value.

In addition to this method of directly defining class attributes, we can also define a class attribute as follows:

class Ten:

We can see that this time the class properties of thexNot a specific value, but a classTenThrough thisTenIt defines a__get__Method to return a specific value.

So we can conclude that: in Python,We can manage the properties of a class to a class, and such a property is a classdescriptor

In short,descriptor It’s aBinding behaviorattribute

And what does that mean?

In retrospect, when we are developing, in general, we willbehaviorWhat is it called?behaviorIt’s a method.

So we can alsodescriptor It is understood that:The attribute of an object is not a specific value, but a method to define it.

Imagine if we use a method to define a property, what’s the benefit of doing so?

With a method, we can implement our own logic in the method. In the simplest way, we can assign different values to attributes in the method according to different conditions, just like the following:

class Age:
    def __get__(self, obj, objtype=None):
        if obj.name == 'zhangsan':
            return 20
        elif obj.name == 'lisi':
            return 25
        else:
            return ValueError("unknow")

class Person:

    age = Age()

    def __init__(self, name):
        self.name = name

p1 = Person('zhangsan')
print(p1.age)   # 20

p2 = Person('lisi')
print(p2.age)   # 25

p3 = Person('wangwu')
print(p3.age)   # unknow

In this case,ageClass properties are managed by another class, which is in the__get__It will be based onPersonProperties of classname, decideageWhat is the value.

Through such an example, we can see that through the use of descriptors, we can easily change the way a class attribute is defined.

Descriptor protocol

Now that we understand the definition of descriptors, let’s focus on the classes that host properties.

In fact, if a class property wants to be managed to a class, the methods implemented in the class cannot be defined casually. It must comply with the descriptor protocol, that is, to implement the following methods:

  • __get__(self, obj, type=None) -> value
  • __set__(self, obj, value) -> None
  • __delete__(self, obj) -> None

As long as the implementation of the above methodsOne of themThen this class property can be called a descriptor.

In addition, descriptors can be divided into data descriptors and non data descriptors

  • It only defines__get___It’s called a non data descriptor
  • Except for the definition__get__In addition, it defines__set__or__delete__It’s called a data descriptor

What’s the difference between them? I’ll explain it in detail below.

Now let’s take a look at an inclusion__get__and__set__Method descriptor example:

# coding: utf8

class Age:

    def __init__(self, value=20):
        self.value = value

    def __get__(self, obj, type=None):
        print('call __get__: obj: %s type: %s' % (obj, type))
        return self.value

    def __set__(self, obj, value):
        if value <= 0:
            raise ValueError("age must be greater than 0")
        print('call __set__: obj: %s value: %s' % (obj, value))
        self.value = value

class Person:

    age = Age()

    def __init__(self, name):
        self.name = name

p1 = Person('zhangsan')
print(p1.age)
# call __get__: obj: <__main__.Person object at 0x1055509e8> type: <class '__main__.Person'>
# 20

print(Person.age)
# call __get__: obj: None type: <class '__main__.Person'>
# 20

p1.age = 25
# call __set__: obj: <__main__.Person object at 0x1055509e8> value: 25

print(p1.age)
# call __get__: obj: <__main__.Person object at 0x1055509e8> type: <class '__main__.Person'>
# 25

p1.age = -1
# ValueError: age must be greater than 0

In this case, the class propertyageIs a descriptor whose value depends on theAgeClass.

From the output, when we get or modifyageProperty, calledAgeOf__get__and__set__method:

  • When callingp1.ageWhen,__get__Called, parameterobjyesPersonexample,typeyestype(Person)
  • When callingPerson.ageWhen,__get__Called, parameterobjyesNonetypeyestype(Person)
  • When callingp1.age = 25When,__set__Called, parameterobjyesPersonexample,valueIt’s 25
  • When callingp1.age = -1When,__set__Failed to pass the verification, throwValueError

Where, call__set__The parameters passed in are easy to understand, but for the__get__Method, byClass or instanceCall. The parameters passed in are different. Why?

This requires us to understand how descriptors work.

How descriptors work

To explain how descriptors work, we need to start with accessing properties.

In the development, I wonder if you have ever thought about such a problem: usually we write such codea.bWhat happened behind it?

thereaandbThere may be the following situations:

  1. aIt may be a class or an instance, which we call objects here
  2. bIt may be an attribute or a method. A method can also be regarded as an attribute of a class

In fact, in any of the above cases, there is a unified calling logic in Python:

  1. Call first__getattribute__Try to get results
  2. If there is no result, call__getattr__

The code is as follows:

def getattr_hook(obj, name):
    try:
        return obj.__getattribute__(name)
    except AttributeError:
        if not hasattr(type(obj), '__getattr__'):
            raise
    return type(obj).__getattr__(obj, name)

We need to focus here__getattribute__Because it is the entry of all attribute search, the order of attribute search implemented in it is as follows:

  1. Is the property to be found a descriptor in the class
  2. If it is a descriptor, check whether it is a data descriptor
  3. If it is a data descriptor, call the__get__
  4. If it is not a data descriptor, the__dict__Find in
  5. If__dict__We can’t find it in. Let’s see if it’s a non data descriptor
  6. If it is a non data descriptor, the__get__
  7. If it is not a non data descriptor, it is found from the class property
  8. If the class does not have this property, throwAttributeErrorabnormal

The code is as follows:

#Gets the properties of an object

If it’s not easy to understand, you’d better write a program to test it and observe the search order of attributes in various cases.

Here, we can see that to find an attribute in an object, we start from the__getattribute__It started.

stay__getattribute__In, it checks whether the class property is a descriptor. If it is a descriptor, its__get__method. But the specific call details and the parameters passed in are as follows:

  • IfaIt’s aexampleThe call details are as follows:
type(a).__ dict__ ['b'].__ get__ (a, type(a))
  • IfaIt’s aclassThe call details are as follows:
a.__ dict__ ['b'].__ get__ (None, a)

So we can see the output of the above example.

Data descriptor and non data descriptor

After understanding how descriptors work, let’s continue to look at the differences between data descriptors and non data descriptors.

In terms of definition, the differences are as follows:

  • It only defines__get___It’s called a non data descriptor
  • Except for the definition__get__In addition, it defines__set__or__delete__It’s called a data descriptor

In addition, we can see from the above sequence of descriptor calls that data descriptors have priority over non data descriptors when searching for properties in objects.

In the previous example, we defined__get__and__set__So those class properties areData descriptor

Let’s take another lookNon data descriptorFor example:

class A:

In this code, we define a property and method with the same namefoo, if you do it nowA().fooWhat do you think the output will be?

The answer isabc

Why are instance properties printedfooInstead of the methodfooWhat about it?

This has something to do with non data descriptors.

We dodir(A.foo)The results were as follows

print(dir(A.foo))

Did you see?AOffooThe method is actually implemented__get__We have learned from the above analysis: only define__get__Method, which is actually a non data descriptor,The method we define in a class is actually a non data descriptor.

Therefore, in a class, if there are properties and methods with the same name, follow the above instructions__getattribute__This property will be obtained from the instance first. If the instance does not exist, it will be obtained from the non data descriptor. Therefore, the instance property is the priority to be found herefooThe value of.

Here we can summarize the knowledge about descriptors

  • Descriptor must be a class property
  • __getattribute__Is the entry to find a property (method)
  • __getattribute__The search order of a property (method) is defined: data descriptor, instance property, non data descriptor and class property
  • If we rewrite it__getattribute__Method to prevent the descriptor from being called
  • All methods are actually non data descriptors because they define__get__

Usage scenarios of descriptors

After understanding the working principle of descriptors, which business scenarios are descriptors generally used in?

Here I use descriptors to implement an attribute checker. You can refer to this example and use it in similar scenarios.

First, we define a check base classValidator, in__set__MethodvalidateMethod to verify whether the property meets the requirements, and then assign the value to the property.

class Validator:

Next, we define two validation classes, inheritanceValidatorAnd then implement its own verification logic.

class Number(Validator):

Finally, we use this validation class:

class Person:

Now, when we talk aboutPersonWhen the instance is initialized, you can verify whether these properties conform to the predefined rules.

Function and method

Let’s take a look at what we often see in developmentfunctionunbound methodbound methodWhat’s the difference between them?

Here’s the code:

class A:

From the results, we can see the difference between them

  • functionIt’s exactly a function, and it implements it__get__Methods, so everyfunctionIs a non data descriptor, and in the class will befunctionput to__dict__Storage in
  • WhenfunctionWhen called by an instance, it is abound method
  • WhenfunctionWhen called by a class, it is aunbound method

functionIt’s a non data descriptor, as we talked about earlier.

andbound methodandunbound methodThe difference lies in the type of the caller. If it’s an instance, then thefunctionThat’s onebound methodOtherwise it is aunbound method

property/staticmethod/classmethod

Let’s look at it againpropertystaticmethodclassmethod

These decorators are implemented in C by default.

In fact, we can also directly use the features of Python descriptors to implement these decorators,

propertyImplementation of Python version of:

class property:

staticmethodImplementation of Python version of:

class staticmethod:

classmethodImplementation of Python version of:

class classmethod:

In addition, you can also implement other powerful decorators.

It can be seen that we can achieve powerful and flexible attribute management function through descriptors. For some scenes that require complex attribute control, we can choose to use descriptors to achieve it.

summary

In this article, we mainly talk about how Python descriptors work.

First of all, we learn from a simple example that a class property can be managed to another class. If the class implements the descriptor protocol method, then the class property is a descriptor. In addition, descriptors can be divided into data descriptors and non data descriptors.

After that, we analyzed the process of getting a property. All the entries are in the__getattribute__In, this method defines the order of finding attributes, where instance attributes take precedence over data descriptor calls, and data descriptors take precedence over non data descriptor calls.

In addition, we also learned that a method is actually a non data descriptor. If we define instance properties and methods with the same name in a class, we can__getattribute__The property search order of the instance is the priority.

At last, we analyze itfunctionandmethodIt can also be realized by using Python descriptorspropertystaticmethodclassmethodAdorner.

Python descriptors provide powerful attribute access control functions, which can be used in scenarios where complex control over attributes is required.

This work adoptsCC agreementReprint must indicate the author and the link of this article