Time zone problem in Python

Time:2021-9-28

Problem background

Python has been used for development for a long time and has not stepped into the time zone pit. Recently, more services have been introduced into the new business, and grpc is used for data communication. Unfortunately, I stepped into the time zone pit. Sure enough, there will be retribution for my laziness. So I sorted out the corresponding time zone problem and found the time zone problem in the previous database Mongo in the system, It is sorted as follows.

Basic concepts

Several time concepts

The first is several common concepts of time

  • GMT: Greenwich mean time, base time
  • UTC time: coordinated universal time, a more accurate benchmark time, which is basically equivalent to GMT
  • CST China base time: UTC time + 8 hours, i.e. 0:00 UTC time corresponds to 8:00 China base time, which is generally referred to as the time of Dongba district

ISO 8601

A standardized time representation method. The representation format is yyyy mm ddthh: mm: SS ± timezone. It can represent the time in different time zones. The time zone part is represented by Z as UTC standard time zone. Two examples:

  • 1997-07-16t08:20:30z represents 8:20:30 UTC time on July 16, 1997
  • 1997-07-16t19:20:30 + 08:00 indicates 19:20:30, July 16, 1997, East 8 District time

time stamp

The time of 00:00:00 UTC + 00:00 time zone on January 1, 1970 is called epoch time, which is recorded as 0. The current timestamp is the number of seconds from epoch time to the present, which is generally called timestamp. Therefore, a timestamp must correspond to a specific UTC time and a certain time in other time zones. Therefore, timestamp can be considered as a relatively safe time representation method.

Datetime practice

Datetime is the most basic time management package in Python. The following uses datetime to practice the corresponding time zone concept

Datetime type

Datetime is divided into two types:

  • Naive, the time of local type. This type is used when no time zone information is specified in datetime. This type of time zone determines the corresponding time zone according to the running environment. Therefore, this type of time will get different timestamps due to different running environments
  • Aware, a time with a time zone type, which corresponds to the determined timestamp because the time and time zone are determined

Examples are as follows:

?
1
2
3
4
5
6
from datetime import datetime, timezone
 
now = datetime.now()
now.tzinfo   # None
utc_now = datetime.now(timezone.utc)
utc_now.tzinfo # UTC

You can see that in the above example, now does not specify a time zone. It is a naive time, and its time zone is related to the running environment. And UTC_ Now specifies the UTC time zone, which is a time of type aware.

Get current time

  • Datetime. Now() can be used to obtain the current time and support setting the corresponding time zone. If the time zone is not set, the local time is obtained by default. According to whether the time zone is specified, it may wear naive type time or aware type time, but the corresponding timestamp is expected.
  • Datetime. Utcnow() is used cautiously to obtain the time corresponding to the current UTC, but the generated datetime object does not specify a time zone, so it uses the local time zone and creates a naive time. Therefore, if the operating environment is Dongba District, the time obtained is the time corresponding to UTC, but the time zone is Dongba District, the final time obtained will be 8 hours earlier than expected, and the time stamp obtained by conversion does not meet the expectation.

Examples are as follows:

?
1
2
3
4
from datetime import datetime
now = datetime.now()
now.timestamp() # 1610035129.323702 unow = datetime.utcnow()
unow.timestamp() # 1610006329.323797

Finally, run the above code in the environment of Dongba district on 2021-01-07 23:58:49. The time stamp obtained by now. Timestamp() is converted to the corresponding time of 2021-01-07 23:58:49 in Dongba District, but the time stamp obtained by UNOW. Timestamp() is 2021-01-07 15:58:49 in Dongba District, corresponding to UTC time 2021-01-07 07:58:49, which is completely inconsistent with the current UTC time.

Timestamp operation

  • Datetime. Timestamp() generates the timestamp corresponding to the current time
  • Datetime. Fromtimestamp() generates the time corresponding to the time zone of the running environment according to the time stamp
  • Datetime. Utcfromtimestamp() is used cautiously to generate the corresponding UTC time according to the timestamp. Because the generated datetime does not specify a time zone, it looks like the timestamp obtained is 8 hours ago

For the above example, we use the current timestamp 1610035129 obtained above to test as follows:

?
1
2
3
4
from datetime import datetime
 
timestamp = 1610035129
d1 = datetime.fromtimestamp(timestamp) # 2021-01-07 23:58:49 d2 = datetime.utcfromtimestamp(timestamp) # 2021-01-07 15:58:49

Finally, D1 is the correct time in the local time zone, but D2 is UTC. Yes, Jin, but there is no specified time zone, so it seems to be the local time 8 hours ago

time zone

The default built datetime has no time zone information. You can set the time zone for the time through datetime. Replace(), but you must ensure that the corresponding time matches the time zone information, otherwise it will lead to an incorrect time zone. A simple example is:

?
1
2
from datetime import datetime, timedelta, timezone
tz_utc_8 = timezone(timedelta(hours=8)) #Create the time zone UTC + 8:00, i.e. the time zone corresponding to the East 8 zone now = datetime. Now() # the default built time has no time zone DT = now. Replace (tzinfo = tz_utc_8) # it is forcibly set to UTC + 8:00

After setting the corresponding time zone, the corresponding date and time will remain unchanged. However, due to the new time zone, if it is different from the previous time zone, the corresponding timestamp will change. Be careful when using this method

Time zone conversion

You can convert a time with time zone information into another time zone through datetime. Astimezone(). A simple example is:

?
1
2
from datetime import datetime, timedelta, timezone
utc_dt = datetime.utcnow().replace(tzinfo=timezone.utc) #Built UTC's current time BJ_ dt = utc_ DT. Astimezone (timezone (timedelta (hours = 8))) # converts the time zone to the time of East Zone 8

After the conversion through astimezone(), although the time changes, it corresponds to the same reference time, so the corresponding timestamp is unchanged,

Grpc practice

In the use of grpc, it is designed to convert the timestamp object to time. The timestamp object supports the construction through the timestamp in Python, that is, the corresponding timestamp seconds of the current time, and also supports the construction through datetime. The corresponding interfaces are as follows:

  • Timestamp. Fromseconds() this method generates grpc timestamp objects based on the timestamp. There is no special place
  • Timestamp. Fromdatetime() uses this method cautiously to generate timestamp objects based on the datetime. It is implicitly expected that the datetime is UTC time. If an error is passed into the time of East Zone 8, it will result in an absolute time after 8 hours

We mixed the two methods in practice, and finally found that the timestamp obtained when calling fromdatetime () was completely unexpected. A simple example is as follows:

?
1
2
3
4
5
6
7
8
from datetime import datetime
from google.protobuf.timestamp_pb2 import Timestamp
 
now = datetime.now()
now_timestamp = int(now.timestamp()) # 1610245593 t1 = Timestamp()
t1.FromSeconds(now_timestamp) # 1610245593
t2 = Timestamp()
t2.FromDatetime(now) # 1610274393

You can see that the order timestamp obtained through fromdatetime () is inconsistent with the expectation. The two are consistent only when the incoming datetime is UTC

The interface converted to datetime object is:

  • Timestamp. Toseconds() this method obtains the corresponding integer timestamp according to the timestamp object. There is no problem
  • Timestamp. Todatetime() is used cautiously. This method generates datetime according to the timestamp object of grpc. The implied output datetime is UTC time, and the generated datetime has no time zone information. It will be processed according to the local time zone by default. If it is not processed, the result is 8 hours ago, and the corresponding timestamp is also wrong

Similar to the above problem, the time obtained through todatetime() is UTC time, but since the obtained datetime does not specify a time zone, the time obtained only in the UTC operating environment is expected.

Pymongo practice

When pymongo was used for data storage, the default setting of pymongo was directly used. The operating environment was set to East Zone 8. In use, datetime without a specified time zone was directly stored in the database, and then taken out for use. It seems that everything is normal. However, an obvious problem is found when viewing the data stored in the database when combing the time zone. The date and time stored in the database seem to be correct, but it is UTC time, that is, the actual stored time is 8 hours later than expected, but why can it work normally? The results after confirmation are as follows:

  • When pymongo does not specify a time zone, it does not consider this time as local time by default. In fact, it considers this time as UTC time. Finally, it will use this time to calculate and store the corresponding timestamp, so the final stored timestamp will be 8 hours late;
  • By default, the time returned from pymongo does not have a time zone, and the time is still UTC, so the calculated time will be 8 hours earlier, so the time looks normal.

How can we ensure that the correct time is saved and the return is in line with expectations?

  • The saved time can be set to the corresponding time zone, that is, avoid saving naive time, save aware time, and avoid entering the time considered UTC
  • Set the output time with time zone in pymongo to avoid the problem of default output time. Pymongo can use TZ_ Aware specifies the time zone of the output time. Tzinfo specifies the time zone of the output time. This setting can be passed in when building pymongo. The correspondence is as follows:
?
1
2
3
from datetime import timedelta, timezone
 
db = MongoClient(settings.MONGODB_DSN, tz_aware=True, tzifo=timezone(timedelta(hours=8))).get_default_database()

summary

According to the above practice, the three parts are used as follows:

  1. During the use of datetime, if the running environment is set to a non UTC time zone, it is recommended to disable the UTC related methods, such as utcnow, utcfromtimestamp(), and try to avoid using naive to ensure that the time is decoupled from the running environment;
  2. Avoid calling methods containing implicit information such as fromdatetime () and todatetime () in the use of grpc, and try to interact with grpc’s timestamp object through timestamp;
  3. Try to import time with time zone in pymongo, and configure time zone output to avoid hidden problems;

A general principle is: when interacting or storing with third-party services, try to use only the absolute mechanism of timestamp, so as to fundamentally eliminate the problem.

The above is the detailed content of the time zone problem in Python. For more information about Python time zone, please pay attention to other related articles in developepper!