4
HANDLING TIMESTAMPS AND TIME ZONES

Time zones are complicated. Most people expect dealing with time zones to involve merely adding or subtracting a few hours from the universal time reference, UTC (Coordinated Universal Time), from −12 hours to +12 hours.

However, reality shows otherwise: time zones are not logical or predictable. There are time zones with 15-minute granularity; countries that change time zones twice a year; countries that use a custom time zone during summer, known as daylight saving time, that starts on different dates; plus tons of special and corner cases. These make the history of time zones interesting but also complicate how to handle them. All of those particularities should make you stop and think when dealing with time zones.

This chapter will outline why dealing with time zones is tricky and how to best handle them in your programs. We’ll look at how to build timestamp objects, how and why to make them time zone aware, and how to deal with corner cases you might come across.

The Problem of Missing Time Zones

A timestamp without a time zone attached gives no useful information, because without the time zone, you cannot infer what point in time your application is really referring to. Without their respective time zones, therefore, you can’t compare two timestamps; that would be like comparing days of the week without accompanying dates—whether Monday is before or after Tuesday depends on what weeks they’re in. Timestamps without time zones attached should be considered irrelevant.

For that reason, your application should never have to handle timestamps with no time zone. Instead, it must raise an error if no time zone is provided, or it should make clear what default time zone is assumed—for example, it’s common practice to choose UTC as the default time zone.

You also must be careful of making any kind of time zone conversion before storing your timestamps. Imagine a user creates a recurring event every Wednesday at 10:00 AM in their local time zone, say Central European Time (CET). CET is an hour ahead of UTC, so if you convert that timestamp to UTC to store it, the event will be stored as every Wednesday at 09:00 AM. The CET time zone switches from UTC+01:00 to UTC+02:00 in the summer, so on top of that, in the summer months, your application will compute that the event starts at 11:00 AM CET every Wednesday. You can see how this program quickly becomes redundant!

Now that you understand the general problem of handling time zones, let’s dig into our favorite language. Python comes with a timestamp object named datetime.datetime that can store date and time precise to the microsecond. The datetime.datetime object can be either time zone aware, in which case it embeds time zone information, or time zone unaware, in which case it does not. Unfortunately, the datetime API returns a time zone–unaware object by default, as you’ll soon see in Listing 4-1. Let’s look at how to build a default timestamp object and then how to rectify it so that it uses time zones.

Building Default datetime Objects

To build a datetime object with the current date and time as values, you can use the datetime.datetime.utcnow() function. This function retrieves the date and time for the UTC time zone right now, as shown in Listing 4-1. To build this same object using the date and time for the time zone of the region the machine is in, you can use the datetime.datetime.now() method. Listing 4-1 retrieves the time and date for both UTC and my region’s time zone.

>>> import datetime
>>> datetime.datetime.utcnow()
  ➊ datetime.datetime(2018, 6, 15, 13, 24, 48, 27631)
>>> datetime.datetime.utcnow().tzinfo is None
  ➋ True

Listing 4-1: Getting the time of the day with datetime

We import the datetime library and define the datetime object as using the UTC time zone. This returns a UTC timestamp whose values are year, month, date, hours, minutes, seconds, and microseconds ➊, respectively, in the listing. We can check whether this object has time zone information by checking the tzinfo object, and here we’re told that it doesn’t ➋.

We then create the datetime object using the datetime.datetime.now() method to retrieve the current date and time in the default time zone for the region of the machine:

>>> datetime.datetime.now()
  ➌ datetime.datetime(2018, 6, 15, 15, 24, 52, 276161)

This timestamp, too, is returned without any time zone, as we can tell from the absence of the tzinfo field ➌—if the time zone information had been present, it would have appeared at the end of the output as something like tzinfo=<UTC>.

The datetime API always returns unaware datetime objects by default, and since there is no way for you to tell what the time zone is from the output, these objects are pretty useless.

Armin Ronacher, creator of the Flask framework, suggests that an application should always assume the unaware datetime objects in Python are UTC. However, as we just saw, this doesn’t work for objects returned by datetime.datetime.now(). When you are building datetime objects, I strongly recommend that you always make sure they are time zone aware. That ensures you can always compare your objects directly and check whether they are returned correctly with the information you need. Let’s see how to create time zone–aware timestamps using tzinfo objects.

Time Zone–Aware Timestamps with dateutil

There are already many databases of existing time zones, maintained by central authorities such as IANA (Internet Assigned Numbers Authority), which are shipped with all major operating systems. For this reason, rather than creating our own time zone classes and manually duplicating those in each Python project, Python developers rely on the dateutil project to obtain tzinfo classes. The dateutil project provides the Python module tz, which makes time zone information available directly, without much effort: the tz module can access the operating system’s time zone information, as well as ship and embed the time zone database so it is directly accessible from Python.

You can install dateutil using pip with the command pip install python-dateutil. The dateutil API allows you to obtain a tzinfo object based on a time zone name, like so:

>>> from dateutil import tz
>>> tz.gettz("Europe/Paris")
tzfile('/usr/share/zoneinfo/Europe/Paris')
>>> tz.gettz("GMT+1")
tzstr('GMT+1')

The dateutil.tz.gettz() method returns an object implementing the tzinfo interface. This method accepts various string formats as argument, such as the time zone based on a location (for example, “Europe/Paris”) or a time zone relative to GMT. The dateutil time zone objects can be used as tzinfo classes directly, as demonstrated in Listing 4-3.

>>> import datetime
>>> from dateutil import tz
>>> now = datetime.datetime.now()
>>> now
datetime.datetime(2018, 10, 16, 19, 40, 18, 279100)
>>> tz = tz.gettz("Europe/Paris")
>>> now.replace(tzinfo=tz)
datetime.datetime(2018, 10, 16, 19, 40, 18, 279100, tzinfo=tzfile('/usr/share/zoneinfo/Europe/
Paris'))

Listing 4-3: Using dateutil objects as tzinfo classes

As long as you know the name of the desired time zone, you can obtain a tzinfo object that matches the time zone you target. The dateutil module can access the time zone managed by the operating system, and if that information is for some reason unavailable, will fall back on its own list of embedded time zones. If you ever need to access this embedded list, you can do so via the datetutil.zoneinfo module:

>>> from dateutil.zoneinfo import get_zonefile_instance
>>> zones = list(get_zonefile_instance().zones)
>>> sorted(zones)[:5]
['Africa/Abidjan', 'Africa/Accra', 'Africa/Addis_Ababa', 'Africa/Algiers', 'Africa/Asmara']
>>> len(zones)
592

In some cases, your program does not know which time zone it’s running in, so you’ll need to determine it yourself. The datetutil.tz.gettz() function will return the local time zone of your computer if you pass no argument to it, as shown in Listing 4-4.

>>> from dateutil import tz
>>> import datetime
>>> now = datetime.datetime.now()
>>> localzone = tz.gettz()
>>> localzone
tzfile('/etc/localtime')
>>> localzone.tzname(datetime.datetime(2018, 10, 19))
'CEST'
>>> localzone.tzname(datetime.datetime(2018, 11, 19))
'CET'

Listing 4-4: Obtaining your local time zone

As you can see, we pass two dates to localzone.tzname(datetime.datetime()) separately, and dateutil is able to tell us that one is in Central European Summer Time (CEST) and the other is in Central European Time (no summer). If you pass in your current date, you’ll get your own current time zone.

You can use objects from the dateutil library in tzinfo classes without having to bother implementing those yourself in your application. This makes it easy to convert unaware datetime objects to aware datetime objects.

Serializing Time Zone–Aware datetime Objects

You’ll often need to transport a datetime object from one point to another, where those different points might not be Python native. The typical case nowadays would be with an HTTP REST API, which must return datetime objects serialized to a client. The native Python method named isoformat can be used to serialize datetime objects for non-Python native points, as shown in Listing 4-5.

   >>> import datetime
   >>> from dateutil import tz
➊ >>> def utcnow():
       return datetime.datetime.now(tz=tz.tzutc())
   >>> utcnow()
➋ datetime.datetime(2018, 6, 15, 14, 45, 19, 182703, tzinfo=tzutc())
➌ >>> utcnow().isoformat()
   '2018-06-15T14:45:21.982600+00:00'

Listing 4-5: Serializing a time zone–aware datetime object

We define a new function called utcnow and tell it explicitly to return an object with the UTC time zone ➊. As you can see, the object returned now contains time zone information ➋. We then format the string using the ISO format ➌, ensuring the timestamp also contains some time zone information (the +00:00 part).

You can see I’ve used the method isoformat() to format the output. I recommend that you always format your datetime input and output strings using ISO 8601, with the method datetime.datetime.isoformat(), to return timestamps formatted in a readable way that includes the time zone information.

Your ISO 8601–formatted strings can then be converted to native datetime.datetime objects. The iso8601 module offers only one function, parse_date, which does all the hard work of parsing the string and determining the timestamp and time zone values. The iso8601 module is not provided as a built-in module in Python, so you need to install it using pip install iso8601. Listing 4-6 shows how to parse a timestamp using ISO 8601.

   >>> import iso8601
   >>> import datetime
   >>> from dateutil import tz
   >>> now = datetime.datetime.utcnow()
   >>> now.isoformat()
   '2018-06-19T09:42:00.764337'
➊ >>> parsed = iso8601.parse_date(now.isoformat())
   >>> parsed
   datetime.datetime(2018, 6, 19, 9, 42, 0, 764337, tzinfo=<iso8601.Utc>)
   >>> parsed == now.replace(tzinfo=tz.tzutc())
   True

Listing 4-6: Using the iso8601 module to parse an ISO 8601–formatted timestamp

In Listing 4-6, the iso8601 module is used to construct a datetime object from a string. By calling iso8601.parse_date on a string containing an ISO 8601–formatted timestamp ➊, the library is able to return a datetime object. Since that string does not contain any time zone information, the iso8601 module assumes that the time zone is UTC. If a time zone contains correct time zone information, the iso8601 module returns correctly.

Using time zone–aware datetime objects and using ISO 8601 as the format for their string representation is a perfect solution for most problems around time zone, making sure no mistakes are made and building great interoperability between your application and the outside world.

Solving Ambiguous Times

There are certain cases where the time of the day can be ambiguous; for example during the daylight saving time transition when the same “wall clock” time occurs twice a day. The dateutil library provides us with the is_ambiguous method to distinguish such timestamps. To show this in action, we’ll create an ambiguous timestamp in Listing 4-7.

>>> import dateutil.tz
>>> localtz = dateutil.tz.gettz("Europe/Paris")
>>> confusing = datetime.datetime(2017, 10, 29, 2, 30)
>>> localtz.is_ambiguous(confusing)
True

Listing 4-7: A confusing timestamp, occurring during the daylight saving time crossover

On the night of October 30, 2017, Paris switched from summer to winter time. The city switched at 3:00 AM, when the time goes back to 2:00 AM. If we try to use a timestamp at 2:30 on that date, there is no way for this object to be sure whether it is after or before the daylight saving time change.

However, it is possible to specify which side of the fold a timestamp is on by using the fold attribute, added to datetime objects from Python 3.6 by PEP 495 (Local Time Disambiguation—https://www.python.org/dev/peps/pep-0495/). This attribute indicates which side of the fold the datetime is on, as demonstrated in Listing 4-8.

>>> import dateutil.tz
>>> import datetime
>>> localtz = dateutil.tz.gettz("Europe/Paris")
>>> utc = dateutil.tz.tzutc()
>>> confusing = datetime.datetime(2017, 10, 29, 2, 30, tzinfo=localtz)
>>> confusing.replace(fold=0).astime zone(utc)
datetime.datetime(2017, 10, 29, 0, 30, tzinfo=tzutc())
>>> confusing.replace(fold=1).astime zone(utc)
datetime.datetime(2017, 10, 29, 1, 30, tzinfo=tzutc())

Listing 4-8: Disambiguating the ambiguous timestamp

You’ll need to use this in only very rare cases, since ambiguous timestamps occur only in a small window. Sticking to UTC is a great workaround to keep life simple and avoid running into time zone issues. However, it is good to know that the fold attribute exists and that dateutil is able to help in such cases.

Summary

In this chapter, we have seen how crucial it is to carry time zone information in time stamps. The built-in datetime module is not complete in this regard, but the dateutil module is a great complement: it allows us to get tzinfo-compatible objects that are ready to be used. The dateutil module also helps us solve subtle issues such as daylight saving time ambiguity.

The ISO 8601 standard format is an excellent choice for serializing and unserializing timestamps because it is readily available in Python and compatible with any other programming language.