[Bug 207604] Re: pytz includes a hard coded list of time zone names

Thu Mar 30 22:44:33 UTC 2023

** Description changed:

  [ Impact ]

  The Debian/Ubuntu packages of pytz include a patch to not install pytz's
  zoneinfo database and instead load data from the system zoneinfo
  database.  This keeps pytz up to date when system timezone updates are
  installed which is generally a good idea.

  The pytz/__init__.py includes common_timezones and all_timezones lists,
  which enumerate the time zones in the database pytz was released with.
  If pytz is instead using the system database, these lists may not be
  correct (e.g. missing newly added time zones).

  all_timezones and common_timezones should be calculated at run time.
  all_timezones will need to use os.walk to discover what timezones exist.
  common_timezones should be loaded from zone.tab, plus a hard coded list
  of extras such as UTC and US/Eastern.
+ 
+ Everytime a new timezone is added to tzdata in an update, this timezone
+ cannot be used in pytz with the hardcoded list of timezones. For jammy
+ and kinetic this applies to America/Ciudad_Juarez (when comparing pytz
+ to tzdata 2023c). For bionic the list of missing timezones is even
+ bigger:
+ 
+ * America/Ciudad_Juarez
+ * America/Nuuk
+ * Asia/Qostanay
+ * Europe/Kyiv
+ * Pacific/Kanton

  [ Test Plan ]

  Two autopkgtest test cases were added:
  1. Run the upstream unittests
  2. Regression tests with all tests that I could come up with.

  There are no manual test, because all those test should also be run when
  there is a tzdata update to prevent regressions.

  [ Where problems could occur ]

  Dynamically determine the timezones relies on a correct environment and
  can fail more easily if the environment is broken. python-tz is used in
  several places (maybe in the installer). The patch might have a
  performance impact, because the list of timezones is determined
  completely (but only once) instead of constructing it lazy.

  [ Other Info ]

  Upstream recommend our approach:
  https://github.com/stub42/pytz/issues/91#issuecomment-1356628324

** Description changed:

  [ Impact ]

  The Debian/Ubuntu packages of pytz include a patch to not install pytz's
  zoneinfo database and instead load data from the system zoneinfo
  database.  This keeps pytz up to date when system timezone updates are
  installed which is generally a good idea.

  The pytz/__init__.py includes common_timezones and all_timezones lists,
  which enumerate the time zones in the database pytz was released with.
  If pytz is instead using the system database, these lists may not be
  correct (e.g. missing newly added time zones).

  all_timezones and common_timezones should be calculated at run time.
  all_timezones will need to use os.walk to discover what timezones exist.
  common_timezones should be loaded from zone.tab, plus a hard coded list
  of extras such as UTC and US/Eastern.

  Everytime a new timezone is added to tzdata in an update, this timezone
  cannot be used in pytz with the hardcoded list of timezones. For jammy
  and kinetic this applies to America/Ciudad_Juarez (when comparing pytz
  to tzdata 2023c). For bionic the list of missing timezones is even
  bigger:

  * America/Ciudad_Juarez
  * America/Nuuk
  * Asia/Qostanay
  * Europe/Kyiv
  * Pacific/Kanton

  [ Test Plan ]

  Two autopkgtest test cases were added:
  1. Run the upstream unittests
  2. Regression tests with all tests that I could come up with.

  There are no manual test, because all those test should also be run when
  there is a tzdata update to prevent regressions.

+ To manually test, a newly added timezone to tzdata should be able to be
+ specified:
+ 
+ ```
+ #!/usr/bin/python3
+ import pytz
+ pytz.timezone("America/Ciudad_Juarez")
+ ```
+ 
+ This should not raise UnknownTimeZoneError.
+ 
+ 
  [ Where problems could occur ]

  Dynamically determine the timezones relies on a correct environment and
  can fail more easily if the environment is broken. python-tz is used in
  several places (maybe in the installer). The patch might have a
  performance impact, because the list of timezones is determined
  completely (but only once) instead of constructing it lazy.

  [ Other Info ]

  Upstream recommend our approach:
  https://github.com/stub42/pytz/issues/91#issuecomment-1356628324

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/207604

Title:
  pytz includes a hard coded list of time zone names

Status in pytz:
  New
Status in python-tz package in Ubuntu:
  Fix Released
Status in python-tz source package in Jammy:
  Incomplete
Status in python-tz source package in Kinetic:
  Incomplete

Bug description:
  [ Impact ]

  The Debian/Ubuntu packages of pytz include a patch to not install
  pytz's zoneinfo database and instead load data from the system
  zoneinfo database.  This keeps pytz up to date when system timezone
  updates are installed which is generally a good idea.

  The pytz/__init__.py includes common_timezones and all_timezones
  lists, which enumerate the time zones in the database pytz was
  released with.  If pytz is instead using the system database, these
  lists may not be correct (e.g. missing newly added time zones).

  all_timezones and common_timezones should be calculated at run time.
  all_timezones will need to use os.walk to discover what timezones
  exist. common_timezones should be loaded from zone.tab, plus a hard
  coded list of extras such as UTC and US/Eastern.

  Everytime a new timezone is added to tzdata in an update, this
  timezone cannot be used in pytz with the hardcoded list of timezones.
  For jammy and kinetic this applies to America/Ciudad_Juarez (when
  comparing pytz to tzdata 2023c). For bionic the list of missing
  timezones is even bigger:

  * America/Ciudad_Juarez
  * America/Nuuk
  * Asia/Qostanay
  * Europe/Kyiv
  * Pacific/Kanton

  [ Test Plan ]

  Two autopkgtest test cases were added:
  1. Run the upstream unittests
  2. Regression tests with all tests that I could come up with.

  There are no manual test, because all those test should also be run
  when there is a tzdata update to prevent regressions.

  To manually test, a newly added timezone to tzdata should be able to
  be specified:

  ```
  #!/usr/bin/python3
  import pytz
  pytz.timezone("America/Ciudad_Juarez")
  ```

  This should not raise UnknownTimeZoneError.

  [ Where problems could occur ]

  Dynamically determine the timezones relies on a correct environment
  and can fail more easily if the environment is broken. python-tz is
  used in several places (maybe in the installer). The patch might have
  a performance impact, because the list of timezones is determined
  completely (but only once) instead of constructing it lazy.

  [ Other Info ]

  Upstream recommend our approach:
  https://github.com/stub42/pytz/issues/91#issuecomment-1356628324

To manage notifications about this bug go to:
https://bugs.launchpad.net/pytz/+bug/207604/+subscriptions