Ticket #44615

generate_packets.py: Add type annotations

Open Date: 2022-05-15 23:33 Last Update: 2022-06-08 20:21

Reporter:
Owner:
Type:
Status:
Closed
Component:
MileStone:
Priority:
5 - Medium
Severity:
5 - Medium
Resolution:
Fixed
File:
1

Details

Part of #43927. Add PEP 484-style type annotations for use with IDEs and static type checkers.

The decision in #44735 renders the rest of the original ticket description moot. It is kept here as context for the discussion.

Problem: Python 3.4 has no special support for type annotations, beyond allowing function argument and return type annotations. In particular, the typing module is only introduced in Python 3.5, and variable annotations only in Python 3.6.
Potential solutions:

  • Leave out any annotations that would require typing
    In particular, this means no generic functions (which some helper functions could be)
  • Add workaround code to use typing only when available, while maintaining 3.4 compatibility
    This might add a certain amount of noise to the script
  • Increase the minimum supported Python version to 3.5 or even 3.6

The first of these options is likely the best one that doesn't require changing dependency versions.

Ticket History (3/12 Histories)

2022-05-15 23:33 Updated by: alienvalkyrie
  • New Ticket "generate_packets.py: Add type annotations" created
2022-05-15 23:51 Updated by: cazfi
Comment

Reply To alienvalkyrie

* Increase the minimum supported Python version to 3.5 or even 3.6

This script is quite fundamental part of our build (it's needed to build any component), though it's not needed when building from the tarball (one can work around python requirements by producing a tarball in another environment). Python-3.5 was released 2015.

Python-3.4 is already the highest (release date wise) requirement we have for building the server, and even for most of the guis.

2022-05-16 01:40 Updated by: alienvalkyrie
Comment

Reply To cazfi

Python-3.4 is already the highest (release date wise) requirement we have for building the server, and even for most of the guis.

...so what you're saying is, there's almost certainly a ton of new, useful features in less outdated versions of our other dependencies as well, that we could be taking advantage of to improve our code base — and the only real cost would be no longer supporting environments which only supply versions of these dependencies that have already been EOL for years anyway?

My point being; I'm not convinced forgoing the very tangible benefits provided by certain new features is worth it just to maintain compatibility with environments that only ship dependency versions which are years out-of-date.

2022-05-16 05:54 Updated by: cazfi
Comment

Reply To alienvalkyrie

Reply To cazfi

Python-3.4 is already the highest (release date wise) requirement we have for building the server, and even for most of the guis.

...so what you're saying is, there's almost certainly a ton of new, useful features in less outdated versions of our other dependencies as well

Not really. For most dependencies the development has been really slow.

2022-05-16 07:11 Updated by: cazfi
Comment

Reply To cazfi

Reply To alienvalkyrie

Reply To cazfi

Python-3.4 is already the highest (release date wise) requirement we have for building the server, and even for most of the guis.

...so what you're saying is, there's almost certainly a ton of new, useful features in less outdated versions of our other dependencies as well

Not really. For most dependencies the development has been really slow.

To clarify: We are talking about mandatory dependencies. Optional dependencies are completely another thing.

2022-05-30 03:31 Updated by: alienvalkyrie
  • Owner Update from (None) to alienvalkyrie
  • Details Updated
Comment

Reply To alienvalkyrie

Leave out any annotations that would require typing

This is the option I'll probably go with, not attempting any try-import shenanigans. I'll also, at this point in time at least, not add any imports for base types from e.g. collections.abc or contextlib – even where those would already be possible in Python 3.4 – for the sole purpose of use in type annotations; I don't think we'll need them very much anyway.

There are still certain considerations:

  • In order to use features supported by external type checkers, but not the runtime (Python 3.4), annotations can be wrapped in strings
    • String annotations are not syntax checked by Python (since they're just strings), which might make it a bit more difficult for contributors not using a type checker to produce code that will pass a type check (or rather, a bit easier to accidentally write syntactically invalid annotations)
      This is not a serious problem IMO, since I currently have no plans to make passing a type check mandatory in any way.
    • From Python 3.11 onwards (and from Python 3.7 with from __future__ import annotations), annotations will always be wrapped in a string (after syntax checking them) rather than evaluated, so this will no longer be necessary.
  • This allows parameterizing built-in generic types (mostly collections) PEP 585-style, which is otherwise only possible from Python 3.9 onwards
  • This also allows PEP 604-style union types without the need to import typing, which is otherwise only possible from Python 3.10 onwards

So long as there aren't any objections to it, I'm going to be making use of these options – the end result will be code that runs under Python 3.4, but uses more recent type checking features where reasonable and subsequently requires an up-to-date type checker supporting Python 3.10 features. I think this is a reasonable tradeoff, since static type checking is entirely optional, even for development.

(Unless, of course, we do decide to raise the minimum Python version to something that has only been EOL for under three years.)

2022-05-30 03:44 Updated by: cazfi
Comment

Reply To alienvalkyrie

we do decide to raise the minimum Python version

One option I have in mind is to start keeping our minimum python version, even for things outside meson build, in line with the minimum python version of our minimum meson version, starting from freeciv-3.3, i.e., in master as soon as S3_2 has been branched.

2022-05-31 19:30 Updated by: cazfi
Comment

Reply To cazfi

Reply To alienvalkyrie

we do decide to raise the minimum Python version

One option I have in mind is to start keeping our minimum python version, even for things outside meson build, in line with the minimum python version of our minimum meson version, starting from freeciv-3.3, i.e., in master as soon as S3_2 has been branched.

Python requirement bump discussion to a new ticket -> #44735

2022-06-07 02:10 Updated by: alienvalkyrie
Comment

With the decision in #44735 to increase the minimum Python version for freeciv-3.2 to Python 3.5, I will be using the typing module for abstract base types, typevars etc.; however, I'll still stick to aforementioned (string-wrapped) use of newer, preferred typing syntax, so "list[int | float]" instead of typing.List[typing.Union[int, float]] – the only point in evaluating annotations at runtime would be to check that they are well-formed, which is already being done by the static type checker.

2022-06-07 04:55 Updated by: alienvalkyrie
  • Resolution Update from None to Accepted
  • Details Updated
2022-06-08 20:21 Updated by: alienvalkyrie
  • Status Update from Open to Closed
  • Resolution Update from Accepted to Fixed

Attachment File List

Edit

Please login to add comment to this ticket » Login