Integer quantities support status

[RBerga06]

The docs mention that it makes sense to write Quantity[int] (or Quantity[np.ndarray[<shape>, <integer dtype>]]).

However, I think this poses some problems, because then a lot of operations can break this type invariance: for example, Quantity(3, 'cm').to_root_units() returns Quantity(0.03, "meter").

The real problem comes when we consider inplace operations:

def foo(x: Quantity[int]):
    reveal_type(x.magnitude)  # int (good)
	x.ito_root_units()
	reveal_type(x.magnitude)  # type checker thinks it's int, but might be float at runtime

I don't think there's any way of telling type checkers that an in-place operation might've changed the type of a variable.

Is this expected behaviour? Should the docs be updated with a note about this?

---

Note: since type checkers think int <: float (and it's still an open problem), as long as MagnitudeT (in Quantity's definition) remains covariant (which simplifies a lot arithmetic method overloads) it's not possible to make the Quantity[int] annotation an error – the best we can do is document it.

However, in the case of numpy arrays, integer and floating dtypes are not in a subtype-supertype relationship, which means we can simply accept floating and complex arrays while the boolean/integer ones will be disallowed by default in the annotations.

---

Edit: apparently x.ito_root_units() fails at runtime if the magnitude is an integer array. The error is:
numpy._core._exceptions._UFuncOutputCastingError: Cannot cast ufunc 'multiply' output from dtype('float64') to dtype('int64') with casting rule 'same_kind'.
This happens because we're essentially doing self.magnitude *= <float> and numpy correctly raises because this operation cannot preserve the LHS type.

However, x.to_root_units() works correctly and returns a float array (following the data type promotion rules).

[andrewgsavage]

I hadn't thought of this but it's a great point.

Adding a note or recommendation to avoid using int would be helpful

Could you expand on the numpy case with an example?

[RBerga06]

See the numpy dtype hierarchy here.
The idea is we can require (via type annotations) magnitude arrays to have dtype np.floating (or even better np.inexact to support complex numbers as well).

Small example:

from typing import Any
import numpy as np

def foo(x: np.ndarray[Any, np.dtype[np.inexact]]):
    pass

a = np.array([1, 2, 3], dtype=np.int64)
b = np.array([1.0, 2.0, 3.0], dtype=np.float64)
foo(a)  # pyright error: `np.int64` is not assignable to `np.inexact`
foo(b)  # ok

(btw, this works because numpy arrays are covariant in their dtype, which makes sense because all float64 arrays are indeed inexact array.)

Edit: I've just checked runtime behaviour and apparently in-place operations like ito_root_units() error out if the magnitude is an integer numpy array: I have updated my previous post accordingly.

I can now see two routes that could be taken:

A. Document the status quo in type annotations (and maybe also in the docs).

Quantities with integer array magnitudes support most methods, except for the mutating/inplace ones, which are restricted to quantities with np.inexact magnitudes, while Quantity.to_root_units() and such perform type promotion like numpy does.

I think this would require a lot more effort than option B, but it would not introduce runtime changes and in particular it would be backwards-compatible.

B. Document integer magnitudes (including arrays) as not supported

In my head, this would mean writing something like:

type Magnitude = float | complex | np.inexact | np.ndarray[Any, np.dtype[np.inexact]]

This denies integer arrays and makes Quantity[np.ndarray[..., np.dtype[np.int64]]] and friends typing errors (because _MagnitudeT_co is bounded to Magnitude).

B'. Additionally, support integer arrays in the constructor only

Of course, I think we should still allow calling Quantity(a) (a from the example):
I propose to add an overload to __new__ encoding Quantity(<int array>) -> Quantity[<float array with the same shape>], in the same spirit of what I just did for Quantity(<datetime>) -> Quantity[float] Quantity(<timedelta>) -> Quantity[float] in #2373 (we can add Quantity(<int>) -> Quantity[float] as well).

However, this would mean:

from typing import reveal_type
import numpy as np
import optype.numpy as npt
from pint import UnitRegistry
ureg = UnitRegistry()

a: npt.Array1D[np.int64] = np.array([1, 2, 3], dtype=np.int64)
q = ureg.Quantity(a, "cm")	# q is inferred to be Quantity[npt.Array1D[np.float64]]
reveal_type(q.magnitude)	# npt.Array1D[np.float64]
print(q.magnitude.dtype)	# np.int64
q.ito_root_units()			# runtime error!

To be consistent, it would be necessary to change runtime behaviour and make Quantity actually convert integer magnitudes to floating-point ones.

That would probably be a backwards-incompatible change, but anyone who was relying on the old behaviour could run into the runtime error by adding a seemingly-innocuous operation. In fact, I myself initially assumed x.ito_root_units() would change the magnitude type in-place, like it does for int -> float, without returning an error.

[andrewgsavage]

I think the inplace operators are not often used, I cannot recall seeing them.

The trouble is people ususally use ints as shorthand for floats. It's very very rare to see problems with physical units where ints are more logical than floats. Often people use int to avoid rounding/formatting...
Though pint is used for non-physical units like pixels where ints make some sense.

A is not fun,
B and B', integers are too widely used to deny

maybe another option is to modify the inplace operators to maintain ints. ie do the operation and convert back to ints, losing some fidelity. Similar to how inplace works:

a=3
a//=2
print(a)
1

Actually never mind; if you use a float it converts the int

a=3
a//=2.
print(a)
1.5

[RBerga06]

Though pint is used for non-physical units like pixels where ints make some sense.

Yeah, that's actually a good point (although for some calculations subpixel distances might make sense as well).

The trouble is people ususally use ints as shorthand for floats
B and B', integers are too widely used to deny

Yes, that's what I imagined. The rationale behind B' would be that Quantity([1, 2, 3], "m") becomes automatically Quantity([1.0, 2.0, 3.0], "m") precisely for that reason, although one might find it surprising that this means Quantity(x, "m").magnitude can be different from x in these cases (but == will return True, or in the case of numpy arrays, an array full of Trues).

[RBerga06]

maybe another option is to modify the inplace operators to maintain ints

On second thought, this might be possible when the RHS of those operations is a float that can be represented as an integer exactly. For example, one might expect this behaviour:

>>> q = Quantity(3, "m")
>>> q.to("mm")
Quantity(3000, "mm")   # still an integer, like 3 * 1000 in Python
>>> q.to("km")
Quantity(0.003, "km")  # now this is a float, like 3 / 1000 which in Python returns float

The question then would be how to treat q.to("mm").to("m"), because in Python 3000 / 1000 -> 3.0 (a float), which means if we want to keep Python's behaviour here q.to("mm").to("m") would return Quantity(3.0, "m"), losing the int-ness even though it could be represented as an integer.

Maybe something similar could also make sense for numpy integer arrays.

I'll note that this would complicate things typing-wise, because the return type for these operations would depend on the ratios between units with the same dimensionality, and I think there's no way that will ever get encoded that in the type system.

---

Whatever gets decided, anyone who wants to extract integer magnitude from a float quantity can always use numpy's .astype(np.int64) on the magnitude (and if we can make sense of integer quantities, it would be possible to also use Quantity.astype(np.int64)).