Is your feature request related to a problem?
The PySpark dependency constraint is pinned to `<=4.0.0`, which excludes PySpark 4.0.1 (Sep 2025) and 4.0.2 (Feb 2026) maintenance releases.
Describe the solution you'd like
Widen from `<=4.0.0` to `<4.1.0` to include 4.0.x patch releases while excluding 4.1.x (which introduces Kafka client incompatibility with CI's Kafka 3.0.0). Also add proper PyPI classifiers for Python versions, license, and topic.
Additional context
PySpark 4.0.1 and 4.0.2 are maintenance-only releases compatible with the existing Kafka 3.0.0 setup in CI. PySpark 4.1.x bundles Kafka client classes that are incompatible.
Is your feature request related to a problem?
The PySpark dependency constraint is pinned to `<=4.0.0`, which excludes PySpark 4.0.1 (Sep 2025) and 4.0.2 (Feb 2026) maintenance releases.
Describe the solution you'd like
Widen from `<=4.0.0` to `<4.1.0` to include 4.0.x patch releases while excluding 4.1.x (which introduces Kafka client incompatibility with CI's Kafka 3.0.0). Also add proper PyPI classifiers for Python versions, license, and topic.
Additional context
PySpark 4.0.1 and 4.0.2 are maintenance-only releases compatible with the existing Kafka 3.0.0 setup in CI. PySpark 4.1.x bundles Kafka client classes that are incompatible.