fix: do not shift naive datetime query parameters by the host timezone - #901
Conversation
joe-clickhouse
left a comment
There was a problem hiding this comment.
Hey @knQzx thanks for this. I agree that this is the right thing to do. Naive datetimes sent verbatim as wall time matches how ClickHouse itself interprets datetime text and how nearly every other database driver handles naive bind params, so I decided to keep your change as the new default.
However, because the old conversion was long standing behavior I pushed some additional work on top of your commits to make it safe in 1.x:
- Added a global setting
naive_datetime_bindingwith valueswallandlegacy. The default iswall, your new behavior.legacyrestores the previous host-local conversion exactly. - Changed the awareness check to
utcoffset() is not None, which is Python's actual definition of an aware datetime. AtzinfowhoseutcoffsetreturnsNoneis still naive and would have reintroduced the shift. - Expanded the tests.
- Documented the new contract and the setting in the driver docs.
One related issue stays out of scope here. The insert path still interprets naive datetimes as host-local, so an insert followed by a filter with the same naive value behaves differently than before on non-UTC hosts. That inconsistency was existing but I'm tracking it in #938.
Thanks again for catching this. CI never sees it because it runs in UTC...which is why it survived so long.
There was a problem hiding this comment.
Pull request overview
Fixes incorrect timezone shifting for timezone-unaware (naive) datetime values used as query parameters, where astimezone(...) previously treated naive values as host-local time and silently offset them on non-UTC hosts. The change introduces an explicit binding mode setting so existing workloads can opt back into legacy host-local behavior.
Changes:
- Update datetime parameter formatting to only call
astimezone(...)for timezone-aware values by default, and add a"legacy"compatibility mode viacommon.set_setting("naive_datetime_binding", ...). - Add unit and integration regression coverage that forces a non-UTC process timezone and validates wall-time vs legacy behavior across scalar and container binds.
- Document the new semantics and setting, and add a Behavior Changes entry to
CHANGELOG.md.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
clickhouse_connect/driver/binding.py |
Implements wall-time binding for naive datetimes by default, with an opt-in legacy mode that preserves the prior host-local conversion behavior. |
clickhouse_connect/common.py |
Registers the new global setting naive_datetime_binding with options wall (default) and legacy. |
tests/unit_tests/test_driver/test_settings.py |
Adds unit coverage for the new global setting validation and option handling. |
tests/unit_tests/test_driver/test_params.py |
Adds focused unit regression tests for naive vs aware datetime formatting and binding, including non-UTC host TZ scenarios and container shapes. |
tests/integration_tests/test_timezones.py |
Updates timezone binding integration tests to exercise wall-time semantics for naive values while keeping aware conversions correct. |
tests/integration_tests/test_params.py |
Adds live integration coverage to ensure naive wall-time binds round-trip correctly against a real server under non-UTC process TZ. |
docs/driver-api.mdx |
Documents wall-time semantics for naive datetime query parameters and the legacy compatibility setting. |
docs/advanced-inserting.mdx |
Clarifies that inserts still interpret naive datetimes as local time, while query parameter binding does not. |
docs/additional-options.mdx |
Documents naive_datetime_binding as a binding-time setting and adds it to the global settings reference table. |
CHANGELOG.md |
Adds an UNRELEASED Behavior Changes entry describing the user-visible semantics change and the legacy fallback. |
Naive (timezone-unaware)
datetimevalues bound as query parameters are silently shifted by the client host's local timezone. the bind paths callvalue.astimezone(server_tz), and for a naive valueastimezonetreats it as host-local time, so on any non-UTC host everyDateTime/DateTime64bind or filter value is offset and matches the wrong rows. CI does not catch it because it runs withTZ=UTC, where local == UTCreproduce: on a non-UTC host (or with
TZ=Europe/Berlin), bind a naivedatetime(2025, 1, 1, 12, 0, 0)to a parameter - the value sent to the server is shifted by the host offset instead of staying2025-01-01 12:00:00. the added regression test forces a non-UTCTZand asserts a naive value is written verbatim while a timezone-aware value is still converted to the server timezonethe fix only calls
astimezonewhen the value is timezone-aware; naive datetimes are formatted as-is