v0.0.47

What's Changed

• Kinesis: Added ctk kinesis CLI group with list-checkpoints and
  prune-checkpoints commands for checkpoint table maintenance.
  Thanks, @hampsterx.
• Dependencies: Permitted installation of click 8.3
• DataQuery: Help agents turn natural language into SQL queries

Details

• CI: Fix cloud integration tests not skipping on fork PRs by @hampsterx in #704
• CI: Fix - Replace sleep-based Kinesis test sync with deterministic wait_ready() by @hampsterx in #703
• Update ASzc/change-string-case-action action to v8 by @renovate[bot] in #702
• Update dependency uvicorn to <0.43 by @renovate[bot] in #705
• Update dependency marimo to <0.22 by @renovate[bot] in #709
• Update dependency aiobotocore to <3.4 by @renovate[bot] in #710
• Update dependency attrs to v26 - autoclosed by @renovate[bot] in #712
• Update dependency dlt to >=1.10,<1.25 by @renovate[bot] in #711
• Chore: Fix Iceberg test leftover after ctk load interface change by @amotl in #715
• Dependencies: Permit installation of click 8.3 by @amotl in #714
• Tests: Use DROP SCHEMA ... CASCADE to streamline data reset operations by @amotl in #717
• Fix Dask ProgressBar writing to closed file in tests by @hampsterx in #719
• Info: Include sys.cluster_health in ctk info cluster diagnostics by @hampsterx in #721
• Tests: Fix flaky test_kinesis_latest_dynamodb_cdc_insert_update by @hampsterx in #722
• Relay: Propagate write failures, bump async-kinesis to 2.4.0 by @hampsterx in #718
• OCI: Don't miss .gitignore in Dockerfile when building OCI by @amotl in #723
• CI: Validate Kinesis adapter on Python 3.10 by @amotl in #724
• Docs: Remove ambiguous file:// prefix from ILP examples by @hampsterx in #720
• Chore: Configure ruff E402 suppression within test files by @amotl in #729
• Update codecov/codecov-action action to v6 by @renovate[bot] in #730
• QA: Migrate type checker from mypy to ty by @amotl in #725
• Update dependency ty to v0.0.26 by @renovate[bot] in #731
• Documentation: Add and configure sphinx-sitemap extension by @amotl in #732
• Documentation: Use site.xml to avoid collision with RTD sitemap by @amotl in #733
• feat(kinesis): add persistent checkpointing for stream consumers by @hampsterx in #728
• Update dependency ty to v0.0.27 by @renovate[bot] in #734
• Update dependency marimo to <0.23 by @renovate[bot] in #735
• Update dependency mcp to <1.28 by @renovate[bot] in #738
• Update dependency ty to v0.0.28 by @renovate[bot] in #739
• Update dependency uvicorn to <0.44 by @renovate[bot] in #740
• Update dependency ty to v0.0.29 by @renovate[bot] in #741
• Update dependency uvicorn to <0.45 by @renovate[bot] in #742
• feat(kinesis): add ctk kinesis CLI with checkpoint listing and pruning by @hampsterx in #736
• Update dependency aiobotocore to <3.5 by @renovate[bot] in #743
• Documentation: Remove special configuration for sphinx-sitemap by @amotl in #744
• Kinesis: Improve SQLAlchemy compatibility by @amotl in #748
• chore(deps): update dependency botocore to <1.43 by @renovate[bot] in #751
• chore(deps): update dependency docutils to <0.23 by @renovate[bot] in #752
• chore(deps): update dependency ty to v0.0.30 by @renovate[bot] in #756
• chore(deps): update dependency dlt to >=1.10,<1.26 by @renovate[bot] in #763
• chore(deps): update dependency ty to v0.0.31 by @renovate[bot] in #765
• chore(deps): update dependency fastapi to <0.137 by @renovate[bot] in #766
• chore(deps): update dependency polars to <1.41 by @renovate[bot] in #767
• chore(deps): update dependency marimo to <0.24 by @renovate[bot] in #745
• chore(deps): update dependency ty to v0.0.32 by @renovate[bot] in #769
• chore(deps): update dependency azure-storage-blob to <12.29 by @renovate[bot] in #750
• chore(deps): update dependency rich to v15 by @renovate[bot] in #746
• chore(deps): update dependency aiobotocore to <3.6 by @renovate[bot] in #771
• chore(deps): update dependency uvicorn to <0.46 by @renovate[bot] in #772
• chore(deps): update dependency pyarrow to v24 by @renovate[bot] in #773
• chore(deps): update astral-sh/setup-uv action to v8 by @renovate[bot] in #775
• chore(deps): update dependency uvicorn to <0.47 by @renovate[bot] in #776
• chore(deps): update dependency kaggle to <2.2 by @renovate[bot] in #777
• This and that 202604 by @amotl in #779

Full Changelog: v0.0.46...v0.0.47

v0.0.46

What's Changed

• I/O: API improvements: ctk {load,save} table became ctk {load,save}
  Also, the --cluster-url option becomes optional. That's a much more
  concise interface now.

Details

• I/O: API improvements: ctk {load,save} table becomes ctk {load,save} by @amotl in #706

Full Changelog: v0.0.45...v0.0.46

v0.0.45

What's Changed

• I/O: Refactored ctk.io subsystem
• I/O: Refurbished documentation of the I/O subsystem
• I/O: Started using ingestr for loading data from Amazon Kinesis
• I/O: Started propagating start_date parameter to ingestr's interval_start
• I/O: Started propagating batch_size parameter to ingestr's page_size
• Packaging: Renamed extras all → full, io-ingestr → io-ingest, and io-all → io-curated

Details

• Tests: Process deprecation warnings of testcontainers package by @amotl in #680
• Update docker/setup-buildx-action action to v4 by @renovate[bot] in #683
• Update docker/metadata-action action to v6 by @renovate[bot] in #684
• Update docker/build-push-action action to v7 by @renovate[bot] in #685
• Tests: Fix testcontainers. CI: Use uv across the board. by @amotl in #686
• Update dependency dlt to >=1.10,<1.24 by @renovate[bot] in #688
• Chore: Refactor ctk.io package by @amotl in #679
• CI: Fix Kinesis integration test race condition in LocalStack readiness and stream reset by @hampsterx in #691
• Update dependency polars to <1.40 by @renovate[bot] in #699
• Update ASzc/change-string-case-action action to v7 by @renovate[bot] in #700
• Refurbish IO documentation by @amotl in #698

New Contributors

• @hampsterx made their first contribution in #691

Full Changelog: v0.0.44...v0.0.45

v0.0.44

What's Changed

• I/O: Added adapter for Apache Iceberg tables
• I/O: Added adapter for Delta Lake tables

Details

• Update dependency cattrs to v26 by @renovate[bot] in #662
• Update dependency responses to <0.27 by @renovate[bot] in #663
• Update dependency marimo to <0.21 by @renovate[bot] in #665
• Update dependency fastapi to <0.132 by @renovate[bot] in #666
• Update dependency fastapi to <0.133 by @renovate[bot] in #667
• Update actions/upload-artifact action to v7 by @renovate[bot] in #670
• Update dependency fastapi to <0.134 by @renovate[bot] in #668
• Update dependency aiobotocore to <3.3 by @renovate[bot] in #669
• Update dependency fastapi to <0.136 by @renovate[bot] in #671
• Update dependency yarl to <1.24 by @renovate[bot] in #673
• Renovate: Configure rangeStrategy=widen by @amotl in #675
• Update dependency dlt to >=1.10,<1.23 by @renovate[bot] in #657
• CI: Update to LocalStack 4.14 by @amotl in #676
• Update docker/login-action action to v4 by @renovate[bot] in #677
• Update docker/setup-qemu-action action to v4 by @renovate[bot] in #678
• I/O: Adapter for Apache Iceberg by @amotl in #444
• I/O: Adapter for Delta Lake by @amotl in #664

Full Changelog: v0.0.43...v0.0.44

v0.0.43

What's Changed

• Retention: Validated integration with Azure blob storage using Azurite
• Fixed missing pandas package that prevented installation/execution without extras

Details

• Update dependency com.rockset:rockset-java to v0.13.0 by @renovate[bot] in #630
• Update dependency org.apache.maven.plugins:maven-compiler-plugin to v3.15.0 by @renovate[bot] in #632
• Update dependency org.codehaus.mojo:exec-maven-plugin to v3.6.3 by @renovate[bot] in #633
• Update dependency markdown-it-py to v4 by @renovate[bot] in #639
• Update dependency node-fetch to v3 by @renovate[bot] in #641
• Update dependency node-fetch to v3 by @renovate[bot] in #642
• Update dependency kaggle to v2 by @renovate[bot] in #644
• Update dependency pytest to v9 by @renovate[bot] in #646
• CI: Save resources by turning off packaging on PRs by @amotl in #652
• Retention: Validate integration with Azure blob storage using Azurite by @amotl in #650
• Update dependency fastapi to <0.130 by @renovate[bot] in #654
• Update dependency uvicorn to <0.42 by @renovate[bot] in #656
• Fixed missing pandas package that prevented working without extras by @amotl in #660

Full Changelog: v0.0.42...v0.0.43

v0.0.42

What's Changed

• Settings: Stop flagging gateway.recover_after_time as a difference
  when both gateway.expected_nodes and gateway.expected_data_nodes are
  unset (-1).
• PyMongo adapter: Adapted to CrateDB 6.2+ which disallows writes to _id columns
• Dependencies: Permitted installation of pyarrow 23
• Dependencies: Permitted installation of duckdb 1.x
• Dependencies: Permitted installation of pandas 3.0
• CI: Validated against Python 3.14
• CI: Validated against InfluxDB 2.8
• CI: Validated against MongoDB 8

Details

• Chore(deps): Bump actions/checkout from 4 to 5 by @dependabot[bot] in #519
• Bump python from 3.12-slim-bookworm to 3.13-slim-bookworm in /release/oci by @dependabot[bot] in #431
• Update marimo requirement from <0.15 to <0.16 by @dependabot[bot] in #530
• Settings: Stopped flagging gateway.recover_after_time as a difference by @amotl in #521
• Update polars requirement from <1.33 to <1.34 by @dependabot[bot] in #532
• Update ruff requirement from <0.13 to <0.14 by @dependabot[bot] in #535
• Update pytest-cov requirement from <7 to <8 by @dependabot[bot] in #536
• Update mypy requirement from <1.18 to <1.19 by @dependabot[bot] in #538
• Update fastapi requirement from <0.117 to <0.118 by @dependabot[bot] in #545
• Update ingestr requirement from <0.14,>=0.13.61 to >=0.13.61,<0.15 by @dependabot[bot] in #543
• Bump actions/setup-python from 5 to 6 by @dependabot[bot] in #552
• Maintenance 20251002 by @amotl in #553
• Update uvicorn requirement from <0.36 to <0.38 by @dependabot[bot] in #550
• Update dlt requirement from <1.16,>=1.10 to >=1.10,<1.18 by @dependabot[bot] in #554
• Bump cratedb-sqlparse from 0.0.15 to 0.0.16 by @dependabot[bot] in #556
• CI: Fix Azurite integration testing by @amotl in #561
• CI: Reorganize workflows by @amotl in #562
• CI: Start using Python 3.14 by @amotl in #563
• Chore(deps): Bump actions/upload-artifact from 4 to 5 by @dependabot[bot] in #566
• Chore(deps): Bump astral-sh/setup-uv from 6 to 7 by @dependabot[bot] in #565
• Chore(deps): Update croud requirement from <1.15,>=1.13 to >=1.13,<1.16 by @dependabot[bot] in #567
• Chore(deps-dev): Update universal-pathlib requirement from <0.3 to <0.4 by @dependabot[bot] in #568
• Update marimo requirement from <0.16 to <0.18 by @dependabot[bot] in #559
• CI: Use Python 3.13 where applicable to speed up builds again by @amotl in #569
• Chore(deps-dev): Update uvicorn requirement from <0.38 to <0.39 by @dependabot[bot] in #570
• Chore(deps-dev): Update duckdb requirement from <1.4 to <1.5 by @dependabot[bot] in #571
• PyMongo adapter: CrateDB's _id columns are not writable by @amotl in #573
• CI: Validate InfluxDB I/O adapter on Python 3.14 by @amotl in #574
• Chore(deps): Update yarl requirement from <1.21 to <1.23 by @dependabot[bot] in #576
• Tests: Fix minio dependency woes about ctor signature changes by @amotl in #578
• Chore(deps-dev): Update testcontainers requirement from <4.13 to <4.14 by @dependabot[bot] in #575
• Chore(deps): Bump actions/checkout from 5 to 6 by @dependabot[bot] in #579
• Chore(deps-dev): Update fastapi requirement from <0.118 to <0.124 by @dependabot[bot] in #580
• Chore(deps-dev): Update minio requirement from <7.2.19 to <7.2.21 by @dependabot[bot] in #581
• Chore(deps): Bump cratedb-sqlparse from 0.0.16 to 0.0.17 by @dependabot[bot] in #582
• Chore(deps-dev): Update fastapi requirement from <0.124 to <0.125 by @dependabot[bot] in #583
• Chore(deps-dev): Update dlt requirement from <1.18,>=1.10 to >=1.10,<1.21 by @dependabot[bot] in #585
• Chore(deps-dev): Update polars requirement from <1.34 to <1.37 by @dependabot[bot] in #584
• CI: Update runner from macos-13 to macos-15-intel by @amotl in #590
• Chore(deps-dev): Update aiobotocore requirement from <2.25 to <3.1 by @dependabot[bot] in #588
• Chore(deps-dev): Update fastapi requirement from <0.125 to <0.128 by @dependabot[bot] in #589
• CI: Use more Python 3.14 by @amotl in #591
• Chore(deps-dev): Update tikray requirement from <0.3,>=0.2 to >=0.2,<0.4 by @dependabot[bot] in #594
• Chore(deps): Bump actions/upload-artifact from 5 to 6 by @dependabot[bot] in #595
• Chore(deps-dev): Update ruff requirement from <0.14 to <0.15 by @dependabot[bot] in #596
• Chore(deps-dev): Update fastapi requirement from <0.128 to <0.129 by @dependabot[bot] in #597
• Chore(deps): Bump vasuki from 0.7.0 to 0.7.2 by @dependabot[bot] in #600
• Chore(deps-dev): Update uvicorn requirement from <0.39 to <0.40 by @dependabot[bot] in #602
• CI/Kinesis: Expand Python version coverage by @amotl in #606
• Chore(deps-dev): Update dlt requirement from <1.21,>=1.10 to >=1.10,<1.22 by @dependabot[bot] in #604
• Tests/Kinesis: Disable test matrix slot for Python 3.11 by @amotl in #608
• Chore(deps-dev): Update undatum requirement from <1.1 to <1.2 by @dependabot[bot] in #610
• Chore: Fix PyMongo software testing by @amotl in #611
• Dependencies: Permit installation of duckdb 1.x by @amotl in #613
• CI: Validate against InfluxDB 2.8 by @amotl in #615
• Dependencies: Permit installation of pandas 2.3 by @amotl in #614
• Dependencies: Permit installation of pandas 3.0 by @amotl in #616
• Dependencies: Update requirements across the board by @amotl in #617
• CI: Validate against MongoDB 8 by @amotl in #618
• Tests/Kinesis: Re-enable test matrix slot for Python 3.11 by @amotl in #609
• Configure Renovate by @amotl in #621
• Dependencies: Adjust test requirements by @amotl in #619
• CI: Reduce amount of Kinesis testing by @amotl in #624
• CI: Remove Dependabot configuration. Has been replaced by Renovate. by @amotl in #625
• Update dependency aiobotocore to <3.2 by @renovate[bot] in #622
• CI: Properly configure versioningit by @amotl in #626

Full Changelog: v0.0.41...v0.0.42

v0.0.41

What's Changed

• I/O: Updated to influxio-0.6.0. Thanks, @ZillKhan.
  • The target table is now the measurement name when importing ILP files.
  • The InfluxDB source URL accepts a timeout query parameter (seconds) to
    configure the network timeout when talking to the InfluxDB API.
  • For ILP imports, the CrateDB URL no longer needs a table component;
    you can point it at the schema only (the measurement determines the table).

Details

• I/O: Update to influxio-0.6.0 by @amotl in #516

Full Changelog: v0.0.40...v0.0.41

v0.0.40

What's Changed

• I/O: Fixed MongoDB CDC invocation. Thanks, Mỹ Duyên.

Details

• I/O: Fix MongoDB CDC invocation by @amotl in #515

Full Changelog: v0.0.39...v0.0.40

v0.0.39

What's Changed

• OCI: Started producing image ghcr.io/crate/cratedb-toolkit-ingest
• I/O: Added drivers for ODBC and Oracle to cratedb-toolkit-ingest
• I/O: Updated BSON library to support ARM64

Details

• Ingestr: Remove "MongoDB to CrateDB" advice by @amotl in #505
• Update botocore requirement from <1.40 to <1.41 by @dependabot[bot] in #511
• Update dlt requirement from <1.14,>=1.10 to >=1.10,<1.16 by @dependabot[bot] in #512
• Status: Make it beta by @amotl in #514
• OCI: Start producing image cratedb-toolkit-ingest by @amotl in #508

Full Changelog: v0.0.38...v0.0.39

v0.0.38

What's Changed

• I/O: Updated to ingestr>=0.13.61
• CFR: Improved log output
• CFR: Fixed double quoting of table name. Thanks, @karynzv.
• CFR: When importing, started using replace strategy instead of append
• CFR: Improved importing data re. type mapping without NumPy
• CFR: Truncated target table before importing, using append
  strategy again, because replace doesn't do the right DDL.
• I/O: Tuned down ingestr, it masked native I/O adapters

Details

• I/O: Improve documentation page about ingestr by @amotl in #476
• Dependencies and documentation by @amotl in #478
• Chore: Fix mypy type checking by downgrading types-docutils by @amotl in #484
• CFR: Assorted fixes and improvements by @amotl in #483
• Update botocore requirement from <1.39 to <1.40 by @dependabot[bot] in #480
• Update dlt requirement from <1.13,>=1.10 to >=1.10,<1.14 by @dependabot[bot] in #485
• Update datasets requirement from <4 to <5 by @dependabot[bot] in #486
• Update fastapi requirement from <0.116 to <0.117 by @dependabot[bot] in #488
• Update mypy requirement from <1.17 to <1.18 by @dependabot[bot] in #489
• Update pyarrow requirement from <20.1 to <21.1 by @dependabot[bot] in #490
• Update testcontainers requirement from <4.11 to <4.13 by @dependabot[bot] in #492
• Update polars requirement from <1.32 to <1.33 by @dependabot[bot] in #497
• Update aiobotocore requirement from <2.24 to <2.25 by @dependabot[bot] in #495
• Dependencies (PyMongo): Fix error with NumPy 2.x by @amotl in #501
• Chore: Remove Python 3.10 from compatibility support for Kinesis nozzle by @amotl in #502
• Bump astral-sh/setup-uv from 5 to 6 by @dependabot[bot] in #493
• Chore: Fix boot-time warning about ingestr by @amotl in #500
• I/O: Tune down ingestr, it masked native I/O adapters by @amotl in #503
• CI: Separate GHA workflow definition for Kinesis by @amotl in #504

Full Changelog: v0.0.37...v0.0.38